Hey there Sunshine
You sound plenty light enough for your height
Women will tend to look heavier than men because we have more fat, and we cannot get down to the same leaness as men without compromising our health.
Thats a physiological fact - so be careful in your consideration of weight loss.

Also, remember that one part of being faster on the bike is the power to weight ration equation.

Yes, losing weight is extremely advantageous to riding faster, however, at some point the 'perfect' power to weight ratio is reached (which is unique for every individual).

If you drop below this weight, your power output begins to decrease.

How long have you been biking for? If its not long - that is - only a year or two, then you have only just established/or begun to establish a good base for improving speeds/times. Perhaps you should consider some specific training such as intervals, or hill reps, or motor-pacing.

Also consider what kind of riding you like - if you like enduro flattish courses, or time trials, then a little extra weight is not as important as if you wanted to be an ace hill-climber.

Specific training may build muscle and you may add a pound or two, but you need muscle to make you stronger and faster. Its a worthwhile compromise.

Good luck