Kajero, I agree with all of what's said above.
I am the same height as you. Do not sell your x small Ruby. I can guaruntee you will be too stretched out on the small. I am also long waisted, so while I am petite, in every other sense of the word, my torso is longer than other women my size. I think that what Rebecca said is important, in that power to weight ratio is the most important thing. Losing weight and maybe doing some targeted strength training will help. I am just a couple of years younger than you, so don't think that you can't improve with age. If I did more than I do, I would be faster, but I also know I am not super slow. I can put the hurt on if I need to. What I've found is that a lot of women don't want to "suffer" and have issues dealing with the pain it takes to improve. I certainly don't want to do this on all of my rides, but sometimes, you need to, in order to see change.