I had the same problem when I started riding. I'm sure the other ladies are right that it does go away with time, but I had to adjust my bike fit so that my weight was not so forward on the bars. That got rid of it completely and I still had a good position.

Also, Specialized makes a women's glove that is very good for nerve pain in the hands - it's one of the ones from their ergonomic line.

I've tried Trek, Pearl Izumi and others and the Specialized has been the best and lasted the longest.