Well, I don't consider myself lady-like at all but it doesn't bother me to be called a lady.

Having said that, I would typically refer to a female athlete as a woman, not a lady. Somehow lady just seems a little weird to me.