Although it is easy to say 'who cares what others think about you', it is much harder to do it.

From personal experience, people who are weak get picked on more. I'm not saying that you are weak, but next time they make a joke on your expense, chime in! Be loud like them. Make fun of them. Just join in! That's just the way men are. If they make a comment about your bike, tell them you can kick Stacy's (that's her name right?) ***. Or any of their asses. They made fun of your jersey? Next time somebody walks in with something god-awful, make a sarcastic comment. Chances are, they don't know that your grandma suffered from diabetes. They make fun of your horse? Point to your legs and say you have no problem climbing hills because of the muscles you've developed riding your horse. Old man from the bike shop downstairs comes upstairs to check you out? Ask him where your coffee is. Doesn't have it? Then why is he up here? If you can't beat them, join them. You don't want to stick out like a sore thumb when the company culture encourages the type of behavior your colleagues exhibit.

The guys I ride with know not to mess with me because I can make lots of sarcastic comments about their shortcomings. When I do, the guy at the receiving end just shuts up and his buddies crack up. Boy dynamics. Same goes for my non-cycling guy friends.