I am also old and slow and I have a somewhat esoteric race bike that might have some people making fun of me.
But, I don't care. I am pretty fit for my age and if some think I'm a wanna be, let them. I actually have received more c**p from people my age who have been riding for years that are just too cheap to buy a bike like mine. It's a particular brand of reverse snobbism that is somewhat common here in New England. So while they ride around with their bikes literally taped together, they make fun of me...
The moral is, don't let others decide for you.