It's probably more to do with personal world-view.
Sometimes the world is more comfortable if one has a few large classes of persons to put down, in order to make oneself higher up.
On a happy note: Sarah Chan's bike blog is one of my very favorite internet things
http://www.girlsandbicycles.ca/ She loves bikes, and she loves clothes, and she loves activism. Through-out her blog posts, you can see that she gets a kick out of being creative with clothes. It's art, it's fun, and she shares her excitement about clothes just as well as she shares her excitement about her bikes.
Perhaps pleasure in clothing and make-up might look fussy, but so might any one us with our detailed concern about our bikes and our knowledge about bike parts and types. Is it fussy for us to choose the "right" bike from our stable for a jaunt to the coffee shop? Is it fussy to choose a different bike for a 50 mile ride-like-the-wind? How about choosing a folder for an evening at the symphony? How fussy is that?
We're all fussy to someone else.
Don't fret over others, don't search for a pecking order.
Ride lots.