It's true that cyclists who look like TDF racers get nothing but disdain from car drivers. And cyclists who are obviously females decked out like screaming fluorescent Xmas trees do get a wider berth from drivers. I think we actually frighten them because we are obviously 'mental cases'.

On the other hand, there are always drivers who won't see you no matter what you wear (though they'd probably see you in a hurry if you weren't wearing anything), and I tend to ride with the default assumption that they do NOT see me.