Momentum magazine has pictures of people riding in all sorts of clothes. I had to get some specific bike clothes after getting so cold I couldn't warm back up for days. I was sweating too much and getting chilled. You shouldn't be made to feel bad about whatever you decide to wear on a ride, it's a personal preference thing. I also get winter hives, so I had to break down and buy windpants. I think I look weird in bike clothes, but they feel good on me. When I got started doing this it was always just sneakers and a t-shirt and regular shorts, until I could slowly afford the other stuff.