I used to wear T-shirts during first 6 years or so of returning to cycling...and I was cycling triple the annual mileage compared to what I've been doing for past 18 months.I think a lot of people hestitate to start with exercise in general because they think they need all this expensive gear, but clothes shouldn't stop you from doing anything you want to do. If you do what you love you won't think about how you look anyway
Then I got into a jersey and did feel more comfortable. I resisted the longest, because I was afraid of looking like an athlete or similar.
For awhile I only had 2 jerseys, 2 chamois-less cycling shorts and 2 tights. Plus 2 cycling jackets. All purchased over 2-3 years. (I say chamois-less because that's what I wear. It works for me but not for everyone. But they are cheaper.)
No need to break the bank on this. Yup, I hardly buy a new fashion, off-bike item. For some people that would be disasterous. It's ok..if I can buy 1 new off-bike top annually or similar because I've made my bottoms last several years longer, it works for me.
Saves my dress/biz wardrobe. I don't buy the argument fully on "cost", when lots of low-income people own and drive cars daily. THAT is alot more expensive..sometimes well justified, sometimes not.