I lost about 20 lbs when I started cycling. I saved one pair of jeans in one size larger. After about 3 years, I threw them out. I didn't want to assume I'd gain the weight back, and I didn't.
I give all of my clothes to Goodwill, or a charity that provides business type clothes to women getting off of welfare (I used to get much more dressed up for work).
I basically go through my closet at the change of seasons. Anything I haven't worn in a year goes to Goodwill. This stops me from buying "faddish" clothes. I spend a lot more on cycling clothes now, but now I am buying a few more things for work... since my only summer type clothes are not totally appropriate for work, even in a casual environment.
2015 Trek Silque SSL
Specialized Oura
2011 Guru Praemio
Specialized Oura
2017 Specialized Ariel Sport