Gee, I would say that Pearl Izumi runs large. But maybe it's because even if they make something in an x small, the stores don't carry it and the smalls are huge on me.
Craft is the only brand that I have found that runs a little small on me. It's more like the xsmall is too long and it is very tight. The small fits, but is waaaaay too long. Gore stuff fits me snugly, but I like that for cycling gear.
Currently, I only buy Terry Bella shorts or Shebeest S pro shorts. I don't even try other styles in the same brand. I can't wear Shebeest jerseys, the top part of the jerseys, even in x small, are huge on me or have weird shoulders, or some other defect.
Like Oak, I find regular clothes getting larger and larger. My body size has not changed much in the last 10 years. But, I've gone down 2 sizes in some cases. Yet, the dress I bought for my older son's Bar Mitzvah in 1995 still fits me, and I suspect that is my "true" size.