There's acclimation and there's aging, and I'm pretty sure both play a part.
It takes me a good month for my body to acclimate to the temperature every time I move. My ex-husband served in the Air Force in Thailand, and he tells a story about coming home to Ohio in August when it was 85° and humid, and being just freezing, literally had to wrap himself in blankets. And he was 20 years old then. Except, although I can get used to humidity, I still can't breathe in it - even the one time we were here for a year and a half, I breathed MUCH better as soon as we came home, and that wasn't just emotional. 
My parents (in their 70s) still keep their house freezing cold and sleep in the basement, but that's stubbornness.
I gave up trying to tell them how the cold and damp in that house was damaging MY health even on short visits oh, about 30 years ago, and now that the damage to THEIR health has accumulated, they just don't care and I'm sick of telling them.
For the most part though, it does seem like people lose the ability to tolerate cold when they get older - my in-laws keep their home super hot, and I can't tell you how many times I've visited elderly clients whose thermostats were set at 78 or 80 in winter, even when they couldn't really afford such a high utility bill. And like you guys, when I was a kid I know I used to go around in the cold out of stubbornness and machismo.
But I think it has as much to do with not feeling like I need to prove anything at my age, as anything else.
But I've read that from a health standpoint, as you get older, while warmth may feel good, your body actually loses the ability to tolerate heat, and the #1 weather-related killer is actually heat, not tornadoes, hurricanes or earthquakes.
Speed comes from what you put behind you. - Judi Ketteler