I love dressing up, but there really are not that many places that require super dressy attire, even here in Boston. Those are places I might go to for a special occasion, where my husband would wear a tie. There are lots of "fine dining" restaurants where a work type of dress or pants are fine. I might dress it up with jewelry or shoes, etc. The rest of the places I go to are just casual and it doesn't matter what you wear, as long as you have shoes and a shirt on.
I think it's important to dress appropriately for whatever the situation is. If you don't like wearing a dress or whatever, then don't go to the places that require it.
I know that when I've been in Europe, I appreciate how nice everyone looks. It's not that they are super dressed up, but people seem to think about this more. Since I spend a lot of my time in sports type apparel, dressing up seems even more special!