My arms get colder than my body, so I wear a bolero over a wool sweater in winter, under my jacket. I like the extra wind resistance. So larger in this case.

A tighter fit would be good if you're wearing it over short sleeves.