Long sleeve jerseys come in all different weights and permeability. I wear them a lot with another layer underneath. Windjackets are reserved for really cold days or rain because it is too easy to get sweaty and then cold. But I have heavy windfront lycra jackets which I adore for my nighttime winter rides. I'll ride to about 30 degrees at the lowest. Colder than that and I'm indoors with my Irish Coffee.

You bet they're worth it.