I wear pants that have wind blocking properties in the front only. Nice akle zippers. They have articulated knees, too. Over cycling shorts or cycling under shorts which are thinner.
I also wear a bright yellow jacket that has pit zips, is waterproof, breathable and can be cinched at the waist or bottom, which are great for regulating heat, you wouldn't believe how great that is so you don't build up a sweat.

Under that I just wear a long sleeve cycling or polypro shirt that wicks really well. Depending whether it has fleece or not, I may wear a thin sleeveless silk or polypro shirt under that. Zippers that go up and down to regulate are also important.

Lobster claw gloves and shoe covers. Sometimes I even put those little warming packets in my shoes--like skiiers. Headband that covers ears or skull cap, thin. Yes, lots of lip balm. But I'm talking below freezing riding. Which by the way is nice when the sun is out, but harder when it is cloudy.

I also recommend good glasses that cut the glare for winter riding because the angles of light make it harder to see (esp. pot holes which proliferate where I live in the winter), and blinking head and tail lights because it is harder to see you.