Winter riding is about layering. There is some trial and error involved, as you need to figure out what works for your riding style, your area, and your individual comfort level.

Here's what's in my closet:
Long-sleeve jerseys of a couple weights (normal long-sleeve, heavy weight, thermal)
Smartwool baselayers (long-sleeve and sleeveless; these things were the best investment I've ever made in bike gear)
Leg and arm warmers
Thermal tights (I like Sugoi Midzero Zaps--they're actually running tights, so no chamois, but that means I don't have to wash them after every ride.)
Gloves (mine are PI Cyclone; they're more a fall glove, but I'd get more use out of them if I used them with a liner.)
Good socks
A jacket
I really should get a set of toe covers or booties too.