I think the key, as others have said, is to have a variety that you can mix and match for different conditions.

Top half of the body:
- Pretty inexpensive, I like the non-wool base layer that runners wear. I have one that is thinner and one that is thicker. I'm allergic to wool.
- A wind-stopping jacket (for cold temps, under 45 or so)
- Long-sleeved jerseys (for warmer temps, above 50)
- Long-sleeved, fleece-lined jersey/jacket (in between)
- Arm warmers to use alone on warmer days with a regular jersey or to layer for cooler temps

Bottom half:
- Knickers (for the warmer end of the winter spectrum)
- Leg warmers (for below 50)
- Tights (for below 40 or so)
(I personally like the tights with a chamois and wind-blocking front, but these can get pricy - as others have said, there are lots of options with tights, some less expensive.)

Hands:
- Thinner full-finger gloves (for warmer temps)
- Wind-blocking full-finger gloves (for those cold to cool 40's)
- Lobster claws (for really cold)

Feet:
- Thick socks (wool, as mentioned above if you can wear wool)
- Toe warmers for the warmer temps
- Shoe covers (neoprene) for the colder temps

Head and neck:
- Ear warmers for warmer temps
- Scull cap for really cold (it has to be really cold for me to cover my whole head - like below freezing)
- Balaklava for really cold
- Neck warmer for relatively cold

I once heard and adhere to this tenet: Your first thought when you start riding ought to be, "I should put on one more layer". You don't want to be thinking that you need multiple extra layers, but if you're just cold enough to think you need one more layer, then you are properly dressed for when you heat up and start sweating.

As noted above, each person is different and wind plays a huge role in how you dress. It can be 50 and windy and you will dress like it's 35 or 50 and calm and dress like it's 60. I note in my riding log what the conditions were, what I wear and how I felt and that helps for future rides.