This might not work for training proper i.e. higher speeds, less stops and more wind, but this works for me for commuting down to about -10 deg C. (14F), up to 1 hour rides.
regular bike shoes, though I've bought winter shoes for this winter
wool socks
neoprene shoe covers
regular bike shorts
PI AmFib winter tights (w/room for a thin wicking base layer under but I don't think I've ever used it)
sports bra
thin wicking base layer top, turtleneck+long-sleeved
sometimes an extra sleeveless thin wicking layer, wool or synthetic
Gore Phantom jacket (LOVE this jacket, it's warm and it doesn't get clammy)
Buff or face mask over my neck and nose
Gore windproof skull cap (very handy)
big fat PI lobster mitts (not too thrilled about these, they're lumpy)
Remove layers as needed
I don't usually wear goggles or glasses, but I'll use some water-free skin cream or Vaseline in my face before going out.
And eating well in advance and being warm before you leave the house really makes a difference. It's very hard to warm up fingers and toes if you're cold inside to begin with.
(Ok, I re-read your post and maybe you weren't really asking about below freezing is-it-still-possible-to-bike gear. I like biking in winter, I get carried away
)
Winter riding is much less about badassery and much more about bundle-uppery. - malkin
1995 Kona Cinder Cone commuterFrankenbike/Selle Italia SLR Lady Gel Flow
2008 white Nakamura Summit Custom mtb/Terry Falcon X
2000 Schwinn Fastback Comp road bike/Specialized Jett