Lots of layers of thin wool that I can put on and take off.
As long as it's daylight, the road is dry and it's above 20F, I can go for a ride.
Balaclava (sometimes 2), several layered gloves, regular pedals and heavy insulated hiking boots with good wool socks. Helps to have a decent sized saddlebag where you can put layers you peel off as you warm up.
I keep my rides shorter, 20 miles or less when it's really cold.