I have a pair of the REI wind/waterproof pants. They are designed for bikies so the ankles are more fitted and the knees have a dart in them so there's room for the pants to move over the knees when pedalling. I wore those all last winter with leg warmers/shorts and was fine.
(ok...I'm a whiney baby when it come to the cold but I wasn't going to give up riding and those worked for me!)
Life should NOT be a journey to the grave with the intention of arriving safely in an attractive and well preserved body, but rather to skid in sideways, champagne in one hand, strawberries in the other, body thoroughly used up, totally worn out and screaming: "Yeah Baby! What a Ride!"