They make clothes for everything now, so as long as you're comfortable, you can ride.
I read that if the temp is below like 50 or 55 degrees your knees should be covered, that they dont take the cold well. I usually wear knickers anyhow (i have leg "issues" and wont go out with sausage legs!) and depending on how cold I feel, i may wear knee socks or throw on my long thermal riding pants.
I have a hard time keeping my hands warm at anytime, so finding gloves that work is step one, and making sure my hands are warm before I put them in is step two. I'll also use those little hand warmer things.
Cold is all about tolerance, and you have to be comfy. There is no special temp that tells me whether I ride or not. Snow on the ground? I'm driving! But only because it's dangerous conditions, not cold. Then again, I'm in Colorado - I LOVE the outdoors in winter!