My threshold for riding in the cold has moved down in recent years, but this year, it seems to have taken a momentary pause.
I'd say, realistically, I don't go out if it's below 35 and it's not going to get much warmer. However, 30 is my absolute limit, and I have started/ended many rides with temperatures between 30-35.
For me, limiting factors are wind and degree of cloudiness/dampness. Also, road conditions. For example, on election day, it was brilliantly sunny, but cold for early November, maybe 40 at the highest. I rode to vote, about 4 miles, then rode about 18 miles for fun, ending at my client's house for work. When I left to go home (5.5 miles), it was dark, extremely dry and about 32. I was cold the whole time, mostly because I needed my warmest gloves, and I had my mid level ones. It took me awhile to get warm after arriving home.
Yesterday, I chose not to ride outdoors, even with temperatures around 48-50. Why? It was cloudy, damp, and there still was wetness on the roads. That means I'm going to get sprayed and I don't like that. For me, dampness/clouds/wind combined with cooler temperatures really take the joy out of riding. I do it in the summer and fall, and a bit in the spring, but I found that after riding through the winter last year, I like a break in my activities, too, and that's part of it.
And like LPH, I have more trouble with heat.