We just rode this past weekend in temps like you mentioned (50-55). I don't mind it and was happy and warm in shorts (I'm a freak), but DH was having serious issues despite wearing tights. I made him back off the pace with me and told him to spin. He does well with spinning up hills, but he likes to push bigger gears in the flats. I told him he could do that...ONCE he was warmed up. He's a southern boy, too...so to him, "warming up" was a 2-3 minute exercise. Once I got him to agree to spin for awhile at a slower overall speed (I led to make sure he couldn't mash), he was fine. I think it was a solid 20-30 minutes before he felt that he'd gotten his leg strength back.

Now he complains about how his early commute takes him longer because he's trying to spin more to warm up. I remind him that he can take an extra 10 minutes per day to get to work now if it means that he'll still be able to ride at 60!

Anyway, my point is to work on spinning for the first part of any ride...but particularly in those temps. Yes, your speed will suffer if you aren't real efficient with spinning, but it's worth it. Once you are truly warmed up, you can kick it into a higher gear.