Well for starters, several people have told me that neoprene booties over your shoes are important for keeping your feet warm. I just bought these and I like them so far. And this is a really good price for a product like this.

http://www.teamestrogen.com/prodLG_1083073.html

I also recently bought this wool cap for under my helmet, and I love it.

http://www.teamestrogen.com/prodIB_852.html

I asked some friends last year for advice on what to wear for cycling in temps between 40 and 50. I got a variety of answers, because everyone tolerates the cold differently, but the basic theme was layers. Many people like to wear multiple base layers and jerseys under a windbreaker. This gives you the opportunity to shed layers if you start to warm up on the ride. Wool is great, very comfortable and fast-drying, if you can afford it.

For my hands I've been happy so far with unlined wind-blocking gloves over a pair of Smartwool glove liners. But I haven't tried riding when it's below 40 yet, so I don't know how well they would work for colder temps.