I've never had any trouble keeping my core warm on the bike, even riding in the 20s. But my extremities do get cold. Your arms aren't really doing a lot on the bike and really catch the wind. Fingers and toes can be just miserable.

On cold days I've been known to wear arm warmers under a long sleeved jersey.