We probably need a FAQ on "layering".

Check out this thread.

For the temps/conditions you are describing, I would choose arm warmers, knee warmers, a wicking sleevless base-layer and a wind-front vest.

If it was in the 30's I'd also start off with an ear-band, magic-gloves (yep those 99 cent wonders) over my cycling gloves and toe covers (full-on insulated booties make my feet sweat if it's over 35F). If you don't own toe covers and your feet get cold, you can try to plastic baggie trick.

From your description it sound like your jacket may be the culprit. It's holding too much heat in and not allowing your sweat to evaporate.