Exactly. It's much easier to STAY warm than to GET warm.
I've never had great clothes to ride in the rain like the Pacific Nwesterners we have on this forum, but I swear by the cheap clear plastic raincoats with velcro on the front that they sell in bike stores. You may get hot from within, but at least it will keep you warm.