A few years ago I did a a 2-week bike trip in Belgium & Netherlands, in Ocotober where it rained nearly every day. I bought a Goretex jacket from Performance with zip off sleeves. I love that jacket and it worked out really well on that trip. I also bought some cheap Goretex rain pants from Sunny Surplus (sadly, out of business now). I wore the jacket every day on that trip, and wear it all winter when I'm riding at home. The pants I wore on many of those days on the trip but have not had them on since. Then again I don't normally start a ride if it's raining. If rain happens, that's another story, I'll deal with it, but I ride for fun and riding when cold, wet, miserable isn't so much fun for me.

Oh - you said affordable. Well, you can use a plastic shower cap over your helmet - will help to keep your head dry, and thus warmer. In a pinch you can use a plastic garbage bag as a ponch, but eventually that's a mess and you'll get soaked.