That kinda sounds like Prickly Heat, if so get some Gold Bond on it and get out of the short ASAP after your rides.