It helps to slap a bandaid over these not just when riding (it'll either stay, or rub off right away), but over the next few days under regular clothes. They seem to need a period of peacefulness to go away.