I'm not a real fast rider, but i've done the same as you when it's windy. I slow down. Then the wind can't push me as far!