I have had this problem, especially when first starting out on a ride in colder weather. I got a different pair of sunglasses that provide more coverage in a more "aero" shape (Tifosi Slip) and that has taken care of the problem. Good luck!