I'm sure you've ruled this out... but sometimes my pump, which is strapped to the down tube, rotates around a little bit as I ride, and my shoe hits it as I go around, between the 12 and 1 position, exactly you describe... I usually just have to get off and cinch it up a little tighter.