I think it's quite normal. I suppose you'll either get used to it, or it will happen more rarely when you get used to cycling..