Sounds like you're not sitting squarely on the saddle. Have someone stand behind you to see if you're hanging off to one side. If you're not centered, you need to address that or it will cause you problems.