What kind of pedals are you using? What kind of shoes?

If you are using clip in pedals, different pedals are more prone to cause this problem and cleat placement is a very important factor no matter what pedal you use.

Often bunions come with a flattened metatarsal arch as well, is that true for you? This is the arch in the ball of the foot and is not the same as the arch in the inner foot area (the medial longitudinal arch), although it is not uncommon to see both arches fallen. If you have an arch issue, you may benefit from an insert with a metatarsal arch support (sometimes called a button).