I've always had handlebar mirrors (Zephal), for years and on several different bikes. Finally, I couldn't take the sloppiness and the constant adjusting, and realized it was really distracting me.

Bought a mirror that hooks on my glasses (CycleAware) and really love it. Took about 1/2 hour to get used to it. The field of view is awesome for me, it doesn't go out of adjustment when I go over bumps, and it doesn't block my view.

I'm much happier with the glasses mirror. Your mileage may vary.