My Take-A-Look worked for me right from day 1. My DH hated it and took it off after the first ride, refusing to try it again. I have no idea what the answer to your question is, except maybe to try a different kind of mirror. My DH now has a handlebar-mounted one and does fine with that one.
Good luck!