Since I always unclip and lean left, pushing a crosswalk button is impossible for me, because I fear exactly what happened to you! The only time I ever have to do this, is if I ride to the gym at 5 AM and there isn't enough traffic to trigger the light to stop traffic on the highway I have to cross. It's a 2 part crossing, with a second signal in the center island. I usually just ride across for the first crossing, without pushing the signal for the crosswalk, as there is rarely traffic coming west at that time in the AM, but there is a lot of traffic going east, toward Boston. In addition, the signal is off to the side, up the curb cut for the sidewalk, nowhere near where I am on the road. However, when I get to the island in the middle, the signal is on my left, and I just reach over and push the button. I ride across this intersection all of the time, during daylight hours, and it's just a regular intersection, but without the cars behind me, the first time I did this at 5 AM, I could not get my bike to trigger the automatic signal for the lights to change.
2015 Trek Silque SSL
Specialized Oura
2011 Guru Praemio
Specialized Oura
2017 Specialized Ariel Sport