Do not wait for another event. Tell your local law enforcement agency now.

Do not go to his home, with or without your husband.
He wants to control you and showing him how nice you are will not change his mind.

Talk to your neighbors about their experiences with this guy. See if he is intimidating others or just stalking you.

Change your riding venue.
When I remove my front tire I find I get better leverage if I stand in front of the handlebars. I hold the bike up by pinching the front tire between my knees. Once I flip down the lever, I move behind the handlebars and bend over them to finish loosening. I can't even budge the quick release if I approach it from behind the bars. This works 100% even if my hubby was the one who last put the tire on.
I have a VW GTI hot hatch and my bike fits great.