How do you come to a stop out of curiosity? Are you unclipping just one foot or both?

I would suggest going to a parking lot and practicing over and over again until it starts to feel more second nature. If you're not already, just unclip one foot to stop. If you're using two, I just think that overcomplicates the process, especially in emergency situations.

I have to admit that this isn't an issue I've had much trouble with. I do unclip a bit before a stop just so I'm ready to put a foot down. That's not to say that what you're going through is highly unusual, but I'd encourage to work on it, rather than avoid it, because you're otherwise just going to develop even more anticipatory anxiety over it.