Cold water on the face tends to stimulate a gasp reflex that can make it hard to breathe and easy to panic. I've dealt with that, and it's something you can get used to. Splashing cold water on the face, dunking the head, etc can prepare the body and reduce the reaction.

Something else I've dealt with is "freak out on go" syndrome Starting out too fast, getting out of breathe, and then panicking.