It's just the different sensation of being horizontal then getting quickly out to vertical. The longer I'm horizontal swimming, the worse it is. Also, if the water is choppy and I'm fighting waves, it's worse. Must be the equilibrium getting off kilter when thrashing about in open water (although it doesn't seem to bother me in the pool that I've noticed- but then again I'm never swiming fast, then jumping out to run to the locker room ).
The more I swim OW, the less it bothers me. Perhaps it's just something we become accustomed to (like the smooshing of the soft tissue on the saddle?)?

I wonder if wearing ear plugs on the swim would make a difference?

I'm sure others will have far more wisdom than I...