Take this for what it is worth, which may not be much. When I swam team as a kid, we used something called 'sweet oil' in our ears. Both before and after swimming. I only remember having swimmer's ear once.

Course, this was a couple of years ago.