Just rinse it a lot - and the tube if it's new.

If the water has been sitting for any reason, I spit out the first mouthful. I also fill it up fresh before every use (which usually doesn't require spitting out the first mouthful)

I don't usually taste plastic (unless it's been sitting for a week or so)