I use the standard wide-mouth bike bottles you can get at any shop. I switch to polar insulated bottles when it heats up.

I get the plastic taste out by rinsing a new bottle with mouthwash (listerine). Amazing how well that works.