I have my Polar bottles. I've never noticed a plastic-y taste, and they haven't started growing anything yet (one's a year old, the other's about 8 months old.) The only problem I've had is that if I'm not paying attention and put the bottle in the cage the wrong way, the loop rubs my leg. I could take the loop off, I guess...
The problem with steel is that I imagine it's harder to get into the bottle cages because it doesn't flex. Or is that just me?