Some people wear their bathing suit and then slip on bike shorts and a top. It's kind of difficult to ADD clothes when you're wet, though, especially bottoms or tight fitting tops. There are triathlon clothes made specifically to wear that fit close to the skin, shed water, and have a little bit of a chamois in them, they come in either one-piece or two-piece outfits. It is easy to put on a looser-fitting top over them in transition if you want a little more warmth and/or skin coverage.
Usually you can find tri clothes in triathlon-specific stores and in a lot of bike shops.
Good luck!!