I always have one pannier on the bike (left side so cars to my left better see how wide I am).

The second one (the right side) gets added if I'm doing a big grocery trip.

I've gone through a few sets of panniers. My favorites have been my Ortliebs. Expensive, but worth it to me. Absolutely waterproof. Ortliebs carry a LOT of stuff, and more than once I've gone to the store with both panniers but only needed one after all.

Never had any problem with balance, in fact I feel my balance and bike handling are much more fabulous with one pannier than with a backpack.