I take up to 30 lbs on one side only all the time with no balance problems. A heavy, unbalanced load is noticeable, but as long as you're not exceeding the load limit on the rack it's not a hazard.

redrhodie, don't worry about what you're wearing. When I see people at the grocery store in athletic clothes, I assume that they've been out for a long ride or somesuch. If you want to wear street clothes, you can get a strap with velcro for your pants leg, to keep it out of your chainring. I also find a sports bra to be indispensable.

If you think you'll occasionally be picking up things like kitty litter that might not fit in your pannier, pick up a bungee cord or two at the hardware store for strapping stuff crosswise across the rack. I love hauling home weird stuff like garden hoses, just for the double-takes from motorists Welcome to the world of transportation cycling!