Here are the grocery bag panniers I put on my bike:
http://harmonias.com/bikes/
See the blue Rivendell Rambouillet there.

They stopped making my panniers, but the others already pointed out on this thread are similar folding ones. You can put extra zip ties in several underneath spots which will deter impromptu thieves from just grabbing them and walking off. Then just take your grocery totes in with you.
I'd think that anything looking like a nice backpack left on your bike would get stolen more quickly.