I have Axiom Seymour panniers. Like most panniers, they don't look very big when unloaded. Each pannier can hold 10lbs of frozen chicken, plus around 4-5lbs of vegetables. I can overload my bike with them - at 45lbs, the handling gets very sketchy. (frozen chicken and vegetables aren't very dense, so other sorts of food loads can weigh more)
I do *want* bigger panniers. It is sometimes inconvenient to carry bulky items like flour or rice in mine. But I worry that bigger panniers would tempt me to overload my bike more often. I can definitely get enough food in them, but sometimes there is squashable stuff that doesn't do well.
As far as books go, mine will handle about 3 standard hardcovers each, with room for additional paperbacks.