first you need a lightweight rack on the back of your bike. You can hold all sorts of things there, including panniers, booster seats, etc. And you can get a little bag that ties on the back of your seatpost as well.
Another possibility is to get a little bag that hangs in front of your handlebars. Hm, I have all of that sutff.