I'm still pretty new, so take everything with a grain of salt.

I used to be in the same boat - so I hand-washed each pair as soon as I took them off. Did this for two months, no problem. But, I just started to get some bad saddle sores, and I'm wondering the same thing - did I not wash them good enough to get rid of whatever bugger causes the saddle sores. I just threw all my shorts in the machine. My plan is to wash them daily by hand, but once every few weeks I'll toss them in the machine.