I still wonder how the word "commando" has come to mean "without underwear," (doesn't say that in my dictionary, I don't *think*) but ya you betcha, it's skin against the chamois.
I, too, have always tossed the shorts in with the laundry, though if I remember, I pull them out and toss 'em on the line instead of putting them through the dryer, which I suspect is the more damaging heat level. I think lycra's a little sturdier than way back in the day when it was the AMAZING replacement for nylon swimsuits (and that one-design-where-the-straps-always-fell-down)
I have also seen what can happen to lycra when it wears out... if you've ever ridden behind somebody whose shorts have gone to the see-through stage in patches...
If I were having yeasties I'd probly try ... oh, a vinegar rinse? There's gotta be a pH that makes 'em die.