I have a friend who calls this "lycra leprosy." I've had it happen more with some shorts than with others. I think it's possible that even a quality fabric can start to wear from abrasion.

I've never had to get rid of any shorts because of this problem. Typically I can still wear them until chamois starts to wear out, and sometimes then I can still use them for short rides. If they're otherwise comfortable they can also be worn under tights in the winter.