Are you hanging to dry? My friend and I got new shorts at the same time, she had been drying hers and they were history after about 500 miles?
I've been washing high end real chamois for almost 20 years (only wearing for less than 2) And have never used a special soap, just cold wash and hang dry!