Chamois can definitely pack out, same thing that happens to a ski boot liner. The materials break down from age, washing, friction, pressure, oxidation. If you are rotating between two identical pairs of shorts with the same amount of wear, and you didn't have problems with them before, I would seriously suspect that the chamois are, indeed, worn out.

Good excuse to buy new shorts !