This does happen, it happened to my first pair of shorts pretty quickly. Seems to me that even if you haven't lost any real weight but if things have been tightening up that the same thing would occur.
Heh, actually, I haven't lost "weight," but I have lost body-fat and inches. Yay for muscle!