My shorts are about that low and come up pretty high in the back. I hate anything touching my bellybutton so I like them that way.
Although, I would like to try some bib shorts. It seems like they would eliminate the problems of the elastic band digging in.
