I get a rash from silicone grippers - it's not chafing, it's a sensitivity reaction, but I can see how they could chafe.
I just turn the grippers up so they're on the outside. They still provide some reinforcement to keep the fabric tight so the legs don't ride up, although if the shorts are already short on you, that will make them shorter. Some brands actually print their logo upside down on the grippers so that it's right-ways up when you do this.