Earlier this summer, I got a wicked sunburn right under the silicone gripper of a pair of Terry shorts. It was this little patch about an inch wide and ~4 inches long- very funny looking! I think that, although I vigorously lathered on the sunblock, the gripper moved and rubbed it off. I didn't realize this until much later...post-burn

Just thinking out loud here... The fact that the others don't bother you makes me think it's not an allergy or sensitivity...