i think Duct Tape will get GOOEY and put gross marks on your shorts. But it won't permanently ruin the saddle, you'll be able to get it off eventually.
Remember, Duct tape doesn't stay stuck if it's wet or hot..