I'm presuming you were riding, not running (I'll let the tri gals address those issues - I haven't gone there yet) -
What brand shorts? How do they fit? They should be very snug. Sounds like they may be too large if they're bunching up? Could be that the outer layer was re-arranging them? I think most of us have had shorts bunch up and rub unpleasantly (I know I have) - they were too big, and I need a more "sculpted" chamois. Current preference = Shebeest SSS Shorts (but I know there are a few that those shorts just don't work for). So - it's all personal preference and, expensively, trial and error. Don't give up on bike shorts just because those didn't work well for you!
Most days in life don't stand out, But life's about those days that will...