You're definitely not being too sensitive. We have enough trouble with the non-cycling public, and shenanigans like that certainly don't help. I can think of at least one ride leader here that would have been screaming his head off at those other riders.