I do less and less group rides as the years go on. Although I ride with a more "social" group and they are very experienced riders, some of them are twitchy enough to make me stay in front of them... there is one particular woman who races me on the downhills and then can't keep up with me the rest of the time. I admit I love dropping her on climbs. She's very twitchy and actually has tried to pass me on the right. There's others like her, and I can't figure out why, after riding for 20+ years, these people don't have better skills.
I think some of it is riding in large groups. With more people, there's more room for problems. The rides I lead tend to be smaller groups, and then of course, I have "control." I've been known to use my teacher voice to "help" people ride safely; like the guy who was trying to make a left turn across a busy 2 lane road from the right lane, when I was sweeping. I can still hear myself saying "Get in the lane!"