I do ride with someone that is slower than me, for most of the rides I don't do with DH. Frankly, I have found it hard to find people who ride at my speed. It's not slow and apparently, it's not fast, either. I also have found one group I am "good" with. But deep down, I like riding with these people because most of them are slower than me! I get compliments from the guys and I can climb with the best of that group.
I don't like competition, but I guess there is a part of me that is competitive. DH keeps me on my toes and forces me to do a faster ride every so often, but the thrill of that has worn off. Getting dropped will always piss me off, so I choose to ride with people where the chances of that are not huge. Like I said, 99% of the time DH and i are riding together and we really are together.