Yes, I've always been in the in between group.
When you average around 15, it's hard to find a "group." It took me a couple of years... but it's true, I ride faster with a faster group. On the other hand, I don't do much group riding anymore. Just with 1-5 people, friends, who either ride around my speed or a little bit faster. Or, I ride with friends who are slower, and I end up waiting.