How can I become strong enough to ride with a group that rides faster than me, if I can't keep up with them? Though I'm usually only about 5 minutes behind them.

And when I ride with the slower group I'm always in front.
So basically I'm in the middle. What do I do?

And what if the fast group is on a sort of if you can't keep up, you shouldn't come kick?

Sometimes I make excuses and ride alone.

Suggestions of all kinds welcome.