Take the long way there, working out hard. Recover on the easy spin on the way home with them.

I have a 12 year old who rides and have found it impossible to get any kind of workout when riding with him. I ride with him becuase I want to spend time with him. I workout without him. It keeps both of us happy.