I think you're doing the right thing. Its just not fun when its a constant struggle and not enjoyable. That's actually the reason I usually ride alone; I want to be able to pace myself so that I can progress at my own pace. Riding with the group will keep you pushing your limits though, so definately do it sometimes; just not all the time.

Yes, it is about enjoying yourself and having fun!

Tracy