I can so relate. It wasn't until this year that I became brave enough to do group rides. I picked the beginner/recovery ride and usually do that one. I also do one that is one level above that. I like to go with someone I know and then, I feel much better and less nervous. I have gotten to know some cool people on the rides and so it feels more comfy for me now.

Don't give up though...It does get better