another thought to consider is that if you are riding in an area with a narrow shoulder, the courteous and right thing to do legally is to ride single file. any time you ride in anything other than single file you should be alert to leaving a space ahead of you for someone on your left to drop back into should there be traffic.

Second thought is use your words,you can always call out gravel, hole, glass just as you pass it. Yes the hand gesture is better but voice works too.

If you develop a strong riding relationship with one more people, you will often find that all of the communication happens non verbally unless your are stopped. You develop a set system of points hand shakes and hand waves, as well as signaling when you are dropping back etc.

Pack riding skills are necessary but come with practice and clear communication with the leaders. Perhaps they could assign someone to wait for you if you get dropped at a stoplight so that you can ride at the back of the pack and learn by watching and pulling forward when you feel ready to.