I too am really sensitive about these sorts of things and tend to feel left out if I am not invited to do something, even if I end up declining because I don't want to do it, and that happened with some frequency back when I was working since I am introverted and pick my social occasions very carefully.

I am glad it was no big deal after all, VBC, but I too would have felt bad at the time.