I have been passed by old guys on mountain bikes, carrying huge backpacks, when I was on my road bike . And not going slowly either (15-22).
When I commute, I don't care who passes me. I see other cyclists once in awhile; one of the roads I go on is a popular training road for cyclists, so I do see them here, usually in the morning, going the other way. I tend to go slower on my commute, especially in the morning. Who cares???