IMPO, I think it's because people in general cannot stand to be behind someone else. They do the same things in cars, and that drives me crazy too. Why would you pass me, make me slow down, and then go slower so that before you know it I am passing you.
I can understand your frustration, that would drive me crazy too. I go on my first organized ride Sat. So I will have to remember that so I don't do it to someone else unknowingly.
Of course I am a fat chick too, so it will probably be everyone passing me. LOL
I think what I would do is pass them back if they don't speed up. Or knowing me I would probably open my big fat mouth and say some smart remark and make everyone hate me. LOL Sorry, but I don't have much patience for inconsiderate people, and I know I am not perfect either, but if someone points out to me something I am doing wrong, I try to remember it and correct it and not do it again. I don't want to be inconsiderate to others and would never do it intentionally.