If I'm riding with somebody significantly slower than I am, then I don't mind at all that they go flying by me... because I'll have a good time catching up, and they won't have to slow down and then speed up again.
If the roles were reversed, I'd explain that I was working too hard to waste *any* energy and knew that person could catch up... tho' I'd at least slow down a bit and try to get a reading on what the other person was thinking.

I'm thinking that if I got ahead, and then the person behind me blew by me... maybe they simply felt like they were doing *exactly* what I had already done to them, and either that was the natural rhythm of things, or they were trying to send me a "see how it feels?????" message. If they were rude... then I was, too.