I often find myself thinking similarly. I'll think, "I have the right of way" when it's between me and a car. Who would lose in that duel?

I think we have to reach a compromise between asserting our "right of way" and our safety. You were there so only you can tell if you did the right thing.