I actually just did a little research and you're right! Uphillers have the right of way for exactly the reasons you stated.

V.