They should be polite about passing. You shouldn't feel apologetic that you don't ride everything yet or that you are 'slow'. So long as you are considerate then don't let the rude ones get to you, they weren't always fast, either. Some obstacles work better taken at speed so that may be part of some riders' deal (they'll live), uphill mostly has right of way but sharing and working with trail conditions is what it's about.


I did inadvertently scare a newer rider on the whoop-dis on Monarch Crest several years ago. Followed her at a distance and passed when I had several feet between us to do so, announcing myself well in advance and several times. I guess she was so white-knuckle on the downhill she was really concentrating on that and I startled her. She let me have it and boy, did I ever feel really bad.
Point of the whole long drawn out story is try to remember to listen to what's happening around you, it's really easy to zone in on what you're doing and forget about everything else....actually, I still sometimes do that and they surprise me too, every once in awhile.