It's so stupid, isn't it? It scares me and then it makes me angry.

You brought up a great point just now. The advantage of a bike is that you can stop and turn around anywhere you want. It's not a car. If you get spooked, you don't have to have a reason, - turn around and ride against traffic till you ditch 'em.

Same deal on a bike path. If two guys climb out from the under bush - turn around.

It's okay to look silly or be wrong. Trust your gut. If it doesn't feel right, that's a good enough reason to get yourself out of there.