Sometimes I can anticipate that and I make a point to be looking anywhere but at the driver in question... especially if I can do that by looking at the other direction cars might be coming from. Other times I have to just wave *them* through.