I have had a couple of close calls in my life - once when riding on a sidewalk (years ago) and once recently at an intersection where a woman looked right at me and turned right in front of me at a light where I had the right away. I swerved out of the way and yelled some profanities.
Anyway, the most common harrassment I get is people yelling at me while passing in a car and I've also had teenage boys yell at me after passing me on BMX bikes. Teenagers seem to be the main culprit here. I've been called a B**** by a car full of teenagers for no reason (I was riding in the bike lane), I was called a "loser" by some hick in a HUGE pick-up (a young male) and I've had a handful of times when someone will yell at me really loud as they pass me which really bugs me because it can be very startling and I don't think drivers realize they could startle you enough to cause you to fall or even swerve into traffic.
I live in a suburb of Detroit and this area is not known for outdoor health or bike riding. I used to own a motor scooter, but I decided to sell that because of all of the verbal harrassement I got for riding that around. It seems like if you're not driving a car in the "motor city", you are fair game for harrassement.