It's hard when someone has headphones in. The couple of times I have had to go by a pedestrian I slow down and very cheerfully say, "I'm coming up behind you on my bike and I just didn't want to scare you". Once by, I say "Thanks!" and keep going.

I wasn't sure how to handle the guy with the iPod though.