I find a bell works very well. I ring it and most joggers and peds give a little wave, and other cyclists move over to the right. The bells I have on each of my bikes are small and inconspicous (for those of you concerned about the perceived uncoolness of having a bell on your $2000 road bike), and yet when I ding it, it's loud.

I find it very disrespectful (and dangerous) when cyclists say nothing and whizz by. The same for when they say "on your left" in a quasi whisper when they are already at your shoulder!