I always announce "passing on your left" on the local bike trail, even if I know someone has seen me coming up on them. If I don't announce my presence pretty far back, some bikers (especially those with kids) will weave all over the path and not leave any room for me to pass. I've had a lot of people thank me for announcing my presence before passing, and only a few get startled/confused - no one has been offended yet! The people startled/confused are usually either kids or with kids who dart the wrong way or into the middle of the path, freaking mom or dad out a bit. That's why I announce where I am some distance behind them, so I have time to stop if necessary and they have time to figure out which way they're going. 
I have only been passed a couple times on the bike path, and I make sure to thank someone if they announce their intentions before flying by me.
On the road, I usually ride with my boyfriend, so we're usually chattering a lot and know exactly where the other is. Whomever is behind will announce "car/truck/motorcycle/etc." if a vehicle is passing, even if it's obvious that the person in front knows about it.
When we ride with the tri club, it seems like everyone is really quiet (maybe because we don't all know each other yet?). It bothers me a little - my boyfriend and I are usually the only ones I see making arm signals for turns/stops, and no one warns that they are right behind you or passing. When I'm coming up on someone in the group, I usually say "coming up behind you" or announce my intentions to pass. Since we're riding in a big group (that splits into smaller chunks based on pace), no one seems too jumpy about another rider unless they get passed closely with no warning. We do, however, have to speak to each other about the routes, since only a few people are familiar with them.
“Hope is the thing with feathers, that perches in the soul, and sings the tune without words, and never stops at all.”
- Emily Dickinson