I listen to music with one earbud in (the right one) on my commutes (all MUT and/or dedicated bike lane). I can hear traffic and other important noises (calls of other trail users on the rare occasion they actually use them) fine.

If I was someone that got so caught up in music that I could not hear my surroundings with one earbud in then I wouldn't wear one.

I don't listen when I'm training as I do end up in areas where the roads are less bike friendly (only a shoulder, low visibility, narrow, mountain roads) or in a large group where I want the extra few tenths of a second warning that not having music in might give me. Also I find if I'm working hard I don't hear the music as well, so I focus more on it, and THAT'S when I get distracted. Not when I'm cruising down the path to work.

IMO, if you pick headphones/headphone arrangement that allow you to hear (they now make headphones that are designed to sit away from your ears for just this purpose) and know yourself and where you are riding (how distracted you get, what you may miss with headphones in a given area and the chances of that, etc) it's ok. It's a calculated risk (moreso in some situations than others), but so is most anything else we do.

I too would be more distracted by talk, so I wouldn't do that personally.