Sometimes when riding alone I wear ONE earbud to listen to spoken-word podcasts or audiobooks. I feel this does not distract me from traffic noises at all. When I ride with even one other person, the iPod stays home.
BTW, just to clarify, I only wear the one earbud doing laps in the traffic-free park. Never out on the streets.
Again, on group rides, it is just rude in my opinion to plug up your ears in any way. You should ride alone if you want to be in your own world!