I think the person who said it was annoying in your peripheral vision is just a style-conscious poser!
I suppose the usefulness of mirrors would depend on the type of riding you do. It's true that sometimes you can't even hear cars approaching from behind but shouldn't the emphasis be on the car avoiding you?

I personally thinks I'd find them annoying for my vision - however, got a feeling this has more to do with me being a style-conscious poser!