To me- I don't see any tradeoff. I ride on 650's and I LOVE them. Plenty of people around here ride 650's (and oh the race tires I'm eyeing on eBay... don't even get me started.) I don't have as many color options should I choose colored tires (which is not an issue for me), and I guess I don't have as many tire options- but again-not a big deal. I've never had a problem finding tires or tubes (but if I flat out on the road and need a new tube and don't have one- then I'm SOL- but that's why I carry everything I need).

For me, I like the quicker power when accelerating. Let me explain that- I can "get up and go" much quicker on my 650's on the Cervelo than I can on my 700's on the Giant. I can outsprint DH from a dead stop on my "teeny tires" (as he calls them). They have much more "giddy up" than my 700's (at least IMO). I know it all evens out in the wash, because with smaller tires you're technically putting in more pedal strokes (because the tire rotation is quicker), but I really don't think it makes a diff.

And plus, the 650's are more aesthetically pleasing to me. On a 48" frame, they look far more proportional than the 700's. I always feel like my Giant bike is just that- a tiny frame (it's an XS) with GIANT wheels.
It looks silly to me. The smaller tires don't make me look like a kid riding on my dad's bike.