I think it has more to do with fit than wheel size. I have both a 700c bike and a 650c bike and I find the opposite of TG to be true for me. Both are racing frames, one is Carbon one is Ti. My 700c wheel bike, for me, is MUCH quicker to accelerate and a much better climber. I am short, with more leg than torso though, so the 700c bike just fits me better.
I'd say really fit each of the bikes comprehensively and figure out which one really is the best fit for you and don't worry about the wheel size. The only big disadvantage to 650's is if you race you'll never be able to get neutral wheels. If you don't race, don't worry about it. If you do race, just be prepared to always put in your own wheels.