This is precisely the reason I bought a custom bike with couplers.
The first two supported tours I did were with Trek Travel. At the time, I had a Trek 5200 and on the first trip, I had my own bike, since we drove to the start, and when we went to the Czech Republic, I had one of the first Madones, which was even nicer than my bike... in a 47 cm. The other tours we have done were ones we could drive to, until we went to Spain in 2009. I had a lower end Orbea in a 48 cm. Now, my carbon bike is a men's frame in a 48 cm, but, I have the short reach brifters, etc. I could barely ride the Orbea for the first day. Seriously, I was so far behind when we took off, because I felt totally unbalanced, and could barely squeeze the brakes. I was miserable.
Yes, it's never an issue for DH, even though he is shorter for a guy (5' 7") there is always a 52 cm for him.
I am so looking forward to going to the San Juan Islands, with my own bike.