I feel exactly the same when I ride my road bike (Bike Friday with narrow drop bars and those 20" wheels) after being used to my mtb. The first few miles are kinda freakish/scary!