The bag did not affect my hand position on the road bike (and it was a tiny road bike).
Catrin, I find that I am aware of a loaded pannier when I start off, but I know it will feel different, so I guess I just do what I do and adapt my technique, because I haven't fallen over yet! My Jamis is so heavy and sturdy that it adapts well to the load. One time I went holiday shopping for my colleagues and after realizing I had to stash some hand made ceramic pieces in my pannier, I thought it might be hard to deal with. But, while it was slower going up the hills, I never felt like it was hard to do anything else. Of course, being 30 degrees out probably made the load feel worse...