What do riders think about the possibility of using a trunk bag (Arkel? Topeak?) instead of a backpack? Or maybe a combo of panniers and trunk bag?

Does shifting a bit of the weight from a panniers-only set-up to panniers-and-trunk-bag either help or hinder maneuverability? Or would the change be minimal?