Another issue to be aware of is heel-strike. Short wheelbase plus small-ish wheels can lead to your heel hitting your pannier.
To the extent that you can choose/adjust your rack for this, or your panniers, you should be okay.
The clip system on Arkel Overdesign panniers allows for fore/aft adjustment, I've found this to be a pretty important feature given the geometry of my bikes.
2014 Bobbin Bramble / Brooks B67
2008 Rodriguez Rainier Mirage / Terry Butterfly Tri Gel
2007 Dahon Speed Pro TT / Biologic Velvet