Back to the original question, just putting on racks/fenders shouldn't affect handling. But the load you put on the rack sure can... On a recent trip to the supermarket, I got carried away, and ended up with less than 5 lbs of fragile stuff in one pannier, and 20+ lbs in the other. Handling was absolutely affected; thankfully it was only 2 miles home.

Enjoy your Surly!