I think the calipers make a big difference... many bikes with middle of the road Shimano (not sure about Campy) group sets have cheap Tektro brakes on them. IMHO they aren't very nice.... I can definitely tell the difference between good calipers and bad ones - mainly in how grabby they are, the good calipers are much more smooth, but yeah maybe in how hard you have to grip the brakes too.

That said, I think long descents can be just as tiring as long ascents.... if its a twisty descent and you have to do a lot of braking, it can be quite hard on your upper body. So, some of it may be the brakes and some of it may be getting used to the descending too.