105 is definitely more durable than that. 105 cassettes and chains are very durable. So are Ultegra. Once you get to DA for those, then you are mostly spending the money on weight savings.

DA cables, on the other hand, are much thicker and last a lot longer. But I digress..

I have experience with 105 and Ultegra brakes, rear drivetrains, front derailleurs, and shifters (not chains and cranksets, personally). I hate 105 brakes. Hate them. As soon as I can justify the upgrade, Ultegra SL or DA is replacing those calipers.

Shifting is better with Ultegra, particularly in the front. Much better. The only down side to my Ultegra FD is that with my compact crankset and the narrower design of the cage, I have to really get the FD position just right to minimize chain rub. It really makes me long for DA shifters with their better trimming than my 105 shifters. Which brings me to shifters. The higher up you go, the more crisply they shift with less effort. The throw is a bit shorter and easier. That can make a big difference. For example, I cannot shift into the big ring from the drops, because I have to put such force into the shift that I wind up swerving from putting that sideways pressure on the levers from that hand position. That is a big pain in my *** when I'm in a criterium with a little ring hill thrown in the middle, because I can't just stay in the drops. That may matter less to you if you don't race, but it is still obnoxious.

I'd say even making your setup partially Ultegra is worth the money. I have a mix of 105, Ultegra, and FSA on my bike (with a KMC chain), and I only complain about my 105 components. Except for my old 105 cassette. That thing lasted forever and did a fantastic job. I expect the Ultegra cassette I just put on to perform well also, but so far, there's no appreciable difference aside from a few grams.