I think there are several factors.

One is that true endurance is built over time. There are certainly genetic factors involved too.

2ndly is patience.

3rdly may be that as we get older some of us don't give a damn what others think. So we may be willing to spend hours on the road in our crazy endeavor.

4thly. It's not a cheap sport.

Some thoughts anywho.