Busy, busy today, but here's one of my favorite quotes (It's mine):

If you do the same thing the same way all the time,
you become very good......at doing the same thing.

The most common training mistake I see is that folks go "hard" all the time. However, because they have no variation in their training, "hard" isn't really the top limit of their performance. In order to improve, we need to vary our training. Unless you go very easy, you will never go very hard. This manifests itself in all your training cycles (intervals -- long & short), day-to-day variation, and seasonal cycles as well.

I see this with racers and recreational riders alike. Most of us see significant improvemnt when we begin riding -- for a year or two. The logical progression for most of us is to increase our distance. However, we don't typically work on our speed. Once our body adapts to the rigors of training/riding, we cease to see improvements. For some folks, instead of a plateau, we actually seem to be losing ground. This is because our body has adapted and we don't create change once we've adapted to the stress/overload of training.

In simple terms, once you've hit a certain level, you need to change your training to continue to improve.

Riding longer does not help you ride faster.

And yes, there is a genetic component as well. You can thank (or curse) your parents for your athletic potential. However, genetics plays the largest role in elite athletes. For most of us, we have lots of untapped potential that we could realize if we were simply training appropriately.