FWIW, I bought my first adult bike (first since I was in high school) in 1998. That was a mountain bike but I rode it mostly on paved trails. In late 2002 I got my road bike. Since then I've done 3 full centuries and countless other long rides (metrics, etc.) But I'm still pretty much the same speed as I was before and I don't expect to get any faster ever. I was built for endurance, not speed, and that's just the way it is. As for bike handling, I don't think I've advanced very far on that either. I still can't bunny hop a curb and I always have to unclip for u-turns and turns in tight spaces. I think my balance has improved but that's probably due as much to working out in the gym as to bike riding.
As for this:
Sometimes it can be good to compete or compare yourself against others. But unless you're racing, I think it's best to only compare yourself to yourself. I know what my average speed tends to be for a certain type of ride, so I focus on matching that or maybe trying to beat it a little. If anyone else is faster than I am, so be it.
And it can help to set goals, but if you don't reach them it's not the end of the world. There are too many things you can't control -- injuries, illness, work- or family-related conflicts -- that can keep you from training. All you can do is try to learn from it and then move on.

