I think age is a factor, but it's very individual. All of a sudden, I need lots of recovery time and my average is going down.... some of it is my particular situation, but there is nothing unusual going on with me compared to a couple of years ago. I didn't start cycling until I was 48, but I was very active in other fitness things. I improved steadily until last year, when health issues interfered. Now, I am frustrated. I am only competitive with myself and myself is not happy thinking that this is age!
I feel like I have to make a decision between whether I will concentrate on distance or speed, because i don't seem to be able to do both well.