How long did it take you to faster?

I'll let you know if I ever get any faster! Probably happen about the same time I get taller!

Zen makes a good point, if not a tongue in cheek one. If there's an active cycling community in your area, then I'd encourage you to join some club/group rides. You'll likely meet some people with similar abilities and/or similar goals. And don't assume that your current friends have no interest in riding with you just because you can't necessarily keep up with them at their top speed. Not every ride has to be a fast one.

I think you're on the right track. You may be guilty of over training a bit though. Ditto what was said about cadence. I would suggest that at least on your shorter rides you never get out of the small chainring. This will force easy spinning, which may feel a little funny, but you'll get used to the decreased resistance and higher cadence. Other than that............time. I got faster last year by riding in pacelines with people who were faster than me. Unfortunately I'm a little behind this year because of a wreck that kept me off the bike for 5-6 weeks, but that's ok. Hang in there, it will happen.

I've been riding for 7 years. I'm barely faster now than when I started. At first I averaged in the 13-14 mph range for rides. Now I'm in the 14-15 mph range. And I have a MUCH lighter, MUCH more expensive bike that fits me MUCH better than back then.
It can sure be frustrating, and literally all my tri friends are so much faster than me that I can't hang with any of them. Most have progressed much better and are consistently riding in the 18-20 mph range now.
Oh well, I got VERY frustrated last season then said to he!! with it. I ride what I ride. Same as running; I run what I run. Even when I was 40 lbs lighter, my running and riding pace wasn't any faster. I am what I am, and now I just enjoy getting on my bike and riding.

It's not always easy to be so blase about it, though. I wasn't for years...

I started at a 12-13 average (I live in a hilly area), got up to about 15.5-16.5 after about 3 years. Then, about 2 years after that I got sick, lost a lot of my strength and took about 1.5 years to get back somewhat back to where I was. But, my attitude has changed. Aging has also kicked in. I need more time to recover and frankly, I just don't care about my speed anymore. My goal is to ride until I croak, and I know that I am not going to keep getting faster at age almost 59 unless I put in some serious training. I am still too fast for "slow" groups and too slow for fast groups. I found one group that I can ride with, and now they often split into the regular group, who averages about 13 on rides and the "spirited" group who averages about 15-16. I am on the edge of that and they are encouraging, but I just don't ride much with the group anymore.
I do short rides (15 miles) where I push the speed, and my hilly riding serves as intervals. Other than that, I don't care. Today I rode 43 miles, with several very steep climbs. Until the lunch stop, my average was 12.5. After lunch, we had a long flat stretch where we were going along at 18-21, so my average was 13.8 for the ride, that is, until my cool down, which involves climbing a 10% grade hill to my house.
I can go along at 18-20 mph if I am on flats, but the thing is, there is no place around here to ride that is flat. I can also climb better than a lot of women my age, and I am not afraid to spin up a hill in a very low gear. I don't like descending, so my average suffers because of that.
Cyclists get very competitive about their speed, etc. I found people to ride with who just aren't into that. They are great riders, but not particularly competitive. This is why you will never see me doing a tri or even a charity ride where some people make it into a race.