Just agreeing with everyone else here - time on the bike/ consistent riding is crucial to your improvement.
Pushing yourself hard (by trying to ride a known course faster, or drafting others who ride faster than you so your pace is lifted) is important to.

Something that I didn't see clearly said here though is do not try and do every ride fast. Do not treat every ride as an attempt at your personal best.

This will tire your body and leave you "over trained" which will actually have a detrimental effect on your overall fitness.

Rest days (no riding) and/or recovery rides/active recovery days (gentle riding where your heart rate is not lifted by much at all... flat courses, or hills ridden at a slow steady pace with minimal exertion) are just as crucial to improving fitness.