I've found cycling is like running. The long, slow rides improve your speed on shorter rides. I forget the name for it, but running books talk about it all the time. My first year I rode 10-12 and after a couple summers, I'm up to 14. Your pace will improve with experience. If you have a GPS, try making one of your shorter rides a "Pace" ride--riding faster for a shorter distance (riding with a faster friend is also good for this--my husband is my training buddy. I'll never be able to keep up, but I can try). Or spending one day a week riding a hillier route will improve your leg strength and make you faster on flat roads.
My first century took 8 hours--there's nothing wrong with that.
Getting in your long mileage is the most important.
"Well-behaved women seldom make history." --Laurel Thatcher Ulrich
'09 Trek WSD 2.1 with a Brooks B-68 saddle
'11 Trek WSD Madone 5.2 with Brooks B-17