I spent a year riding on a mtb with slicks, doing the same 3 or 4 15 mile rides. It got me comfortable on lower traffic roads, but it took my husband forcing me to go up some big climbs on that bike and to do a 20 mile ride. When I saw I could do that, I got a road bike. I worked on increasing my speed for about 2 more years. I find that I need to mix up the kind of riding I do, or I get a "bad attitude." I still am not too confident about my mechanical abilities and i actually ride by myself less now than I did at the beginning. But, I can ride pretty much anywhere, now and deal with traffic, too, although I don't particularly love that. So some days I go out for a fast, short ride, sometimes I ride for fun with a friend who is slower than me, and sometimes I ride with groups. Riding with my husband either pushes me or we both decide to go at a certain speed and stick to it. A big part of the problem is shutting off that voice inside your head that is constantly saying "this su*ks. I still hear it after 7 years and if you let it get to you, it will kill any joy in riding. Then, of course, after I have finished the ride, i feel very accomplished. This happens to me a lot in situations where i don't know the terrain and the type of climbing involved. So, I take that into consideration and realize that any time that is the first time I've done that route, especially not in my local area, I tend to get a little negative.
Keep working at it!