Some people think that you lose too much in the upper gears (read: speed) by having the really low gears.
In reality, it depends on the particular triple, and it depends on the particular double. Well actually it depends on the chainring in combination with the rear cassette.
If you want to know more about how gearing affects the ride of your bike, go here:
http://sheldonbrown.com/gear-theory.html
He explains all about gearing, crossover, shift patterns, etc. in a way that's comprehensible to those of us who are not gearheads. (Unintentional pun, btw)
http://www.sheldonbrown.com/gears/
Above is Sheldon Brown's gear ratio calculator, just plug your particular bike info in there and it will tell you how many gear inches you have in each gear. He's got about 30 or 40 standard sprocket combinations in the drop down for the rear cassette plus a custom setting. You'd need to count the teeth on each sprocket in your rear cassette (and ditto for the chainring). It's PROBABLY in the user's manual that came with your bike, if you still have that stuff or know where it is.
Heck, include your crank length and he's got it set up so you can even figure your speed for each ring/sprocket combination at a particular cadence.
The highest gear on a road bike is usually something like 108", and I think the highest gear on my triple is like 104". Unless you want to get serious about racing I don't see that there's much to be gained by going to a double from a (good) triple.
The weight of the bike is probably a bigger concern for a recreational rider who wants a bit more speed. Truth be told, for many of us it's the weight of the RIDER that's the biggest concern, LOL!
I don't think a triple is something you have to "grow out" of is all I'm saying. Of course all triples are not the same, I can't say what yours is giving you in those upper gears, but I've never maxed my 3rd ring out and I use those low gears all the time. Wouldn't give 'em up for love nor money.
I don't know how long Sheldon's site is going to be up, but I would strongly encourage people to browse and save all you can from it. The cycling world really lost something when we lost Sheldon. He explains all sorts of technical things in ways that are easy to understand.
You might be able to get a bit more out of the bike that you have just by swapping a sprocket or two out, or by changing one or more of your chain rings. There are technical issues that need to be addressed when you do something like that but a good mechanic should know what is and isn't feasible.