My first bike was a Specialized Globe. It was a comfortable, easy to ride hybrid. Within 2 months I realized that it was just too slow and heavy. I sold it and bought a Specialized Vita WSD. This is the bike I would recommend that you try. It has flat bar handlebars, but is lighter weight and rides much better than the Globe without being any more difficult. Not sure about off road, but you could request cyclocross tires to replace the ones it comes with (before you buy the bike so you don't have to pay for additional tires - just the difference).
Since then, I bought a Specialized Ruby carbon road bike that was almost new on craigslist. I love this bike, but you can't add child seats and fenders to a carbon bike. Since I got Ruby, I have stopped having 'bike envy' and looking for bikes. It's a better bike than I ever thought I would have, so, there is no up. I did get a Bike Friday for travel.
I think it is very easy to buy the wrong bike when you start out, because you don't really know how much you will get into cycling. So, you just don't know what you need.
2007 Ruby Comp/Specialized Dolce
2004 Bike Friday Crusoe/Specialized Dolce