I am not a fan of chamois butter. The one time I used it was the one time I got a sore. Go figure. I know lots of folks like it, though.
I second the idea of a 'cross bike, or even a road bike. Could you keep your hybrid and use it for the short trips around town and trails (if it fits, that is).
At the very least TEST lots of bikes, including some nice road bikes. You might find that the different position and bike are such a nice change that you are out there training on the roads before you know it. You could keep your hybrid for commuting.
Sounds like you are enticed to do the longer rides. If you enjoy the longer endurance riding, a nice road bike will be a treat for you, and actually help you improve faster, IMO. And, Knot's right...a nice 'cross bike could do double duty.
Good luck, and great for you getting out there! 
Edited to add: you might check the ingredients in the chamois butter. Some people are irritated by things like lanolin...maybe a different lube would help?
"The best rides are the ones where you bite off much more than you can chew, and live through it." ~ Doug Bradbury