You definitely need some fuel in your system. As you gradually build up your endurance, you'll be able to sustain longer rides.