I think some of it's down to conditioning. How long are your rides normally? Are your trails hilly or flat? A banana and toast wouldn't be enough for me before a ride, but I think you'd notice it ON the ride first.