CC - How long are you riding when it sets in? Is this consistent? What changes do you make when you start to get your ooooomph back?
This sounds like a nutritional issue, but I'm not sure I have a suggestion. If you give us a little more info, I might be able to come up with something better than...
Try changing what you eat. You may need a faster acting/more easily digestable form of carb. I prefer gels, but there's lots of options.
My nutrition schedule is usually one gel 15 min before riding if I haven't had anything to eat in the last 1.5 to 2 hours. Then I'll have another after 20 min on the bike and then every 40 to 45 min after. I also drink gatorade on long rides (not my favorite drink, but it's what they give out at IM so I have to get used to it).
I would also wonder if you're not getting enough calories back in you, soon enough after a ride and if this is affecting your glycogen stores. This might mean you run out of muscle glycogen sooner than you should and it takes a while for your liver to release glycogen to "catch up".
By the way, this is all speculation. I really don't know all that much about nutrition. Just thought I'd share my random thoughts based on my tri experience.
Living life like there's no tomorrow.
http://gorgebikefitter.com/
2007 Look Dura Ace
2010 Custom Tonic cross with discs, SRAM
2012 Moots YBB 2 x 10 Shimano XTR
2014 Soma B-Side SS