Afraid its just a fact, if you use heaps of energy you need to dedicate time to replenishing yourself which means more sleep.
I am the same too - I often want a nap in the middle of the day after a ride, or if I have ridden hard the day before.
I try and manage it by getting to bed at reasonable times and when I can, allowing a cat nap after work, or during the day at weekends...