Boy, can I relate!
First, when I started mountain biking, I saw my weekly mileage plummet and FREAKED! Then I did a reality check and saw mtb vs. road and took a deep breath. It is still hard sometimes when I see other peoples' mileage, but I know what I am doing and I work hard not to compare myself-
Second, I am the queen of "if I am having a recovery day, I am doing nothing to help make myself stronger". Oh, yes, I know that recovery is important blah blah blah blah. I can recite it chapter and verse to people who are overtraining. But for me, I had and still have a hard time taking a day off. One suggestion by a friend of mine was to take out my road bike and ride flat roads (not easy finding flat roads in Vermont), even if you have to do an out-back, and ride slowly, in an easy gear, for even as little as half an hour. And it really made a difference to me, emotionally. For some reason, even though I never even worked up a sweat, just being on the bike helped me get over it. No real mileage, no hard gear, no hills, just light spinning. And eventually, you can work yourself into a rest day or so, not on the bike. But for me, emotionally, I needed that bike connection!
Hang in there!
The best rides are the ones where you bite off much more than you can chew--and live through it.