I have to fight off thoughts like that (though admittedly on a smaller scale LOL).
Like Lisa, I want to enjoy myself and that is my primary motivation. Speed comes in dead last with distance coming in ahead of speed! I'd like to train for a century this year but am afraid all those competitive types will leave me in the dust and close up shop before I even get to mile 50!!!
So as I get those feelings of insecurity I remind myself that I'm doing it for the enjoyment and that alone is how *I* win 
I did push myself to do 31mi today and it feels great!
Alison - mama of 2 (8yo and 6yo)
2009 Independent Fabrication steel Crown Jewel SE
1995 trek 800 steel MTV