We tell everyone that racing is supposed to be first and foremost fun! We pay to do this, not the other way around.
That said 1. you do not need to win to have fun. 2. your age alone probably shouldn't be holding you back
There are so many reasons you don't have to be winning to have fun in a bicycle race! Just having goals and seeing yourself progress toward those goals is great. Don't aim for the moon. If you haven't finished a race with the pack then make that your goal. If you've never tried to put in an attack make that your goal. Helping your teammates can be just as rewarding as winning. I have teammates who rarely finish in the pack that still have helped me sooo much and I hope I let them know that often enough. Even if you can only stay in for a lap if you can keep another teammate out of the wind or help to control the front of the race you are important. If you put in one good attack and then blow up you've still helped. Make the other racers chase you, tire them out too! Just try to make what you are able to do count and not only will you feel like you've been a contributor you will likely be on your way to being stronger and more confident too! Even being there at the finish to cheer for your teammates helps - you would not believe how much you feel like you can give just that little bit more when someone is yelling your name!
As far as age goes. I'm sure genetics must have something to do with it, but age alone certainly shouldn't hold you back - unless its mental! We have so many women out here (including me - I'll be 35 this year) who have only recently come to racing. To tell the truth many of the women who do very, very well are not the young chickies, but the masters. I have many teammates who are over 35, even in their 40's and 50's and they are certainly not struggling - many of them can kick my arse around.(I regularly am beaten in TT's by two women who qualify for the senior games )
Don't feel bad to treat road racing as training for mt bike racing either. We have some people who focus in other areas - tri's, track, cross, mt. bike but still race road just since its easier to get a really good workout when its competition... (and of course because its fun to come out with the team!), but of course its a whole lot less pressure when you know its not your focus.
It doesn't sound like you really want to quit road, but still its getting you down. Give yourself a break, take your time getting your fitness back and don't listen to those nasty folks who want to put you down. Do they truely think that no one for whom racing doesn't come naturally should not try. How would anyone ever get to know if they could be good if they gave up so easily.