You've received some good advice. This is a hard decision to make. I would listen to SheFly. I am not a racer, but I have been around racing "people" for about ten years. Mostly, I see guys who are out there training, putting in the time, and their s.o.'s are very resentful. But, I even see this in people like me, who ride for fitness/fun/lifestyle choice, particularly, in the group I ride with. There are some very strong older men in this group, who have been riding for years; they are 50-75 years old. They ride incredible distances, do charity rides, and some don't have cars. I know some of them are married, and I have never seen their wives. On the other hand, there are also several very strong women (similar stories to the men) in this group. They are mostly single. Some are divorced, some never married. I wonder what effect cycling had on their status. Or maybe they are just single by choice and cycling has nothing to do with it. There are also quite a few couples like my DH and I; riding is the biggest part of our lives. Our vacations, weekends, etc. are planned around riding or some other outdoor activity. We do have a few friends that don't share this passion, but they sort of accept us for who we are.
For years, my DH sat on the couch, while I taught at the gym and was fanatic about my fitness. He play a little racquetball and tennis. No matter what I did, he complained about outdoor stuff. Then, he started riding when my son started racing as a young teen. I suddenly know how he had felt. We planned our separate activities at times that didn't interfere with family stuff, but eventually, I came around. It was a real role reversal and I hated being less strong than him. Now I accept it, although we are fine riding together.
Good luck.