I must add that when I needed to, I got a babysitter, so I could keep exercising. Mostly, this occurred when I was teaching at the gym and my DH was traveling. I had a wonderful teenage babysitter who would come to my house at 4:40 AM, so I could teach my class at 5 (very hard to get subs at that hour), and stay until I either came home and got ready for work and my nanny came, or, stayed until the nanny came, if I had to be at work early and had to shower at the gym. She would drive to my house, and get ready for school there. She was from a family of 5 kids who had foster babies, and I also used her older sister and younger brother to watch my kids at times. We started with her when she had just turned 12, and she knew more about child care than most moms! She now is 42 and has 4 kids of her own.
I guess what I am saying, is that develop a support system, whether it be sitters, relatives, or other moms. I am sure some thought I was selfish, but it was "my time," and it kept me sane. Also, exercising at that early time didn't really interfere with other family responsibilities. My DH used to play racquetball at lunch.