I would have been BS.
I don't have any real constructive advice, but I am sure part of the reluctance to change is that he cannot imagine your schedule. And, I also know from experience that schools don't like it when your DH stands up for you. Teachers are usually wimps when it comes to stuff like this and you dared to question.
I'd turn off my phone or unplug it if it's a land line and then schedule a mtg with him when you have thought about it more. Educators are not good at having difficult conversations (meaning him) so my bet is he will listen.