Sorry it didn't go well for you yesterday - we had a similar situation with our son when he was in the trailer. The things that worked for us were - he picked his own helmet, we refused to move if he took it off and we kept the rides short and to child friendly destinations. Now he's 5 and his sister 7 they are the ones who are the "helmet police" checking everyone has one on before we set off.
We didn't really bother with helmets particularly on short quiet road rides until we had the kids when we felt that if we wanted them to wear a helmet we had to set an example. It is fortunate for me that we did - a few years ago I was only cycling the kids to their pre school a mile or so away. I had the trailer on the back and hit a patch of black ice. I came off and my head hit the road very hard. My helmet cracked badly, but my head was perfectly fine. Thankfully the trailer stayed upright and the kids were OK.