Just a helpful suggestion- you should maybe always go down hills in front of him?
It's not good to encourage people to descend hills faster than what they are comfortable with doing. For whatever reason, your DH feels the need or preference to go down the hill more slowly than you- best to just let each of you choose your preferred speed for downhills. 35mph is fast, and can be dangerous if something unexpected happens. How would you feel if he grumbled at you about how you weren't going fast enough?
It's wonderful that you two are out there riding together!
