Could the tires be inflated too much? It sounds counter intuitive because higher inflation means less sidewall flex and thus less energy lost flexing the sidewalls. However, overinflation also makes you bounce over every little road/trail irregularity. Lifting you up a millimeter and then dropping back down takes a lot more energy than you might save in reduced sidewall flex.
A nearby university has a sidewalk / utility road where one side is concrete and the other is asphalt. Riding on the concrete is annoying because of the constant thump-thump across each expansion joint. However, it is the easier and faster ride. In contrast, the asphalt has constant rolling irregularities that just sap all my energy and speed.

