"Only" riding the bike 5 days a week? I don't know how long or hard your rides are, but that's a lot of days to be riding. I find my performance is best when I am riding more like every other day.

Given how physical your job is, are you sure you are eating enough good quality food to fuel your workdays in addition to your training?