Are you sure you're not overtrained? It's more than your specific training load that contributes to that, it's your whole life - work stress, family, everything. How many recovery days are you incorporating into your week? Maybe add another one. How's your resting heart rate the morning after one of your "good, hard" rides? Hard training does suppress the immune system, that's why professional athletes get colds so often.