How does the trailer attach?
I wouldn't use one that attaches to the seatpost or to the chainstays. MAYBE would consider one that attached through the axel as the stress is better spread out.
That said, I had a trailer (attached to the axel) get hit and flipped over. It sheared the hitch and pulled my 33lb, thick aluminum mountain bike unrideabley out of alignment...
That's not likely to happen under normal circumstances (unless maybe you flip it into a ditch and somehow prevent the bike from following). Mine got hit by a snowplow while locked to a rack...
It's not really so much the carbon fiber (at least for me), but just the skinny,thin,fragile tubing that typically makes up high performance road bikes. It's just not overbuilt enough for me to trust with "unusual" forces. Now, if you were talking about a carbon fiber downhill mountain bike (yes, they make those), I'd say sure, hook an axle bolt trailer to it, no problem.