My favorite wool t-shirt for riding is an asphalt-grey. Bummer, but gray is one of my favorite colors. I always (always ALWAYS) wear my screaming yellow something over it. (usually my PI whisper vest. Thanks TE!)
Back in the old days when it was very hard to find bright colored biking gear I wore a hunter-orange sweater my G'pa won at a mass motorcycle blessing. He was very big on wearing full protective gear and being visible.
I figure wear whatever you love, but put something gawd-awful bright on over it if you need to be seen.
"If Americans want to live the American Dream, they should go to Denmark." - Richard Wilkinson