A few of the people I ride with sometimes wear non-cycling workout wear to ride in, and it sometimes is black or some other dark color. Although, just because it is cycling specific doesn't necessarily mean it is a very visible color. I've noticed that a lot of the blue jerseys just fade into the highway.
When I"m looking at the riders either ahead or behind me at a distance, you can really see a huge difference in visibility between the riders in high vis colors vs the ones in dark colors. Yellow shows up for miles. I am so paranoid about being seen that I wouldn't even enjoy a ride if I wasn't wearing nice bright colors with my blinkie light flashing! My next purchase is going to be a yellow vest.
Grits
2010 Trek 5.2 Madone WSD, SI Diva Gel Flow
2002 Terry Classic, Terry Liberator