In most states, it's only forbidden to cross the double yellow to pass another "motor vehicle" (which excludes farm equipment and horse-drawn buggies as well as bicycles). Vehicles in the road that are not defined as "motor vehicles" by state law are considered "obstructions" (more equivalent to a fallen tree, a piece of truck tire or an enormous pothole), and it's perfectly legal to cross the double yellow (when safe) to pass us.
(This is a general rule - I haven't researched the law of every state.)
ETA: one point nobody's mentioned is that when there's a long line of vehicles behind you, whether you're riding a bicycle or driving a car below the speed limit, it's common courtesy to pull over and let them pass when you have a safe opportunity to do so. Yes, it's inconvenient, and a long paceline makes it very difficult to do, but it's still common courtesy. Sometimes there just isn't a safe place or time to pull over (e.g. in rush hour traffic, once you pulled over, you'd never get back in line), and in those cases the cars do just have to deal with it IMO.
Several years ago I was looking at the California vehicle code (which is much more detailed than most states'), and IIRC, even when you're in a car, if you're holding up five or more vehicles, you are required to pull over. Something to think about.
Additional edit: here's the California statute:
Turning Out of Slow-Moving Vehicles
21656. On a two-lane highway where passing is unsafe because of traffic in the opposite direction or other conditions, a slow-moving vehicle, including a passenger vehicle, behind which five or more vehicles are formed in line, shall turn off the roadway at the nearest place designated as a turnout by signs erected by the authority having jurisdiction over the highway, or wherever sufficient area for a safe turnout exists, in order to permit the vehicles following it to proceed. As used in this section a slow-moving vehicle is one which is proceeding at a rate of speed less than the normal flow of traffic at the particular time and place.
Amended Ch. 448, Stats. 1965. Effective September 17, 1965.