Suze, I think someone could write a book on this. Here are a few that get discussed a lot:

1) Fear. Even if men and women are equally fearful, I think women tend to be socialized to express it more readily. I hear a lot of "It's too dangerous to ride on the roads!" I live in an area with a very nice network of separate bike paths, and I see more women, but it's still not a 50-50 breakdown

2) Children. Like it or not, women still get the burden of childcare. That means that many don't have the option of going out for a 2-3 hour ride on a regular basis. It's sad, but even in two-parent families, if both ride, it turns into the mother having to ask the father to babysit. (There's a quote from a cycling coach that says pretty much this. I'll have to go dig it up). I've seen a number of families walk into bike shops, and the man will start chatting with the shop guys about the ride he did yesterday while the woman will be attempting to keep two or three children from destroying the shop.

3) While the sport may not be inherently biased toward men, the marketing arm of it sure is. In most bike shops I've been to with my boyfriend, the staff will address him first, even if it's ME who wants something. It's hard to find a modestly-spec'ed WSD road bike the shop floor. LBSes outside of cycling hotspots like Arizona or SoCal tend not to have a wonderful selection of women's clothes.

4) Comfort, psychological and physical. And fashion.
A number of women I talk to (40-60ish) in my department like riding bikes, but really don't like bike saddles. I have yet to convince them that it's a matter of finding the right one. If we're talking road bikes, it has (until relatively recently) been difficult to find geometries that better fit the "typical" women's build (long legs, short torso). (Note the quotation marks. ) If you're not comfortable, you're not going to ride.
Finally, many women may not be comfortable wearing lycra (since they may see it as a requirement of the sport) for a variety of reasons (body image, cultural, etc). In addition, as silly as it may seem to some of us, there's a fear of helmet hair.