I'm going to throw out another vote for fear of the mechanical aspects: I used to be terrified to ride alone because OMG what if I had to change a tire? Or if something broke? Practice, and always carrying a cell phone, help a lot.

Also, yeah, the aggression/competitive thing.