The only reason you'd need a jersey is for the back pockets. Otherwise just look for a high-vis tee or tank.

You could also buy regular bike shorts and wear a skirt over them.

Depends on what kind of riding you're doing. There's no need to drop a load of cash. Keep an eye out here for second-hand clothing. I've gotten some really nice stuff here.