I wear Lycra and capris at that.

If you like the thought of baggies but want something that is more fitted, take a look at this short. Dirtrag did a review of the men's version, the Betasso short, and said the legs were form fitting and would not snag on the bike seat.