Wool pants

[KnottedYet]

These look quite nice (I'm too frickin' fat to wear them at the moment) and tailored, but you will have to strap them down at the ankles on the bike: http://www.teamestrogen.com/prodIB_0455.html

The fabric is dense and heavy enough that it doesn't cling.

[Crankin]

I have a pair from Jones Ware that look almost exactly like the Ibex ones. And, you can go semi-custom for 10.00 extra... i.e. have them hem to a different length or change the waist measurement. You do have to clip the right leg to wear for riding, but I have done that and it was fine. I would think you could wear these at work, if the atmosphere wasn't too dressy. They have a wide, flat waist band and are quite comfortable. I wear them snow shoeing and x country skiing, too.
But, I have worn them out to lunch, etc when riding in let's say March, when it's in the 40s.