I ride around in jeans because it's comfortable. Not sure how comfy it would be on a long ride, but for 2-5 mile stretches, jeans and a brooks is definitely a nice way to go.

Took me about 200 miles on average to break in a Brooks, btw.
My husband bought one once that we couldn't break in. We sold it to a 200 pound guy who loved it.