sizes are a joke!

I can wear a size 4 at banana republic, but need a 10 in most patagonia pants. The rest of my clothes are in between, but a 4 is so far from a 10 it seems a little crazy!