I just got a Timbuk2 "Dumpling wallet" for this purpose. Works very well, enough room for cash, cards, keys (there's a tether), phone and still fits in my jersey pocket.

It even comes in pink !