I'm guessing it's because inside your shifters you have a disc of some sort, with retractable springloaded "hooks", that catch the gearing lever when you gear by tightening the wire, but allow the lever to slide past when gearing the other way, loosening the wire. These hooks can freeze in place or get very slow when it's cold. But I'd think it would be a problem when shifting up, not down

At least this happens to the pawls in hubs, so I'm guessing this is similar.