Seems like a viable compromise might be to switch to MTB shoes and pedals. MTB shoes are generally much more "walkable" than road bike shoes.
I've always preferred mt. bike specific bike shoes for safety reasons. Forget getting assaulted or having to walk 3 miles from a flat. I'm more worried about slipping and breaking my neck going down a flight of stairs!