Big benefits of going clipless?

[sundial]

I also have a bum ankle and I'm worried I won't be able to act quickly enough to get it out of the clip.

If you feel you can't unclip fast enough, I would suggest you consider BMX type platform pedals with the pins that secures the sole of your shoe to it. Since you are already acclimated to using toe clips I think the platform pedals is a natural progression and it will give you the confidence needed when you need to bail fast. There are stiff soled mtb shoes that would work nice with the platform pedals and one of the benefits of that type of shoe is that the cleats are recessed and you can walk around in them.