Clips make sense to me on pavement. Where you need more responsiveness to conditions like you might on gravel, it does not make as much sense. I don't even like the feeling of slipping on ice in a car so I know I won't on dirt but I can figure it out. I have fallen before. I have fairly pedestrian flat pedals on my hybrid; what the bike came with. I might look at different pedals if I get grabbed by it.
Have you ever done any bigger rides on gravel? I think in Kansas there are some less hard-core more recreational gravel rides...
2011 Trek Madone 4.5 WSD
2011 Trek FX7.2--What can I say? It was on sale!