North Woods Gal, Kansas is actually home to a rather famous gravel race (which I do not aspire to) called the Dirty Kanza. They have various distances including a shorter community ride. But it takes place in the Flint Hills which means there is some climbing. I live in the tail end of the Flint Hills and it is possible to find yourself climbing. One of my first goals is a ride to Coronado Heights (where legend has it, Coronado actually visited.) This is entirely do-able since on my road bike I ride by it but to get to it, you must ride gravel. But again, like you, I like being off the beaten path. I was at another site looking at what you need to know about gravel rides and they said you have to have a light grip on the handlebars as you come in to heavier gravel so you can "float" across the gravel. Bet I know what the opposite of floating is.
I had anxiety reading about maneuvering gravel but then I remembered that I have overcome other cycling anxieties.
Do you wear clips to ride gravel? My hybrid bike has pedals on it and I won't change that. But if I liked gravel and got a dedicated gravel bike, I might do something different.
2011 Trek Madone 4.5 WSD
2011 Trek FX7.2--What can I say? It was on sale!