I had a very similar fear after I crashed doing 40 mph on a mountain descent. I left the road on the 3rd switchback and hit a rock. The bike stopped and I kept going It was a long time before I could do a descent of any kind without my heart racing and my throat tightening. The anxiety was paralyzing.
I just took it slow. I took deep breaths, made myself relax and kept making myself do downhills until I got my confidence back. I don't descend with the wild abandon I used to and am much more cautious but I've got my downhill legs back.
Just try to relax and keep practicing doing downhills until you gain some confidence. If you are more comfortable keeping one foot unclipped, then do so but try to work toward staying clipped in. My best advice is to sit back in the saddle, trying to keep more of your weight on the back wheel, break with your back brakes, not your front, stay down in the drops, pinch the top tube with your knees, and just keep practicing. Also make sure your bike fits your correctly. If you are riding a bike that doesn't fit you, you can really feel out of control on a downhill.
"We don't stop playing because we grow old; we grow old because we stop playing." George Bernard Shaw
Luna Eclipse/Selle Italia Lady
Surly Pacer/Terry Butterfly
Quintana Roo Cd01/Koobi Stratus
1981 Schwinn Le Tour Tourist
Jamis Coda Femme