It took me a long long time to be able to ride down hills without anxiety. I still get freaked if I let myself go too fast but at least I am now able to feather my brakes rather than ride them. Hang in there it will come. Maybe you'll eventually turn into a down hill bomber or a speed demon. Odder things have happened.