the only way to fight fear is to do it.
Keep riding where you are comfortable, and then go a little further. You will find that a lot of us fight fear all the time.