I'd have to agree with that advice. Asking directly may be the way to go and definately talk about it. We all react differently to our fears. My significant has gotten better about the fear of my riding which I think was hard because I'd already had a very severe and life threatening accident. I explain how I ride to protect myself and how I react to different situations that occur.
Good luck and be glad he is concerened despite how it comes across. Let him know you appreciate his concern.