Call the police anyways. And keep calling them every time he threatens you and your family. It's their job to take care of public safety, and this guy certainly seems like a threat. The more calls about him that they receive, the less they can ignore the situation. And to be fair, they might not know about all of his antics if no one calls in about him.

Best of luck to you.