Best to break it off right away vs. just not returning his calls.
I would go somewhere public like a cafe, over lunch. That way when you break up he might not make a scene. Just don't tell him over dinner, that's too much like another date and he could be saddened.
You could call him but let him down gently, say "It's not you, it's me" :p
Texting him that you don't want to see him again is just bad manners. :cool: