You know, we never know what we have until we lose it. Maybe he will realize that in time. Maybe you will still be willing to work on things if that time comes and maybe you won't. If you still want to make things work and love this man, then it is totally irrelevant what anyone else thinks.
You go for it, if the time comes, no one has to live your life but you.