You can simply state you have another obligation during the time you will be gone on vacation and leave it at that.. there's not really any reason to give them more information.. where you are going or why really isn't any of their concern.