It's definitely simpler to switch between two sets of wheels, but cheaper to just have the extra tires and if you're not switching frequently that might make more sense. I chose to pick up a used wheelset and put slicks on it for my rigid MTB, because I have that bike set up as a sort of all-purpose bike and sometimes I ride pavement, other times the local easy trails. I switch back and forth often enough that changing tires would be a real pain. Maybe you could find out how much it would cost to set up a second wheelset for your bike, then make your decision based on weighing cost vs. convenience/simplicity.