I don't personally have much experience with cats, but have heard that they are ultra-sensitive to household smells that we humans cannot detect: colognes, air fresheners, cleaners, etc.

As someone else suggested, maybe the kitty doesn't like that brand of litter-or there is something (noises, foot traffic?) upsetting them within the litter box location, etc.