Quote Originally Posted by sah View Post
The strange part is it only happens when I am not pedaling.
You mentioned that it happens on rough sections of pavement. Perhaps your chain is smacking against the chainstay as you go over bumps. If you're pedaling over those same bumps, there could be enough tension in the chain to prevent it from smacking.