Keep trusting your instincts. If your gut says don't ride by this man's property, then don't do it. I agree with the drive to ride advice. You'll at least get to ride, even though it is more of a hassle.

Stay safe!