I think riding alone or with a partner that is less stress-inducing is a fine idea. By the way, the original post made it sound like he's drafting you, given that he's a foot behind your back tire. If he's such a strong rider, why is he sucking on your back wheel? How about having him pull you?