I wouldn't ride with him ... first time he got too close to me or made me too nervous I'd pull over and wait for him to finish... if he didn't finish I'd tell him I'll make the trip another time. I have, however, a low tolerance for unpredictable bicycle movements (and people who act like little boys and need to be dangerously dominant).