It's my understand that drafting without asking is rude. I haven't had it happen to me but I would probably treat if like a tailgater - if he won't get off your wheel, then just pull over and stop. Take a break, eat something, catch your breathe - let him go on by. It isn't worth it to ruin your legs becuase of him - coz I'm guessing by the time he finished His ride He wasn't burned out!