I suggest you work out a compromise where you two sometimes ride together and sometimes ride on your own. That works well when two people don't ride at the same speed. Think of it as a pleasant thing that adds variety to riding.