I used to have a "friend" who would make such loaded comments all of the time, and more. After a couple of years I felt like I had to walk on egg shells around her. I know I am far from perfect myself so I've a pretty thick skin for such things - but finally realized that she was no friend and sadly walked away. How you deal with this depends on how important the relationship is to you - but it sounds like you did exactly the right thing.