I've had similar experiences... usually in a store and once in synagogue . Usually, one of my clients from the gym where I taught aerobics would come up to me in the mall or grocery store and say loudly, "Oh, I didn't recognize you in clothes."
The time at the synagogue involved a man who regularly came to my 6 AM class. This was about 15 years ago. He came up to me in the lobby and said, "What are you doing here? It looks funny to see you with clothes on!"
His wife was not amused. It turned out that they are friends with some of my good friends and over the years we have ended up doing holidays with this couple. We always laugh about that encounter.