I just think its cool you went to church regardless of what you had to wear. I've done the same on trips and the church goers are very appeciative of the effort.

I go to all sorts of restaurants and stuff in my car gear and no one seems to care. The most uncomfortable day I ever had if once I biked to work and since I was only staying a few hours stayed in my bike stuff. The suit and tie crowd gave me the once over that's for sure!