Your touring company should be able to give you recommendation on clothing if they are suggesting the sites to be visited.

When I toured (supported) Belgium and Netherlands a couple years ago, we did visit churches, cathedrals, museums, galleries and were welcomed everywhere. It did not matter if we were wearing bike shorts or street clothes. I don't think anyone on the trip brought a skirt or wrap to put on over bike shorts either. There is probably a greater risk of offending by wearing tasteless t-shirts or muscle shirts (the guys) and flip flops.

More important I suppose is to conduct yourself accordingly, and give due respect to the place you are visiting. No yelling, laughing, running, talking during presentations, no snacking inside the church etc.

If you don't give people a reason to dislike the "Ugly American" they will simply remember the nice group of Americans that visited in their bike shorts and asked thoughtful and appropriate questions, as opposed to the noisy American cyclists that rudely talked through the tour, ate candy bars, left behind their trash.