There are really two questions:
- Did you do anything offensive? Not in my book
- Might they have been offended? It depends on whether or not they have the same "book" But, if so, it's because they had a different expectation and not because of anything overt that you did.