You are so not crazy. My saddle height got messed up a couple months ago and it took 4 or 5 rides before I was able to get it back to where it was comfortable.

There's a reason that Lance's nickname was "Mr. Millimeter."

If you can, it might be easiest to go back to the shop that did the earlier fitting for you and ask them to help.