I have never seen a flat part of Vermont.... but I guess along the river will be.
I agree with Tuckerville. This is a lot of riding. I might consider A) either leaving from Poughkeepsie or B) take the train or other transportation there and ride home.
I say this because your tour is most likely very hilly. I would want fresh legs to start the tour. And Adventure Cycling can help you with the storage of your gear/picking you up, if you ask. Every tour I have been on has been more than helpful with these types of issues.