I don't think you're mad. A 3 day backpack trip is a great fitness base and prepares you for climbing hills on the bike too. I would arrange some contingency plans so you have bail out options at a few points along the way just in case. But I'd go for it.