I don't think it's crazy at all! In fact, I agree with Teigyr that it's a great idea.
I trained for my first IM 9 months out. I had never even done a sprint at that point. I had done marathons, but nothing tri related. My first sprint was at the beginning of May, my first IM was at the end of September. I did fine and so will you!

With all the training and life altering that has to go on to prepare for an IM, I personally think that doing it together is smart. The time you spend away from your spouse training can be tough (I know that for sure), and it can be hard when one is so incredibly focused on a goal and the other is just along for the ride (DH is very supportive and does tris, but it was still tough with the volume of training for the IM).
I think it's a great idea. You'll both have the same goals in mind and will spend a great deal of your training time bonding over that common goal (and spending time with hubby is always a good thing- not like "I'm going out to train for 9 hours- see ya later").

I think it's GREAT!