I agree, thoughtful of you, but I would not worry about it.
I'm Type 1 diabetic, my mom is Ciliac - we both know darned well what is in things! I wouldn't eat a "sugar free" pie in that circumstance, because, quite often, "sugar free" means honey. My point is, if he's observant, than he knows darned well what *could* be in the pie.
Next time, might be nice to have a little sign that says what's in it. There is a woman in my building at work who's ciliac (I have no idea who it is, though) and every work potluck we have there are now little signs on several of the dishes "no gluten, contains soy", for example. These little notes are nice and helpful.