Valve caps are an important part of air retention in a Schrader valve. Those should always be in place and in good condition.

On a Presta valve, they're kinda pointless. A waste of time IMO, rather than a waste of weight. Mainly they keep the valve stem from poking through the wall of a new tube in the box or in the seat bag. I suppose they might give the stem a tiny bit of protection on 'cross bikes, but for the most part, skinny Presta tubes don't get ridden through the kind of debris that might threaten a valve stem anyway.

I think tossing the nut is kind of a weight weenie thing though. To me, they're useful for positioning a new tube, and at that point it's a waste of time to take it off again.