I agree that the main difference is in climbing, and that in any case the aerodynamics of you + bike is more important than the weight on the flats.
I agree with Limewave's DH theory: it's better to eat less icecream (or one less toast) than to try to shave grams.
However, the lighter you are, the more difference it makes. A 5 lbs difference is 5% of a 100-lbs person's body weight. You wouldn't that person to loose five pounds. That's the thing with racing cyclist, especially pros. You wouldn't want these guys to cut in their own weight, they'd disappear!!
In other words: your mileage may vary.