Maybe a little sensitive, but a bit insensitive on his part too. Maybe he was reacting to your negative statement? As in "I don't like you feeling so down on yourself"?