My last husband obsessed over those cute little aerobic bunnies on the morning exercise shows to the point I went on a starvation diet to try and look like them. Bottom line: He's now an ex!! and I'll Never look like them!! It's taken me close to 6 yrs to recover from the self-image damage done during that time, and some days I'm not over it. But I learned the advice Mick gave - if my jeans fit and I can ride 5000 miles a year, then who cares what I look like! I even keep 2 jean sizes in the house so on those days when the smaller size doesn't feel comfortable (things like water weight, last nights food binge, whatever), I just switch out jeans.......I don't see it as a setback - it's just another place my body is at right now. and I amaze myself that I'm in better shape and more active now at 47, 5.2" and 140-150 (I get heavier in the winter lol) than I was at 27, 5.2" and 125. and I have Yet to have a dr tell me I need to "lose a few more pounds" when they see my exercise/workout schedule! If you're fussing over the junk you're eating at work, bring in your own foods and/or go out for a lunchtime ride/walk. See if that doesn't help you over the "binge" craving. Whatever, don't stress it, your accomplishments Shout for themselves, embrace them for their worth!! 
Oh and clothes size: it's just a number as you said, one brands 10,12 whatever is another brands 12,14 or 8,10, I don't even know half the time what size I'm wearing, but I know they're comfortable!
Life should NOT be a journey to the grave with the intention of arriving safely in an attractive and well preserved body, but rather to skid in sideways, champagne in one hand, strawberries in the other, body thoroughly used up, totally worn out and screaming: "Yeah Baby! What a Ride!"