My guess is it's muscle. I'm probably about the same height/weight/build as what you described, and when I'm getting good regular exercise my clothes will start to feel snug -- I especially notice this with work clothes. Muscle is dense, and it can be surprising to some people that when you develop muscle, it is often accompanied by weight gain.

I don't own a scale, but I do know that my weight has varied with my relative health/fitness. If I'm not training for anything specific, I'm usually in the 105-108 range. I may have weighed a little over 120 after my first sprint tri, and that is probably the most I have ever weighed.

IMHO, you would KNOW if there were something to be concerned about -- your energy levels might be low, for example. It sounds to me that you are very fit.

That said, if you're really concerned that something other than your physical activity is responsible for the weight gain, then by all means you should go to the doctor.

(They will probably tell you to keep up the good work and that they wished all their patients were as active as you!)