Squirrel,
I found that when my gloves were tight, it impaired my ability to use my fingers,
so a couple months ago when I rode in 32 degree weather with 2 pairs of gloves
i had hand pain for quite a while because the fatness and tightness of the gloves made it hard to grip and to operate the levers.

Hands are definitely a limiting factor for me as well!