I think that the reason there's a weight limit and the reason that petite folks might take it harder, is simply that you donate the same amount no matter the size of your body. I asked on my first trip if big tall guys donated more, but no, 500 ml is the standard size. So small people donate a slighter larger percentage of the total amount of blood in their body.

I also think (lotsa thinking going here!) that being fit is going to help you recover from donating, but fit people live active lives and will feel the lack of energy a lot more keenly.