I had the exact problem when I started riding. Actually , it lasted for quite a while!

Then suddenly it just got easier. One thing I KNOW I was doing wrong every time was that I was afraid to press the pump on far enough. This is because right off the bat I had bent / damaged one, and thought I was just being too rough with it. But now I seem to have a feel, and the secret for me is just simply pressing in on straight and far enough on. (Oh, and getting the little flip stem thingy in the right position to lock it on... that is always challenging for me! ).

Hang in there!