Toughen up and stick with it!

As long as you're not getting discouraged when you do get dropped, then use them for motivation and to get better. Ignore the jerks (who probably just feel threatened) and socialize with the nice ones. It's awesome that you're keeping up with these guys and the fact that you keep at it should give you a lot of satisfaction.