How long have you been swimming and how much swimming (time and distance) do you do in each session? Endurance take a while to develop. What kind of swimming plan are you using? I've had good success with the 0-1650 plan. Keep at it at least twice a week and it will get easier.