Try taking them with food? I take capsules, and they don't bother me if I take them first thing in the morning with orange juice, then eat something easily digestible afterwards like a sandwich. If I forget and have to swallow them afterwards I think I have more chance of them "coming back to haunt me".

I used to take liquid cod liver oil, which tastes truly revolting, but I found out that drinking a little orange juice first, then swallowing a spoonful of oil, then a lot more orange juice, helped "cut" the taste. Coffee did NOT work, there's some fat in coffee and it increased the taste.