No, it doesn't.  Fresh goat's milk (I can't speak for sheep's milk) tastes as sweet and delicious as cows milk.  In fact, it's better. I does somewhat depend on breed, what the goats eat and how the milk is processed, but if it's done right, it's not goaty at all.   I don't like goaty cheese even a little bit, but I do like the fresh milk. Store bought stuff tastes like dirt...it's gross.
I grew up with a gag reflex for whole milk.  As a small child, I wouldn't drink it.  My doctor told my mom to try 2% or skim and it worked.  I grew up on 2% and switched to skim in college (where 2% wasn't available and whole milk seemed like paint to me). Up until a year ago, it was the only way I could handle milk...even in coffee.  
Now I use whole goat's milk in my coffee and smoothies and in recipes, but I don't drink it as a beverage.  Different goat breeds have different fat content in their milk.  Ours is richer than whole cows milk which is generally 4% butterfat.  Our goat's milk is about 6%.  It is downright amazing what you can get used to!  Now skim milk seems so blah, flat and boring to me. 
 
And we drink our milk raw, too.