Depending on the garment part wool can be fine. Socks for instance. You need some nylon for durability (knitters have added a nylon running thread to the heels of socks for years) and elastic or Lycra can help them hold their shape (not sag, the bane of my hiking/skiing life).

Nylon and polyester used appropriately can make a far more durable garment and help the garment hold its shape, especially in a woven fabric. You just need to learn how to gauge when it's used appropriately and when it's just a way to cut cost and, therefore, quality.