I was super sensitive to wool as a kid and while I tons of wool sweaters, I could only wear them with a heavy turtle neck underneath.
I have to have the finest (17-18 micron) merino to have it next to my skin. The Ibex tees I bought are on the edge of bothering me, but I wear them. But, I can't layer my Ibex Kates with a Shak, because the wool over wool makes me itch. I can only wear wool pants if they are lined and even a stretchy cotton/lycra button down shirt will itch me enough that it's OK alone, but not with a merino sweater over it.
So, from my experience, you probably can wear the lightest weight Merino with no issue. I find Ibex and Smartwool the least bothersome. Haven't done so well with Icebreaker, though.
Of course a seam that is too thick will irritate me and make me break out in welts, so take this for what it's worth.