Is your pillow (mattress/bedspread...) down? Feathers do it to me, and I don't have an 'allergy' to them.
Also, an irritant, so dog/cat hair or dust/dust mites all can be irritants... sterilize your bedroom.