I'd suspect mold too. If it was building materials of any kind, I'd expect two things: (1) that your symptoms would get less pronounced over time as the concentration decreases, and (2) that you'd react in other buildings too.