There are also food intolerances -- no allergic reaction, but the GI tract has trouble digesting a certain food due to a missing enzyme or some other reason, resulting in various stomach and intestinal issues. I have this problem with tomatoes. I can generally have them in small quantities, like a few slices on a sandwich. But recently I had fair amount of stewed tomatoes with dinner on a Monday followed by tomato soup for lunch on Wednesday, and I was sick for a week.
The problem with food intolerance is that other than lactose intolerance, most people are not familiar with it so it's a pain in the a** to explain, especially in a restaurant. So it's easier to just say you're allergic, even though you might be able to eat a small quantity without problems.

