I would think so. At the very least, you might lower your functionality for several days. That's what happens when I eat one of my allergy foods.
I understand need and not wanting to have anything go to waste though especially when so many of us are living in poverty.
Even today, I was considering whether to start using my wood stove again even though I have MCS (multiple chemical sensitivities) in order to cut down on my heating bill somewhat since we have very cold temps right now and the furnace keeps cycling on.
I think it depends on the individual. One person might develop serious symptoms from being in a room with a spot of mold somewhere. Another person might eat many slices of moldy bread without showing any symptoms (other than odd taste). I can't remember ever showing any signs of sensitivity to mold.
The question for you is: what is your tolerance to a given substance?