I had severe unrelenting anxiety with panic attacks since as far back as I can remember. I definitely think it wore my body down and made me more susceptible to illnesses, and eventually developing CFS.
I no longer have it as of almost two years. The primary driver for me is a strong sensitivity to dietary amines (histamines, tyramines etc) and glutamate, closely followed by effects of severe streptococcal bowel overgrowth. I had a massive permanent decrease in anxiety following ampicillin to treat the strep.
These two driving factors also appear to have disrupted my ability to absorb and manage nutrients, because I also had significant decreases in anxiety when supplementing melatonin, tryptophan, niacin and high doses of Zinc. I need to take these continuously otherwise I backslide, but its never ever as bad as the old days (shudder). Unless I eat histamines/glutamates, well then...