worry about something else being wrong
If you do think that you're having problems with anxiety, then I think that it would be worth trying to improve that somehow, regardless of other health problems. If it's not something you feel comfortable talking to your doctor about, and you cannot see someone privately, it could still be worth doing some independent reading on it. To me, it seems like a lot of problems with 'anxiety' can be helped by psychological techniques. A degree of anxiety is often reasonable, but if you find yourself getting caught up in it, or struggling to think as clearly as you'd like about things because of your anxiety levels, then learning about different management techniques could be a worthwhile use of your time.
I hope things start to improve for you soon. Good luck with everything.
edit: There is also, of course, plenty of quackery around psychological responses to anxiety.
I've read Jon Kabat-Zinn's 'Full Catastrophe Livining', and thought it was interesting and worthwhile - although I didn't agree with it all. I've seen a number of other CFS patients mention it positively too.
Someone I respect said that this website is pretty good, but with a few bits they didn't agree with:
http://www.llttf.com/
Shinzen Young does online/over the phone mindfulness meditation retreats cheap, and I think that his ideas are interesting too.
I think that there are aspects of CBT/mindfulness type approaches which can be understood as an offshoot of philosophy intended to clarify thought and improve one's pursuit of truth, with some useful tricks for avoiding common errors/instincts/prejudices. Unfortunately, they can commonly be misapplied, and used/promoted by quacks (as we've often seen with CFS). Hopefully everyone will be able to find what is most helpful for them.