He then suggested I cut out yet more types of food.
I've cut out so many food groups through the years with this disease. Lots of us have had to.
It is overwhelming to do and I would say it makes it a little easier if you do just one food group at a time.
Then after you've taken it out for a while (7-10 days for instance) evaluate if something/anything has improved for you.
If not, trial that food group back in using small doses and see if you crash or get other symptoms like headaches, brain fog, depression or aggression, etc. (The list is long so just try to be very aware of what's going on with your body. Even something like lips and/or eyes drying out can be a indicator.)
If you don't seem to get any reaction at all, then I would say that food is probably (maybe/possibly) not a problem for you (at least for now) and then move to the next food group and do the same thing.
I've been trialing foods back in this last year or so since being on antibiotics and antibiotic herbs and I've found I can eat potatoes and natural ferment sauerkraut (even some types of mushrooms) again but dates, eggs, and soy all still crash me very badly and for several miserable days afterward. (I still stay away from cow's milk dairy, yeast, and gluten.)
Also be aware that eliminating these things probably won't "cure" the Lyme or ME/CFS but exposure to a problem food (or environmental trigger) can cause a lot of additional problems that make this disease even harder to deal with.
Unfortunately, it gets to feeling like one step forward; two steps back sometimes.
Hang in there and just do what you can handle. Also remember the ME/CFS moto "Start low and go slow" on anything you do try.