Discussion about Armin labs (Split Thread).

[JAH]

But in my experience you can tell when people are ahead of the game because they have everything openly documented and in general make no special claims, simply give their case. The controversial Lyme testing labs in no way fit that picture to me. There are a whole lot of other reasons why I take the view I do but they are best kept confidential.

I am not suggesting anybody takes my opinion as carrying weight, simply that other opinions may be ill-founded and that it is sensible to consult with specialists who have no financial interest in the matter.

Your opinion carries weight to many of us. The opinion of another doctor I respect disagrees with you, so here I am. I understand your discretion, but wish you could share the root of your doubts about Armin.

Armin does consider PCR as the gold standard - in that a positive PCR is a positive, just that there are many false negative PCR tests.

A Lyme researcher has told me that new tests are on the horizon, I hoping that is true...JAH

 

[msf]

I am not sure of the precise biochemical dynamics, but the accuracy of medical tests is expressed in terms of their sensitivity and a specificity. If a test was 100% accurate every time, it would have both a sensitivity of 100%, and a specificity of 100%. But in the real world, tests are never perfect, so you never get a 100% figure.

Lyme tests for example have a sensitivity of up to around 80% for the best tests. That means 80% of patients actually infected with Borrelia will get a correct positive result on the test when tested. But 20% will get an incorrect negative result (a false negative).

And if the specificity of a Lyme test is around 95%, that means 95% of patients who are not infected with Borrelia will get a correct negative result on the test when tested. But 5% will get an incorrect positive result (a false positive).


This issue of false positives and false negatives is problem in Lyme testing because the testing technology is not very accurate.

If we compare to HIV testing, the HIV test is much more accurate, with a sensitivity of 99.7% and specificity of 98.5%. That means only 0.3% of patients (3 patients in 1000) tested for HIV will get a false negative, and 1.5% of patients (15 patients in 1000) tested will get a false positive.

Yes, I think this is a problem with LTT testing, and it is connected to the problem I outlined above, that of discerning an active infection from a past or latent one. However, I think adjusting the cut off of the ELISA so that less than 5% of healthy blood donors test positive (see my last post) is an even bigger problem, in that it is completely arbitrary, and leads to ridiculous situations like the one mentioned in the video where people test positive in Alsace but not in Paris, or one where two seroprevalence studies using the local ELISA cut-off values will find exactly the same level of seropositivity for Lyme in Alsace and Paris.

 

[Daffodil]

how can anyone say that less than 5% of healthy blood donors are positive for lyme when lyme tests are supposed to be so inaccurate?

 

[Vitalic]

I was just watching the video below, in which at around 14.00 he says that EUCALB (European Concerted Action for Lyme Borreliosis) recommend that the cut off value for an ELISA is adjusted so that...wait for it...less than 5% of healthy blood donors are positive for Lyme...is this the validated testing that I hear people talk about?

That presentation underlines the difficulties Lyme/Chronic Lyme patients are faced with. There seems to be enough evidence to assert beyond any reasonable doubt that it is possible to be a seronegative Lyme patient, and so at what point do you turn your back on it as the potential root cause while it remains one of the only plausible explanations. After one course of antibiotics? What if you had been sick for years prior? What should be the cut off for a rational person trying to approach the problem in an evidence based way. All we have is opinions, and mutually exclusive/contradictory opinions at that, whichever choice you make it requires making a judgement call on insufficient evidence.

 

[Vitalic]

At the time you tested positive for the second Elispot LTT, were you taking (or had you taken) any medications or supplements that were different than you'd taken when you tested negative for the first Elispot LTT? For example, did you undergo antibiotic treatment in between the two tests?

Yes, I did four weeks of Doxycycline in between. The reason for that was that I had tested positive for Mycoplasma + Tularemia (via RED Labs). Are you suggesting that the treatment was significant?

I should add, the report on the initial test said the following:

Perhaps the combination of those two things are responsible for the change in result, but I guess if there turned out to be fundamental issues with the test as some suspect then why the result changed is not all that important, insofar as that if the chance of a false positive is unduly high then I would have likely always got a positive after a certain number of retests.

 

[Hip]

I think adjusting the cut off of the ELISA so that less than 5% of healthy blood donors test positive (see my last post) is an even bigger problem, in that it is completely arbitrary, and leads to ridiculous situations like the one mentioned in the video where people test positive in Alsace but not in Paris,

It is arbitrary to an extent, but you have to place your positive/negative cutoff line somewhere.

The alternative is to give patients results like "you are slightly Lyme positive", or "you are very Lyme positive". In other words, quantify how Lyme positive they are.

But that does not make much sense either, because the positive/negative result indicates whether the patient gets treatment or not. You cannot really have a situation where, when you are only slightly Lyme positive, you only get a little bit of antibiotic treatment. You either get antibiotic treatment or you don't. That's why you have to create an arbitrary positive/negative cutoff line somewhere.


As mentioned in this earlier post, where you choose to set the positive/negative cutoff point should be based on two factors: what the consequences are for not treating a patient who actually has Lyme disease; and what the consequences are for treating a patient who does not have Lyme disease.

Also, you have to consider how successful treatments are. If the treatment has minimal benefits, then the case for treatment is not a strong as a treatment which is highly effective.

In an ideal world, cost would not be factor in this decision where to place the positive/negative cutoff line, but in practice, that is something that is probably also taken into account by health authorities: if a test cutoff is set so that it produces a large number of false positives, then treating those false positive patient can be considered an expensive waste of money.


Of course, none of this would be necessary if a nearly 100% accurate test for Lyme disease were available.

 

[justy]

Labs like Arminlabs may have moved their cutoff line to the left to reduce false negatives

This is again just speculation. MAY HAVE? LABS LIKE?

Do you think a man of Armins professional standing does not know this? Why don't you ask Armin himself?

As far as I can tell these discussion are bordering on ridiculous when people are making (not you specifically) posts that could be slanderous with absolutely nothing concrete to substantiate their claims.

 

[msf]

That presentation underlines the difficulties Lyme/Chronic Lyme patients are faced with. There seems to be enough evidence to assert beyond any reasonable doubt that it is possible to be a seronegative Lyme patient, and so at what point do you turn your back on it as the potential root cause while it remains one of the only plausible explanations. After one course of antibiotics? What if you had been sick for years prior? What should be the cut off for a rational person trying to approach the problem in an evidence based way. All we have is opinions, and mutually exclusive/contradictory opinions at that, whichever choice you make it requires making a judgement call on insufficient evidence.

Yes, in the absence of a gold standard, you just have to go for the approach that makes the most sense to you - this usually means a choice between doing something and doing nothing. For more cautious people, doing nothing will seem like the better bet, but that might be just as risky (it´s just that the risks are old risks rather than new ones). It might interest you to know that the ILADS guidelines were accepted by the National Clearinghouse in spite of the fact that the evidence for their suggestions was ´very poor´ - the justification for this was that the suggestions were deemed to be low-risk.

They obviously did not include financial concerns in their risk assessment, but then again you have to weigh up the risk of not doing anything (and probably not working) and doing something (and definitely spending lots of money but possibly getting back to work).

 

[msf]

It is arbitrary to an extent, but you have to place your positive/negative cutoff line somewhere.

The alternative is to give patients results like "you are slightly Lyme positive", or "you are very Lyme positive". In other words, quantify how Lyme positive they are.

But that does not make much sense either, because the positive/negative result indicates whether the patient gets treatment or not. You cannot really have a situation where, when you are only slightly Lyme positive, you only get a little bit of antibiotic treatment. You either get antibiotic treatment or you don't. That's why you have to create an arbitrary positive/negative cutoff line somewhere..

No, the alternative is not to use the superfluous and misleading first part of the two-tier test. The CDC has said that this should only be used for research purposes, and that the Western Blot on its own should be used for diagnostics.

 

[msf]

As mentioned in this earlier post, where you choose to set the positive/negative cutoff point should be based on two factors: what the consequences are for not treating a patient who actually has Lyme disease; and what the consequences are for treating a patient who does not have Lyme disease.
Also, you have to consider how successful treatments are. If the treatment has minimal benefits, then the case for treatment is not a strong as a treatment which is highly effective.
In an ideal world, cost would not be factor in this decision where to place the positive/negative cutoff line, but in practice, that is something that is probably also taken into account by health authorities: if a test cutoff is set so that it produces a large number of false positives, then treating those false positive patient can be considered an expensive waste of money.
Of course, none of this would be necessary if a nearly 100% accurate test for Lyme disease were available.

Perhaps this should be how the cut-off point is determined, but that´s not how it works. The cut-off is adjusted to give the best possible (albeit plausible) specificity. Of course, there is external pressure from health authorities to keep the specificity high, as this means they don´t have to pay for people to be treated for a complex disease.

 

[Hip]

Do you think a man of Armins professional standing does not know this?

Of course he knows it, but obviously my discussion about the mathematical issues related to testing was not directed at Dr Armin Schwarzbach, but to people in this thread.

This is again just speculation. MAY HAVE?

It is my speculation that Dr Schwarzbach may be adjusting the cutoff point so as to reduce false negatives but increase false positives; it is speculation because this information is not to be found on Dr Schwarzbach's website, where I would have liked to have seen these details.

Why don't you ask Armin himself?

OK, I will send him an email and ask.

I will ask him what gold standard he uses to validate his tests. I understand that Borrelia culture is the gold standard test, where a sample of blood is taken from the patient, and using this sample, an attempt to grow Borrelia bacteria in a petri disk with the appropriate growth medium as made. Ref: 1

I will ask Dr Schwarzbach if he uses this Borrelia culture gold standard to validate his tests.

I don't think he does, because for example, in his webpage for the LTT EliSpot test he says "the sensitivity of EliSpot is estimated at 84%, and the specificity is 94%."

If you were using the Borrelia culture gold standard, presumably you would not have to estimate the test sensitivity; you would know it.


I will also ask Dr Schwarzbach what his approach is to setting the positive/negative cutoff point for his tests, and whether compared to the other testing labs, he sets his cutoff point in such way as to reduce false negatives at the expense of increasing false positives.



Can anyone else suggest any other useful questions I should put to Dr Armin Schwarzbach when I email him?

 

[Hip]

Perhaps this should be how the cut-off point is determined, but that´s not how it works. The cut-off is adjusted to give the best possible (albeit plausible) specificity. Of course, there is external pressure from health authorities to keep the specificity high, as this means they don´t have to pay for people to be treated for a complex disease.

I agree, there may well be pressure to adjust the cutoff so as to keep the specificity high (which means fewer false positives), due to the cost of unnecessarily treating those who do not have Lyme.

But that is not the only factor. The patient themself, who has symptoms, does not benefit from being told that they are positive for Lyme when in fact they are not. That not only means the patient my then pursue an unnecessary Lyme treatment, but moreover, may mean the patient no longer searches for other causes for his symptoms, because he has incorrectly attributed those symptoms to Lyme, due to a false positive test result.

There may be people on this forum who are Lyme positive in their testing, but do not actually have Lyme, because they received a false positive result. In these cases, the patient may stop looking for other explanations, such as enterovirus infections which are linked to ME/CFS, herpes virus reactivations, or other diseases like anemia, hypothyroid, celiac disease, lupus, hepatitis C, which can produce symptoms resembling Lyme.

Thus too many false positives no only incurs increased health costs, but also may prevent the patient finding the true cause of their symptoms.

 

[duncan]

The flip side of that coin arguably is also dire: A false negative might result in Lyme going undiagnosed, and the patient worsening without good and timely treatment - potentially resulting in profound illness, disability or death.

If I had a say, I know which side I'd prefer to err on.

 

[halcyon]

The flip side of that coin arguably is also dire: A false negative might result in Lyme going undiagnosed, and the patient worsening without good and timely treatment - potentially resulting in profound illness, disability or death.

Long courses of unnecessary abx isn't a pretty picture either though.

 

[duncan]

Either way, it's usually the patient that loses.

 

[Hip]

I'd like to know what is happening with the Advanced Laboratory Services Borrelia culture test. Since this is a bacterial culture, it presumably means you are testing using the gold standard, or at least using a technique close to the gold standard (as I am nore sure if ALS are using some shortcuts in their culturing methodology).

Soon after this ALS culture test came out in 2011, it was criticized by the CDC as not being a properly validated. I then read (in this thread) that two universities were going validate it, but since then I have not heard any more news regarding this validation.

A commercially-available validated Borrelia culture test like ALS's should be the most accurate testing method, albeit an expensive method. Though I am not sure if this ALS culture test is suitable for detecting European species of Borrelia — does anyone know?


Moreover, if Borrelia bacteria can actually be grown in a culture from the blood of patients who have had Lyme disease symptoms for many years (even after antibiotic treatment), it will pretty much settle the argument over whether Borrelia can persist in the body on a long-term basis.

 

[barbc56]

What is the possible serological mechanism for a false positive

Many reasons. The test may be reacting to other bacteria and/or viruses. A previous infection may be involved as the antibodies stay with you for life. Sometimes an autoimmune condition can create a false positive. I don't know if sloppy lab techniques would be more prone to false positives or false negatives. I would think it's the latter but don't quote me.

I thought it interesting that this article mentions that EBV can cross react to a lyme test.

@Hip
I would like to see the results of the same person submitting samples to different labs. I think this has been done and will look.

Once again I want to make it clear that I believe Lyme exists and it can be a devastating condition. The tests have inaccuracies which means a clinical decision which involves weighing many complex factors is often needed. It's rarely an either or diagnosis. It's in the best interests of patients to get an accurate diagnosis and treated appropriately which is the bottom line!

We probably all agree with this. It's how you get there where there are differences.

But I shouldn't have to say that.

Barb

 

[nandixon]

Yes, I did four weeks of Doxycycline in between. The reason for that was that I had tested positive for Mycoplasma + Tularemia (via RED Labs). Are you suggesting that the treatment was significant?

I should add, the report on the initial test said the following:

Perhaps the combination of those two things are responsible for the change in result, but I guess if there turned out to be fundamental issues with the test as some suspect then why the result changed is not all that important, insofar as that if the chance of a false positive is unduly high then I would have likely always got a positive after a certain number of retests.

You may be right. I was asking because I think several people on the forum have mentioned that their LTT testing turned from negative to positive after taking an antibiotic. So there are at least the following possibilities for this (in addition to the notation included with your first test results about the low lymphocytes):

1. It may be coincidence if the test inherently has a high false positive rate, as you suggested.

2. The antibiotic (and possibly other drugs or supplements) may have sensitized the memory T-cell receptor, perhaps resulting in some form of cross-reactivity to the Borrelia antigen used for the LTT and thus causing a false positive.

3. The antibiotic may have somehow, paradoxically, increased the chances of a true positive.

Regarding #2 above, it's interesting that attempts have been made over many years to use the LTT for drug (hyper)sensitivity testing. See this article for more info: The lymphocyte transformation test in the diagnosis of drug hypersensitivity

What's also interesting is the possibility that the false positive rate for the LTT in ME/CFS patients might be higher than it is in healthy individuals. If that's true, I think it might suggest an underlying problem in some ME/CFS patients that could be mediated by memory T-cells.

 

[msf]

Of course he knows it, but obviously my discussion about the mathematical issues related to testing was not directed at Dr Armin Schwarzbach, but to people in this thread.





It is my speculation that Dr Schwarzbach may be adjusting the cutoff point so as to reduce false negatives but increase false positives; it is speculation because this information is not to be found on Dr Schwarzbach's website, where I would have liked to have seen these details.





OK, I will send him an email and ask.

I will ask him what gold standard he uses to validate his tests. I understand that Borrelia culture is the gold standard test, where a sample of blood is taken from the patient, and using this sample, an attempt to grow Borrelia bacteria in a petri disk with the appropriate growth medium as made. Ref: 1

I will ask Dr Schwarzbach if he uses this Borrelia culture gold standard to validate his tests.

I don't think he does, because for example, in his webpage for the LTT EliSpot test he says "the sensitivity of EliSpot is estimated at 84%, and the specificity is 94%."

If you were using the Borrelia culture gold standard, presumably you would not have to estimate the test sensitivity; you would know it.


I will also ask Dr Schwarzbach what his approach is to setting the positive/negative cutoff point for his tests, and whether compared to the other testing labs, he sets his cutoff point in such way as to reduce false negatives at the expense of increasing false positives.



Can anyone else suggest any other useful questions I should put to Dr Armin Schwarzbach when I email him?

I wouldn´t ask those questions if I were you, unless that is you want to sound ignorant about Lyme testing.

Re: your first point, whether Armin has adjusted the cut off or not isn´t as important as how he and other physicians interpret it. This is related to the point I made earlier about how to tell a active infection from a past or latent one. The new Infectolab test (note this was after Armin left) I mentioned is supposed to do this by measuring the ratio of Interferon Gamma to IL-2.

This method has been evaluated for Q fever, but not so far (as far as I know) for Lyme. The impression I got from the paper was that the ratio depends mainly on the level of Interferon Gamma, i.e. the level of this changes whilst the level of IL-2 stays pretty stable (at least in chronic cases). I think this is because the IL-2 is produced by the memory T-cells.

So I assume that Armin has not adopted this new test because you can use the level of Interferon Gamma on its own to distinguish an active infection from a past or latent one. So, if the level of Interferon Gamma is very low, then it might be a past or latent infection, and this will show as a low SI value on the LTT-ELISPOT.

The issue then becomes how you interpret the low end of the spectrum. Do you say (as they did to improve specificity in the LTT paper) that SI-3 is an active infection? What if the sample hardly had any lymphocytes in it? This, as KDM has pointed out, will affect the result. What about the possibility that some people may have a very weak T-cell reaction to Lyme (and therefore low levels of Interferon Gamma are produced)? This has been demonstrated in Yersinia, and might explain why I am IgA positive for Yersinia but my Yersinia LTT-ELISPOT result was ony SI-2.

These are all inferences I have made about the way the test is interpreted. If you are going to email Armin, I would suggest you ask whether these inferences are correct.

Re: your second point, as I have pointed out several times in this thread, there is no gold standard in Lyme. That means no one uses culture as the standard. This is because so far it has been very difficult to culture Lyme. In lieu of a gold standard, clinically confirmed cases of Lyme are used, along with healthy controls (who are assumed not to have an active Lyme infection). This is what you will find if you read the paper on the LTT test. In that study, they also tested people who had been treated for Lyme, to demonstrate that the LTT test can show whether an infection is no longer active (something that the two-tier test isn´t very good at).

 

[duncan]

@barbc56 , I have been tested for Lyme at different labs over the past few years and have tested positive at each one. Not every time, but most times.

I even had one time where I tested at 2 labs from the same draw on the same day. One lab had me four bands IgG positive; the other also had me four bands positive - but two of the bands registering were different. If you combined the two test results - remember, same draw same day - I would have been positive then, as well.

Labs I have tested 2T positive include Mayo, Labcorp, Quest, Imugen, Immunetics, Stony Brook, and BioReference Labs.

I have always tested conventional ELISA positive, tested WB positive about 15 out of 20 times, and tested C6 positive always.