False negative results in MLV-related publications

[natasa778]

I haven't seen this posted here, but could have missed it, pls delete if duplicate thread...

False negative results from using common PCR reagents

http://www.biomedcentral.com/1756-0500/4/457

The sensitivity of the PCR reaction makes it ideal for use when identifying potentially novel viral infections in human disease. Unfortunately, this same sensitivity also leaves this popular technique open to potential contamination with previously amplified PCR products, or "carry-over" contamination. PCR product carry-over contamination can be prevented with uracil-DNA-glycosylase (UNG), and it is for this reason that it is commonly included in many commercial PCR master-mixes. While testing the sensitivity of PCR assays to detect murine DNA contamination in human tissue samples, we inadvertently discovered that the use of this common PCR reagent may lead to the production of false-negative PCR results.

....
None of the 38 MLV-related publications we examined used internal PCR controls. Given the data presented in this manuscript, we would propose that in cases such as the debate regarding the existence of novel infectious viruses, failure to find a virus is not the equivalent of evidence against its existence. The potential for false negative results needs to be considered and carefully controlled in PCR experiments, just as the potential for false positive results already is.

 

[natasa778]

I guess this is the research she refers to in her blog

I am not entirely convinced that polytropic or other MLVs aren't infecting humans. I think that the same level of healthy (usually) skeptism regarding positive findings of these viruses should have been extended to the negative finding papers, as well. I do not believe that it has; ill-defined levels of detection and poorly described methods attest to this. As I mentioned in my post, we found that it was unbelievably easy to generate false-negative results using PCR detection - we accidentally found this while testing our levels of detection for mouse DNA contamination in patient samples.

http://okeefe-lab.blogspot.com/2011/11/so-lets-get-this-straight-cfs-patients.html

 

[currer]

Hi Nastasa, this was put up a while ago, but keep it here, because this is a useful fact to remember, and one that is disregarded.
After all it is difficult to say, when a study has negative results..."ah, but false negatives are easily generated" The whole subject of false negatives has never been addressed by the research community, apart from this one reference from O'Keefe. A paper investigating the problem would be most interesting, but.....would it get published?

 

[RustyJ]

I never saw it the first time, so thanks Natasa. Interesting isn't it. Imagine that, there's actually a reagent that makes MLVs go away. I agree with O'Keefe - the skepticism has been all one way.

 

[natasa778]

Imagine that, there's actually a reagent that makes MLVs go away.

I wonder if the same thing applies to some of the other 'rumour' viruses that many love to dismiss.

 

[barbc56]

The potential for false negative results needs to be considered and carefully controlled in PCR experiments, just as the potential for false positive results already is.

From what I remember this study simply means that there are errors either way. It would be highly unlikely that all negative samples would be positive. Do they say the statistical probability that this would happen? Just because it can happen does not make it so. With all the other prevailing evidence, I don't think it negates any of the negative studies. However, I need to do some reviewing from my files.

Still this is interesting and probably should be followed up at some time.

Can someone point me to the original thread?

RRM, Firestorm, et. al.? What is your take.

Thanks.

Barb C. :>)

 

[natasa778]

It is obviously entirely possible that all the "prevailing evidence" is based on a simple PCR error. None of the negative studies took this possible error into account.

 

[barbc56]

It is obviously entirely possible that all the "prevailing evidence" is based on a simple PCR error. None of the negative studies took this possible error into account.

Possible but is it probable? I have a feeling it isn't. But TBH,I don't know and that is why I am asking the above people to give their feedback or someone point me to comments in the original post.

Thanks.

Barb C.

 

[Bob]

Hi Nastasa, this was put up a while ago, but keep it here, because this is a useful fact to remember, and one that is disregarded.
After all it is difficult to say, when a study has negative results..."ah, but false negatives are easily generated" The whole subject of false negatives has never been addressed by the research community, apart from this one reference from O'Keefe. A paper investigating the problem would be most interesting, but.....would it get published?

Good points currer...

I'd forgotten this paper as well.

Thanks for reposting it Nastasa.

 

[RRM]

First, regarding the negative studies, the possibilty of false PCR negatives has been addressed and considered by the research community. After all, did the research stop after the first negative study? Or the second? I've lost count, but I'd say that around 15 negative blood studies have been published thus far. And still, the original researchers had/have been given two chances (BWG and Lipkin study) to show that others have been producing false negatives after all.

Second, the conclusions in this paper are directly contradicted by experimental evidence. For instance, the original labs have compared their assays with other labs on two occasions for the BWG study (for Phase 1 and Phase III) and both times other labs showed equal or better sensitivity. Proponents attributed this to the possibility of sequence diversity and the fact that 'wild virus' may not be as 'detectable' like spiked material. Fair enough, but in the context of this O'Keefe paper this argument does of course not apply: 'weakened sensitivity' due to PCR inhibition was not an issue.

Last (although this doesn't really relate to the finding itself), this paper's suggestion of alternative controls does not seem sound advice to me. After all, it involves 'meddling' with both the patient samples as well as with the PCR assay. In a field where there is so much discussion about the tiniest bit of detail possibly influencing results (the kind of tubes, thawing and refreezing, etc), I don't think experiments like the one proposed will persuade any person that isn't already convinced at this time.

That's not to say that this isn't a good finding. It's good to know for researchers that these factors can affect assay sensitivity. However, it cannot explain the discrepant XMRV PCR findings thus far, and that's why you will not see a surge of new validation papers anytime soon because of it.

 

[natasa778]

Fthe possibilty of false PCR negatives has been addressed and considered by the research community. After all, did the research stop after the first negative study? Or the second? I've lost count, but I'd say that around 15 negative blood studies have been published thus far

yes but HAVE those negative studies actually addressed the specific PCR error point that O'Keefe paper brought up? if so, which studies were those?

 

[RRM]

yes but HAVE those negative studies actually addressed the specific PCR error point that O'Keefe paper brought up? if so, which studies were those?

Yes. All studies that showed assay sensitivity, to be precise. Which would probably be all negative studies.

After all, all of these studies determined their detection limit, i.e. the level of spiked XMRV that could still be reliably detected. If this specific PCR error was an issue in those studies, it would already have been reflected in these experiments demonstrating assay sensitivity.

Only when you hypothesize that somehow, this sort of reagent contamination was not at all an issue during assay testing but was suddenly totally an issue during actual testing, you could 'escape' this circumstance. However, in just one study this would already be extremely unlikely to have happened, let alone in 15.

One of the authors of the negative studies has replied in the comment section of this paper, by the way. This author explains why PCR inhibition was not issue for them (and like I said, this argument really applies to probably all of them):

[...] we have been able to detect traces of contaminating murine DNA by nested PCR in several publications (see references below), indicating that PCR inhibition is not an issue in our assay.

 

[natasa778]

Yes. All studies that showed assay sensitivity, to be precise.

versus

None of the 38 MLV-related publications we examined used internal PCR controls. Given the data presented in this manuscript, we would propose that in cases such as the debate regarding the existence of novel infectious viruses, failure to find a virus is not the equivalent of evidence against its existence.

(unless you are really really desperate for a novel infectious retrovirus not to exist :innocent1

 

[RRM]

versus



(unless you are really really desperate for a novel infectious retrovirus not to exist :innocent1

That quote relates to the controls the authors suggest. I have explained earlier why this I don't think this way of 'meddling' with both samples and assays will yield anything useful.

And clearly, no one has 'picked up' on this suggestion because of this (i.e. this is not an issue with the studies in question), and no one probably will.

Like I argued before, the finding is in itself useful, but the authors are clearly overstating their findings/conclusions.

 

[Bob]

Like I argued before, the finding is in itself useful, but the authors are clearly overstating their findings/conclusions.

That's exactly my opinion about the Paprotka study. (Useful, but not conclusive.) Every paper adds to the pool of knowledge.

 

[barbc56]

Thanks, RRM. My memory is slowly coming back about this particular paper, LOL!!

Barb C. :>)

 

[barbc56]

yes but HAVE those negative studies actually addressed the specific PCR error point that O'Keefe paper brought up? if so, which studies were those?

Yes they have. It's an important point scientiest need to keep in mind when doing studies.

(Unless you are really desperate for a retrovirus to exist.)

I think this quote from RRM is the bottom line, so to speak.

That's not to say that this isn't a good finding. It's good to know for researchers that these factors can affect assay sensitivity. However, it cannot explain the discrepant XMRV PCR findings thus far, and that's why you will not see a surge of new validation papers anytime soon because of it.

 

[Bob]

yes but HAVE those negative studies actually addressed the specific PCR error point that O'Keefe paper brought up? if so, which studies were those?

Yes. All studies that showed assay sensitivity, to be precise. Which would probably be all negative studies.

After all, all of these studies determined their detection limit, i.e. the level of spiked XMRV that could still be reliably detected. If this specific PCR error was an issue in those studies, it would already have been reflected in these experiments demonstrating assay sensitivity.

Only when you hypothesize that somehow, this sort of reagent contamination was not at all an issue during assay testing but was suddenly totally an issue during actual testing, you could 'escape' this circumstance. However, in just one study this would already be extremely unlikely to have happened, let alone in 15.

One of the authors of the negative studies has replied in the comment section of this paper, by the way. This author explains why PCR inhibition was not issue for them (and like I said, this argument really applies to probably all of them):

Hmm... Aren't you being rather simplistic and rushing to a conclusion?

It seems to be me that there are many variables involved.

I don't understand how you can be so sure.


For example, do you know that the assays were treated exactly the same for both the spiked sensitivity tests and for the main part of the studies?

If the assays were new when testing for sensitivity then maybe they sometimes were treated uracil-DNA-glycosylase (UNG) only after the sensitivity test?

(I don't know enough about this to speak about it confidently, but it's a variable that needs to be ruled out.)


Also, the level of sensitivity is a crucial issue here, so if the actual samples had lower titres than the spiked samples then this 'false-negative PCR' research seems very relevant, and I don't see how it can easily be concluded that uracil-DNA-glycosylase (UNG) could not have affected the study.

 

[natasa778]

(Unless you are really desperate for a retrovirus to exist.)

Why on Earth would anyone want a retrovirus to exist??

I think this quote from RRM is the bottom line, so to speak.

That's not to say that this isn't a good finding. It's good to know for researchers that these factors can affect assay sensitivity. However, it cannot explain the discrepant XMRV PCR findings thus far, and that's why you will not see a surge of new validation papers anytime soon because of it.

Only his subjective opinion versus O'Keefe's group saying that it COULD explain the negative findings.

 

[RRM]

For example, do you know that the assays were treated exactly the same for both the spiked sensitivity tests and for the main part of the studies?

If the assays were new when testing for sensitivity then maybe they sometimes were treated uracil-DNA-glycosylase (UNG) only after the sensitivity test?

Like I said, we are not talking about one or two studies with a small number of samples here. We are talking about (approx.) 15 independent studies (that not all used UNG BTW) involving hundreds of CFS patients' samples.

Even if one lab was stupid enough to change reagent protocol for the experimental testing, it's realistically impossible that most/all labs did this AND that most/all labs did then contaminate all of their experimental samples with UNG or another reagent AND that basically all samples that were contaminated then produced a false negative while all controls consistently produced a positive signal.

Also, the level of sensitivity is a crucial issue here, so if the actual samples had lower titres than the spiked samples then this 'false-negative PCR' research seems very relevant,[..]

Although O'Keefe doesn't really mention this, in some studies these controls were actually serially diluted to reflect the reported positivity level and thus should also (at least sometimes) come back as negative (that is, if you have contaminated your samples to such an extent that all of them became false negatives because of it). For instance, in the Switzer et al. study, they had positive controls that consisted of just 3 XMRV copies in 250ng of DNA that were "reliably" positive.

The BWG Phase III (Simmons et al.) findings also speak against this idea. After all, the other labs reported better sensitivity than WPI for the (serially diluted) spiked controls that were mixed in with the patient and negative control samples. This basically shows that their PCR assays were sensitive enough.

Only his subjective opinion versus O'Keefe's group saying that it COULD explain the negative findings.

It's not "just" me. It's really me and the overwhelming majority of the research community that think the combination of all negative (and other) studies together paint a convincing picture against XMRV being present in those samples "versus" O'Keefe's findings.

If Mikovits/Ruscetti/Lo again cannot reliably discriminate between patients and controls, the search for XMRV in ME/CFS is effectively over (except perhaps for a study that was already in progress). This finding by O'Keefe does nothing to change about that fact, and rightly so.