I think we are talking about two different studies and I may not have been clear about that. I was in editing hell at the time.
My sympathies for the difficulties you're having with editing. But don't worry, we are talking about the same study, the one you linked to. The bullet points were from Racaniello's blog about that study, as you've clarified.
I cited the following study as someone asked how the patients could have different outcomes in the positive studies.
Ila Singh finds no XMRV in patients with chronic fatigue
You did, and indeed it is highly relevant to the theme, here's the link to the PDF:
http://jvi.asm.org/content/early/2011/05/04/JVI.00693-11.full.pdf html
This is from Raceinallo's blog about the Singh paper:
In the introduction to their paper, published in the
Journal of Virology, the authors note other problems with many of the studies of XMRV in CFS patients:
- Too small control populations
- Patient and control samples collected at different times
- Investigators generally not blinded to sample identity
- PCR assays that rely on conservation of viral sequence mainly used
- Limits of detection, reproducibility, and precision of assays unknown
- Controls for each step that would identify analysis not done
- Insufficient numbers of negative controls included
- No study included positive samples from the original 2009 patient cohort of Lombardi et al
http://www.virology.ws/2011/05/04/ila-singh-finds-no-xmrv-in-patients-with-chronic-fatigue-syndrome/
The last point is specifically talking about the Mikovitz Limbardi original paper. The other points were indeed talking about all the positive papers.
No, the points definitely weren't talking about all the positive studies, this is definitely a list of problems with
all the studies. Some of them apply to Lombardi et al, most of them don't, most of them apply to the negative papers.
As your own quote states: "the authors note other problems with
many of the studies of XMRV in CFS patients".
This does not refer to the positive papers only. If it did mean that, likely it would have said so. And since it mentions problems with "many" studies, I ask you, how many positive studies were there, to classify as "many"?
You can read the actual source for these bullet points of Racaniello's in the pdf:
http://jvi.asm.org/content/early/2011/05/04/JVI.00693-11.full.pdf html
Lines 65-78 describe the problems with the negative studies, and notably Singh echoes the many valid criticisms made of those studies by forum members here at the time. It is most heartening to see her do so, because regardless of the end result of the science she firmly vindicates the response of patients here to the poor methodology of those negative studies. All of these points were highlighted by patients here; I'll take you through them line by line...(I am dealing here only with the critique of the negative studies; the critique of the positive studies is in the previous lines).
65 - 67: "In
all but two studies that failed to detect virus in association with CFS, only PCR-based assays were used, thus relying heavily on conservation of retroviral sequences"
67 - 69: "The limits of detection, reproducibility and precision of the assays used in different studies were not known,
making it difficult to distinguish the lack of ability to detect XMRV from a genuine absence of XMRV from samples."
70 - 71: "Furthermore,
tests that had resulted in more frequent detection of XMRV, such as growth of virus in cultured cells (14),
were not used in subsequent studies."
72 - 73: "Adequate controls for each step of the analysis, such as controls that would flag contamination occurring during the nucleic acid extraction process, were mostly lacking.
73 - 76: "Furthermore, the number of negative controls should equal or exceed the expected prevalence of the virus in the control population. It is not clear if
any of the studies employed more than one negative control per experiment, which would be important for the detection of a low incidence of sample contamination.
76 - 78: "Finally, none of the studies tested samples from the same patients that were found to be positive in the original study by Lombardi et al (12).
I'm grateful to you for bringing these details of Singh's paper to my attention Barb, because they are a damning indictment of most of the negative studies, and as I've said, it echoes the criticism from patients here and elsewhere, and they illustrate very well why so many patients had such serious concerns about the negative studies, particularly the early ones.
As Singh explains, and as we commented at the time:
- the negative papers only used PCR assays, focusing on a specific sequence and omitting the other methods used in Lombardi et al (so they weren't a proper replication of Lombardi et al);
- the precision and reproducability of the assays used in the negative papers were unknown (so it was difficult or impossible to say whether there was no XMRV there or whether they just couldn't find it).
- the tests that had worked best in Lombardi et al - especially the crucial culture test - weren't used in any of the subsequent negative studies
- controls, such as controls to flag contamination, were lacking
in all the studies (so the criticism and innuendo heaped on Dr Mikovits over that issue was quite unfair because nobody else controlled for contamination any better)
- it wasn't clear whether
any of the studies used any more than one negative control (so again, that criticism applies to the other studies just as much as to Lombardi et al).
- none of the studies re-tested samples from the original Lombardi study (and very clearly indeed, that criticism can hardly be levelled at the Lombardi study itself!)
So when you say:
The last point is specifically talking about the Mikovitz Limbardi original paper. The other points were indeed talking about all the positive papers.
Well yes, the last point specifically talks about the Lombardi paper, in that it blames all the other papers for not retesting samples from that study. But the other points listed above were definitely, emphatically talking about the papers that
failed to detect XMRV (prior to Singh's own study).
The only two of your bullet points that refer to weaknesses in Lombardi et al and Lo/Alter are these, which are discussed in the section before the one I've highlighted above, in lines 57-65 of the paper (and Racaniello missed the first of these I think):
- Patient and control samples collected from different geographical regions (11, 12)
- Patient and control samples collected at different times (11)
- Investigators generally not blinded to sample identity (except in a subset of 12)
Note "generally", because there was
some blinding (in Lombardi et al - 12). And the second of these two bullets applies to Lo/Alter (11), not to Lombardi et al (12).
So in fact, excepting the flaws that were present in every study, including the negative ones, the only one of these bullet points that applies to Lombardi et al is the point about collecting samples from different geographical regions. And even in that case, I am not quite sure that Singh is quite correct on this point, because the Lombardi samples were collected from about 10 different sites across the US (in surgeries in ME outbreak areas), but I thought Mikovits had clarified that, although not mentioned in the paper, the blood from controls was collected from the same locations.
So I'm glad I've done the full analysis there, in the end, because the point I've been highlighting has quite unexpectedly got considerably stronger along the way: It turns out that, in fact, every item in Singh's list of criticisms of all XMRV studies up to that date either applies to the negative studies
only, or applies equally to
all the XMRV studies, including the negative ones.The only exceptions are the three listed above, of which two points apply to Lo/Alter only, and
only one (geographical regions) applies to Lombardi et al - and I think even that point may well be incorrect!
So in Singh's very thorough analysis of the XMRV studies, there's not a single direct hit on Lombardi et al's methodology, and the only criticisms which
may apply to Lombardi et al are explicitly applied to
all the studies, including all the negative ones. Rather a different picture from the image of weak methodology in Lombardi et al which one sees presented all too often...
