XMRV Replication Vs Validation Studies

[Cort]

My thoughts on these two types of studies

Replication vs Validation Studies - the Imperial College and CD tests are validation 'studies' not replication studies and there's a big difference between them. Replication studies follow the first studies techniques to the letter.They're using the same processes to find the same genetic sequences that the original study found.

Validation studies, on the other hand, simply try to validate the first studies assertion - in this case the WPI's assertion that they found XMRV. They're not trying to replicate the original techniques; in fact, it's better if they don't. Virus hunters can use any number of techniques to find a virus; if a virus is there any of these techniques should be able to find it. Thus, they're often using different kinds of PCR tests and are often looking for different genetic sequences than in the original study. When Imperial College or CD attempts to detect the virus using a different means than the WPI does, its trying to validate their assertion that they've found XMRV - not find the same genetic sequences WPI did.

Notice that the WPI did not take Imperial College to task for saying whether or not they found XMRV - they took them to task for not replicating their results.

In order for them both to be right WPI must have found something a bit different from XMRV - a possibility - or one of the two groups made a mistake. Given that the WPI was working with 2 other important labs - if a mistake was made I would suggest it was probably Imperial that made it.

I would think the most probable conclusion, though, was that they both did their tests right and WPI found something different than what Imperial was looking for: ie standard XMRV. (How standard XMRV is is in question since it was undoubtedly gleaned from a few prostate cancer samples. )

The Science Paper - Thank god the Science Journal pushed them so hard because if studies start rolling in showing that no one can find 'XMRV' the whole thing could concievably crash to a halt right there with the research community believing it was nothing but a contaminant. But the WPI showed that if you put an infected cell next to an uninfected cell that uninfected cell gets infected - and contaminants, by default - broken up bits of endogenous retroviruses - cannot, to my knowledge, infect other cells. That makes it intriguing for retrovirologists and hard to dismiss their findings even if labs cannot find 'XMRV'. Something in their slides was infecting cells - and that appears to be quite unusual.

 

[CBS]

I would think the most probable conclusion was that they both did their tests right and WPI found something different than what Imperial was looking for: ie standard XMRV. (How standard XMRV is is in question since it was undoubtedly gleaned from a few prostate cancer samples. )

In January 6th, 2010 edition of Science Now, Dr. Coffin writes in response to the UK article and the WPI response:

One distinct possibility, is that both papers are right. He called the PLoS ONE paper too "preliminary" to settle the debate and said XMRV could show more genetic variety, and thus be harder to detect, than anyone assumed. It's also possible that distinct strains of XMRV appear in different parts of the world, like the retroviruses HIV and HTLV (a leukemia virus).

 

[starryeyes]

There's still the matter of the patient cohort the IC used. I doubt that the WPI, the NCI or the CC could find any XMRV in that cohort either.

I just learned that I'd be exempted from being tested for XMRV by the IC and so would Dysautonomia, the member here. Why? Because we both have sluggish pupil responses which is common in ME/CFS.

That alone would exempt us. I'm sure there are many other abnormalities we have that would have exempted us also. I'm not saying we both have XMRV but if this is the kind of symptom they eliminated no wonder they found 0% of XMRV in the UK.

 

[thefreeprisoner]

Actually, I think it is unexpected that they found 0% in the UK - but good for us.
Why?
Most studies that involve XMRV have found at least 1 sample of XMRV in healthy controls, even in Europe and Japan. (Percentages range from about 1% - 3%, admittedly less than the US.) So if they were doing their test properly, they should have found at least one infection.

It puzzles me why they were so smug, confident, and quick to report, because if they wanted to skew their results without getting closer scientific scrutiny, they should have found about 1-4% positive in their sample to show they'd done it properly. This, to me, points to a genuine error in testing.

Which should really prompt us to ignore all the political wrangling about the cohorts and focus on why they failed to find anything. If we can uncover that they failed to find anything because the test was flawed, then the whole study falls apart, good cohort or bad cohort. If we focus on the cohort, then we're bound for a dead end, sadly.

 

[Countrygirl]

[

I just learned that I'd be exempted from being tested for XMRV by the IC and so would Dysautonomia, the member here. Why? Because we both have sluggish pupil responses which is common in ME/CFS.

That alone would exempt us. I'm sure there are many other abnormalities we have that would have exempted us also. I'm not saying we both have XMRV but if this is the kind of symptom they eliminated no wonder they found 0% of XMRV in the UK.

Hello teejay,

As I see it, abnormal physical signs typically found in M.E. would not have excluded us from the I.C. research for the simple fact that Wessely does not screen for them. Somewhere.......yesterday, I posted the response from a King's patient of Wessely in which he strongly disputed S.W.'s claim that they have been checked for physical abnormalities as S.W. stated. He just uses the standard battery of blood tests that show ziltch. This means that some of the I.C. group of patients probably did have M.E. This probably weakens our argument about patient selection.

 

[flex]

[



Hello teejay,

As I see it, abnormal physical signs typically found in M.E. would not have excluded us from the I.C. research for the simple fact that Wessely does not screen for them. Somewhere.......yesterday, I posted the response from a King's patient of Wessely in which he strongly disputed S.W.'s claim that they have been checked for physical abnormalities as S.W. stated. He just uses the standard battery of blood tests that show ziltch. This means that some of the I.C. group of patients probably did have M.E. This probably weakens our argument about patient selection.

You can easily screen for neuro signs and then leave people out of the cohort. The samples were already on a seperate database in my opinion. Wessely would not leave himself open to that. One or two genuine ME people may attend his clinic but they wont appear in any studies. That as he already stated is done under CDC criteria which can be easliy matched to Oxford criteria. All you need to qualify for his studies is "fatigue "and some other vague tick sheet answers.

He stated "we do not use the Canadian criteria as it is not internationally validated". Many other studies are going to do the same thing simply because they dont have the stored blood available for patients that meet Canadian Criteria (real ME).

The main aim according to alot of agencies right now is to do quick studies under the CDC definition. This creates the same issues.

 

[Cort]

Generally researchers when they do tests to exclude people from research studies simply do the standard, simple, cheap blood tests to pick up specific major abnormalities/ major illnesses. Dysautonomia should not prevent someone from participating from a study because its not in the list of excluded illnesses in the Fukuda definition. There's a myth that you can't have any other abnormalities and still have CFS or participate in a research study. The only abnormalites that are excluded from research studies are other conditions that also cause severe fatigue. The exclusionary factors from Fukuda are below:

1. Any active medical condition that may explain the presence of chronic fatigue [31], such as untreated hypothyroidism, sleep apnea, and narcolepsy, and iatrogenic conditions such as side effects of medication.
2. Any previously diagnosed medical condition whose resolution has not been documented beyond reasonable clinical doubt and whose continued activity may explain the chronic fatiguing illness. Such conditions may include previously treated malignancies and unresolved cases of hepatitis B or C virus infection.
3. Any past or current diagnosis of a major depressive disorder with psychotic or melancholic features; bipolar affective disorders; schizophrenia of any subtype; delusional disorders of any subtype; dementias of any subtype; anorexia nervosa; or bulimia nervosa.
4. Alcohol or other substance abuse within 2 years before the onset of the chronic fatigue and at any time afterward.
5. Severe obesity [32, 33] as defined by a body mass index (body mass index = weight in kilograms/[height in meters]2) 45.

It is true that Fukuda simply gave guidelines. Here are the complete exclusionary factors from 2003:

Examples of permanent medical exclusions include the following: 1) organ failure (e.g., emphysema, cirrhosis, cardiac failure, chronic renal failure); 2) chronic infections (e.g., AIDS, hepatitis B or C); 3) rheumatic and chronic inflammatory diseases (e.g., systemic lupus erythematosis, Sjogren's syndrome, rheumatoid arthritis, inflammatory bowel disease, chronic pancreatitis); 4) major neurologic diseases (e.g., multiple sclerosis, neuromuscular diseases, epilepsy or other diseases requiring ongoing medication that could cause fatigue, stroke, head injury with residual neurologic deficits); 5) diseases requiring systemic treatment (e.g., organ or bone marrow transplantation, systemic chemotherapy, radiation of brain, thorax, abdomen, or pelvis); 6) major endocrine diseases (e.g., hypopituitarism, adrenal insufficiency); 7) primary sleep disorders (e.g., sleep apnea, narcolepsy).

Temporary medical exclusions include treatable conditions that require evaluation over time to determine the extent to which they contribute to the fatiguing illness. These encompass four general categories: 1) conditions discovered at onset or initial evaluation (e.g., effects of medications, sleep deprivation, untreated hypothyroidism, untreated or unstable diabetes mellitus, active infection); 2) conditions that resolve (e.g., pregnancy until 3 months post-partum, breast feeding, major surgeries until 6 months post-operation, minor surgery until 3 months post-operation, and major infections such as sepsis or pneumonia until 3 months post-resolution; sleep disorders such as restless leg syndrome and periodic limb movement should be considered temporary exclusions for research criteria, if they are severe, but not if the degree of the sleep problem is insufficient to explain the severity of the fatigue); 3) major conditions whose resolution may be unclear for at least 5 years (e.g., myocardial infarction, heart failure); and 4) morbid obesity (body mass index [BMI] > 40). The 1994 CFS case definition specified a BMI > 45. While both cut-off values are arbitrary, a BMI > 40 defines morbid obesity and is a more inclusive contributing factor to explain chronic fatigue.

Permanent psychiatric exclusions include lifetime diagnoses of bipolar affective disorders, schizophrenia of any subtype, delusional disorders of any subtype, dementias of any subtype, organic brain disorders, and alcohol or substance abuse within 2 years before onset of the fatiguing illness. The 1994 case definition stated that any past or current diagnosis of major depressive disorder with psychotic or melancholic features, anorexia nervosa, or bulimia permanently excluded a subject from the classification of CFS. Because these illnesses may resolve with little or no likelihood of recurrence and only active disease

 

[Cort]

Generally researchers when they do tests to exclude people from research studies simply do the standard, simple, cheap blood tests to pick up specific major abnormalities/ major illnesses. Dysautonomia should not prevent someone from participating from a study because its not in the list of excluded illnesses in the Fukuda definition. There's a myth that you can't have any other abnormalities and still have CFS or participate in a research study. The only abnormalites that are excluded from research studies are other conditions that also cause severe fatigue. The exclusionary factors from Fukuda are below:

1. Any active medical condition that may explain the presence of chronic fatigue [31], such as untreated hypothyroidism, sleep apnea, and narcolepsy, and iatrogenic conditions such as side effects of medication.
2. Any previously diagnosed medical condition whose resolution has not been documented beyond reasonable clinical doubt and whose continued activity may explain the chronic fatiguing illness. Such conditions may include previously treated malignancies and unresolved cases of hepatitis B or C virus infection.
3. Any past or current diagnosis of a major depressive disorder with psychotic or melancholic features; bipolar affective disorders; schizophrenia of any subtype; delusional disorders of any subtype; dementias of any subtype; anorexia nervosa; or bulimia nervosa.
4. Alcohol or other substance abuse within 2 years before the onset of the chronic fatigue and at any time afterward.
5. Severe obesity [32, 33] as defined by a body mass index (body mass index = weight in kilograms/[height in meters]2) 45.

It is true that Fukuda simply gave guidelines. Here are the complete exclusionary factors from 2003:

Examples of permanent medical exclusions include the following: 1) organ failure (e.g., emphysema, cirrhosis, cardiac failure, chronic renal failure); 2) chronic infections (e.g., AIDS, hepatitis B or C); 3) rheumatic and chronic inflammatory diseases (e.g., systemic lupus erythematosis, Sjogren's syndrome, rheumatoid arthritis, inflammatory bowel disease, chronic pancreatitis); 4) major neurologic diseases (e.g., multiple sclerosis, neuromuscular diseases, epilepsy or other diseases requiring ongoing medication that could cause fatigue, stroke, head injury with residual neurologic deficits); 5) diseases requiring systemic treatment (e.g., organ or bone marrow transplantation, systemic chemotherapy, radiation of brain, thorax, abdomen, or pelvis); 6) major endocrine diseases (e.g., hypopituitarism, adrenal insufficiency); 7) primary sleep disorders (e.g., sleep apnea, narcolepsy).

Temporary medical exclusions include treatable conditions that require evaluation over time to determine the extent to which they contribute to the fatiguing illness. These encompass four general categories: 1) conditions discovered at onset or initial evaluation (e.g., effects of medications, sleep deprivation, untreated hypothyroidism, untreated or unstable diabetes mellitus, active infection); 2) conditions that resolve (e.g., pregnancy until 3 months post-partum, breast feeding, major surgeries until 6 months post-operation, minor surgery until 3 months post-operation, and major infections such as sepsis or pneumonia until 3 months post-resolution; sleep disorders such as restless leg syndrome and periodic limb movement should be considered temporary exclusions for research criteria, if they are severe, but not if the degree of the sleep problem is insufficient to explain the severity of the fatigue); 3) major conditions whose resolution may be unclear for at least 5 years (e.g., myocardial infarction, heart failure); and 4) morbid obesity (body mass index [BMI] > 40). The 1994 CFS case definition specified a BMI > 45. While both cut-off values are arbitrary, a BMI > 40 defines morbid obesity and is a more inclusive contributing factor to explain chronic fatigue.

Permanent psychiatric exclusions include lifetime diagnoses of bipolar affective disorders, schizophrenia of any subtype, delusional disorders of any subtype, dementias of any subtype, organic brain disorders, and alcohol or substance abuse within 2 years before onset of the fatiguing illness. The 1994 case definition stated that any past or current diagnosis of major depressive disorder with psychotic or melancholic features, anorexia nervosa, or bulimia permanently excluded a subject from the classification of CFS. Because these illnesses may resolve with little or no likelihood of recurrence and only active disease

 

[Hope123]

I'm glad you decided to separate this thread out since these are interesting points Cort.

As much as I respect the WPI, I don't think the press releases by the WPI or the way that they presented their data was the clearest it could be to avoid controversy. They should have avoided putting out figures (like the 95% positive figure for antibodies) when this isn't published officially yet - lots of people have confused this number with the 67% PCR/DNA figure and the 99% figure for ANY positive test given at the CFSAC meeting. I am not a microbiologist but have a background in biology.

My understanding is that the WPI researchers in the Science study used a multitude of techniques to look for XMRV but they were not applied uniformly to all 100-odd subjects with CFS. It would have been interesting if they had applied all the techniques to every subject but time/ money/ urgency considerations probably played into this and this is a preliminary study.

1. PCR was used on all 101 patients (68/101; 67% positive), 218 controls
2. The remaining 33 PCR negative patients had cultures, antibodies, and transmission tests done and almost all had ONE test which was positive (not reported in Science but
at CFSAC) (19/33 antibody, 30/33 transmissible, 10/33 protein expression positive) ; 99/101 total with at least one test positive = 99%
3. 19/30 patients had XMRV protein expression (don't kow if this refers to the 33 in #2)
4. transmission of XMRV from CFS-infected cells to other cells (no N given)
5. transmission of XMRV from CFS-plasma (no cells) to other cells (no N given)
6. electron microscopy of budding virus from transmission study (no N given)
7. 9/18 patients with antibodies to XMRV Env

I agree with some of your points. I suspect there must be some people in the IC cohort who would also fit the Canadian Criteria and have positive findings so the cohort issue is still an issue but I'm more concerned about technique. (Personally, I would not have been recruited into IC cohort either as my ESR has always been a bit elevated and the ESRs in the study were normal.)

With validation, some sort of gold standard has to be established against which future tests will be judged. For many bacterial studies, it's common to use isolation/ culture/ transmission as gold standards to measure other tests by, especially if you catch a bacteria where it normally is not found. I would have thought viral culture to be the gold standard and, like you, thought that the transmission part of the studies was important but I am not a virologist and would like to hear more from Kurt about how endogenous retroviruses may mess up identification of viruses that have been cultured. I'm not clear from the current explanation. Do human endogenous retroviral genes or proteins cross-react with XMRV-associated gene and protein probes (things used to identify a protein or gene)? How specific are the probes being used?

I think it was not unreasonable for the IC reseachers to use a probe which was thought to be highly conserved in XMRV but this is with the underlying assumption already that the XMRV seen in CFS is like the XMRVs we are familiar with so John Coffin's comments are well-taken. It would have been interesting for them to look at both the conserved as well as WPI sequences in their study. Indeed, looking back at the opposing papers to Defreitas in the 1990s, authors were still open in the paper discussion to an "unknown" virus that they were not picking up with their probes despite the eventual consequence that the whole Defreitas matter was dropped prematurely.

 

[starryeyes]

It would only have taken a few seconds for a doctor or nurse to flash that pen-light thing in the patient's eyes like they regularly do.

 

[kim500]

My understanding is that the WPI researchers in the Science study used a multitude of techniques to look for XMRV...

Thank you, Hope.

Excellent comments and analysis in your long post.

 

[gerwyn morris]

spot on-my only comment would be that given the genetic variability and the slow replication rate the scientific method would call for validation by replication as you know thats why scientific papers are written the way they are Alternatively they could have validated their assay proceedures using a positive whole blood contol-They willfully chose to do neither-Why???

 

[Cort]

It may be that they didn't have a positive blood control. The WPI stated, as I remember, that the virus was genetically almost uniform in their patient samples. I don't know if they're basing that on the 2 1/2c samples they sequences or small changes in the sequences they were looking at but I would think given that - that you wouldn't expect much variability. They also did conclude, based on the sequencing of 2 1/2 strains that it was very, very close to XMRV - no reason, I wouldn't think, to assume that another lab looking at a very conservative region of the genome wouldn't pick it up if it was there. It is rather mysterious!

 

[gerwyn morris]

depression is not an exclusion criterea in the Oxford or CDC criterea.It,s rich that Wesselly,s reason for not using the canadian criterea is that they are not internationally recognised-the oxford criterea used in the imc study are not recognised by the WHO as being able to diagnose CFS the neurological disorder-patients diagnosed by these criterea are included in the mental health section of the WHO classification-Is this commonly known?

 

[gerwyn morris]

Indeed the key point being "if it was there"- poorly conducted tests-i doubt it-Inadequate amplification /detection assays-a distinct possibility looking at the faint bands-depressed patients even more likely-diagnosis according to the Oxford critera-certain

 

[natasa778]

Gerwyn, are you thinking/saying that the most likely reason UK study failed to find xmrv is because of patient selection criteria? If so, would you still expect them to find xmrv present in at least a small percentage of patients (regardless of their CFS/ME status), given that WPI found it in 4% of healthy individuals?

Great points raised from many sides, I hope some of these can be included on Q&A session with Judy M on 22 January! - please do write in