Feb. Retrovirology Paper-CDC: It's the patients not the test!

[Stephen12]

I haven't read all the threads & posts on this new bombshell from CDC, but few folks are mentioning what seems to be the real problem w/ CDC replication of WPI's work. I believe the CDC is likely selecting patients using their own CFS Empirical Definition in their published studies and implying they are following the Fukuda criteria. This has the added benefit of letting the reader assume Fukuda was used while not disclosing they have added psychiatric criteria, which would blow up the entire comparability effort to WPI/Lombardi.

The CDC has found XMRV in the BWG testing, so they know how to find it. But they mysteriously have not found it in any published scientific study. That's b/c the Empirical Definition doesn't select the same patients that used in WPI/Lombardi (Fukuda and Canadian Critieria).

The question everyone should be asking in the CDC's XMRV studies to date is which patients are they really studying. The following is taken directly from the article abstract:

"The majority of CFS patients (31/45, 69%) had a minimum of 6 months of post-exertional malaise and a high degree of disability, the same key symptoms described in the Lombardi et al. study."

The words of interest here are "key symptoms described in the Lombardi et al study." They are specifically highlighting the commonalities w/ WPI/Lombardi while not disclosing what's different. I honestly can't believe this quality of work passes for real science. I don't know of any clinical definition stating 6 months of post-exertional malaise (perhaps they meant 6 months of unexplained fatigue--Fukuda).

I went back to check the actual wording of the Empirical Definition and it does include all criteria of the Fukuda Definition, while adding in substantial psychiatric criteria in the form of depression. The description above can technically be correct while not conforming to Fukuda, as they imply, and using Empirical Criteria for its patient population.

This was highlighted by none other than Dr. Lo in his confirmatory PNAS article in August. 2010:

"Finally, it is also quite possible that there is heterogeneity in the patients diagnosed with CFS in different studies. CFS is a syndrome defined exclusively by a group of nonspecfic symptoms and thus has an ill-defined phenotype. Future Studies should adhere to consensus case definitions such as that developed by the Centers for Disease Control and Prevention." -- In which he specifically references the Fukuda criteria.

Unless and until the CDC states explicitly which clinical criteria it's using to select patients for its XMRV studies, I don't believe any of their published work should carry weight w/ the XMRV scientific community. And that goes for the editors of Retrovirology journal for not requiring the CDC's authors to state exactly who they are choosing for their patient samples.

It's the patients, not the test that's the primary problem here. While there may be issues w/ the test methods, the CDC will never find XMRV in its studies when using the Empirical Definition.

 

[August59]

And yet the CDC will be presenting at the Retrovirus Conference in Boston this week conveying to the researchers and physicians why XMRV does not exist in CFS patients. These researchers and physicians for the most part have no idea what Emperical, Fukuda, Canadian means. All they will take away is that there is no XMRV in CFS and they will carry it with them for a long time unfortunately.

 

[eric_s]

But even in those cohorts there should be at least the same prevalence as in healthy controls, i guess. So they should have had some positives at least, if they are able to find it and it was there.

 

[wdb]

I'm sure selection criteria is a big factor but it seems quite unlikely that amongst a large sample of patients who meet the CDC Empirical Definition that there would not be a single case who also meets the Canadian criteria.

 

[bullybeef]

Has the CDC tested prostate cancer patients for XMRV yet? They can’t get that patient group wrong, so I wonder what their excuse would be then?

This springs to mind due to the UK “researchers” suggesting XMRV is now not even pathogenic, after looking at prostate cancer!!

So what is it? Is it real; is it in humans; is it pathogenic; is it a contamination that can infect humans; or is something that new to science, they are way behind the likes of the WPI, the Cleveland Clinic, and the NCI?

Just the fact they’re all on difference pages but trying to prove different negative hypothesises implies they’re somewhat reaching, and looking for multiple options to bury this.

BB

 

[Jemal]

I don't think it's the patients. To me it looked like this study did not target the usual bunch of depressed people the CDC liked to do their studies on. And I agree with eric_s that at least some people should have tested positive for the virus, given the prevalence of the virus in healthy control groups in positive studies.

Either XMRV really is contamination or the CDC still can't detect it. That's my 2 cents...

 

[Angela Kennedy]

I'm sure selection criteria is a big factor but it seems quite unlikely that amongst a large sample of patients who meet the CDC Empirical Definition that there would not be a single case who also meets the Canadian criteria.

The thread below demonstrates how CC, neurological ME suffers can be TOTALLY selected out of RESEARCH cohorts.

http://forums.aboutmecfs.org/showthread.php?10219-PACE-study-and-oxford-criteria

 

[alex3619]

Hi,

My bias is is that it is the test not the patients. Many studies have used bad patient selection, for some of the reasons discussed here and elsewhere, but even by chance some positives should have been found. Almost everyone is doing blood tests. Some pretend, like the CDC, that plasma tests are equivalent to blood tests - they are not. Its time we insisted that all research aimed at CFS, and not blood bank security, use biopsies from tissues known to be rich in XMRV in infected patients.

Bye,
Alex

 

[eric_s]

I think this issue is much more important than XMRV. Whatever XMRV is, not everyone with a current CFS diagnosis has the same illness. We need some scientific and unimpeachable way of diagnosing whatever it is that people with CFS have. The current situation allows scope for far too much "shenanigans", whether this is intentional on the part of researchers or not. It's no coincidence that one of the most accomplished researchers ever to become involved in CFS, Dr Harvey Alter, highlighted this central issue.

Yes, that's why i think those CSF studies are so interesting. On the other hand, if they can prove everybody with ME/CFS has XMRV that will be even a better biomarker, probably. Either way, i'm hoping too, that they will quite soon find a better way to diagnose ME/CFS than those catalogues/definitions we have now.

 

[eric_s]

Hi,

My bias is is that it is the test not the patients. Many studies have used bad patient selection, for some of the reasons discussed here and elsewhere, but even by chance some positives should have been found. Almost everyone is doing blood tests. Some pretend, like the CDC, that plasma tests are equivalent to blood tests - they are not. Its time we insisted that all research aimed at CFS, and not blood bank security, use biopsies from tissues known to be rich in XMRV in infected patients.

Bye,
Alex

What are those tissues, what do you think? Or do they already know, from humans, and i've missed it? I think KDM is looking at the stomach. He always seems to be looking in that region, what i find a bit strange, but who knows, he might be right.

 

[LJS]

Everyone is so quick to find fault but the latest paper from the CDC is a well done study with much better patient selection. They required that patients have PEM. Even if the patient selection was not good they should have found XMRV in healthy controls considering how sensitive there latest XMRV assay is. Everyone is ignoring the elephant in the room which is it is possible that the WPI is not actually finding XMRV in patients but there is something wrong with there test. We will know this year with the BWG study but I am getting tired of everyone jumping to the conclusion that negative papers some how used poor scientific practices.

 

[Angela Kennedy]

I'm thinking the problems are around both testing methodology AND patient selection.

 

[eric_s]

Everyone is so quick to find fault but the latest paper from the CDC is a well done study with much better patient selection. They required that patients have PEM. Even if the patient selection was not good they should have found XMRV in healthy controls considering how sensitive there latest XMRV assay is. Everyone is ignoring the elephant in the room which is it is possible that the WPI is not actually finding XMRV in patients but there is something wrong with there test. We will know this year with the BWG study but I am getting tired of everyone jumping to the conclusion that negative papers some how used poor scientific practices.

But what about Lo and Alter, the Spanish researchers at IrsiCaixa and others who have found XMRV in people with ME/CFS? It's not just one group. Could it be that there is something, a potential source of contamination, that they all have shared without recognizing the contamination? I find that hard to believe, but if you have an idea it would be interesting to hear. Ruscetti will be presenting a new assay at CROI, maybe this will help anwering the contamination issue, but i don't know.

 

[SOC]

Everyone is so quick to find fault but the latest paper from the CDC is a well done study with much better patient selection. They required that patients have PEM. Even if the patient selection was not good they should have found XMRV in healthy controls considering how sensitive there latest XMRV assay is. Everyone is ignoring the elephant in the room which is it is possible that the WPI is not actually finding XMRV in patients but there is something wrong with there test. We will know this year with the BWG study but I am getting tired of everyone jumping to the conclusion that negative papers some how used poor scientific practices.

Pointing out legitimate flaws in a research paper is not "jumping to conclusions"; it is highlighting facts about the study that readers need to keep in mind. Readers of scientific papers should always keep in mind the limits of the study at hand.

Yes, the CDC should have found XMRV in some of their samples if their assay is as good as they claim. That they didn't suggests more about the quality of their methodology than the existence of XMRV

The quality of the research showing some level of XMRV infection in the population at large is still much higher than the quality of research showing there is none at all. The preponderance of evidence is still on the side of the existence of XMRV, so the question that arises from this study is why the CDC is not finding it, not whether or not it exists at all.

The preponderance of evidence may eventually shift in favor of the nonexistence of XMRV, but that will not happen until someone does true replication of the Lombardi and Alter/Lo work and fails to find XMRV. Even then, nonexistence is not proved until multiple replication studies fail to achieve the same conclusion. That's proper science.

The scientists at the CDC seem to be forgetting a fundamental science principle -- you can't prove something isn't there, you can only prove you can't find it.

The current studies finding no XMRV at all are largely a waste of time and money. Although there is some benefit in showing methodologies that can't find XMRV, it is more critical at this time to get the replication work done.

 

[alex3619]

What are those tissues, what do you think? Or do they already know, from humans, and i've missed it? I think KDM is looking at the stomach. He always seems to be looking in that region, what i find a bit strange, but who knows, he might be right.

The various animal studies show which tissues, from lymph nodes to the pancreas if I recall correctly. Further research on this is being done by Singh. XMRV has specificity to certain tissue types in theory as well, all these things can be tested. To my knowledge multiple tissue biopsies have never been conducted on humans to identify the specific human tissues that are best suited for this.

KDM is indeed looking at the stomach, and I hope it works, but I think colon biopsies would be better, if a little more difficult.

Multiple lines of evidence show that looking for XMRV in blood under certain conditions is problematic, whether you find it or not. Good biopsy research will end the debate.

Plasma testing is not highly problematic, it is extremely problematic, and it is bad science at this point to pass it off as reliable. If you do this, expect a contraversial result, no matter what you find.

Bye
Alex

 

[WillowJ]

I'm sure selection criteria is a big factor but it seems quite unlikely that amongst a large sample of patients who meet the CDC Empirical Definition that there would not be a single case who also meets the Canadian criteria.

It's not at all unlikely. From the 2010 Switzer paper, (emphasis added)

The Lombardi et al. study specifies that samples were selected from patients fulfilling the 1994 international CFS case definition [23] and the 2003 Canadian Consensus Criteria for CFS/ME [25]....The 1994 International CFS case definition and the Canadian Consensus Criteria are different and do not necessarily identify similar groups of ill persons. Most notably, the Canadian Criteria include multiple abnormal physical findings such as spatial instability, ataxia, muscle weakness and fasciculation, restless leg syndrome, and tender lymphadenopathy. The physical findings in persons meeting the Canadian definition may signal the presence of a neurologic condition considered exclusionary for CFS and thus the XMRV positive persons in the Lombardi et al. study may represent a clinical subset of patients [11].

Switzer et al. find immunologic and neurologic signs and symptoms, even a diagnostic-supportive manifestation (tender lymphs) of Fukuda and noted findings from the 2003 modification, to be suspect findings indicating that the patient in question probably doesn't have "CFS" (as they have redefined it).

The CDC has evidently pulled the same old Wessely school trick of first eliminate persons with the actual Disease, then apply the selection criteria. That way you won't have to bother with the complicated and troublesome neurological G93.3 patients who have actual immune and neurologic abnormalities, and get worse with GET, and thus mess up all your studies which are meant to show NO immune or neurologic abnormalities and improvement with GET. Plus, you can still claim you are using Fukuda and have a similar patient set to those of the Troublesome Researchers who will keep making Findings.

 

[LJS]

But what about Lo and Alter, the Spanish researchers at IrsiCaixa and others who have found XMRV in people with ME/CFS? It's not just one group. Could it be that there is something, a potential source of contamination, that they all have shared without recognizing the contamination? I find that hard to believe, but if you have an idea it would be interesting to hear. Ruscetti will be presenting a new assay at CROI, maybe this will help anwering the contamination issue, but i don't know.

Well it seems that mouse DNA is everywhere in city drinking water which is used in labs and it is insanely hard to filter out even three stage reverse osmosis filters can not get rid of it. It has also been found in some heparin which is used in blood collection tubes. Since XMRV PCR tests cross react with mouse DNA this is a whole new level of possible contamination that no other virus studies in the past have had to deal with.

Pointing out legitimate flaws in a research paper is not "jumping to conclusions"; it is highlighting facts about the study that readers need to keep in mind. Readers of scientific papers should always keep in mind the limits of the study at hand.

Yes, the CDC should have found XMRV in some of their samples if their assay is as good as they claim. That they didn't suggests more about the quality of their methodology than the existence of XMRV

The CDC should have only found XMRV in healthy controls if it is actually present in healthy controls. Each of the recent negative studies have tried a new way to find XMRV with more and more sensitive tests and have found nothing which suggest that XMRV is actually not present in the general population and something is wrong with the WPI test.

The quality of the research showing some level of XMRV infection in the population at large is still much higher than the quality of research showing there is none at all. The preponderance of evidence is still on the side of the existence of XMRV, so the question that arises from this study is why the CDC is not finding it, not whether or not it exists at all.

How is the quality of research showing XMRV higher then that done in all the negative studies? Each negative study has it weaknesses but so do the positive studies. I would argue the preponderance of evidence is on the side of XMRV not present in the general population. The amount of data showing issues with XMRV or lack of finding of XMRV outweighs the papers finding it at this point.

The preponderance of evidence may eventually shift in favor of the nonexistence of XMRV, but that will not happen until someone does true replication of the Lombardi and Alter/Lo work and fails to find XMRV. Even then, nonexistence is not proved until multiple replication studies fail to achieve the same conclusion. That's proper science.

The latest CDC study did a pretty good job at replicating some of the aspects of the WPI study and should have found it if it was there. They even tested for antibodies to XMRV based off the work in the positive control rhesus macaques study. Everyone including myself was unhappy that no one was looking for antibodies to XMRV but now people are and are not finding them.

The scientists at the CDC seem to be forgetting a fundamental science principle -- you can't prove something isn't there, you can only prove you can't find it.

How much evidence showing something, say X is not there is needed to prove that X is truly not there? It gets to a point where no matter how much evidence you show that something is not there a small subset of people will not accept the science and cling to their personal belief. It is not at the point yet and I am not at that point yet but it is getting close quickly. If the BWG study comes back fully negative I would be pretty confident we have reached the point where it is safe to say there is overwhelming evidence showing XMRV not to be present in the general population and CFS research should move on. That is saying a lot considering I tested positive for XMRV and have donated to WPI.

 

[WillowJ]

The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.

What is this all about? Does this seem unusual to anyone else?

It doesn't seem like it's the "seems unlikely XMRV is related to CFS" conclusion that CDC is distancing itself from... so it's the PEM and the talk of patient stratification that brought this on? They didn't do that on the Sorensen et al. transcription control of complement activation study, that I recall...

 

[WillowJ]

I am getting tired of everyone jumping to the conclusion that negative papers some how used poor scientific practices.

The early negative papers DID use poor scientific practices, like not using any controls, using databanks which had been prescreened to remove patients with any signs of infection, and so on.

 

[alex3619]

The CDC has evidently pulled the same old Wessely school trick of first eliminate persons with the actual Disease, then apply the selection criteria. That way you won't have to bother with the complicated and troublesome neurological G93.3 patients who have actual immune and neurologic abnormalities, and get worse with GET, and thus mess up all your studies which are meant to show NO immune or neurologic abnormalities and improvement with GET. Plus, you can still claim you are using Fukuda and have a similar patient set to those of the Troublesome Researchers who will keep making Findings.

Hi Willow, I completely agree with you on this, I have said the same about biopsychosocial studies in the past. The cohorts are bad. But by chance alone there should be some positive cases, but few if any are rarely found, and often these are really contaminated cases. This applies to the controls even more.

The tentative conclusion I draw again and again is that one group is consistently wrong - either the ones who find XMRV, or the ones who don't. The balance of evidence has consistently led me to conclude that it is the ones who don't who are wrong. This means they are doing something wrong, even if we don't know what it is.

At some point one of the scientific groups will find the issue, and we will all be wiser.

Bye
Alex

ps From the paper:

"PCR analysis was performed on PBMC DNA using three previously described tests (Table 2), two
for the polymerase (pol) gene, and one for the gag gene used in Urisman et al., Lombardi et al., and Lo et al.

A subset of 48 plasma samples were tested for viral RNA sequences by RT-PCR using primers from the nested gag assay and also by using a new quantitative RT-PCR test that generically
detects MuLV and XMRV gag sequences.

Since antibody responses are hallmarks of retroviral infection, we also used a newly
modified Western blot (WB) test to detect anti-XMRV antibodies in plasma"


I don't have a huge problem with the first one, except there is debate elsewhere (and I never checked the math) that insufficient DNA quantity was used. The second and third look like smoke and mirrors to me. Take a test that is likely to fail and then look, magically it fails. Plasma is problematic until someone proves it isn't. From the animal studies we know that even when the blood tests fail completely in absolutely known positive cases, tissue lights up like a christmas tree. At such high levels of XMRV nested PCR may not be necessary and so the supersensitivity of nested PCR goes away, and you can mostly forget about low level contamination, it becomes a non-issue. We need biopsy studies. They are not being done. Why? Cost comes to mind, so does invasiveness (damned ethics committees!) but this endless blood debate is causing massive delays in research.

Bye
Alex