1. Patients launch $1.27 million crowdfunding campaign for ME/CFS gut microbiome study.
    Check out the website, Facebook and Twitter. Join in donate and spread the word!
The Pathway to Prevention (P2P) for ME/CFS: A Dangerous Process
Gabby Klein gives an overview of the P2P process, shedding light on the pitfalls with advice as to what we can do in protest ...
Discuss the article on the Forums.

XMRV CFS UK study #II

Discussion in 'XMRV Research and Replication Studies' started by Orla, Feb 15, 2010.

  1. Gerwyn

    Gerwyn Guest

    They would certainly have to use the same methods in the controls.I would be very suprised if they did not. I dont think that would have survived the science peer review process.Their methods are what you expect if they were looking for a virus mostly in its latent phase.The Methodology in the IC and Groom studies were juvenile at best.The control methodology in the Groom study was totally bizarre.The detection methodology in the IC study went against all the published protocols for recovering XMRV partcularily in the transfection stage
  2. natasa778

    natasa778 Senior Member

    Messages:
    1,409
    Likes:
    1,208
    London UK
    Could that be down to a different blood collection technique used at SGUL, as Gerwyn pointed out the exact procedure of drawing and processing blood is crucial for virology purposes. Maybe they somehow did the correct thing at SGUL and messed up with other samples, drawn elsewhere?
  3. Gerwyn

    Gerwyn Guest

    Agreed in full in all parts and the very fact that they had to use multiple amplification steps makes the Uk methodoly frankly silly
  4. Mithriel

    Mithriel Senior Member

    Messages:
    564
    Likes:
    40
    Scotland
    I haven't gone over the papers again as I don't feel up to it today, but this was my impression.

    In the original study, 67% of patients were positive and 3%(?) controls were by PCR.

    Once these figures were established, some of the positive samples from both were given extra tests where they looked at the virus in more detail. One patient sample had the entire genome of the virus worked out. They used culture at this stage.

    I am sure the virus from the control did not have env, but I would have to check.

    After the paper was presented, they did more testing on the patients that were negative. It was at this stage that the four samples came into it. Some patients were negative until they had been tested four separate times. One patient did not test positive by any of this extra testing all the rest had the virus. It may be that she meant they found patients positive by culture but they had to run PCR on four separate samples until PCR matched culture.

    They did not have to test the controls in such detail. The difference between patients and controls was established by the original tests and the studies by the blood people and the CDC are looking at healthy people.

    This paper was examined and refereed for months. The science was done properly. The Imperial College study was very rushed and I am dubious about it, in part because of the unscientific claims that were made, but the Kerr study did proper science but the design was not ideal.

    I can't understand the antipathy to the WPI when everything they did was held up to so much scrutiny but so many well respected scientists.

    Mithriel
  5. starryeyes

    starryeyes Senior Member

    Messages:
    1,513
    Likes:
    4
    Bay Area, California
    Thanks Bob. That pretty much sums it up.

    I see the question coming up here a lot about whether or not the WPI used the Canadian Definition on all of their test subjects for XMRV. Of course they did! They know that if they didn't they couldn't say for sure that the patient has ME/CFS.
  6. kurt

    kurt Senior Member

    Messages:
    1,132
    Likes:
    176
    USA.Earth
    Mithriel, if this is how WPI worked, and the multiple tests were only for those who initially were negative by PCR then I agree, there was no requirement to test the controls in detail. However this is the exact point that is not clear to me. Where did Mikovitz state that multiple tests were only conducted on those negative by initial PCR? Her statements seem unclear.

    Your statement that the paper was examined for several months is correct, however, a paper is not analogous with a study, papers can pass review even when the study has an underlying flaw. The reviewers do not go to the lab and witness how a study is conducted, they rely on what the authors write in the report. If something important is left out of the report, the reviewers may not catch it. If multiple tests were run on some samples and not reported, even as a simple oversight and not with the intention of scientific fraud, that would not be caught in reviews.

    As for antipathy to WPI, I have no dislike of WPI, challenging someone's research is an important part of the scientific process and is not personal. In fact, you are dubious of the IC study, is that antipathy or simply scientific objectivity? I am just trying to be fair and make certain we hold WPI to the same standard of scrutiny as every other XMRV study, and that is not antipathy. Some statements have been made that do not add up, I am trying to sort that out. There is no malice in scientific questioning, research must be validated through many means including questioning methods.

    Gerwyn, As I just said above, this is only true if the methods were all stated, if some part of a method is not stated in the paper no review will catch any related problems. A peer review is restricted to the paper itself and is not a review of a research program.

    Raul, Some interesting comments. I believe your analysis is right if the false negatives are only due to poor test sensitivity or ultra low viral levels. However if false negatives are due to reagent issues then the risk is additive. And I don't believe that is known right now, the cause of false negatives, although WPI has made the assumption that only low viral count is involved. That has to be proven with validation studies which so far have failed.

    Mark, I generally would not consider this type of flaw to be realistic in a study as well organized and carefully reviewed as the Science study. That is why I did a real double-take when Mikovitz revealed that there were extra testing steps not included in the study writeup. Note that she revealed this in the context of explaining why ALL of the UK study results failed, therefore she considers this multiple running of test to be critical in finding XMRV. After hearing that I started working through the implications of some of her statements for the WPI study and realized it just was not clear which samples had been tested multiple times. If only the failed PCR samples were tested multiple times then that probably settles my question about re-testing. Until we have a definite response to that this issue remains, and it is a potentially serious problem.

    Also, questioning basic assumptions and methods is not an insult in science. Sometimes rather large errors are made unintentionally in the rush to write-up and publish studies, and when that happens they must be revealed as early in the process as possible. And an error is not automatically fraud or swindle, please do not attribute that type of accusation to my comments, I am saying nothing of the sort. I respect what WPI is trying to accomplish and just want some technical questions answered in a clear language that we can all understand, that is all. Considering that the WPI study has NOT been validated yet, and also considering how critical WPI has been of the UK studies, and others here on this forum have also been, don't you think it is only fair to hold WPI to the same standard of scrutiny as any other XMRV study?
  7. starryeyes

    starryeyes Senior Member

    Messages:
    1,513
    Likes:
    4
    Bay Area, California
    Dr. Judy Mikovits gave me permission to post this:

    tee: May I please post your answer on the Phoenix Rising forum?

    Dr. Judy: Yes you may. Thank you for your support


    Dr. Judy's answer (bolding mine):


    "The issue is DNA from unstimulated whole blood and PBMC...If the methods used in the science paper are used as they are done at the WPI and VIPDx then you will detect XMRV infection ALL the time. We have always tested healthy controls side by side as is indicated in every figure of the Science paper. The issue with all other studies to date is that they only do PCR on banked DNA and then are not using enough cellular DNA. No one has even attempted to go beyond figure 1 of our paper!

    To clarify..if we looked at banked DNA from unstiumlated white blood cells, we would often have to look at multiple samples from different dates from the same patient..XMRV is simply not incorporated into the DNA at high enough copy numbers to detect unless the cells are dividing, allowing the virus to multiply..that is retrovirology 101.."
  8. julius

    julius Watchoo lookin' at?

    Messages:
    785
    Likes:
    5
    Canada
    Too bad she didn't really answer the second question. That's the one I really want to know.

    But good to know that the first is answered.
  9. Gerwyn

    Gerwyn Guest

    kurt a peer review does look at methodology if there is a hole or ambiguity a peer review worth its salt will pick it out.It cant be done in 24 hours of course.Science reviewers repeatedly asked for more info untill they were satisfied.I
    fail to see why you are pusuing this avenue at the expense of commenting on the far more obvious and published flaws in the two british studies.Any deviations from protocols in the WPI approach are hypothetical the deviations from known established science in the British studies are absolute fact
  10. starryeyes

    starryeyes Senior Member

    Messages:
    1,513
    Likes:
    4
    Bay Area, California
    Julius,

    Doesn't this answer my second question?

  11. kurt

    kurt Senior Member

    Messages:
    1,132
    Likes:
    176
    USA.Earth
    Thanks for posting this. Judy Mikovitz's comments do help make clear that a control was run along side every CFS sample test. She did not state this directly but I assume from her comment that if a CFS sample was run several times, so was that control. Anyway I hope that is what she meant.

    But what if a different sample was used for a CFS patient, a second or third sample, did they have a different sample from that same control? Or did they just test a different control? That remains unanswered, and I do not know how relevant that point is, but it should be addressed.

    And she says that if you use banked samples without stimulating DNA replication you may have to look many times (earlier I believe she suggested four samples, with up to four tests each). So live or recent samples should be easier to test, finding more live virus. So which were used for the 101 samples and the controls also in the Science study, banked or live samples, or a mixture? That also is not answered.

    And she says that when they looked at banked samples without stimulating the WBCs, they had to run multiple samples from multiple dates from the same patient, due to low viral counts. I assume from this that the study reported in Science included only stimulated WBCs then and this was a single pass on those samples?

    I still would like to know the details of ALL of the samples used in the Science PCR study, both CFS samples and controls. How many times were each run, which samples were stimulated and which were not, and were the SAME controls tested alongside any multiple tests of the CFS samples?
  12. kurt

    kurt Senior Member

    Messages:
    1,132
    Likes:
    176
    USA.Earth
    Gerwyn, I am not speaking of a problem with the methodology, but rather a possible problem with the reporting. Reviewers can not evaluate information they are not given.

    As for the British studies, there is not much to evaluate, they did not find anything using their methodology. There were no 'flaws' in their studies other than not knowing how hard XMRV was to find, which was not entirely their fault. There were apparently some rather important details in the Science article that were cryptically written and those labs did not appreciate. Anyway, regardless of the reasons for their failure to find XMRV, their conclusions were incorrect, statements that there was no XMRV in ME patients in the UK would require far more extensive epidemiological testing and use of a validated methodology. At this point in the science they should not be conducting epidemiological research as there is no standardized XMRV test yet. So they were simply running a validation attempt (regardless of what they called it, that is what it was, certainly not a replication study) and they tried to draw epidemiological conclusions from that. The validation attempt was an acceptable part of the scientific process in a new finding like this, you have to run various different tests that prove that what was found by WPI was what they said it was. But the epidemiological claims showed that there was some convolution in the purpose of running the study.

    And neither can WPI with any credibility claim that XMRV is a factor in CFS at this point, when there are no validation studies that have confirmed their results, and no confirmed causal model. We are still quite far removed from any scientific consensus about XMRV in CFS, so the claims being made should be 'we found or did not find XMRV using this methodology' and the methodology should be made explicitly clear. We also should not be getting new information from WPI about how to find XMRV, that all should have been in the Science report.
  13. julius

    julius Watchoo lookin' at?

    Messages:
    785
    Likes:
    5
    Canada
    The fact that she specified "...in the Science paper..." leaves it a little ambiguous to me.
  14. Cort

    Cort Phoenix Rising Founder

    Messages:
    7,025
    Likes:
    439
    Raleigh, NC
    I asked Dr. Mikovits in an email: My thinking was that you used culture techniques with the original Science cohort - and these enabled you to pick up the virus with a single pass or did it take multiple attempts with the Science cohort as well?

    She also said:

    Nobody knew that it took them three or four times to find the virus in the CFS patients until they stated that recently.

    On the other hand, the fact that they were using culturing suggested that viral loads were very low - still making it perplexing to me why researchers wouldn't at least follow that part of the study.

    She also said


    My apologies about stating the Groom study was a UK study; I read somewhere that the MRV was akin to the NIH.
  15. kurt

    kurt Senior Member

    Messages:
    1,132
    Likes:
    176
    USA.Earth
    Thanks Cort for passing that along. And this explains why they are focused on the antibody studies right now, as they are a more reliable match to the stats in the Science paper. But her response confirms some of my worries, so they did run more tests on some of the CFS samples than on the controls. Yes, the 4% positive rate on the antibody tests is definitely supportive of their overall statistics, but this still will basically invalidate the PCR part of the finding, at least if I am understanding this issue correctly. And the antibody results, while compelling, are less specific, they can be due to cross-reactivity.

    This shows a serious disregard for the fact that the lack of detail in the Science article may be partly responsible for the failure of validation studies. They SHOULD lose some sleep over this because valuable resources are being wasted. They may be somebody else's resources and not WPI's, but think about this, if a bunch of labs spend a lot of money chasing down XMRV and nothing works out for them, how likely are they to invest in the next hypothesis that comes around for CFS? A lack of attention by WPI to this serious replication/validation problem could hurt our cause in the long-run.
  16. starryeyes

    starryeyes Senior Member

    Messages:
    1,513
    Likes:
    4
    Bay Area, California
    She also said: "it certainly is not present in Maryland controls where repeated sampling of the same donors is possible." - thank you for sharing her email to you Cort. This shows that Dr. Judy is positive XMRV is not present in at least these Controls.

    I get the feeling that no matter how open and transparent Dr. Judy is and no matter how many questions she answers, some of you here will never believe the WPI's Study is completely correct until replication is proven and that's okay, time will tell.
  17. raul

    raul

    Messages:
    16
    Likes:
    0
    Kurt, if by additive risk you mean that you can calculate the total detection rate by multiplying the detection rate of a single pass times the number of passes I think that is not the case regardless of the nature of false negatives. If it was, you would get total detection rates above 100% in some instances. Say the detection rate of a single pass is 50%. After 3 passes you would get 3x50%=150%
  18. Gerwyn

    Gerwyn Guest

    Kurt the british methodology is full of flaws I,m suprised you cant see therm lack of amplification proceedures normally used to detect a latent virus but one glaring one diagnostic criterea an other laced controls harvesting way to soon using non permissive cells and this is just thesimple stuff peer review done in 24 hours etc looking in the wrong place despite published evidence to the contrary I could go on. Judy,s analysis is spot on from a microbiologists viewpoint.The british methods are frankly baffling considering what they were purportedly trying to achieve they made assumption after assumption without testing any of them
  19. Sasha

    Sasha Fine, thank you

    Messages:
    8,029
    Likes:
    6,797
    UK
    I think you're right that it's a UK study, Cort - the MRC (I think that's what you meant) is the Medical Research Council, a research body funded entirely by the UK government.

    ETA: Oh, just seen ME Agenda's post that you were responding to - I think the issue was not about it being a UK study but a UK govt funded study. It got its funding partly from govt (the MRC) and partly from charities, two (the Wellcome & the Cunningham) being general medical charities and the other an ME charity.
  20. oerganix

    oerganix Senior Member

    Messages:
    611
    Likes:
    5
    KFG said: "We were told that as a result of re-examining their cohort over Christmas, they had discovered that it actually included patients from all over the world, contrary to what was published in "Science". "

    You called that an error. What I heard her say in the first public presentation was that when the cohort was unblinded (not the same as 're-examined') in December she was surprised to find patients from UK, Ireland, Germany and Australia. What they said in Science was that patients were from US clinics. What she didn't know at that time was that those US clinics had treated some patients from other countries. This seems like something that, maybe, Dr Peterson should have caught before publication, but maybe those patients were not from his clinic either. Not an error, just something she found out after the study was complete, for good reasons. And what difference does it make, anyway? We've since had 2 groups of UK patients tested by WPI methods and about 50% have tested positive, and the testing methods are still being refined. As stated in another thread, a European study will be published in March and they are finding XMRV in CFS patients. Even as a non-scientist, I found the idea that there is 0 XMRV in UK to be ludicrous.

    Regarding other points made about the WPI study by others, the fact that neither UK study seems to have asked for any help or collaboration from WPI, although WPI has repeatedly offered it, seems to me to be a contributing factor and something those other studies could have done if they were as serious about actually validating the original study. Their behavior doesn't seem very cooperative to me. They seemed to have started from a Reeves/Wessely point of bias: "nope, no XMRV here", before they did any research. So, IF any money was wasted, it is not WPI's fault that the other studies did not use all the resources available to them.

    If WPI started from a bias that there must be a retrovirus or virus in these patients, I think that's justified by the clinical finding of doctors who have been treating patients for decades. Dr Bell said of XMRV: It just fits. Of course, we're all afraid of finding out this is not THE virus or retrovirus, but since WPI was the first group in a long time that was actually looking, many of us want to give them the benefit of the doubt, at least for now. Which is not to say that Kurt and others are not justified in examining WPIs work in detail. Obviously, good people and good scientists can have different opinions of the research and different impressions of the people doing it.

    KFG, I don't think anyone here is going to accuse you of asking stupid questions, so fire away.

See more popular forum discussions.

Share This Page