1. Patients launch $1.27 million crowdfunding campaign for ME/CFS gut microbiome study.
    Check out the website, Facebook and Twitter. Join in donate and spread the word!
Hunting down the cause of ME/CFS & other challenging disorders - Lipkin in London
In a talk to patients in London on 3rd September, Dr. W. Ian Lipkin described the extraordinary lengths he and his team are prepared to go to in order to track down the source of an illness, with examples ranging from autism to the strange case of Kawasaki disease.
Discuss the article on the Forums.

shock: Top journal publishes failure to replicate key finding

Discussion in 'Other Health News and Research' started by Simon, Nov 8, 2013.

  1. Simon

    Simon

    Messages:
    1,531
    Likes:
    4,911
    Monmouth, UK
    this is a bit of an opinion piece by me, but it is short

    Replication of studies really is the gold standard of scientific research, yet rarely happens leaving a scientific literature that is awash with interesting but unconfirmed findings (see Once Is Not Enough). It's a huge problem in CFS research. One big reason that people don't even attempt repliction - across life sciences - is that journals want novelty, and replication is the exact opposite of that. But here is a case where it did happen:

    Nature Biotechnology has made a lot of noise about publishing a replication study with no novelty value, though as they point out it has considerable importance. The new study represents a failure to replicate a very surprising finding, which implied the gene regulation mechanisms inside some plants, if eaten, could affect human gene expression too. Think they were also having a pop at a rival journal (Cell Research) that published the original paper, and turned down the failed replication on the grounds that “it is a bit hard to publish a paper of which the results are largely negative.” Sadly, this attitude is all too common.

    Perhaps this high profile publication will make it easier for researchers to publish replication studies, and that will be doing science a big favour.
     
    Last edited: Nov 8, 2013
  2. Sparrowhawk

    Sparrowhawk Senior Member

    Messages:
    454
    Likes:
    236
    West Coast USA
    Thanks @Simon, that's refreshing and encouraging to see such an editorial stance, and furthermore the decision to publish, thus admonishing others to do the same.
     
    WillowJ and Simon like this.
  3. Bob

    Bob

    Messages:
    8,917
    Likes:
    12,635
    South of England
    Last edited: Nov 8, 2013
    Helen, Valentijn, alex3619 and 5 others like this.
  4. Esther12

    Esther12 Senior Member

    Messages:
    5,387
    Likes:
    5,903
    There really needs to be more openness around peer review. Some journal's processes are an absolute joke.
     
    barbc56 likes this.
  5. barbc56

    barbc56 Senior Member

    Messages:
    1,581
    Likes:
    984
    I though replication is a key component of science.

    Yes, absolute transparency!

    Very interesting and an important question.
     
  6. alex3619

    alex3619 Senior Member

    Messages:
    7,727
    Likes:
    12,658
    Logan, Queensland, Australia
    I substantially agree with your arguments. There is a move to have online journals for replication studies that have only technical requirements for submission, basically that a study is scientifically sound. So it doesn't matter if its popular, or publishable in a paper journal, the study is still available and can be found by searching or indexing engines. This is an essential step to create continued progress in science. Otherwise science risks becoming just another failed religion.
     
    ahimsa, leela, Helen and 4 others like this.
  7. barbc56

    barbc56 Senior Member

    Messages:
    1,581
    Likes:
    984
    There's also Pub. Med but it's not as user friendly.
     
  8. Simon

    Simon

    Messages:
    1,531
    Likes:
    4,911
    Monmouth, UK
    I'd heard that the worthy goal of a 'replication' journal was struggling to attract any interest from publishers or editors, despite it's obvious usefulness. PLoS One will take anything deemed 'methodologically sound', regardless of importance, and Jonathan Kerr publishd his failed replication of earlier CFS 'identifier' gene expression findings there. But authors have to pay to publish, making it an even less attractive option.

    The Nature Biotech editorial points to some other interesting developments though:
    • one idea being floated by certain funders is to set aside a portion of a research grant specifically for independent verification of the main study's results before publication; in this scheme, submission to a journal would proceed only after the results were corroborated.
    • This summer, the journal Cortex started offering yet another means of improving reproducibility and reducing bias. The mechanism, termed a “Registered Report,” involves peer review of an investigator's experimental design before data are collected. If the scientific question and methods are deemed sound, then authors are offered “in principle acceptance” of their article, irrespective of the study's outcome. [I think pre-review of methodology is in any case a great way to improve research]
    • The Reproducibility Initiative, A collaboration between the Science Exchange and PLOS ONE, the initiative offers to broker independent validation of a researcher's work in return for a fee, with subsequent publication in the journal. In October, the Laura and John Arnold Foundation provided $1.3 million to the initiative to authenticate 50 high-profile cancer papers from the past two years (only ~$20,000 per study).
     
    Helen, WillowJ, Bob and 1 other person like this.
  9. Gijs

    Gijs Senior Member

    Messages:
    145
    Likes:
    141
    Falsification is the key factor for science.
     
    leela and alex3619 like this.
  10. Firestormm

    Firestormm Guest

    Messages:
    5,824
    Likes:
    5,982
    Cornwall England
    Is this not a problem only for those seeking professional advancement through publication? What I mean to say is, and to take let's say Rituximab as an example; it is increasingly apparent to me that private funding institutions are now taking over a greater share of the funding of science.

    In our little world, any 'interesting' papers, should now see their replication attempts funded without too much effort I would think. Unless of course those holding the money are also uninterested in funding replication studies?

    I can see that in the scientific world of competing academics there might be little appeal in replication when it comes to attracting a publisher - though this seems completely daft to me as well - but in our world: there should be less of a barrier to willingness.

    Or maybe this does explain why - even when funding is available - few seem willing to approach with a proposal. I mean there is an ocean of 'interesting' studies on ME that should surely be replicated. I just can't see that this publication bias is the only barrier to not replicating.

    @Jonathan Edwards Do you have any thoughts about this? Thanks :)
     
    aimossy likes this.
  11. Hip

    Hip Senior Member

    Messages:
    3,270
    Likes:
    3,161
    In terms of lack of replication, the fact that Dr John Chia's findings of enterovirus infections in the stomach of ME/CFS patients have not been replicated is a major problem.

    Chia's methods were solid, and he demonstrated a strong connection between enterovirus infection and ME/CFS.

    Yet in spite of the fact that this enterovirus link is the only real lead lead we have on viral ME/CFS, nobody has bothered to replicate Chia's work. This is crazy. During the XMRV debacle, there were loads of replication studies (far more than were actually needed); yet for some inexplicable reason, nobody has thought to replicate Chia's excellent enterovirus results. Why didn't just one of these researchers try to replicate the enterovirus studies, instead of wasting time and money on an excess amount of XMRV study replications?

    Nobody in the ME/CFS community even talks much about these enterovirus results. I cannot understand why there is this lack of interest. Do we want a cure or not?
     
    Last edited: Nov 9, 2013
  12. Sparrowhawk

    Sparrowhawk Senior Member

    Messages:
    454
    Likes:
    236
    West Coast USA
    Good bloody question, @Hip. Definitely agree it's worth pursuing.
     
  13. alex3619

    alex3619 Senior Member

    Messages:
    7,727
    Likes:
    12,658
    Logan, Queensland, Australia
    Falsification is what distinguishes science from nonscience. Freudian psychiatry was a classic case. It could not be properly tested, so Karl Popper called it nonscience, and I think Kuhn agreed - which is itself remarkable because Kuhn and Popper operated in competing paradigms.

    I agree we need the enterovirus findings replicated. However to some extent Chia's findings were the actual replication. This research goes back to the 1980s. There was evidence of coxsackie virus in muscle, I think it was viral nucleotides. Indeed after I tested positive an article on this was handed to my by my doctor, who said that CFS (she was up to date on Holmes at least in 1989) might be caused by an enterovirus. This is not a new hypothesis.
     
    Last edited: Nov 11, 2013
  14. Helen

    Helen Senior Member

    Messages:
    576
    Likes:
    590
    @Hip and @alex3619 Maybe I´m quite wrong, but doesn´t Ian Lipkin´s research include enteroviruses when he and his group is examinating blood, saliva, faeces and spinal fluid (and I think other fluids too)? Two researchers in Sweden who where in the XMRV circus (as someone here called it) are still researching ME/CFS. I would guess they are eager for revenge...At least I know they are examinating spinal fluid with all analyses that are possible according to the assistent who invited me to take part in the study.
     
  15. alex3619

    alex3619 Senior Member

    Messages:
    7,727
    Likes:
    12,658
    Logan, Queensland, Australia
    @Helen, there is a difference between acute virus infection and chronic tissue infection. Blood tests only work on acute infections, and with antibody testing can show recent or past acute infections. Slow tissue infections are invisible without biopsy, but I think there are secondary markers for many of them.
     
    Helen likes this.
  16. Bob

    Bob

    Messages:
    8,917
    Likes:
    12,635
    South of England
    I think Chia studied gut tissue biopsies, didn't he?
     
    Helen likes this.
  17. Hip

    Hip Senior Member

    Messages:
    3,270
    Likes:
    3,161
    @Helen
    I understand that Ian Lipkin did include enteroviruses in his recent study, but as Alex points out, chronic enterovirus infections are unfortunately very hard to detect, unless specific types of testing are used. Lipkin's study has not yet been published I believe, so I am not sure exactly what type of enterovirus testing he performed.

    If Lipkin just used blood tests and cerebrospinal fluid tests for enterovirus antibodies, this does not count. Most laboratory blood tests for enterovirus antibodies come out negative in ME/CFS patients, and I would think that cerebrospinal fluid tests for enterovirus antibodies will also come out negative. This is because these tests are not sensitive enough for detecting chronic enterovirus infections.

    However, Dr John Chia developed and pioneered a new testing method that reliably detects enterovirus infections in ME/CFS patients. His new method uses tiny stomach tissue samples.

    What John Chia did was to take a tiny tissue sample (biopsy) from the stomach of ME/CFS patients (using an endoscope), and then he tested that tissue sample for the presence of enteroviruses. Using this method, Chia found that 82% ME/CFS patients have significant enterovirus infections in their stomach tissues, whereas only 20% of the controls had such an infection. Ref: 1

    So this is a major finding, as it demonstrates a strong association between enterovirus infection and ME/CFS.

    In order to replicate John Chia's results, you would need to conduct a similar study that also tested stomach tissue samples for enteroviruses. This is not a particularly difficult thing to do, and in fact Chia routinely tests his ME/CFS patients for enterovirus infections using the stomach biopsy method.

    So it would be very straightforward for another researcher to replicate Chia's results, but nobody seems to have bothered to do this.

    To me this is crazy, because Chia found a very positive, strong link between enterovirus infection and ME/CFS. You would have thought that the whole of the medical scientific community would sit up and take note. You would have thought that they would spring into action and investigate further. But nothing further has been done by the wider scientific community.

    And even ME/CFS patients, who are usually the most vociferous advocates for further research, have largely ignored Chia's groundbreaking findings. It makes no sense to me. As I say, do we want a cure or not?
     
    Last edited: Nov 12, 2013
    alex3619, Valentijn and Helen like this.
  18. Helen

    Helen Senior Member

    Messages:
    576
    Likes:
    590
    @Hip , @Bob and @alex3619 Thank you very much for your posts. You opened up my eyes to Dr. Chias indeed interesting stomach biopsy research.
     
    Bob likes this.

See more popular forum discussions.

Share This Page