Severe ME Day of Understanding and Remembrance: Aug. 8, 2017
Determined to paper the Internet with articles about ME, Jody Smith brings some additional focus to Severe Myalgic Encephalomyelitis Day of Understanding and Remembrance on Aug. 8, 2017 ...
Discuss the article on the Forums.

Most research spending is wasted on poor study design. Someone wants to change that

Discussion in 'Other Health News and Research' started by Snowdrop, Oct 4, 2015.

  1. Snowdrop

    Snowdrop Rebel without a biscuit

    Messages:
    2,895
    Likes:
    10,072
    Sorry don't know where to put this.

    http://www.vox.com/2015/10/4/9440931/arnold-foundation-meta-research

    Scientific research is often conducted in a highly unscientific manner. About $200 billion — or about 85 percent of global spending on research — is routinely wasted on poorly designed and redundant studies. As much as 30 percent of the most influential original medical research papers later turn out to be wrong or exaggerated.
     
    alkt, Helen, Johannawj and 7 others like this.
  2. PatJ

    PatJ far and free I gaze

    Messages:
    1,825
    Likes:
    5,058
    Canada
    Nice find Snowdrop. This article provides more information about the Laura and John Arnold foundation and the billionaires behind it.

    I think that CFS/ME and other difficult health conditions aren't dealt with properly due to a health care and funding system that just doesn't work properly for many reasons. Someone without brainfog, who has the skills, knowledge, and ability might do well by creating a document that outlines the many ways that CFS/ME funding, research, and treatment reflects a broken health-care system. This document could then be presented to the Laura and John Arnold foundation to see if they would be interested in taking on an interesting and very difficult problem where solutions could potentially benefit many conditions all at once.

    Laura and John Arnold have an interest in working on high-level problems like this. They directly fund projects on a personal level but the foundation focuses on using data and analysis to fix problems at system and policy levels. They aren't afraid of very big, very long-term projects.

    Does the idea for this sort of proposal make sense? Is there a CFS/ME organization that might be suitable and interested in this suggestion?
     
    Last edited: Oct 6, 2015
    alkt, sillysocks84 and Woolie like this.
  3. Simon

    Simon

    Messages:
    1,915
    Likes:
    14,405
    Monmouth, UK
    Wow, this is fascinating stuff, and I'd love to blog about it if I had more energy :(.

    It's about Laura and John Arnold, a couple who made their billions in their thirties and now want to do something different. Laura Arnold was interviewed:

    Most research spending is wasted on bad studies. These billionaires want to change that. - Vox

    Asked why they didn't focus on the more obvious targets for philanthropy, such as breakthrough research projects and buildings they could literally put their name on, Arnold said:
    They've already funded Brian Nosek's famous reproducibility project, which found that most of 100 chosen psychology research studies didn't reproduce, and Stanford Center for Reproducible Neuroscience because Brain imaging research is often wrong (that almost certainly includes some of the fMRI work on mecfs too).
     
    alkt, SOC, barbc56 and 7 others like this.
  4. Snowdrop

    Snowdrop Rebel without a biscuit

    Messages:
    2,895
    Likes:
    10,072
    Yes, I thought it was an exciting development as well.

    And I have also believed from early on that developing links with other groups for advocacy or even just to support research being done for other issues would be to our benefit as well. Finding and highlighting research that's not ME specific so that the ME research community is aware of other developments--but there's always been the problem of any study being crap science.

    Even something like raising awareness of poverty and how this group is perceived by others as marginal people who deserve their fate can impact our situation which is often similar.

    I hope they are successful.
     
    Last edited: Oct 5, 2015
    alkt, barbc56 and PatJ like this.
  5. Woolie

    Woolie Gone now, hope to see you all again soon somewhere

    Messages:
    1,930
    Likes:
    14,521
    @Snowdrop, thanks for the links, very interesting!

    Plus, on one of those links you gave I found this interesting article:

    Why we've probably overestimated the benefits of psychotherapy for major depression

    This article addresses the problem of publication bias, which is the fact that studies finding significant effects are more likely to get published than those that don't find significant effects. So in other words, what you see out there in print isn't a true representative sample of the research that's been done, but rather a selective sample of the most "flattering" results.

    They tried to estimate just how big this bias is, and what would happen if we took into account all the hidden, unpublished studies. The result: the beneficial effect of therapy is probably much smaller than we've been led to believe, at least for major depression.

    Bear in mind that the article considers only publication bias. There are all sorts of other problems with psychotherapy research that might have also led to inflated claims of its effectiveness. It doesn't even start to consider these (for example, use of subjective measures, poor control conditions, selective reporting of only the "best" outcome measures, etc.).
     
    Last edited: Oct 7, 2015
    alkt, SOC, barbc56 and 8 others like this.
  6. Snowdrop

    Snowdrop Rebel without a biscuit

    Messages:
    2,895
    Likes:
    10,072
    Yes, I came across the depression one too on twitter and I agree with your post. It boggles the mind how people can acquire an advanced degree and not have a clue about how to design a study to yield useful data. And while I'm thinking about it...who the heck is funding all this crap?
     
    alkt, SOC, barbc56 and 1 other person like this.
  7. Snowdrop

    Snowdrop Rebel without a biscuit

    Messages:
    2,895
    Likes:
    10,072
    alkt, SOC, barbc56 and 1 other person like this.
  8. Woolie

    Woolie Gone now, hope to see you all again soon somewhere

    Messages:
    1,930
    Likes:
    14,521
    Great stuff, @Snowdrop.
    Wish they'd analyse doctors' biases in making clinical diagnoses more.
     
    alkt, SOC and barbc56 like this.
  9. barbc56

    barbc56 Senior Member

    Messages:
    3,637
    Likes:
    4,992
    While I don't have a doubt that a lot of studies aren't valid, I wonder what kind of studies are taken into consideration?

    A preliminary study that shows a positive result may not bear out when you go to the next phase of testing. That's just part of the process of proving a hypothesis.This is why we often see in the news that x is healthy only to be replaced or negated later. The media picks up the results of these studies but don't necessarily realize or state that the resuls are just preliminary. In that case the money isn't wasted.

    When we see studies where the actual design or interpretation is flawed, it's a different story.

    I need to read more of the links in this thread plus review Ionides as I just don't remember at the moment.

    Great thread @Snowdrop. This looks hopeful.

    Barb
     
    Last edited: Oct 8, 2015
    PatJ, Woolie and alkt like this.
  10. Snowdrop

    Snowdrop Rebel without a biscuit

    Messages:
    2,895
    Likes:
    10,072
    Such a document exists if I understand what you're getting at. In my signature is a link to a doc by Mary Dimmock. It's long and thorough and is about the history and politics of ME. :)
     
    PatJ and alkt like this.
  11. SOC

    SOC

    Messages:
    7,802
    Likes:
    16,459
    Amen. I find myself frequently saying (often to an empty room :oops:), "Where the heck did these people go to college? We had courses in design of experiments and statistical analysis as undergraduates. How could they get through graduate school without a clue about designing rigorous, useful, logical research studies?"

    It simply makes no sense to me that any self-respecting university would graduate science students without a clue how to design a scientific study and correctly analyze and report the data. And yet they do... in droves, apparently.
     
    Snowdrop and Helen like this.

See more popular forum discussions.

Share This Page