May 12, 2017 Is International ME/CFS and FM Awareness Day
International ME/CFS and FM Awareness Day is May 12th, 2017. Jody Smith shares some information about upcoming events and ways you can be heard ...
Discuss the article on the Forums.

Is anyone working on the Cochrane reviews issue after PACE debunking?

Discussion in 'Action Alerts and Advocacy' started by Kati, Oct 22, 2016.

  1. Kati

    Kati Patient in training

    Messages:
    5,450
    Likes:
    19,496
    I was wondering if anybody is working on sending appeals to Cochrane reviews following the reception of the data to Alem Matthees, and I was wondering if this would not be the next logical step?

    Cochrane reviews affects how we are cared for, all around the world.

    Thanks.
     
    MEMum, Jan, barbc56 and 10 others like this.
  2. medfeb

    medfeb Senior Member

    Messages:
    469
    Likes:
    1,852
    Good idea.

    In addition to the issues with the PACE trial, I think we can also leverage the concerns raised with the use of the Oxford definition by both the NIH and the AHRQ Evidence Review and also the 2016 AHRQ Addendum which found that there was no evidence of effectiveness of GET and barely any for CBT once the Oxford studies were excluded. I have not done a side by side comparison of the AHRQ list of studies and the Cochrane list of CBT and GET studies but I would expect that the same issue exists for the Cochrane reviews as well.

    PACE will claim they also characterized by Fukuda and London but at least for Fukuda, they acknowledged they modified the criteria to only require symptoms for one week and that that could affect who was selected. I believe there are similar issues for London ME criteria.
     
    MEMum, Jan, A.B. and 8 others like this.
  3. shannah

    shannah Senior Member

    Messages:
    1,343
    Likes:
    1,578
    There was talk of someone tackling this back in August when the Tribunal came out against PACE. I can't remember the outcome of the discussion though.
     
    MEMum, barbc56 and Sean like this.
  4. Esther12

    Esther12 Senior Member

    Messages:
    8,449
    Likes:
    28,509
    Dolphin, barbc56, Hutan and 1 other person like this.
  5. Kati

    Kati Patient in training

    Messages:
    5,450
    Likes:
    19,496
    barbc56 and Esther12 like this.
  6. Esther12

    Esther12 Senior Member

    Messages:
    8,449
    Likes:
    28,509
    Yes - whoops. Meant to make that clear in my post, but forgot about that while I was finding the link!
     
    barbc56 likes this.
  7. barbc56

    barbc56 Senior Member

    Messages:
    3,633
    Likes:
    4,966
    Interesting question. I need to refresh my memory how Cochrain works. Is it updated as needed or do you go to the report with the latest date? I think it might be the latter but not sure.
     
  8. Esther12

    Esther12 Senior Member

    Messages:
    8,449
    Likes:
    28,509
    I think that they only do big updates of the data every 5+ years or so. The last 'update' was just a response to two of the comments submitted.
     
    Dolphin and barbc56 like this.
  9. barbc56

    barbc56 Senior Member

    Messages:
    3,633
    Likes:
    4,966
    Thanks Esther. I did find this site which might provide more information but I haven't had time to look through it.

    http://www.cochrane.org/
     
    Esther12 likes this.
  10. Anne

    Anne Senior Member

    Messages:
    295
    Likes:
    723
  11. medfeb

    medfeb Senior Member

    Messages:
    469
    Likes:
    1,852
    The AHRQ Addendum reanalysis evaluated the evidence including all studies including Oxford studies and then after excluding Oxford studies.

    The addendum stated:
    "This addendum has delineated differences in treatment effectiveness and harms according to case definitions, highlighting studies that used the Oxford (Sharpe, 1991) case definition and how these studies impacted our conclusions.... Our sensitivity analysis would result in a downgrading of our strength of evidence on several outcomes which can be attributed to the decrease in power, dominance of one large trial, or lack of trials using criteria other than the Oxford (Sharpe, 1991) case definition for inclusion. Blatantly missing from this body of literature are trials evaluating effectiveness of interventions in the treatment of individuals meeting case definitions for ME or ME/CFS. "

    Examples of the findings for some of the specific measures are below.

    In both the initial analysis and the addendum, PACE was evaluated as a good study with no evidence of bias. So this reassessment is not based on a reevaluation of PACE - only on excluding the Oxford studies from the evidence review.
    The Addendum gives the following justification for excluding Oxford from the reanalysis
    "The Oxford (Sharpe, 1991) case definition is the least specific of the definitions and less generalizable to the broader population of patients with ME/CFS. It could identify individuals who have had 6 months of unexplained fatigue with physical and mental impairment, but no other specific features of ME/CFS such as post-exertional malaise which is considered by many to be a hallmark symptom of the disease.3 As a result, using the Oxford case definition results in a high risk of including patients who may have an alternate fatiguing illness or whose illness resolves spontaneously with time. In light of this, we recommended in our report that future intervention studies use a single agreed upon case definition, other than the Oxford (Sharpe, 1991) case definition...The National Institute of Health (NIH) panel assembled to review evidence presented at the NIH Pathways to Prevention Workshop agreed with our recommendation, stating that the continued use of the Oxford (Sharpe, 1991) case definition “may impair progress and cause harm.”

    I would think the same issue applies with the Cochrane reviews although I have not compared the studies in the AHRQ evidence review to those in the Cochrane review.

    Details from the review.
    CBT - evidence for function improvement: Unlike the positive results of the Oxford based trials, the results of the trials fulfilling the CDC criteria are mixed and would provide insufficient evidence to determine the effectiveness of CBT

    CBT - evidence for fatigue improvement: The overall analyses of fatigue outcomes, including the single study using Oxford case definition inclusion criteria, provided low strength of evidence that CBT improves fatigue. In removing the Oxford case definition based study, we are left with four fair-quality studies, three finding benefit (n=327) and one finding no benefit (n=65), and one poor-quality study finding no benefit (n=58). The results are generally consistent with the overall conclusion and would provide low strength of evidence that CBT improves fatigue.

    GET - evidence for improvement - The results are consistent across trials with improvement in function, fatigue, and global improvement and provided moderate strength of evidence for improved function and global improvement, low strength of evidence for reduced fatigue and decreased work impairment, and insufficient evidence for improved quality of life (no trials) (Table 7). By excluding the three trials using the Oxford (Sharpe, 1991) case definition for inclusion, there would be insufficient evidence of the effectiveness of GET on any outcome. .
     
    actup, mango, Hutan and 5 others like this.
  12. TillyMoments

    TillyMoments

    Messages:
    18
    Likes:
    80
    Invisible Woman, actup and slysaint like this.
  13. user9876

    user9876 Senior Member

    Messages:
    2,582
    Likes:
    18,162
    moderated with many others

    This is an update to the previous review where they fail to answer more of the points raised by @seaturtle

    This analysis should be doing some real harm to Cochrane's reputation. Not only are they rating PACE as a good trial and ignoring the outcome switching they are also doing their own outcome switching to promote their favoured results.

    There stats are all wrong as well as mean differences is not a valid measure for the CFQ but I'm not sure they are capable of understanding that their assumptions about the scale being an interval scale are simply wrong.

    There should be a different review coming out at some point involving the individual patient data. But I wouldn't hold your breath they published the protocol sometime ago so they are either very slow or the results didn't say what they wanted. Before the tribunal QMUL pointed to this saying 'independent' groups had access to the data and claiming they supported the PACE results even though nothing had been published. During the tribunal they had to drop their claim of 'independent' as White and the other PACE PIs were involved in the design of this individual patient protocol. This is where White (QMUL really) funded one protocol design meeting.
     
  14. Esther12

    Esther12 Senior Member

    Messages:
    8,449
    Likes:
    28,509
    More briefly on this - White didn't provide funding for the review just published, but did for a meeting about the forthcoming Cochrane Individual Patient Data review.

    PS: It does seem that White is having to rely ever more on appeals to the authority of Cochrane as PACE is getting ripped apart. The Cochrane Common Mental Health disorder group are causing major problems, and their refusal to engage with the concerns raised by patients is very worrying. Not sure what we can do about it. The comments by Courtney/@seaturtle were about as good as we could hope for and I don't know how to push them to make the review authors take them more seriously.
     
    Last edited: Jan 3, 2017
  15. TillyMoments

    TillyMoments

    Messages:
    18
    Likes:
    80
    Thank you that is helpful to understand the big picture of the landscape they have painted. I must say it feels as if we are fly's in a web. Can anyone see these threads as it seems that they stay one step ahead and in control? What sort of reputation do Cochrane have? Sorry to be so green to it all.
     
    Invisible Woman and Esther12 like this.
  16. Esther12

    Esther12 Senior Member

    Messages:
    8,449
    Likes:
    28,509
    So widely respected that many see reading the abstract of a Cochrane review as being synonymous with critically examining the evidence!
     
  17. TillyMoments

    TillyMoments

    Messages:
    18
    Likes:
    80
    FLUFF :-( I did find this and wondered if this could be used? http://bmjopenbeta.bmj.com/content/6/4/e010606.long
     
    Esther12 likes this.
  18. Snow Leopard

    Snow Leopard Hibernating

    Messages:
    4,613
    Likes:
    12,426
    South Australia
    *cough* Especially on Wikipedia, when quotations from the abstract are taken totally out of context *cough*
     
    Esther12 and Invisible Woman like this.
  19. JayS

    JayS Senior Member

    Messages:
    194
    Likes:
    534
    Cochrane certainly 'outranks' the IOM report, that's for sure.
     

See more popular forum discussions.

Share This Page