Severe ME Day of Understanding and Remembrance: Aug. 8, 2017
Determined to paper the Internet with articles about ME, Jody Smith brings some additional focus to Severe Myalgic Encephalomyelitis Day of Understanding and Remembrance on Aug. 8, 2017 ...
Discuss the article on the Forums.

David Tuller / Trial by Error: The Science Media Centre’s Desperate Efforts to Defend PACE

Discussion in 'General ME/CFS News' started by Cheshire, Aug 2, 2017.

  1. Cheshire

    Cheshire Senior Member

    Messages:
    1,121
    Likes:
    8,999
    http://www.virology.ws/2017/08/02/t...dia-centres-desperate-efforts-to-defend-pace/
     
    Binkie4, MEMum, slysaint and 40 others like this.
  2. Valentijn

    Valentijn Senior Member

    Messages:
    14,281
    Likes:
    45,814
    Good to see the SMC getting slammed. My comment:
     
  3. Esther12

    Esther12 Senior Member

    Messages:
    8,449
    Likes:
    28,522
    Thanks. So much to read right now!
     
    Binkie4, MEMum, Demepivo and 7 others like this.
  4. adreno

    adreno PR activist

    Messages:
    4,843
    Likes:
    11,020
    Another excellent - and scathing - post by Tuller.
     
    Daisymay, MEMum, Mary and 10 others like this.
  5. dangermouse

    dangermouse Senior Member

    Messages:
    406
    Likes:
    2,258
    Thanks for posting :thumbsup:
     
    MEMum, Sancar, Mary and 5 others like this.
  6. BurnA

    BurnA Senior Member

    Messages:
    2,086
    Likes:
    9,861
    Would someone mind explaining this to me and the relevance to the outcome changes?
     
  7. Valentijn

    Valentijn Senior Member

    Messages:
    14,281
    Likes:
    45,814
    It means they saw the data (and even could have analyzed it) before they made the changes, rather than declaring the thresholds beforehand. So they could have selected the thresholds which they knew would give them the results they wanted.
     
    Orla, Daisymay, snowathlete and 16 others like this.
  8. BurnA

    BurnA Senior Member

    Messages:
    2,086
    Likes:
    9,861
    But how is that established?

    If a recovery criteria requires post hoc analysis, why does changing it, imply they saw the data.

    Sorry I am just confused with the terms.

    I am trying to figure out the 'evidence' that they saw the data or at least were aware of it.
     
    MEMum and Luther Blissett like this.
  9. Valentijn

    Valentijn Senior Member

    Messages:
    14,281
    Likes:
    45,814
    If it's not declared beforehand (such as in the protocol), it's always going to be under suspicion of being done post-hoc. And in the case of the recovery data, the PACE team won't even claim that it was pre-planned or pre-approved, plus their protocol had different standards which they abandoned. So it would be naively generous to assume that the analysis wasn't post-hoc, and it's essentially up to them to prove it if it wasn't.
     
    Tally, Jill, Daisymay and 14 others like this.
  10. BurnA

    BurnA Senior Member

    Messages:
    2,086
    Likes:
    9,861
    Thanks.

    One more question, if the recovery threshold was weakened, or indeed any outcome measure, would that change be expected to apply equally to all arms of the trial?
     
    Luther Blissett and MEMum like this.
  11. Valentijn

    Valentijn Senior Member

    Messages:
    14,281
    Likes:
    45,814
    Yes, the same lowered thresholds applied to everyone. But the glory of post-hoc analysis is that researchers can play with the numbers until they find the best ones. So maybe lowering the SF36-PF to 75 doesn't help, and maybe 70 makes pacing look most effective, but 60 comes out looking great for CBT or GET.

    It might be interesting to look at the data released from the FOIA to see how other thresholds would have stacked up against the original 85 and the eventual 60, come to think of it :D
     
    Kati, Orla, Jill and 16 others like this.
  12. Keela Too

    Keela Too Sally Burch

    Messages:
    900
    Likes:
    3,985
    N.Ireland
    Good point!
     
  13. alex3619

    alex3619 Senior Member

    Messages:
    12,489
    Likes:
    35,090
    Logan, Queensland, Australia
    Here was my comment on David's blog -
     
    Kati, Jill, Luther Blissett and 24 others like this.
  14. Wonko

    Wonko Senior Member

    Messages:
    1,439
    Likes:
    4,742
    The other side.
    Never going to happen IMO but I like the sentiment.
     
    Luther Blissett, MEMum and Skycloud like this.
  15. alex3619

    alex3619 Senior Member

    Messages:
    12,489
    Likes:
    35,090
    Logan, Queensland, Australia
    What is needed and whether or not it happens are two different things. One thing I will say is that if nobody pushes for change then nothing changes.
     
    Tally, Keela Too, Jill and 12 others like this.
  16. Dolphin

    Dolphin Senior Member

    Messages:
    10,671
    Likes:
    28,178
    The normal range analyses i.e. SF-36 physical functioning score of 60 or more and Chalder Fatigue questionnaire Likert score of 18 or less were called post-hoc in the Lancet 2011 paper. That means they were calculated after seeing the data. These two normal range analyses were then used in the 2013 recovery paper as alterations to the recovery criteria in the protocol.
     
  17. SilverbladeTE

    SilverbladeTE Senior Member

    Messages:
    2,909
    Likes:
    3,562
    Somewhere near Glasgow, Scotland
    Only revolution will change thing in UK, as has been proven, time after time, "Inquiries" are a load of white wash and bollocks, veering things up.
    Whole system is involved in grotesque cronyism, nepotism, corruption, and every kind of corruption, perversion and exploitation including treasonous under funding/lunatic cutbacks of personnel of the military and essential civil infrastructure.
    If we want genuine democracy, we'll have to fight for it
     
    Luther Blissett likes this.
  18. BurnA

    BurnA Senior Member

    Messages:
    2,086
    Likes:
    9,861
    So, are you saying that when the changed the recovery definition it lead to greater results for the non SMC arms only?


    Trying to work out how the reanalysis of the recovery paper as per the original protocol, resulted in a non statistical difference between arms, when there was much more of a difference under changed protocol in the original paper.

    Another way of saying it, does the argument that lowering the recovery threshold results in more patients beings recovered, hold true - if someone could say it also means the SMC arm should have more patients recovered.

    Or are you saying the recovery threshold could have been choosen exactly to give more patients recovered in the CBT and GET arms and not the SMC arm ?

    Apologies if this doesnt make sense.
     
    Luther Blissett and MEMum like this.
  19. Londinium

    Londinium Senior Member

    Messages:
    166
    Likes:
    1,372
    In a blinded trial that would be correct: lowering the threshold for success wouldn't be sufficient in and of itself to skew the results to the treatment arm. In an unblinded trial, where some level of placebo effect is expected, lowering the success threshold might be just enough to capture the increase due to placebo effect in the treatment arm. The control arm (SMC) would not get the same placebo effect* and so lowering the success threshold may well then turn a null result into a successful trial. Reanalysis of the 'recovery' paper shows that had the trial outcome for success not been altered the difference recovery rates for all four arms would not have been statistically significant.

    *separate issue, but lots of reasons why SMC as designed would not have given same level of placebo effect.
     
  20. Esther12

    Esther12 Senior Member

    Messages:
    8,449
    Likes:
    28,522
    I most liked Tuller's response to the MRC comment, which was maybe the one which was least infuriating. He did make me feel more infuriated by it than I was though. With Macleod's comment being so completely rubbish and misguided it's hard to really go through calmly explaining what's wrong with it. It would need everything wrong with the PACE trial to be re-explained.

    I wonder if an FOI request could be lodged with Oxford to find out who signed off on the statement, and what evidence they have to support the claims made within it?

    They could have been, but we wouldn't be able to know that. The changes made it easier for those in all groups to be classed as 'recovered', but particularly favoured those groups most likely to have positively biased the self-report outcomes that were used to define recovery.
     

See more popular forum discussions.

Share This Page