Severe ME Day of Understanding and Remembrance: Aug. 8, 2017
Determined to paper the Internet with articles about ME, Jody Smith brings some additional focus to Severe Myalgic Encephalomyelitis Day of Understanding and Remembrance on Aug. 8, 2017 ...
Discuss the article on the Forums.

New criticism of PACE

Discussion in 'General ME/CFS News' started by JohntheJack, Feb 23, 2017.

  1. JohntheJack

    JohntheJack Senior Member

    Messages:
    185
    Likes:
    967
    Swansea, UK
    Yogi, Sea, Woolie and 37 others like this.
  2. adreno

    adreno PR activist

    Messages:
    4,841
    Likes:
    11,024
    Wow. A devastating critique.
     
    Yogi, trishrhymes, GodGenghis and 4 others like this.
  3. shannah

    shannah Senior Member

    Messages:
    1,366
    Likes:
    1,638
  4. A.B.

    A.B. Senior Member

    Messages:
    3,750
    Likes:
    23,190
    Great to see another professional picking up this story and agreeing that PACE is rubbish.
     
    Binkie4, Yogi, Tyto alba and 14 others like this.
  5. Dolphin

    Dolphin Senior Member

    Messages:
    10,680
    Likes:
    28,207
    I have to admit finding some of it a bit dense. Maybe I tried to read it too quickly. What points do people think are good in it?
     
    Last edited: Feb 24, 2017
  6. alex3619

    alex3619 Senior Member

    Messages:
    12,527
    Likes:
    35,236
    Logan, Queensland, Australia
    Remember the short ditty from Total Recall (the Arnie version?): Recall, recall, recall.

    Sing it with me: Retract, retract, retract.

    The faults with PACE are severe, and plentiful, and while some are technically arcane, only obvious to experts, many are blatantly obvious. The entire study should be retracted. Other similar studies (though smaller) need to be examined for retraction as well.
     
  7. user9876

    user9876 Senior Member

    Messages:
    2,584
    Likes:
    18,184
    I think there are some new criticisms in there particularly around how the randomization was bad but I think some are quite technical. There seems to be an overall criticism that the next patient allocation model is not a good randomization technique. Also the use of different people doing initial assessments.

    It would be much easier to follow if written as a document rather than done as a slide set.

    There are some interesting details which don't quite come out around methodology and the bad statistics used in the trial design and bad scales. I think these are all points we have discussed but not all come out in analyses.

    The timeline near the end is good.
     
    Snow Leopard, Binkie4, MEMum and 6 others like this.
  8. anciendaze

    anciendaze Senior Member

    Messages:
    1,815
    Likes:
    4,677
    I think an important point has slipped by. The PACE authors referenced the Oxford definition, and more or less followed this in what they published, but also said the results applied to patients diagnosed by other criteria including the CCC for CFS and ICC for ME. Out of the 3158 patients referred for treatment by PCP/GP doctors, they excluded 2260. At a later date they modified entrance requirements in a way that permitted more patients to enroll in the trial from that date on.

    All this reveals that the criteria used by the PACE study were considerably different from those understood by the referring physicians. We also have the problem that Oxford with massive exclusions plus loosened requirements is not the same as Oxford criteria themselves. Detailed analysis of the course of diagnosis and exclusion, where possible, indicates this study used ad hoc criteria to define patients suitable for study that do not exactly match any published criteria.

    At this point we come to the great bait-and-switch: PACE authors then claimed their results applied to all patients diagnosed with CFS or ME by any criteria. Statements that they used strict diagnostic criteria notwithstanding, those authors repeated public statements implying their results applied to all patients considered CFS patients by GP/PCP doctors. They also repeatedly said this was a "massive study" with 641 patients, exploiting the fact that most news media would not realize that 3/4 of the patients in the study did not receive any given therapy, and no patients received the touted combination of CBT+GET.

    Coupled with the change in metrics during the study this produced a mismatch that is comical. By the PACE authors own original protocol, applied in the reanalysis paper with Wilshire, Kindlon. Matthees and McGrath, we discover that GET may have produced useful results in 2 more patients than the specialist care the authors considered a control. No competent researcher is going to claim statistically-valid results based on a couple of individuals. This low rate of benefits, by the original protocol, makes concerns about diagnostic criteria vitally important because a tiny rate of misdiagnosis could completely confound any results.

    Did the authors claim they could not have misdiagnosed 2 patients out 3158? If so, where are the published criteria they used? What evidence do they present for such a phenomenally low rate of diagnostic error about a particularly controversial illness, when some of them have published claims that other doctors are in error 30% of the time? They are implicitly claiming accuracy that pathologists examining samples under a microscope do not achieve.

    This is not merely a single error among a plethora of questionable practices, it is an internal inconsistency in the argument that patients not included in the study would also benefit from the touted therapies. You cannot simultaneously claim to apply very narrow diagnostic criteria, as shown by the exclusions, and very broad interpretation of results to patients that don't meet such narrow criteria.
     
  9. alex3619

    alex3619 Senior Member

    Messages:
    12,527
    Likes:
    35,236
    Logan, Queensland, Australia
    Yet they make such claims again and again. Its also often missed by advocates of their approach that such results cannot in any case be generalized to patients with higher levels of illness severity.
     
    Jan, Webdog, Solstice and 4 others like this.
  10. anciendaze

    anciendaze Senior Member

    Messages:
    1,815
    Likes:
    4,677
    Ah Alex, you are simply thinking about those damned patients. What about the poor British taxpayer? If these results apply only to a narrow subset of patients selected from 3158, then the reported results of these therapies are nearly irrelevant to the public health problem presented by those not covered in this research. Suggested headline based on this research: "Over 90% of CFS patients are not suitable candidates for CBT/GET. Massive public health problem remains unresolved."
     
  11. Demepivo

    Demepivo Dolores Abernathy

    Messages:
    406
    Likes:
    2,228
    JohntheJack likes this.
  12. Woolie

    Woolie Senior Member

    Messages:
    1,930
    Likes:
    14,559
    I wasn't keen on this presentation. It doesn't have a lot of focus, just tries to smear everything. It must be a Masters project or something. It looks laid out as if the author used some sort of textbook or intro chapter about "good practice", then just went down the list, looking for as many possible violations as they could.

    It also didn't feel like an objective piece, it came over as though the author was personally motivated to find as many faults as possible, without any attempt to evaluate their importance to the overall conclusions. So that will also detract from its impact.

    (but thanks for posting @JohntheJack , good to know!)
     
    Snow Leopard and Webdog like this.
  13. alex3619

    alex3619 Senior Member

    Messages:
    12,527
    Likes:
    35,236
    Logan, Queensland, Australia
    Ummm, how about the typical European, Canadian, Australian, and American taxpayers. Plus probably a lot more countries. Bad policy that costs money and does not fix problems has become a norm in much of the world, particularly in areas of medicine such as with ME. MUSes are a case in point. Non-science, poor reasoning, a complete lack of evidence, and emotional and persuasive argument ... its the epitome of economic stupidity. Not to mention irrational, non-scientific and unethical.
     
    Jan, slysaint, TiredSam and 2 others like this.
  14. Solstice

    Solstice Senior Member

    Messages:
    617
    Likes:
    2,066
    That's what you get when everyone is just managing their own shop. Here in the Netherlands you have the UWV that decides over disability payments. They get a certain budget to work with, so a lot of sick people are getting turned down. Those sick people then end up in bijstand(wellfare?) and still cost the tax payers money, they just have a lot less rights. It costs money either way and people in bijstand may get pushed over their limits looking for work and get even sicker, which ends up costing more money. But atleast the UWV stayed within budget.
     
  15. Barry53

    Barry53 Senior Member

    Messages:
    2,277
    Likes:
    13,705
    UK
    But even if that is true it would back up the notion that PACE is becoming recognised within education as a good example of bad science. That the medical professionals and scientists of tomorrow recognise it for what it is. I rather hope it is such a project.
     
    Comet and trishrhymes like this.
  16. BurnA

    BurnA Senior Member

    Messages:
    2,086
    Likes:
    9,863
    It is very interesting that no patients received the combination of GET and CBT, yet this is the recommended treatment combination, so to speak.
    How do you conclude that 3/4 patients did not receive any given therapy?
     
    Jan likes this.
  17. hinterland

    hinterland

    Messages:
    49
    Likes:
    133
    One point I've seen here popping up at 0.43 seconds, but not elsewhere, is about selection bias. This is a key point in the selection of participants for any BPS study or clinic; that essentially it is a self-selecting sample of people willing to put themselves forward and participate in these treatments. Is this likely to include individuals whose illness experience informs them that stipulated rehabilitation is harmful? No, absolutely not. Is it more likely to select for patients who suspect, or are aware of ways, that psychosocial factors influence them? Yes, I expect so.

    I was offered a place on the PACE trial. My GP handed me a thick wad of papers to read about it. I filed them away, I was already very aware of what these researchers believed about my medical condition and in particular didn't want to volunteer for a trial that included that possibility of being randomised to graded exercise therapy. At my next appointment I told my GP I probably wasn't interested. I don't think he was particularly impressed by my reply, but in that moment that's all I could manage to say about it. We didn't discuss it again.
     
    Orla, SamanthaJ, ScottTriGuy and 9 others like this.
  18. Barry53

    Barry53 Senior Member

    Messages:
    2,277
    Likes:
    13,705
    UK
    Typo?
     
  19. Yogi

    Yogi Senior Member

    Messages:
    1,131
    Likes:
    6,885
    This is difficult to fully understand but seems another mail in the coffin for pace. If possible Sonia Lee should talk through our give a written commentary to her slides as it would be useful addition to the PACE debate.
     
  20. alex3619

    alex3619 Senior Member

    Messages:
    12,527
    Likes:
    35,236
    Logan, Queensland, Australia
    I wrote a blog on this issue, called something like The Doggy Treat Model of Why it is So. Local optimization versus global failure.
     
    Solstice likes this.

See more popular forum discussions.

Share This Page