• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

New criticism of PACE

shannah

Senior Member
Messages
1,429
Sonia Youtube vide.PNG
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
Remember the short ditty from Total Recall (the Arnie version?): Recall, recall, recall.

Sing it with me: Retract, retract, retract.

The faults with PACE are severe, and plentiful, and while some are technically arcane, only obvious to experts, many are blatantly obvious. The entire study should be retracted. Other similar studies (though smaller) need to be examined for retraction as well.
 

user9876

Senior Member
Messages
4,556
I have to admit finding some of it a bit dense. Maybe I tried to read it too quickly. What what points to people think are good in it?

I think there are some new criticisms in there particularly around how the randomization was bad but I think some are quite technical. There seems to be an overall criticism that the next patient allocation model is not a good randomization technique. Also the use of different people doing initial assessments.

It would be much easier to follow if written as a document rather than done as a slide set.

There are some interesting details which don't quite come out around methodology and the bad statistics used in the trial design and bad scales. I think these are all points we have discussed but not all come out in analyses.

The timeline near the end is good.
 

anciendaze

Senior Member
Messages
1,841
I think an important point has slipped by. The PACE authors referenced the Oxford definition, and more or less followed this in what they published, but also said the results applied to patients diagnosed by other criteria including the CCC for CFS and ICC for ME. Out of the 3158 patients referred for treatment by PCP/GP doctors, they excluded 2260. At a later date they modified entrance requirements in a way that permitted more patients to enroll in the trial from that date on.

All this reveals that the criteria used by the PACE study were considerably different from those understood by the referring physicians. We also have the problem that Oxford with massive exclusions plus loosened requirements is not the same as Oxford criteria themselves. Detailed analysis of the course of diagnosis and exclusion, where possible, indicates this study used ad hoc criteria to define patients suitable for study that do not exactly match any published criteria.

At this point we come to the great bait-and-switch: PACE authors then claimed their results applied to all patients diagnosed with CFS or ME by any criteria. Statements that they used strict diagnostic criteria notwithstanding, those authors repeated public statements implying their results applied to all patients considered CFS patients by GP/PCP doctors. They also repeatedly said this was a "massive study" with 641 patients, exploiting the fact that most news media would not realize that 3/4 of the patients in the study did not receive any given therapy, and no patients received the touted combination of CBT+GET.

Coupled with the change in metrics during the study this produced a mismatch that is comical. By the PACE authors own original protocol, applied in the reanalysis paper with Wilshire, Kindlon. Matthees and McGrath, we discover that GET may have produced useful results in 2 more patients than the specialist care the authors considered a control. No competent researcher is going to claim statistically-valid results based on a couple of individuals. This low rate of benefits, by the original protocol, makes concerns about diagnostic criteria vitally important because a tiny rate of misdiagnosis could completely confound any results.

Did the authors claim they could not have misdiagnosed 2 patients out 3158? If so, where are the published criteria they used? What evidence do they present for such a phenomenally low rate of diagnostic error about a particularly controversial illness, when some of them have published claims that other doctors are in error 30% of the time? They are implicitly claiming accuracy that pathologists examining samples under a microscope do not achieve.

This is not merely a single error among a plethora of questionable practices, it is an internal inconsistency in the argument that patients not included in the study would also benefit from the touted therapies. You cannot simultaneously claim to apply very narrow diagnostic criteria, as shown by the exclusions, and very broad interpretation of results to patients that don't meet such narrow criteria.
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
You cannot simultaneously claim to apply very narrow diagnostic criteria, as shown by the exclusions, and very broad interpretation of results to patients that don't meet such narrow criteria.
Yet they make such claims again and again. Its also often missed by advocates of their approach that such results cannot in any case be generalized to patients with higher levels of illness severity.
 

anciendaze

Senior Member
Messages
1,841
Yet they make such claims again and again. Its also often missed by advocates of their approach that such results cannot in any case be generalized to patients with higher levels of illness severity.
Ah Alex, you are simply thinking about those damned patients. What about the poor British taxpayer? If these results apply only to a narrow subset of patients selected from 3158, then the reported results of these therapies are nearly irrelevant to the public health problem presented by those not covered in this research. Suggested headline based on this research: "Over 90% of CFS patients are not suitable candidates for CBT/GET. Massive public health problem remains unresolved."
 
Messages
3,263
I wasn't keen on this presentation. It doesn't have a lot of focus, just tries to smear everything. It must be a Masters project or something. It looks laid out as if the author used some sort of textbook or intro chapter about "good practice", then just went down the list, looking for as many possible violations as they could.

It also didn't feel like an objective piece, it came over as though the author was personally motivated to find as many faults as possible, without any attempt to evaluate their importance to the overall conclusions. So that will also detract from its impact.

(but thanks for posting @JohntheJack , good to know!)
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
What about the poor British taxpayer?
Ummm, how about the typical European, Canadian, Australian, and American taxpayers. Plus probably a lot more countries. Bad policy that costs money and does not fix problems has become a norm in much of the world, particularly in areas of medicine such as with ME. MUSes are a case in point. Non-science, poor reasoning, a complete lack of evidence, and emotional and persuasive argument ... its the epitome of economic stupidity. Not to mention irrational, non-scientific and unethical.
 

Solstice

Senior Member
Messages
641
Ummm, how about the typical European, Canadian, Australian, and American taxpayers. Plus probably a lot more countries. Bad policy that costs money and does not fix problems has become a norm in much of the world, particularly in areas of medicine such as with ME. MUSes are a case in point. Non-science, poor reasoning, a complete lack of evidence, and emotional and persuasive argument ... its the epitome of economic stupidity. Not to mention irrational, non-scientific and unethical.

That's what you get when everyone is just managing their own shop. Here in the Netherlands you have the UWV that decides over disability payments. They get a certain budget to work with, so a lot of sick people are getting turned down. Those sick people then end up in bijstand(wellfare?) and still cost the tax payers money, they just have a lot less rights. It costs money either way and people in bijstand may get pushed over their limits looking for work and get even sicker, which ends up costing more money. But atleast the UWV stayed within budget.
 
Messages
2,391
Location
UK
It must be a Masters project or something.
But even if that is true it would back up the notion that PACE is becoming recognised within education as a good example of bad science. That the medical professionals and scientists of tomorrow recognise it for what it is. I rather hope it is such a project.
 
Messages
2,087
They also repeatedly said this was a "massive study" with 641 patients, exploiting the fact that most news media would not realize that 3/4 of the patients in the study did not receive any given therapy, and no patients received the touted combination of CBT+GET.

It is very interesting that no patients received the combination of GET and CBT, yet this is the recommended treatment combination, so to speak.
How do you conclude that 3/4 patients did not receive any given therapy?
 
Messages
66
I have to admit finding some of it a bit dense. Maybe I tried to read it too quickly. What points do people think are good in it?
One point I've seen here popping up at 0.43 seconds, but not elsewhere, is about selection bias. This is a key point in the selection of participants for any BPS study or clinic; that essentially it is a self-selecting sample of people willing to put themselves forward and participate in these treatments. Is this likely to include individuals whose illness experience informs them that stipulated rehabilitation is harmful? No, absolutely not. Is it more likely to select for patients who suspect, or are aware of ways, that psychosocial factors influence them? Yes, I expect so.

I was offered a place on the PACE trial. My GP handed me a thick wad of papers to read about it. I filed them away, I was already very aware of what these researchers believed about my medical condition and in particular didn't want to volunteer for a trial that included that possibility of being randomised to graded exercise therapy. At my next appointment I told my GP I probably wasn't interested. I don't think he was particularly impressed by my reply, but in that moment that's all I could manage to say about it. We didn't discuss it again.
 

Yogi

Senior Member
Messages
1,132
This is difficult to fully understand but seems another mail in the coffin for pace. If possible Sonia Lee should talk through our give a written commentary to her slides as it would be useful addition to the PACE debate.