The PACE Trial Invalidates the Use of CBT & GET in ME/CFS: A Review

[Sean]

and the number of patients in receipt of income protection or private pensions had actually doubled in the CBT and GET groups [45]

Didn't know that. Or had forgotten it.

A stark finding that, together with the increase in the number of patients in the CBT and GET arms claiming state sick pay and disability benefits, completely refutes any notion of meaningful real-world recovery.

 

[alex3619]

I don't think there is any evidence to show why so few participants did the follow-up step test and walking test. They may have been harmed by the "treatments", they may have gotten sicker because that is the natural course of the disease for many of us, or there may be other reasons for not doing the follow-up tests.

Which is why we need the data. This is yet another area with poor methodology that could hide the important issues. Generally its considered suspect if too many drop out of anything in the study. You cannot simply presume or assume its without relevance to the study itself.

 

[Dolphin]

Didn't know that. Or had forgotten it.

A stark finding that, together with the increase in the number of patients in the CBT and GET arms claiming state sick pay and disability benefits, completely refutes any notion of meaningful real-world recovery.

From: http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0040808

 

[Dolphin]

Yet as concluded by Wood et al. [19], “In trials with subjectively assessed outcomes lack of adequate allocation concealment or of blinding tend to produce overoptimistic estimates of the effect of interventions”

 

[Dolphin]

However, Evans [22] also noted that “sometimes new information may come to light that could merit changes to endpoints during the course of a trial” and “Such changes can allow incorporation of up-to-date knowledge into the trial design. However,” they “can also compromise the scientific integrity of a trial” [22]. But “Changes in long-term trials” should be considered “as medical knowledge evolves or when assumptions made in design of the trial appear questionable” but only if the decision “to modify an endpoint” is made “independent of the data obtained from the trial to date,” and only an “external advisory committee that has not reviewed data from the trial” should make those changes. It is “not appropriate” that “decision makers” (i.e., “study sponsors, investigators, and DMCs” who may have “impressions” “of the trial to date,” which “may influence decisions regarding changes in endpoints”) make endpoint “revisions during the trial” [22].

 

[Dolphin]

Quibble:
I'm not convinced that most patients with severe ME would give themselves a score of just 18 on the Chalder fatigue questionnaire (Likert scoring) like he did

 

[Dolphin]

Mistake:

He is under the impression that they changed the entry for the criteria on the Chalder fatigue questionnaire to using the Likert scoring from the bimodal score of 6 or more but this is not correct:

Table 1 shows that I score 4 on the binary and 18 on the Likert scale. My binary score indicates that I would have been ineligible, and therefore not ill enough, to enter the ME/CFS PACE trial, despite having severe ME, but the Likert score indicates that I would have been eligible for entry. However, one cannot be ill enough and not ill enough simultaneously. A Likert score of ≤ 18 was also one of the recovery criteria [11]; therefore, without having received any treatment, I would also be classed as recovered on the Chalder Fatigue Questionnaire even though my medical situation has not changed and I remain bedridden and dependent on others.

Same error here:

Oxford criteria
“To satisfy the third criterion for severity of fatigue and disability, participants had to meet trial entry thresholds for fatigue (a binary score of ≥ 6 out of 11 on the CFQ) and abnormal levels of physical function (a score of ≤ 65 out of 100 on the SF-36 physical function subscale)” [5]. However, the original score required for the SF-36 was “a score of 60 of 100 or less” [11]. And the bimodal Chalder Fatigue Questionnaire score of ≥ 6 out of 11 was changed to a Likert scoring of 18 or more, therefore, the Oxford Criteria were also changed during the trial.

And here:

As mentioned previously and shown in Table 1, my bimodal Chalder Fatigue Score indicated that even though I am bedridden and dependent on others, I would not have been eligible for the trial. During the trial, the scoring system was changed, and without changing my answers, I suddenly scored the minimum (of 18) points required to enter the trial. Yet with the same Likert score of 18, and without having received any treatment or any change to my medical situation, I was also classed as recovered, despite remaining bedridden. Is that a conservative definition of recovery?

And here:

Yet even though I am bedridden and dependent on others, I fulfill the single criterion for recovery using the Chalder Fatigue Questionnaire, the same score also classified me as sufficiently ill to enter the trial; however, if a patient fulfills only one of several criteria for recovery, the patient has not recovered, regardless of the definition of recovery.

 

[Dolphin]

He points out that improvements on the fatigue questionnaire could be the due to improvements in comorbid psychiatric conditions/depression:

Furthermore, I did not score many points, despite having severe ME, because too many items were depression related, which is irrelevant in ME/CFS. Yet it is not surprising, as the scale, which does not provide a comprehensive reflection of fatigue-related severity, symptomology, or functional disability in ME/CFS [39], was developed by mental health professionals [40]. This also means that improvements could simply be improvements in comorbid psychiatric disorders, present in 47% of trial participants [11], emphasizing the issues involved in failing to exclude people with psychiatric disorders. The National Institute of Health therefore concluded that “The Oxford Criteria…are flawed and include people with other conditions, confounding the ability to interpret the science” [41] and that “continuing to use the Oxford definition may impair progress and cause harm” and “we recommend that the Oxford definition be retired” [41,42].

 

[Dolphin]

Small mistake: he should have said "minimal clinically important difference" in the last sentence here rather than "none of these improvements reached statistical significance".

The 6MWT provides a good representation of the patient’s ability to perform submaximal activities of daily living and may be useful in serial evaluation of patient status and/or response to therapeutic interventions [47]. The results of the 6MWT showed improvements with all 4 treatments. The GET group improved most and walked 379 m within 6 min, with an adjusted difference of 35 m beyond that of the SMC group after 52 weeks. The CBT group walked 354 m, which was an improvement of 4 m beyond that of the APT group but 1.5 m less than that of the SMC group [11]. None of these improvements reached statistical significance, for which an increase of at least 86 m was required [48].


Wise RA, Brown CD (2005) Minimal clinically important differences in the six-minute walk test and the incremental shuttle walking test. COPD 2: 125-129.

 

[Dolphin]

As soon as therapists tailor exercise programs to patients’ symptoms GET is not GET anymore but has become a form of pacing.

I think this is a good point.

The protocol in the PACE Trial says what participants are asked to do is determined by “their planned physical activity, and not their symptoms” (p.20); similarly, “a central concept of GET is to MAINTAIN exercise as much as possible during a CFS/ME setback” (p.51) and “if the participant reports an increase in fatigue as a response to a new level of exercise, they should be encouraged to remain at the same level for an extra week or more” (p.66).

 

[Dolphin]

Small quibble:

I'm not convinced that the adverse events noted in most people in the APT and SMC-only groups are due to the exercise tests. The trial was over 12 months and people could have experienced increased symptoms from all sorts of things.

It is particularly interesting to note that these numbers are also very high in the adaptive pacing group, in which patients were told to remain within their limits, and the SMC group, which did not involve any exercise, so that very low rates of adverse events would be expected in both groups. The very high proportions of adverse events in these 2 groups suggest that many individuals experienced difficulty completing the step test and 6MWT as expected in patients with ME/CFS, who experience abnormal responses to exercise, as highlighted by 2-day exercise testing [66,67], abnormal gene expression and immunity following exercise [68,69],

 

[Dolphin]

Small quibble:


"Withdraw from the study" has a specific meaning. Not doing the 6 minute walking test or step test is not the same as withdrawing from the study.


I'm not sure what he means by "these 29% and 38% constitute the tip of the iceberg". I'm guessing he might be building on his earlier point that most or all of the adverse events were due to the 6 minute walking test/step test, but we don't have evidence either way for this I think.

The Supplement to the secondary mediation analysis shows that 90% of patients in the APT group completed the primary outcome questionnaires at follow up, but only 71% performed the 6MWT, and only 62% completed the step test [57]; however, the trial reported that only “33 (5%) of 640 participants were lost to follow-up, but rates did not differ between groups” [11] and “by 52 weeks, only 33 (5%) were missing primary outcome data, with no significant difference between treatment groups” [5]. This suggests that 29% and 38% of the participants, respectively, withdrew from the study because the first step test and 6MWT exacerbated their symptoms, and as 93% reported adverse events, it suggests that these 29% and 38% constitute the tip of the iceberg.

 

[Dolphin]

Quibble:


I don't think much of what he has said proves that the recovery score should be 100 specifically (see last sentence).

The recovery article states that “We changed our original protocol’s threshold score for being within a normal range on this measure from a score of ≥ 85 to a lower score as that threshold would mean that approximately half the general working age population would fall outside the normal range. The mean (S.D.) scores for a demographically representative English adult population were 86.3 (22.5) for males and 81.8 (25.7) for females. We derived a mean (S.D.) score of 84 (24) for the whole sample, giving a normal range of 60 or above for physical function” [5].


Having recovered differs from being within the normal range, and the PACE trial did not examine the “demographically representative English adult population.” Furthermore Bowling et al. [72] did not involve the “general working age population,” as the participants ages ranged from 16 to 85, with 28.6% aged 65 years or older. The PACE trial cohort’s mean age was 38 years [11] and therefore participants should be compared to healthy adults of the same age. According to the survey by Bowling et al. [72], which the PACE trial used, the mean physical functioning score for adults aged 35–44 years was 93.3, with a standard deviation of 13.4 [72] providing a normal score of ≥ 79.9, rather than ≥ 60, for physical functioning.


However, SF-36 physical functioning scores for healthy 38 year olds are not normally distributed but skewed to the right, with nearly everyone in the maximum range; and according to the BMJ’s statistical resources for readers, standard deviations will then be grossly inflated, are not a good measure of variability anymore and are therefore inappropriate for use [73], as highlighted by the fact that, Bowling et al.’s [72] survey of 2,000 patients, 1,200 and 400 had median scores of 100 and 90, respectively. The SF-36 physical functioning score decreases with age and ill health, almost 600 patients (28.6%) were aged 65 years or older, and 22% and 16% had chronic and acute health problems, respectively; therefore, the SF-36 physical functioning recovery score should have been 100 rather than 60 or more.

 

[Dolphin]

I don't follow the logic that leads up to the claim in the last sentence "highlighting the fact that recovery from the current episode of illness does not necessarily constitute recovery".

The recovery article also stated, “it is important to note that recovery does not mean being free of all symptoms” [5]. However, if the symptoms of a disease remain, than it is an overly optimistic definition of recovery, including ill people. The following statement acknowledges this: “The prevalence of the case-level international (CDC) definition of CFS may have been inaccurate because we only examined for accompanying symptoms in the previous week, not the previous 6 months” [5], as most symptoms that have been present for less than a week are self-limiting and symptoms that have been there for 6 months or more are not. “We therefore use the term ‘recovery’ in this paper to mean recovery from the current episode of the illness” [5]. According to this, instead of being able to walk 5 or 6 yards to the toilet twice daily, I would be able to do so 3 times per day and otherwise remain bedridden highlighting the fact that recovery from the current episode of illness does not necessarily constitute recovery.

 

[Dolphin]

The last sentence in this section is not correct:

The London criteria


“Specifically, these criteria included... no ‘primary’ depressive illness and no anxiety disorder present (which we interpreted as no co-morbid mood disorder of any kind)” [5]. However, these criteria also state that “the existence of a parallel disease would be grounds for disqualification” [35], which was omitted from the version used in the trial, without the knowledge of the original authors according to Goudsmit and Shepherd [36,37], 2 of the original authors. All psychiatric diseases should therefore have been excluded which did not occur.

In the Lancet paper on the PACE Trial (by White et al (2011)), it said:

"Participants were also assessed by international criteria for chronic fatigue syndrome,12 requiring four or more accompanying symptoms, and the London criteria13 for myalgic encephalomyelitis (version 2), requiring postexertional fatigue, poor memory and concentration, symptoms that fluctuate, and no primary depressive or anxiety disorder (interpreted as an absence of any such disorder)."

I was not 100% sure what "no primary depressive or anxiety disorder
(interpreted as an absence of any such disorder)" meant in relation to the
percentage we were given for "any psychiatric disorder" i.e. could there be
an overlap.

The following is an extract of a letter that clarifies it (see asterisked
bit) - the letter was written by PD White, KA Goldsmith, AL Johnson, R
Walwyn, HL Baber, T Chalder, M Sharpe, on behalf of all the co-authors (of
the PACE Trial)

---------

The trial did not study ME/CFS (pages 12-18)

The selection of patients was for CFS operationalised using the broadest criteria (the Oxford criteria). No sensible neurologist would apply the diagnosis of CFS (or indeed ME) to patients who had "proven organic brain disease", such as Parkinson's disease. For the purposes of this trial ME was not regarded as a "proven organic brain disease". In order to ensure balance between the trial arms in those participants who met alternative criteria for CFS and ME, randomisation was stratified by the International (Centers for Disease Control) criteria (which require additional symptoms) and by the London ME criteria (based on Melvin Ramsay's original description, and which excludes co-existing "primary" psychiatric disorders [****which we interpreted as any psychiatric disorder****] and emphasises post-exertional fatigue). We were provided with the second revised version of the London ME criteria; we did not invent our own. We considered use of the Canadian criteria for ME but we found it impossible to operationalise them adequately for research purposes; to our knowledge they have not been used in a major research trial. We studied the results for differently defined subgroups and they were similar to those in the entire group.

(source: http://www.meactionuk.org.uk/whitereply.htm )

 

[Dolphin]

He has a point that the CGI change score doesn't really measure recovery. But he doesn't really seem to say why a score of 2 is not suitable


The self-rated CGI change score

“We considered scores of 1 (‘very much better’) or 2 (‘much better’) as evidence of the process of recovery, rather than our original protocol threshold of a score of 1 only, because we considered that participants rating their overall health as ‘much better’ represented the process of recovery” [5]. However, everyone who’s had a flu-like illness knows that, while one feels much better during convalescence, relative to the day upon which one fell ill, one is in the process of recovery but has not recovered yet which highlights the fact that the self-rated CGI change score is a global measure of change and clinical progress [75], rather than a measure of recovery, and is also “unreliable and too general to measure... treatment responses validly” [76].

 

[Dolphin]

The recovery article also stated that “Although it seemed that slightly smaller proportions had recovered from the illness as a whole, when the criterion ‘not meeting the London criteria for ME’ was applied, we found that the differences were due to missing data rather than to change in recovery status” [5]. But how did they know this if the data were missing?

They could know it if no one was excluded because they didn't satisfied the London criteria

 

[Dolphin]

Not sure that the last sentence is true. Just because exercise increases symptoms doesn't necessarily mean that avoiding exercise is the appropriate course of action (though I think caution is certainly warranted in ME/CFS). Though what the PACE trial authors said in their BMJ e-letter is somewhat annoying.

In January 2015, the secondary mediation analysis was published, and the authors stated that “Fear avoidance beliefs are characterised by fears that activity or exercise will make symptoms worse” [43] and concluded that “Our main finding was that fear avoidance beliefs were the strongest mediator for both CBT and GET. Changes in both beliefs and behaviour mediated the effects of both CBT and GET, but more so for GET. The results support a treatment model in which both beliefs and behaviour play a part in perpetuating fatigue and disability in chronic fatigue syndrome” [43]. Yet in a rapid response in the BMJ two weeks later, the authors stated, “nor did we say that fear of exercise in CFS was “irrational”” and ME/CFS is “an illness where exercise increases symptoms” [77], thereby acknowledging that avoiding exercise is the appropriate course of action.

 

[Dolphin]

“The main finding of this long-term follow-up study of the PACE trial participants is that the beneficial effects of the rehabilitative CBT and GET therapies on fatigue and physical functioning observed at the final 1 year outcome of the trial were maintained at long-term follow-up 2•5 years from randomisation” [44]. However, this was not the main finding which was that there was no difference in efficacy between treatments and none of them were effective [44].

 

[Dolphin]

With respect to the definition of recovery, the PACE trial authors stated, “it is important to note that recovery does not mean being free of all symptoms” [5]. In other words, “recovery” was synonymous with “no recovery”. The long-term follow-up study showed that patients in all 4 groups still fulfilled the entry criteria for fatigue and physical functioning, the 2 primary outcomes, following effective treatment [44]. So that, similar to the above-mentioned objective outcomes, both primary outcomes showed that patients remained sufficiently ill to re-enter the trial. At long-term follow up, no differences were found between the 4 treatments, which is known as a null effect, and none of the treatments were effective [44]. And as noted by Ioannidis “investigators working in any field are likely to resist accepting that the whole field in which they have spent their careers is a “null field”” [88]; which might explain the authors’ reluctance to release trial data.

Just because the average results were poor doesn't tell us much if anything about the individual results