New paper on FINE in PLOS: "Therapist Effects and the Impact of Early Therapeutic Alliance"

[Dolphin]

In case there is any doubt about what pragmatic rehabilitation is about:

Pragmatic rehabilitation was first reported in 2001 [31]. It is an educational self-help treatment for CFS/ME which starts by providing patients with a detailed explanatory model for their symptoms, supported by a patient manual containing references to the evidence base. While pragmatic rehabilitation has aspects in common with cognitive behavioural therapy and graded exercise therapy, unlike those therapies, it was designed to be delivered by non-specialists who have been taught the explanatory model. After the presentation of the model, the programme of rehabilitation is worked out collaboratively by the patient and the therapist, and is tailored to the patient’s specific goals, but it always includes a programme of graded increases in activity.

 

[Dolphin]

ambulatory status (whether or not the patient used a mobility aid on most days)

I don't think this is the best definition. Some people could be virtually bedbound but have no mobility aids. And indeed there were people with SF-36 physical functioning scores of 0 (!) who were classed as ambulatory while another person was classed as "non-ambulatory" with a SF-36 physical functioning score of 40.

 

[Dolphin]

In the SL arm, the mean number of sessions was 9.5 (S.D. 0.8), and in the PR arm, the mean number of sessions was 9.6 (S.D. 0.9) [5]. This reduces the effect of noncompliance from the randomized protocol in this analysis.

This was out a maximum of 10 sessions from a quick check of the protocol:

Pragmatic rehabilitation and supportive listening treatments each consist of five face-to-face sessions delivered in the patient's home (on weeks 1, 2, 4, 10 and 19), interspersed with five 30-minute telephone calls to the patient (on weeks 3, 6, 8, 12 and 15), in accordance with a standard operating protocol. The first face-to-face session lasts approximately 90 minutes, while the other four face to face sessions last approximately 60 minutes. The telephone calls last a maximum of 30 minutes. A record is kept of the dates on which treatment sessions are delivered, and of the length of each session.

 

[Dolphin]

(Small point):

In the FINE trial, the patient-rated early therapeutic alliance scores were, on average, higher when participants received PR (which has similarities to CBT), than when they received SL. However, this effect was only significant for therapist three*. For therapists one and two, the differences in the mean alliance formed when delivering PR compared to SL were small. Comparison to other studies [2, 23, 27] suggested that the therapeutic alliance scores were typical for this patient group; meaning the results of the present study cannot be explained by unusually high or low alliances being formed.

*They don't give overall figures to see if combined there is a difference, that I recall.

 

[Dolphin]

A limitation of the study is that the conclusion is potentially only valid for the range of recorded alliance levels, and that alliance was measured at the end of the first therapy session, which may have been too early for the therapeutic relationship to have formed.

It is a bit odd all right to use scores after just one session.

 

[Dolphin]

I'm not sure it is mentioned anywhere but there looks to be more missing data in the PR group versus Supportive Listening for the two outcome measures, Chalder Fatigue scale and SF-36 physical functioning.

This would be relevant to the results of the FINE Trial in general.

 

[Dolphin]

The table of individual scores shows some 11 scores. 11 is a score a healthy person would typically give. Scores of less than 11 give the impression people have less fatigue than healthy people. Four people give scores of 0 (though they are spread even between PR and SL) at 70 weeks.

A fifth person had a score of 0 (likert) at 20 weeks (pragmatic rehabilitation) but were at 19 (likert)/9 bimodal at 70 weeks.

 

[Dolphin]

Odd score:
#29:
(20 weeks)
Chalder Fatigue scale (0123): 15 ("back to normal" according to PACE as <=18)
yet
SF-36 PF: 0

 

[Dolphin]

Some deteriorations:
SF-36 physical functioning:

SL
#33: 90 (20 weeks) --> 70 (70 weeks)
#42: 75 (20 weeks) --> 15 (70 weeks)
#55: 75 (20 weeks) --> 15 (70 weeks)
#55: 35 (20 weeks) --> 15 (70 weeks)

#49:45 (baseline) --> 0 (20 weeks) --> 15 (70 weeks)

 

[Dolphin]

we require that patients are sufficiently functionally impaired to score 70% or less on the SF-36 physical functioning scale [35].

http://bmcmedicine.biomedcentral.com/articles/10.1186/1741-7015-4-9

Row #47 (number 24) was 75 at baseline i.e should have been excluded.

 

[Dolphin]

Some deteriorations:
SF-36 physical functioning:

Pragmatic Rehabilitation:
#130: 35 (baseline) --> 55 (20 weeks) --> 25 (70 weeks)
#131: 30 (baseline) --> 30 (20 weeks) --> 10 (70 weeks)

 

[Sam Carter]

PACE trial post-hoc recovery thresholds increase the recovery rate by a factor of 6.

18 FINE participants met PACE trial post-hoc recovery thresholds at week 20 (SF36 ≥ 60 and CFQ likert ≤ 18) compared to only 3 participants who met the original recovery thresholds (SF36 ≥ 85 and CFQ bimodal ≤ 3).

"Recovery" is usually not sustained.

By week 70, 12 out of the original 18 had relapsed such that they no longer met recovery thresholds.

Bimodal and likert scores are often contradictory.

Of the 33 participants who met the PACE trial recovery threshold for fatigue at week 20 (likert CFQ score ≤ 18), 10 had a bimodal CFQ score ≥ 6 so would still be fatigued enough to enter the PACE trial, and 16 had a bimodal CFQ score ≥ 4 (the accepted definition of abnormal fatigue).

So, for this cohort, if a person met the recovery threshold for fatigue at week 20 they had (roughly) a 50% chance of still having abnormal levels of fatigue, and a 30% chance of being fatigued enough to enter the PACE trial.

Some participants were confused by the wording of the Chalder Fatigue Questionnaire.

A healthy person should have a likert score of 11 out of 33, yet 17 participants recorded a likert CFQ score of 10 or less at some point (i.e. they reported less fatigue than a healthy person), and 5 participants recorded a likert CFQ score of 0.

 

[Bob]

@Sam Carter
That's a very interesting analysis you've done of the FINE data, Sam. Thanks for that.

 

[A.B.]

A healthy person should have a likert score of 11 out of 33, yet 17 participants recorded a likert CFQ score of 10 or less at some point (i.e. they reported less fatigue than a healthy person), and 5 participants recorded a likert CFQ score of 0.

I think that might be more due to brainwashing, or desire to please the therapist for whatever reason, than being confused about wording. There's also a study of CBT for MS where the MS patients had better fatigue scores than healthy people.

Regardless, these methods need to be retired.

 

[Bob]

I think that might be more due to brainwashing, or desire to please the therapist for whatever reason, than being confused about wording.

The aberrant scores below 11 might reflect a number of issues, but the wording of the Chalder fatigue questionnaire is very confusing, IMO. I think responders could easily select "less than usual" (e.g. in response to "Do you have problems with tiredness?") thinking that 'usual' means how they usually feel rather than how they felt when they were last well. ('Usual' is supposed to refer to the last time that the patient was well.)

 

[Dolphin]

The aberrant scores below 11 might reflect a number of issues, but the wording of the Chalder fatigue questionnaire is very confusing, IMO. I think responders could easily select "less than usual" (e.g. in response to "Do you have problems with tiredness?") thinking that 'usual' means how they usually feel rather than how they felt when they were last well. ('Usual' is supposed to refer to the last time that the patient was well.)

In the PACE Trial (and probably some other trials also), it said:

We would like to know more about any problems you have had with feeling tired, weak or lacking in energy in the last month. Please answer ALL the questions by ticking the answer which applies to you most closely. If you have been feeling tired for a long while, then compare yourself to how you felt when you were last well.

evaluatingpace.phoenixrising.me/PACE_Protocol.pdf

Either way, it looks like the likert scoring system is problematic when wanting to compare to healthy people (the bimodal scoring system doesn't have this problem as "less than usual" and "no more than usual" have the same scores).

 

[user9876]

There is seems to be something very strange with having two scoring methods for the Chandler Fatigue Questionnaire. I'm hoping that its not due to my data conversion from the sata format to a spreadsheet.

Patient #106
SF36 - 25, 35, 20
CFQ_binomial 10, 10, 11
CFQ_Likert 27, 26, 24

Changes
SF +10, -15 (Overall -5)
CFQ_binomial 0, +1 (overall +1)
CFQ_likert -1, -2, (Overall -3)

Hence someone got worse on the sf36 scale and both improved and got worse on the CFQ scale with two different marking schemes. (worse on the binomial and better using the likert scoring)

Its not the only one
Patient #207
SF36 - 45, 75, 60
CFQ_binomial 7, 8, 9
CFQ_Likert 24, 20, 21

Changes
SF +30, -15 (Overall 15)
CFQ_binomial 1, +1 (overall +2)
CFQ_likert -4, +1, (Overall -3)

In this example someone improved overall with the sf36 scale but they got worse with the binomial scale and better on the likert scale.

This suggests that there is an issue with the CFQ measuring both improvements and worsening at the same time. Perhaps one or the other marking scheme is ok but which one - I think most of the 'validation' was done using the binomial scoring. It certainly suggests that both cannot be ordinal or interval scales hence questions should be asked about the way stats are quoted for the CFQ with both the FINE and PACE trials. PACE justified their protocol changes of moving from binomial scoring to get more sensitivity

Before outcome data were examined, we changed the original bimodal scoring of the Chalder fatigue questionnaire (range 0–11) to Likert scoring to more sensitively test our hypotheses of effectiveness.

However clearly this does not produce more sensitivity but different results!

 

[Woolie]

A healthy person should have a likert score of 11 out of 33, yet 17 participants recorded a likert CFQ score of 10 or less at some point (i.e. they reported less fatigue than a healthy person), and 5 participants recorded a likert CFQ score of 0.

Good point. I think this suggests that some patients may have compared their current fatigue levels not to pre-illness levels, but instead to their level at the commencement of the trial. This might have further skewed the results in favour of the authors' hypothesis (although obviously, the results still weren't that impressive - I'm saying they may have been even worse were it not for these instances)

 

[Dolphin]

From: PubMed Commons

http://www.ncbi.nlm.nih.gov/pubmed/23363640#cm23363640_14248

Sam Carter2016 Feb 15 5:23 p.m. (58 minutes ago)

Exploring changes to PACE trial outcome measures using anonymised data from the FINE trial.

When the results of the PACE trial were published (1, 2) it was noted that the primary outcome measures and the definition of "recovery" described in the trial's published protocol (3) had been abandoned and replaced with markedly less stringent criteria.

The fully anonymised data set from the FINE trial(4), considered to be the PACE trial's "sister" study, makes it possible to explore how these changes may have affected the reported efficacy of the PACE trial's interventions.

At week 20 (assessment 2), 18 FINE trial participants met PACE trial post-hoc recovery thresholds (SF36 PF ≥ 60 and CFQ Likert ≤ 18) compared to only 3 participants who met the stricter, protocol-defined recovery thresholds (SF36 PF ≥ 85 and CFQ bimodal ≤ 3). Therefore, at assessment 2, the post-hoc changes increased the "recovery" rate by a factor of 6.

By week 70 (assessment 3), between 10 and 12 of the original 18 had relapsed so that they no longer met the post-hoc recovery thresholds (data are missing for two participants). Such a high rate of relapse within a year shows that the post-hoc recovery thresholds, said to represent a "strict criterion for recovery" in a Comment (5) which accompanied the original publication of PACE trial results, are neither strict nor reliable indicators of sustained wellbeing.

Regarding the Chalder fatigue questionnaire, White et al wrote that "we changed the original bimodal scoring of the Chalder fatigue questionnaire (range 0–11) to Likert scoring to more sensitively test our hypotheses of effectiveness" (1). However, data from the FINE trial show that Likert and bimodal scores are often contradictory and thus call into question White et al's assumption that Likert scoring is necessarily more sensitive than bimodal scoring.

For example, of the 33 FINE trial participants who met the post-hoc PACE trial recovery threshold for fatigue at week 20 (Likert CFQ score ≤ 18), 10 had a bimodal CFQ score ≥ 6 so would still be fatigued enough to enter the PACE trial and 16 had a bimodal CFQ score ≥ 4 which is the accepted definition of abnormal fatigue.

Therefore, for this cohort, if a person met the PACE trial post-hoc recovery threshold for fatigue at week 20 they had approximately a 50% chance of still having abnormal levels of fatigue and a 30% chance of being fatigued enough to enter the PACE trial.

A further problem with the Chalder fatigue questionnaire is illustrated by the observation that the bimodal score and Likert score of 10 participants moved in opposite directions at consecutive assessments i.e. one scoring system
showed improvement whilst the other showed deterioration.

Moreover, it can be seen that some FINE trial participants were confused by the wording of the questionnaire itself. For example, a healthy person should have a Likert score of 11 out of 33, yet 17 participants recorded a Likert CFQ score of 10 or less at some point (i.e. they reported less fatigue than a healthy person), and 5 participants recorded a Likert CFQ score of 0.

The discordance between Likert and bimodal scores and the marked increase in those meeting post-hoc recovery thresholds suggest that White et al's deviation from their protocol-specified analysis is likely to have profoundly affected the reported efficacy of the PACE trial interventions.

An independent re-analysis of anonymised PACE trial data as described in its published protocol is urgently required to quantify the effects of the revised outcome and recovery criteria.

References
(1) White PD et al (2011) Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet Mar 5;377(9768):823-36.
(2) White PD, Goldsmith K, Johnson AL, Chalder T, Sharpe M (2013) Recovery from chronic fatigue syndrome after treatments given in the PACE trial. Psychol Med. Oct;43(10):2227-35.
(3) White PD, Sharpe MC, Chalder T, DeCesare JC, Walwyn R (2007) Protocol for the PACE trial: a randomised controlled trial of adaptive pacing, cognitive behaviour therapy, and graded exercise, as supplements to standardised specialist medical care versus standardised specialist medical care alone for patients with the chronic fatigue syndrome/myalgic encephalomyelitis or encephalopathy. BMC Neurol Mar 8;7:6.
(4) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4685991/bin/pone.0144623.s002.dta
(5) Bleijenberg G, Knoop H. (2011) Chronic fatigue syndrome: where to PACE from here? Lancet. Mar 5;377(9768):786-8.

 

[Sean]

A further problem with the Chalder fatigue questionnaire is illustrated by the observation that the bimodal score and Likert score of 10 participants moved in opposite directions at consecutive assessments i.e. one scoring system
showed improvement whilst the other showed deterioration.

Now that's a red flag.