PACE Trial and PACE Trial Protocol

[Esther12]

How could we have missed this before?

Part of me thinks that we would have made more of a fuss about this at the time... but I've forgotten more about PACE than I currently know. I just searched this thread for 'normal' to see if I could find an earlier discussion: there are 245 posts to go through! There's so much content in this thread: no wonder lazy science journalists prefer to just assume patients vilify researchers because they're psychologists, rather than try to understand our actual concerns.

 

[oceanblue]

Like Dolphin, I don't recall any requirement for improvement to be classed as normal. I'm pretty sure at least one submitted letter did pick up on the fact that someone could join the study with severe disabling fatigue and then be counted as 'normal' without having improved a jot (and could even have deteriorated 5 points on SF36 score). And it's immensley gratifying to think that I might not be the only one here with a flaky memory.

 

[Dolphin]

Like Dolphin, I don't recall any requirement for improvement to be classed as normal. I'm pretty sure at least one submitted letter did pick up on the fact that someone could join the study with severe disabling fatigue and then be counted as 'normal' without having improved a jot (and could even have deteriorated 5 points on SF36 score). And it's immensley gratifying to think that I might not be the only one here with a flaky memory.

This published letter mentioned (last paragraph) that people could score the same on the fatigue scale and be counted in the normal range and the authors' reply didn't mention that people had to improve to be counted:

The Lancet, Early Online Publication, 17 May 2011

doi:10.1016/S0140-6736(11)60688-0

The PACE trial in chronic fatigue syndrome

Sarah M Feehan a, on behalf of the Liverpool ME Support Group

Peter White and colleagues1 say that normal fatigue is represented by a figure of 18 or less on the Chalder fatigue questionnaire (Likert scoring), rather than the validated definition of fatigue caseness (4 or more, bimodal scoring) used in the trial's protocol.2, 3 A score of 18 represented the mean plus 1 SD (142 + 46) for a control group who had attended their general practitioner in the previous 12 months.4 This figure almost certainly would have been lower if those who had not attended their general practitioner had also been included when deriving population data. Indeed, normative data from a Norwegian study gave a mean of 122 (SD 40).5 Interestingly, the Norwegian data were stratified by health condition (unfortunately, only means were published): No disease/current health problem: 112; Past or current disease: 121; Current health problem: 125, and Disease and current health problem: 142.

Furthermore, 176% of chronic fatigue syndrome patients diagnosed at the Chronic Fatigue Unit (South London and Maudsley NHS Trust) had a score of 18 or less before they were treated.4 This suggests either that the Chronic Fatigue Unit diagnoses and treats fatigue problems in patients with normal levels of fatigue or, alternatively, that the threshold of 18 to represent normal fatigue is not suitable.

Given this information, and the fact that those with a Chalder fatigue questionnaire Likert score of 18 could still meet the trial's entry criteria (bimodal score of 6 or more),1, 3 it would be good if White and colleagues would now recalculate the data using the original definition of fatigue caseness.2, 3

I declare that I have no conflicts of interest.

References

1 White PD, Goldsmith KA, Johnson AL, et alon behalf of the PACE trial management group. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet 2011; 377: 823-836. Summary | Full Text | PDF(309KB) | CrossRef | PubMed

2 Chalder T, Berelowitz G, Pawlikowska T, et al. Development of a fatigue scale. J Psychosom Res 1993; 37: 147-153. CrossRef | PubMed

3 White PD, Sharpe MC, Chalder T, DeCesare JC, Walwyn Ron behalf of the PACE trial group. Protocol for the PACE trial: a randomised controlled trial of adaptive pacing, cognitive behaviour therapy, and graded exercise, as supplements to standardised specialist medical care versus standardised specialist medical care alone for patients with the chronic fatigue syndrome/myalgic encephalomyelitis or encephalopathy. BMC Neurol 2007; 7: 6. CrossRef | PubMed

4 Cella M, Chalder T. Measuring fatigue in clinical and community settings. J Psychosom Res 2010; 69: 17-22. CrossRef | PubMed

5 Loge JH, Ekeberg O, Kaasa S. Fatigue in the general Norwegian population: normative data and associations. J Psychosom Res 1998; 45: 53-65. CrossRef | PubMed

a Liverpool ME Support Group, Bootle, Liverpool L20 9LD, UK

 

[Graham]

On the meetup site I have a page on the Chalder scale linking bimodal with Likert scoring, and have plotted the survey results so far. You will see that two of the 40 in the survey would have qualified for the trial, but equally would have qualified for being within normal range.

 

[Jenny]

On the meetup site I have a page on the Chalder scale linking bimodal with Likert scoring, and have plotted the survey results so far. You will see that two of the 40 in the survey would have qualified for the trial, but equally would have qualified for being within normal range.

What a great piece of work Graham! I haven't read it all yet - there's a lot to get to grips with here, but it shows very clearly some of the problems with the scale. Just one thing - Likert scales are usually considered to provide ordinal data, not continuous. This is because we can't assume that respondents see the differences between all pairs of adjacent scale points as the same. For example, the difference between 'no more than usual' and 'more than usual' can't be assumed to be the same as the difference between 'more than usual' and 'much more than usual'. As you said some respondents commented on this.

Purists would say you shouldn't use parametric statistical techniques with ordinal data, but in fact a lot of psychologists do. Not sure how all this affects the PACE analyses, but perhaps it should be borne in mind.

Jenny

 

[oceanblue]

For bedtime reading (oh what a sad geek I am!) I have been working through the AfME files of lists of studies etc. relating to ME/CFS for each year, looking for how many relate to the severely affected (so far only one out of about 900). But I came across this study by Nijrolder looking at people who had attended GPs surgeries with fatigue that had lasted for more than 6 months (in effect, the Oxford criteria). He actually used the sf-36 as one of the assessments, and monitored them over a year.

I'm not that well up on the sf-36 form, but it looked a lot like the results from the GET and CBT groups in PACE, only these groups didn't have any specific treatment: but there was strong advice that sleep problems were common and should be resolved. Have you discussed this elsewhere?

The Wessely study from 1997 found a prevalence of 11.6% for Chronic Fatigue against only 2.2% for Oxford Criteria so it probably isn't safe to use Chronic Fatigue as a proxy for Oxford Criteria CFS. I think there are some studies around suggesting that the prognosis for CF is much better than for CFS (but don't ask me for references!) so the Nijrolder results might not be so surprising.

 

[Bob]

Like Dolphin, I don't recall any requirement for improvement to be classed as normal. I'm pretty sure at least one submitted letter did pick up on the fact that someone could join the study with severe disabling fatigue and then be counted as 'normal' without having improved a jot (and could even have deteriorated 5 points on SF36 score).

Thanks ocean. I don't think that I read all of the letters, so that's helpful to know.

And it's immensley gratifying to think that I might not be the only one here with a flaky memory.

I think you're in very good company there, ocean!

I can't even remember what it's like to have a good memory any more!

 

[Bob]

This published letter mentioned (last paragraph) that people could score the same on the fatigue scale and be counted in the normal range and the authors' reply didn't mention that people had to improve to be counted:

The Lancet, Early Online Publication, 17 May 2011

doi:10.1016/S0140-6736(11)60688-0

The PACE trial in chronic fatigue syndrome

Sarah M Feehan a, on behalf of the Liverpool ME Support Group

Peter White and colleagues1 say that normal fatigue is represented by a figure of 18 or less on the Chalder fatigue questionnaire (Likert scoring), rather than the validated definition of fatigue caseness (4 or more, bimodal scoring) used in the trial's protocol.2, 3 A score of 18 represented the mean plus 1 SD (142 + 46) for a control group who had attended their general practitioner in the previous 12 months.4 This figure almost certainly would have been lower if those who had not attended their general practitioner had also been included when deriving population data. Indeed, normative data from a Norwegian study gave a mean of 122 (SD 40).5 Interestingly, the Norwegian data were stratified by health condition (unfortunately, only means were published): No disease/current health problem: 112; Past or current disease: 121; Current health problem: 125, and Disease and current health problem: 142.

Furthermore, 176% of chronic fatigue syndrome patients diagnosed at the Chronic Fatigue Unit (South London and Maudsley NHS Trust) had a score of 18 or less before they were treated.4 This suggests either that the Chronic Fatigue Unit diagnoses and treats fatigue problems in patients with normal levels of fatigue or, alternatively, that the threshold of 18 to represent normal fatigue is not suitable.

Given this information, and the fact that those with a Chalder fatigue questionnaire Likert score of 18 could still meet the trial's entry criteria (bimodal score of 6 or more),1, 3 it would be good if White and colleagues would now recalculate the data using the original definition of fatigue caseness.2, 3

I declare that I have no conflicts of interest.

References

1 White PD, Goldsmith KA, Johnson AL, et alon behalf of the PACE trial management group. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet 2011; 377: 823-836. Summary | Full Text | PDF(309KB) | CrossRef | PubMed

2 Chalder T, Berelowitz G, Pawlikowska T, et al. Development of a fatigue scale. J Psychosom Res 1993; 37: 147-153. CrossRef | PubMed

3 White PD, Sharpe MC, Chalder T, DeCesare JC, Walwyn Ron behalf of the PACE trial group. Protocol for the PACE trial: a randomised controlled trial of adaptive pacing, cognitive behaviour therapy, and graded exercise, as supplements to standardised specialist medical care versus standardised specialist medical care alone for patients with the chronic fatigue syndrome/myalgic encephalomyelitis or encephalopathy. BMC Neurol 2007; 7: 6. CrossRef | PubMed

4 Cella M, Chalder T. Measuring fatigue in clinical and community settings. J Psychosom Res 2010; 69: 17-22. CrossRef | PubMed

5 Loge JH, Ekeberg O, Kaasa S. Fatigue in the general Norwegian population: normative data and associations. J Psychosom Res 1998; 45: 53-65. CrossRef | PubMed

a Liverpool ME Support Group, Bootle, Liverpool L20 9LD, UK

Thanks for that, Dolphin.

This has some helpful references for normative, and other, Chalder scores.

 

[oceanblue]

I can't even remember what it's like to have a good memory any more!

I have this vague recollection it was a useful thing to have, but can't be sure.

 

[Dolphin]

I can't even remember what it's like to have a good memory any more!

I have this vague recollection it was a useful thing to have, but can't be sure.

What's the PACE Trial?

 

[Esther12]

What's the PACE Trial?

I'd love to be that forgetful.

Well... probably not actually. It was a poor joke, and now I've belittled the suffering of alzheimer's patients! When will I learn?!

 

[Graham]

Thanks Jenny. I find the labels quite confusing as a mathematician, because "continuous" means infinitely divisible - scores of 0 to 3 don't even get down to fractions. I got the "continuous" label from the PACE trial (page 6 - "we used continuous scores for primary outcomes..." and onwards) and assumed that they were using the term in an accepted way. I think I need to edit "continuous" out and replace it with Likert for clarity.

Ordinal makes sense to me, and that is the way that I have thought of it, which is why it isn't really possible to create an equation mapping Bimodal to Likert, or vice versa: nor is it appropriate to look at a regression relationship. But with enough data from the survey, I hope to be able to create separate average Likert scores for each Bimodal level that may just about be useful somewhere. It's pretty obvious that the scale is ordinal if you look at 11/33. That is so much of a sink, absorbing a whole range of degrees of illness. It's also pretty obvious that people who have gone through really bad spells don't see a level 2 in the same way that someone like me does.

 

[Graham]

The Wessely study from 1997 found a prevalence of 11.6% for Chronic Fatigue against only 2.2% for Oxford Criteria so it probably isn't safe to use Chronic Fatigue as a proxy for Oxford Criteria CFS. I think there are some studies around suggesting that the prognosis for CF is much better than for CFS (but don't ask me for references!) so the Nijrolder results might not be so surprising.

Thanks Oceanblue - I'll steer clear of mentioning it then. I really do appreciate the back-up!

 

[Dolphin]

On the meetup site I have a page on the Chalder scale linking bimodal with Likert scoring, and have plotted the survey results so far. You will see that two of the 40 in the survey would have qualified for the trial, but equally would have qualified for being within normal range.

What a great piece of work Graham! I haven't read it all yet - there's a lot to get to grips with here, but it shows very clearly some of the problems with the scale. Just one thing - Likert scales are usually considered to provide ordinal data, not continuous. This is because we can't assume that respondents see the differences between all pairs of adjacent scale points as the same. For example, the difference between 'no more than usual' and 'more than usual' can't be assumed to be the same as the difference between 'more than usual' and 'much more than usual'. As you said some respondents commented on this.

Purists would say you shouldn't use parametric statistical techniques with ordinal data, but in fact a lot of psychologists do. Not sure how all this affects the PACE analyses, but perhaps it should be borne in mind.

Jenny

That may be why there is the bimodal scale for the Chalder fatigue scale.
However in the PACE Trial, they summed the individual scores. So 2x3s is the same as 3x2s.

Also they estimated the normal range by using mean + 1 SD.

 

[Dolphin]

Thanks Jenny. I find the labels quite confusing as a mathematician, because "continuous" means infinitely divisible - scores of 0 to 3 don't even get down to fractions. I got the "continuous" label from the PACE trial (page 6 - "we used continuous scores for primary outcomes..." and onwards) and assumed that they were using the term in an accepted way. I think I need to edit "continuous" out and replace it with Likert for clarity.

Continuous is often used in this way which is very odd for me who like you has a mathematical background - a lot of the (mathematical) analysis and topology was about continuous functions and the like (initially epsilon/delta proofs).

It is often used to mean the opposite of categorical data - which could be 2 categories (the word "dichotomised is often used in this context) or more than 2 categories. When one has categorical data, one uses tests like Chi-squared, Fisher's exact, etc.

 

[Bob]

So I've been thinking about the 'normal range' a bit more, and searching through the text of the published paper.
I still can't see anything about a participant needing to have improved before being assigned to the 'normal range'.

If this is so, then the figures given for the 'normal range' are completely meaningless, unless the numbers in each group are compared at baseline and at 52 weeks.
The percentages currently given for the normal range are just isolated numbers which have no meaningful comparisons.
(i.e. How do we know if, at baseline, those within the normal range, in the SMC group for example, was 10%, 30% or 60%?)

Why hasn't the Lancet picked up on this?

The only way I can see that it has any meaning at all, is if we assume that each therapy group would have had similar average scores at baseline.
But we can't assume that, because participants were assigned to each group randomly, and the 'normal range' data was not calculated at baseline.

It is meaningless to compare the SMC group at 52 weeks with the CBT group at 52 weeks, if we don't know how many participants were in the normal range in each group at baseline.

Am I right about all this? I'm confused!

 

[Bob]

Below, are the numbers of participants who improved from baseline for each of the primary outcomes (as opposed to the post-hoc analysis which used both primary outcomes)

This is probably an obvious question, but my brain is on a go-slow, so...
Looking at these, can anyone work out how the post-hoc analysis for both primary outcomes can have worked out larger percentages for the difference between SMC-GET (16%) and SMC-CBT (14%)? Would this be because the SMC group improved relatively less well in both of the primary outcomes?


This data is taken from table 3...

Table 3

Number improved from baseline (Participants improved from baseline by two or more points for fatigue and eight or more for physical function.) (I assume this is at 52 weeks)

Fatigue

APT 65% CBT 76% GET 80% SMC 65%

Difference between SMC and CBT = 15%
Difference between SMC and GET = 11%


Physical function

APT 49% CBT 71% GET 70% SMC 58%

Difference between SMC and CBT = 13%
Difference between SMC and GET = 12%

 

[Graham]

Yes, that's probably it. Imagine if the SMC group had nobody that improved on both the Chalder and the SF-36 scores, only on one or the other. That would boost the overall claims of GET and CBT, but would also indicate that SMC was effective over a wider range of patients.

 

[Dolphin]

This is probably an obvious question, but my brain is on a go-slow, so...
Looking at these, can anyone work out how the post-hoc analysis for both primary outcomes can have worked out larger percentages for the difference between SMC-GET (16%) and SMC-CBT (14%)? Would this be because the SMC group improved relatively less well in both of the primary outcomes?

Yes. You have to satisfy both. For each group, think of a Venn Diagram, with two circles, with the numbers who were above the threshold for the SF-36 PF in one circle, and the CFQ threshold in the other and the intersection are those that are in both. BTW, those that satisfy neither are outside the circles.

One can't assume anything, but reporting biases may have been at play given these are subjective outcome measures.

 

[Dolphin]

Yes, that's probably it. Imagine if the SMC group had nobody that improved on both the Chalder and the SF-36 scores, only on one or the other. That would boost the overall claims of GET and CBT, but would also indicate that SMC was effective over a wider range of patients.

Good point. One can calculate the figures e.g. with a Venn diagram e.g.
for GET: #{CFQ\SF-36 PF}=29; #{SF-36 PF\CFQ}=14; #{CFQ n SF-36 PF}=94; # neither= 17 (11%). Total=154.
for CBT: #{CFQ\SF-36 PF}=26; #{SF-36 PF\CFQ}=18; #{CFQ n SF-36 PF}=87; # neither= 17 (11%). Total=148.
for APT: #{CFQ\SF-36 PF}=35; #{SF-36 PF\CFQ}=11; #{CFQ n SF-36 PF}=64; # neither= 43 (28%). Total=153.
for SMC: #{CFQ\SF-36 PF}=30; #{SF-36 PF\CFQ}=20; #{CFQ n SF-36 PF}=68; # neither= 34 (22%). Total=152.

So 78% of those in the SMC only group "improved" on at least one of the two improvement measures. This suggests to me the levels required to improve were too low if one wants to test what a therapy or treatment can do over and above SMC.