Do more people recover from chronic fatigue syndrome with CBT or GET than with other treatments?

[Tyto alba]

The PF-36 questionnaire is terrible for this illness. It is designed for predictable, static health problems that don't fluctuate and don't change in response to activity levels. Depending on how I interpret the questions, I get vastly different levels of impairment.

Indeed. I wonder how they managed to accurately capture functional ability in PACE using this scoring method.

 

[Valentijn]

Or do they answer no because they have taken the PEM into consideration?

They may very well have been taught not to take it into consideration, with CBT and/or GET.

 

[Snow Leopard]

We desperately need to develop better PROMs - ones that make sense to patients (and cater for different levels and dimensions of impairment), rather than those who are trying to make their interventions look more impressive than they really are (whether pharmacological or not).

 

[Dolphin]

Scoring:
Yes, limited a lot = 0
Yes, limited a little = 5
No, not limited at all = 10

Questions:
1.Vigorous activities, such as running, lifting heavy objects, participating in strenuous sports
2.Moderate activities, such as moving a table, pushing a vacuum cleaner, bowling, or playing golf
3. Lifting or carrying groceries
4. Climbing several 0f lights of stairs
5. Climbing one flight of stairs
6. Bending, kneeling, or stooping
7. Walking more than a mile
8. Walking several blocks
9. Walking one block
10. Bathing or dressing yourself

As I have said before on another thread, I think that graded activity-oriented CBT and GET could bias responses to such a questionnaire.

People doing these interventions are very often encouraged to go for a walk several times a week. Some studies suggest that people actually do less during the rest of the day to enable them to go for the walk. So then when you ask people about their ability to walk certain distances, they may feel more confident about walking a certain distance because they can do it once. However it doesn't necessarily mean they could do it several times.

Other people who spread out their activities more evenly across the day may feel less confident about walking a certain distance without necessarily being any more impaired.

 

[Barry53]

Indeed. I wonder how they managed to accurately capture functional ability in PACE using this scoring method.

I would think the simple answer to that is ... they did not.

 

[user9876]

The PF-36 questionnaire is terrible for this illness. It is designed for predictable, static health problems that don't fluctuate and don't change in response to activity levels. Depending on how I interpret the questions, I get vastly different levels of impairment.

It isn't even good for static health problems as a scale there are real issues. There is a normalised version which may be slightly better and it may be ok with large population studies.

The point I was trying to pull out is that when talking about a score in both the sf36 and cfq they reflect question answers and have some meaning. Because they are not real scales a given score does not reflect one particular set of answers but there is a set of possible answers. In looking at results it can be useful to look at the actual answers rather than just scores.

 

[Dolphin]

Here is a paper they refer to. Note that the recovery criteria are much stricter than those used in the 2013 PACE trial recovery paper.

Predetermined criteria for “complete recovery” required that patients no longer met chronic fatigue syndrome criteria, were employed full-time, and scored less than 4 on the Fatigue Questionnaire and more than 83 on the Medical Outcomes Study Short-Form General Health Survey physical functioning scale.

Deale A, Husain K, Chalder T, et al. Long-term outcome of cognitive behaviour therapy versus relaxation therapy for chronic fatigue syndrome: a 5-year follow-up study. Am J Psychiatry. 2001;158:2038–2042.

https://www.researchgate.net/profil...Fatigue-Syndrome-A-5-Year-Follow-Up-Study.pdf

Anyone know why 3 different percentages for completely recovered are given in Deale et al.

"Seven (23%) of the cognitive behavior therapy patients and none of the relaxation therapy patients were deemed completely recovered (p=0.03, Fisher’s exact test)."

Table 2: Complete recovery: 6/25 (24%)

"Finally, only 26% of the patients who received cognitive behavior therapy were judged completely recovered after 5 years,"

 

[Keith Geraghty]

Taken from my upcoming reply to the PACE authors -- if you see any faults please let me know

Protocol changes and Recovery
The original trial protocol suggested that to be deemed ‘recovered’ a participant would need to reach a score of 85 of the SF-36 quality of life survey (in addition to the other conditions, like not being deemed to meet the criteria for CFS). After starting the trial, the authors lowered the SF-36 threshold from a score of 85 to a score of 60 (a 25 point drop). The PACE team have subsequently stated that they altered this measure of improvement and recovery (sanctioned by their trial ethics and steering committees) as the original level chosen was ‘too stringent’. The PACE authors want us to accept that when they conceived and drafted their trial analysis plan they opted for markers of recovery that were ‘too strict’. This long list of researchers, many very senior researchers with considerable experience, chose an inappropriate measure of improvement/recovery. This is hard to believe and clinical trials are meant to have strict criteria outlined prior to the trial phase starting. The quality of life threshold the PACE team first decided upon was not picked from thin air, as it just happens to be the mean SF-36 physical health subscale score for the ‘general population’ (Komarof et al., 1998). Essentially, this is the level the average person in a population would score if asked about their physical health (which seems a fair recovery marker). However, the PACE authors abandoned this mid-trial in favour of a lower SF-36 score of 60 – to a level roughly around where a patient with congestive heart failure might fall (hardly a good marker of recovery). It is worthwhile noting here that the FINE trial also used an SF-36 score of 85 as the threshold for recovery and a score of 75 as a marker of clinical improvement (Wearden et al. protocol). Additionally, in the FINE trial the score needed to enter the trial (be deemed ill enough) was below 70 (yet in PACE a score above 60 was within the recovery banding). These facts point to disarray and inconsistencies in methodologies used to assess recovery PACE and in trials of CBT and GET.

 

[Esther12]

I didn't spot any real faults, but thought I'd point out some things.

Are you using 'strict' in two slightly different ways here? If so, I think that could be confusing.

The PACE authors want us to accept that when they conceived and drafted their trial analysis plan they opted for markers of recovery that were ‘too strict’. This long list of researchers, many very senior researchers with considerable experience, chose an inappropriate measure of improvement/recovery. This is hard to believe and clinical trials are meant to have strict criteria outlined prior to the trial phase starting.

I think that the first 'strict' is meant to mean 'demanding', and the second one 'precise', but it could read like they're meant to have the same meaning, 'demanding', and it's not necessarily the case that clinical trials are meant to have particularly demanding criteria for recovery, rather than just a criteria which is meaningful and reasonable.

The quality of life threshold the PACE team first decided upon was not picked from thin air, as it just happens to be the mean SF-36 physical health subscale score for the ‘general population’ (Komarof et al., 1998).

The PACE protocol actually says:

A score of 70 is about one standard deviation below the mean score (about 85, depending on the study) for the UK adult population [51, 52].

Essentially, this is the level the average person in a population would score if asked about their physical health (which seems a fair recovery marker).

re using the mean as a cut off - I think that this could be considered overly demanding for a recovery criteria if scores were normally distributed, particularly as it's combined with the Chalder Fatigue scale (and the other parts of the recovery criteria). For the SF36-PF they're not, and far more that half of the population have a score of 85 or over (isn't it over 80% for the working-age Bowling data they cited?). Personally, I'd make some reference to that to strengthen this point.

It is worthwhile noting here that the FINE trial also used an SF-36 score of 85 as the threshold for recovery and a score of 75 as a marker of clinical improvement (Wearden et al. protocol). Importantly, in the FINE trial the score needed to enter the trial (be deemed ill enough) was below their70 (yet in PACE a score above 60 was within the recovery banding).

I don't know if you've got a tight word count, but I I found that section a bit confusingly phrased, even though I already knew those figures and the point you were making.

Hope you're having fun with it!

 

[Keith Geraghty]

It is worthwhile noting here that the FINE trial also used an SF-36 score of 85 as the threshold for recovery and a score of 75 as a marker of clinical improvement (Wearden et al. protocol). Importantly, in the FINE trial the score needed to enter the trial (be deemed ill enough) was below their70 (yet in PACE a score above 60 was within the recovery banding).

any suggestions for improving

 

[Dolphin]

The PACE team have subsequently stated that they altered this measure of improvement and recovery (sanctioned by their trial ethics and steering committees) as the original level chosen was ‘too stringent’.

Have they said this regarding the recovery criteria? Or does this just apply to the improvement criteria?

 

[Dolphin]

It is worthwhile noting here that the FINE trial also used an SF-36 score of 85 as the threshold for recovery and a score of 75 as a marker of clinical improvement (Wearden et al. protocol).

http://bmcmedicine.biomedcentral.com/articles/10.1186/1741-7015-4-9

I don't recall a figure of 85 being use regarding recovery in the FINE Trial

 

[Esther12]

any suggestions for improving

Nothing great!

I just looked back at the FINE protocol, and they have the same 'increase of 50%' PACE did for their primary outcome too, so that could make this a less useful point for you. I had thought there was unreleased outcomes from FINE, but I can't remember the details, and now I think Dolphin was right to say FINE did not mention a recovery critieria.

I tried a brief re-write, but maybe it would be worth cutting this section?

"It is worth noting that the FINE trial used an SF-36 score of 75, or an increase of 50% from baseline, as a marker of "improvement" (Wearden et al. protocol). Importantly, in the FINE trial the score needed to be considered sufficently disabled to enter the trial was 70 or under. In PACE, patients with a score of just 60 could be classed as recovered, despite those with a score of 65 at baseline having been entered into the trial."

 

[trishrhymes]

I hope you are able to use graphical illustrations.

A simple graph to show. SF36 scores set for entry, improvement and recovery at original protocol stage in PACE and FINE and later modified levels used in published papers, alongside mean levels for healthy population of same age group as PACE patients , whole population and heart failure patients.

If not a graph, then a table of figures.

I think most readers find figures quoted in paragraphs of text get confused and are hard to hold in the mind long enough to compare easily once more than 2 or 3 figures are quoted.

It might also be worth pointing out that FINE was a study of housebound patients, yet the entry criteria for that was 10 points higher at 70 than the recovery criteria for PACE at 60.

 

[user9876]

The PACE team have subsequently stated that they altered this measure of improvement and recovery (sanctioned by their trial ethics and steering committees) as the original level chosen was ‘too stringent’.

I'm not sure that they have ever claimed that the recovery criteria were approved by the trial ethics committee. I think they used some words which suggested that was the case but never quite claimed it. I also think that the changes to the recovery criteria were not approved. I'm pretty sure I couldn't find a mention of recovery in the stats analysis plan so they may claim that the committee that approved the stats analysis plan approved them doing a post hoc analysis. However I suspect that the committee never gave the stats plan much consideration and didn't compare with the original protocol - I seem to remember that it doesn't talk about changes hence they may have gone unnoticed.

The PACE authors want us to accept that when they conceived and drafted their trial analysis plan they opted for markers of recovery that were ‘too strict’. This long list of researchers, many very senior researchers with considerable experience, chose an inappropriate measure of improvement/recovery. This is hard to believe and clinical trials are meant to have strict criteria outlined prior to the trial phase starting. The quality of life threshold the PACE team first decided upon was not picked from thin air, as it just happens to be the mean SF-36 physical health subscale score for the ‘general population’ (Komarof et al., 1998). Essentially, this is the level the average person in a population would score if asked about their physical health (which seems a fair recovery marker).

I think saying the "this is the level the average person in a population " is misleading. This suggests the typical (median) value to me which I think is 100 for the working age population and 95 for the whole population. (using defn working age as <65).

 

[Barry53]

It might also be worth pointing out that FINE was a study of housebound patients, yet the entry criteria for that was 10 points higher at 70 than the recovery criteria for PACE at 60

Good point.

 

[Esther12]

I'm not sure that they have ever claimed that the recovery criteria were approved by the trial ethics committee. I think they used some words which suggested that was the case but never quite claimed it. I also think that the changes to the recovery criteria were not approved. I'm pretty sure I couldn't find a mention of recovery in the stats analysis plan so they may claim that the committee that approved the stats analysis plan approved them doing a post hoc analysis. However I suspect that the committee never gave the stats plan much consideration and didn't compare with the original protocol - I seem to remember that it doesn't talk about changes hence they may have gone unnoticed.

I think your right, PACE have been really evasive on this and it was Wessely who went further:

Author: Simon wessely
Comment:
Dave, all of this has been covered extensively in the trial FAQs here http://www.wolfson.qmul.ac.uk/current-projects/pace-trial/#faq. Nothing dodgy and nothing to hide. It is perfectly acceptable to make changes to your analytic plan or indeed protocols during the conduct of a trial so long as you explain why, get TSC/DMEC approval, document it fully and make it clear in the publication. All of that was done. The 2012 Lancet paper doesn't deal with recovery anyway, that was covered in a secondary later publication. In essence though they decided they were using a overly harsh set of criteria that didn't match what most people would consider recovery and were incongruent with previous work so they changed their minds - before a single piece of data had been looked at of course. Nothing at all wrong in that- happens in vast numbers of trials. The problem arises, as studies have shown, when these chnaged are not properly reported. PACE reported them properly. And indeed I happen to think the changes were right - the criteria they settled on gave results much more congruent with previous studies and indeed routine outcome measure studies of which there are many. And re analysis proves the wisdom of that to be honest. But even then, using criteria that were indeed incongruent with previous work and clinical routine outcome studies, the overall pattern remains the same. CBT GET superior to pacing SMC. Most people who work in the field agree with them by the way - the criteria in the recovery paper itself (not the main lancet paper which never dealt with ) approximate with all the previous work - reasonable and meaningful but not the new penicillin. Am not going to respond to more comments on the same lines except to say it's very likely that this has been addressed before and covered in previous responses by the authors and the trial FAQs. If you want to use the original recovery criteria you can, but I think that the ones that the trialists finally decided upon are more realistic to patients and clinicians alike . And finally it's not a surprise that if you use extremely harsh criteria you get lower frequencies- no one has pointed out that the original criteria did indeed give recovery rates for Pacing and SMC that are also pretty daft and don't match patient experience of either. People do get better on pacing and even just TLC- it's just that more get better with GET and CBT, but still not enough. OK, enough already. Back to day job.

They classed this FOI to clarify things as vexatious:

https://www.whatdotheyknow.com/request/timing_of_changes_to_pace_trial

 

[slysaint]

"In essence though they decided they were using a overly harsh set of criteria that didn't match what most people would consider recovery"
and the new one's did?