I've just given the LTFU paper a proper read. tbh, I didn't follow a lot of the stats they referred to, but still have some thoughts.
1 Recovery and their use of this Cairns paper:
6. Cairns, R and Hotopf, M. A systematic review describing the prognosis of chronic fatigue syndrome. Occup Med (Lond). 2005; 55: 20–31
The prevalence of chronic fatigue syndrome is between 0·2% and 2·6% of people worldwide,5 and, if untreated, prognosis for recovery is poor.6
Several naturalistic follow-up studies have assessed outcomes in patients with chronic fatigue syndrome. A 2005 systematic review6 found 14 studies of people meeting operational criteria for chronic fatigue syndrome with a range of follow-up durations and outcome measures and concluded that prognosis was generally poor; improvement was reported by a median of 40% of participants and recovery by only 7%.6
So in the LTFU paper they're using Cairns to show how poor prognosis for recovery is if CFS is left untreated.
Now lets look at how they (mis)used the same Cairns paper to justify their new recovery criteria, and their failure to release results for the recovery criteria laid out in the trial's protocol:
We changed some of the thresholds for measuring recovery from those of the original protocols (White et al. 2007); we made the changes before analysis and to more accurately reflect recovery. Our finding that only 7% recovered after the minimal treatment of SMC, exactly the same proportion as the median recovery rate found without treatment (Cairns & Hotopf, 2005), supports these revised thresholds.
The 7% and 8% recovered after both APT and SMC were similar to the 7% reported in a systematic review after no treatment, suggesting a lack of efficacy of these treatments (Cairns & Hotopf, 2005).
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3776285/
As their LTFU paper now indicates that the addition of CBT and GET to SMC led to no additional benefits at 2.5 years, which is closer to the average follow up of the Cairns paper, that would seem to place them in a bit of a bind.
Also, their recovery paper had stated:
We considered scores of 1 (‘very much better’) or 2 (‘much better’) as evidence of the process of recovery, rather than our original protocol threshold of a score of 1 only, because we considered that participants rating their overall health as ‘much better’ represented the process of recovery.
It seems that SMC alone was as effective at initiating the 'process of recovery' as when CBT and GET were added.
Their recovery paper was always such a joke - other PACE papers just serve to highlight that fact.
2 Their use of Powell 2004, while ignoring FINE:
No follow-up study of GET has been done, although a 2 year follow-up of a randomised trial of an educational intervention that included advice on graded activity showed benefits to be maintained.10
10. Powell, P, Bentall, RP, Nye, FJ, and Edwards, RH. Patient education to encourage graded exercise in chronic fatigue syndrome. 2-year follow-up of randomised controlled trial. Br J Psych. 2004; 184: 142–146
It was not mentioned this trial was followed by the larger FINE trial, which found that this approach led to no improvement over a poor control. Despite this trial being so much more expensive, it seems unlikely that we will see LTFU data - which would be likely to only further embarrass the trial's researchers. David Tuller wrtoe a piece on the way in which the results from FINE are routinely overlooked:
http://www.virology.ws/2015/11/09/t...-sister-trial-been-disappeared-and-forgotten/
3 Pre-specified? Employment status?:
They state:
We report the findings of a pre-specified long-term (minimum 2 years after randomisation) follow-up of the PACE trial participants.
Does anyone know where these were pre-specified?
The trial's full protocol (the searchable version that's was gained through an FOI) stated:
9.2.1 Long term follow-up
Permission will be sought from the participant to be contacted annually for follow-up
information regarding the participant's health and employment status. The participant will
also be invited to remain in contact so that.the results may be disseminated to them once
published.
I've not found any more details than that, and we've not been provided with data on participant's employment status.
In their response to concerns about their recovery claims the PACE authors stated:
In addition, follow-up at 6 months after the end of therapy may be too short a period to affect either benefits or employment. We therefore disagree with Shepherd (2013) that such outcomes constitute a useful component of recovery in the PACE trial.
http://www.meassociation.org.uk/201...ournal-of-psychological-medicine-august-2013/
I've seen this argument used by others too. That being the case, it would be nice to have data from LTFU, which they had said they would collect.
4 Wish it was open peer review:
In an exploratory post-hoc subgroup analysis requested by a reviewer, the within-group improvements were observed whether participants had received additional CBT or GET or not; however, this finding must be interpreted with great caution, as we discuss later.
I would love to see their initial submission too, and how they tried to use the argument about follow-up CBT/GET before they included this data.
There was some evidence from an exploratory analysis that improvement after the 1 year trial final outcome was not associated with receipt of additional treatment with CBT or GET, given according to need. However this finding must be interpreted with caution because it was a post-hoc subgroup analysis that does not allow the separation of patient and treatment factors that random allocation provides.
This is cheeky, and Coyne was good on this:
However, why is this analysis singled out has exploratory and to be interpreted with caution because it is a post-hoc subgroup analysis when similarly post-hoc subgroup analyses are recommended without such caution?
The investigators finally get around to depicting what should be their primary finding, but do so in a dismissive fashion.
Between the original groups, few differences in outcomes were seen at long-term follow-up. This convergence in outcomes reflects the observed improvement in those originally allocated to SMC and APT, the possible reasons for which are listed above.
The discussion then discloses a limitation of the study that should have informed earlier presentation and discussion of results
http://blogs.plos.org/mindthebrain/...ace-chronic-fatigue-syndrome-follow-up-study/
I don't think it stood out to me so much on my first quick read through, and I wonder if my standards for PACE are now so low that I forget to be concerned by this sort of spin.
5 Arguably:
They ran a non-blinded trial with subjective self-report outcomes to assess two interventions intended to alter how patients viewed and thought about their symptoms, and the trial's objective outcomes failed to support their claims about CBT/GET's value to patients. You'd think that they'd have come up with some vaguely plausible sounding excuses for this, but instead we get this:
Third, the outcomes were all self-rated, although these are arguably the most pertinent measures in a condition that is defined by symptoms.
They keep referring to an argument on this rather than making it, eg:
First, recovery from chronic fatigue syndrome (CFS), which is defined by a patient’s reported symptoms, is arguably best measured by multiple patient-reported outcome measures, rather than a single performance test.
http://www.meassociation.org.uk/201...ournal-of-psychological-medicine-august-2013/
Whilst some correspondents have complained that the primary outcomes of the trial were self-rated, we argue that these are the most appropriate measures to judge improvement in an illness that is currently defined by symptoms.
http://www.bmj.com/content/350/bmj.h227/rr-24
If it's so arguable, why don't they present the argument?
6 What does it all mean?! (I don't know - worthless speculation, feel free to join me):
In some ways, the failure of CBT/GET to lead to improvements in self-reported fatigue and physical function slightly challenges my own assumptions that claims made as a part of CBT/GET lead to patients positively biasing self-report outcomes. If this biasing were to 'wear off' as patients became more sceptical over time, with the positive claims made by therapists not being supported by their own experiences, then wouldn't scores decline? Instead, we saw scores for APT/SMC increase. Could a desire to be a part of the trial and receive 'evidence based' interventions lead to patients exaggerating symptoms at baseline - as a way of emphasising that they need help? Those receiving CBT/GET had it made clear to them that this was as good as it gets for CFS treatments, which could reduce the need to 'call for help'? Did the passing of time lead to improvements in self-reported symptoms at roughly the same rate that the positive biasing resulting from CBT/GET diminished?
Something weird has happened here and I don't have a satisfying explanation, but I guess I don't have to provide one. PACE and claims about the value of CBT/GET are a mess, and that's not satisfying.
I re-read Keith Law's second blog on this, and appreciated it more this time. I think that a lot of patients have become so cynical about the value of PACE's subjective self-report outcomes that we can try less hard to interpret them.
http://keithsneuroblog.blogspot.co.uk/2015/11/song-for-siren.html
CBT seeming 'ineffective' post GET could reflect them having a shared mechanism of action? But then GET was associated with improvement post CBT.
This is making my head hurt. I'm going to eat a yoghurt. I'd like to think that this paper is the death-knell of CBT/GET/PACE-spin and I'll never have to read another medical paper again. Suspect that may not be the case.
