This is now open access:
http://journals.sagepub.com/doi/full/10.1177/1359105317707531
This piece is critical of PACE, but I wasn't so sure about some of it.
They defended the use of these therapies with arguments based on a series of false dilemmas: treatments are either effective or ineffective; the result is either black or white; the opponents are wrong and they are right. Unfortunately, they have not shown how effective CBT and GET are. I believe this is the crucial point in the debate between Geraghty and White et al. Let us consider the shades of grey by studying Cohen’s d effect sizes.
Have they shown CBT and GET are genuinely effective, if only slightly? I don't think that they have.
For pragmatic reasons, I decided to use the 0123 coding scheme in my effect size analysis: the data are readily available from White et al., and it produces more precise results for fatigue than the 0011 scheme.
Likert and bimodal scoring of the Chalder Fatigue questionnaire are measuring different things, and I don't think it's right to just assume that likert is a more precise measure of fatigue than bimodal.
edit: While the Chlader Fatigue Scale is so rubbish it doesn't really matter, I'd suspect that bimodal scoring might be better at mitigating some of the problems with it than likert, and so would be a 'more precise' approach. Maybe looking at PACE data and seeing how likert/bimodal scoring correlates with more objective outcomes could provide some evidence on this?
This leads to the interesting hypothesis that the effect size of CBT and GET reduces as the objectiveness of the outcome increases.
To investigate this hypothesis, I added to the analysis the only objective test which I could find in White et al.’s study: the distance covered in a 6-minute walking test after 12 months.
I don't really understand why he didn't also use the fitness and employment data that has been released.
This has been discussed elsewhere, eg:
http://www.bmj.com/content/350/bmj.h227/rr-10
Stulemeijer et al. studied the effects of CBT on subjective fatigue, subjective functional impairment and school attendance in young people with CFS. Their control group consisted of patients on a waiting list for receiving CBT. To deal with issues around missing data, they carried forward the last observations for all variables, except for school attendance. Their rapid response reveals that the final choice of the method for analysing school attendance was made after inspecting the trial data (
Stulemeijer et al., 2005). This suggests that their analysis was not in line with the trial protocol. If they had carried forward the last observations for the missing school attendance data too, the results for CBT would have shown that it was not an effective treatment for this primary outcome (
Stouten, 2004).
I thought that this was interesting, and I don't think I'd read the earlier BMJ RR he'd written on this:
http://www.bmj.com/rapid-response/2011/10/30/question-statistical-advisors-bmj
After publication, the authors agreed that, according to my suggestion, recoding the Chalder fatigue scale from 0011 to 0123 gives more precise results (
Stouten, 2010;
Wearden et al., 2010b).
Wearden et al. (2010b) subsequently demonstrated a modest improvement in fatigue that is statistically significant in favour of pragmatic rehabilitation.
Yet this seems to be contradicted by the results released in the Larun Cochrane review, supposedly calculated from raw FINE data. It would have been good if this had been mentioned.
Maybe I'm being overly-critical on this, as my expectation is that this issue should be an easy win for us, but I feel like this paper uneccessarily muddies the water by omitting some important pieces of infomation. As the author cites some of his own comments from 2004, I wondered if maybe he has been paying less attention to issues recenty, and is a bit relying on old knowledge? If I was speaking with the author I would express my gratitude for him writing it, but I didn't think that this was great tbh, and I have some concerns that it gives the PACE authors some valuable tools for their own response.