A further comment by Robert Courtney and a reply have finally been published.
Again the reply is unsatisfactory.
http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD003200.pub7/full
Feedback submitted, 12 May 2016
Summary
Again the reply is unsatisfactory.
http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD003200.pub7/full
Feedback submitted, 12 May 2016
Summary
Comment: A query regarding the way outcomes for physical function and overall health have been described in the abstract, conclusion and discussions of the review.
I would like to query the way that the outcomes for both physical function and overall health have been reported in the abstract, conclusion and in the main discussion section of the current version (version 4) of the Cochrane review by Larun et al., dated 7 February 2016 [1].
The abstract, conclusion and main discussion section unambiguously indicate that there was a positive treatment effect on both physical function and overall health, in relation to exercise therapy compared to a passive control.
For example, with respect to exercise therapy versus passive control, the "authors' conclusions" in the abstract state without qualification that: "A positive effect with respect to sleep, physical function and self-perceived general health has been observed[...]". Another section of the review ("What does evidence from the review tell us?") asserts that: "Exercise therapy had a positive effect on people’s daily physical functioning, sleep and self-ratings of overall health." The "summary of main results" unequivocally states that: "A positive effect of exercise therapy was observed both at end of treatment and at follow-up with respect to sleep (Analysis 1.12; Analysis 1.13), physical functioning (Analysis 1.5; Analysis 1.6) and self-perceived changes in overall health (Analysis 1.14; Analysis 1.15)." (Please see the appendix, below, to read these quotes in full.)
However, upon careful consideration of the relevant analyses, it seems that there were not consistent positive treatment effects for either physical function or overall health in relation to exercise therapy versus passive control. Instead, for both of these variables, there was a significant treatment effect only at end of treatment, but not at follow-up.
The relevant analyses are 1.5 (end of treatment) and 1.6 (follow-up) for self-report physical function, and 1.14 (end of treatment) and 1.15 (follow-up) for self-report overall health.
Analysis 1.5 assessed the pooled treatment effect on physical function at end of treatment for all eligible studies, and demonstrates a significant effect. Analysis 1.6 used the same criteria but at follow-up, and demonstrates that there was not a significant effect for physical function at follow-up.
Analysis 1.14 assessed the pooled treatment effect on overall health at end of treatment for all eligible studies, and demonstrates a significant effect. Analysis 1.15 used the same criteria but at follow-up, and demonstrates that there was not a significant effect for overall health at follow-up.
The lack of a significant treatment effect at follow-up is clearly illustrated by analyses 1.6 and 1.15.
These outcomes are also confirmed in the analysis section of the review where, in relation to the difference between exercise therapy versus passive control, for physical function at follow-up, it is confirmed that: "[...] little or no difference cannot be ruled out." And for overall health at follow-up, it is confirmed that "the confidence interval implies inconclusive results".
I believe that these outcome are not reflected accurately in the abstract, the main discussions or the conclusions of the review; specifically the extracts that are quoted above and in the appendix below. For example, the "summary of main results" specifically claims that positive treatment effects are demonstrated by analyses 1.6 and 1.15, but these analyses actually demonstrate an absence of significant treatment effects. The discussion claims: "A positive effect of exercise therapy was observed [...] at follow-up with respect to [...] physical functioning ([...]Analysis 1.6) and self-perceived changes in overall health ([...]Analysis 1.15)."
It is generally understood that a "positive" treatment effect equates to a significant effect, and I believe that the Cochrane text should reflect this, or at least clarify that the term "positive effect" is being used to indicate a lack of significance.
It is likely that many readers will not read the full report or scrutinise each individual analysis but will read only the abstract, main discussions or conclusions, so I believe it is important for the discussions to carefully and accurately reflect the outcomes of the analyses.
Cochrane has a reputation for upholding the highest standards including with respect to explaining outcomes in accurate and straightforward language. With this in mind, I request that the Cochrane review team kindly review the apparent disparities described above and amend the text of the discussions and conclusions where appropriate, in order to reflect the lack of a significant treatment effect for physical function and overall health at follow-up with respect to exercise therapy versus passive control.
---
Appendix
Quotes from the review:
Abstract > Authors' conclusions
"Patients with CFS may generally benefit and feel less fatigued following exercise therapy, and no evidence suggests that exercise therapy may worsen outcomes. A positive effect with respect to sleep, physical function and self-perceived general health has been observed, but no conclusions for the outcomes of pain, quality of life, anxiety, depression, drop-out rate and health service resources were possible."
What does evidence from the review tell us?
"Moderate-quality evidence showed exercise therapy was more effective at reducing fatigue compared to ‘passive’ treatment or no treatment. Exercise therapy had a positive effect on people’s daily physical functioning, sleep and self-ratings of overall health."
Summary of main results
"[...] A positive effect of exercise therapy was observed both at end of treatment and at follow-up with respect to sleep (Analysis 1.12; Analysis 1.13), physical functioning (Analysis 1.5; Analysis 1.6) and self-perceived changes in overall health (Analysis 1.14; Analysis 1.15)."
---
Reference
1. Larun L, Brurberg KG, Odgaard-Jensen J, Price JR. Exercise therapy for chronic fatigue syndrome. Cochrane Database Syst Rev. 2016; CD003200.
---
I do not have any affiliation with or involvement in any organisation with a financial interest in the subject matter of my comment
Reply
Thank you for your ongoing and detailed scrutiny of our review. We have the greatest respect for your right to comment on and disagree with our work, but in the spirit of openness, transparency and mutual respect we must (again) politely agree to disagree.
All the included studies reported results after the intervention period and this is the main result. The results at different follow-up times are presented in the text. It can be noted that the quality of the evidence is higher for the end-of-treatment time point because more trials are included, and hence, we do not agree that it is wrong to give higher weight to these results in the abstract. Additionally, it is often challenging to analyse follow-up data gathered after the formal end of a trial period. There is always a chance that participants may receive other treatments following the end of the trial period, a behaviour that will lead to contamination of the original treatment arms and challenge the analysis.
Cochrane reviews aim to report the review process in a transparent way, which enables the reader to agree or disagree with the choices made. We do not agree that the presentation of the results should be changed. We note that you read this differently.
Contributors
Feedback submitted by: Robert Courtney
Response submitted by: Lillebeth Larun
