Harms of outcome switching in reports of randomised trials: CONSORT perspective

[Dolphin]

Harms of outcome switching in reports of randomised trials: CONSORT perspective
BMJ 2017; 356 doi: https://doi.org/10.1136/bmj.j396 (Published 14 February 2017) Cite this as: BMJ 2017;356:j396

Douglas G Altman, professor1,

1. David Moher, senior scientist2,
2. Kenneth F Schulz, distinguished scientist3 4

1. ↵1Centre for Statistics in Medicine and UK EQUATOR Centre, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
2. Correspondence to: D G Altmandoug.altman@csm.ox.ac.uk

The outcomes reported for trials often differ from those specified in the protocol. Douglas Altman, David Moher, and Kenneth Schulz call for journals to do more to ensure that authors follow guidelines

Rigorous and transparent conduct of clinical trials is essential for confidence in their results. Randomised trials must have a protocol that gives details of the planned study design and outlines the intended methods of analysis, reflecting substantial discussions among the investigators. Key elements of the study design should also be included in an entry on a clinical trials registry. The core principles, as set out in the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) statement,1 are that the protocol should give full details about objectives and planned conduct; the protocol should be adhered to; and any changes from what was initially planned should be reported and, ideally, explained. The SPIRIT guidance for protocols intentionally overlaps considerably with CONSORT recommendations for what information should be included in the report of a randomised trial.2

A particular concern is that researchers should specify the primary and secondary outcomes that will be assessed, including the method and timing of measurement. When the trial’s findings are published, the main reported results should be for those predeclared outcomes and any differences should be highlighted and explained. Adherence to a prespecified plan avoids bias, gives credibility to the results, and avoids confusion for the reader. Conversely, when authors substitute other outcomes after the trial has started there must be concern that such changes were done with knowledge of the data. That casts doubt on the reliability and integrity of the results.

Switching outcomes
Despite this guidance, Chan and colleagues identified frequent discrepancies between the outcomes specified in trial protocols and those reported in subsequent journal articles.3 In particular, in over half …

Continues at: http://www.bmj.com/content/356/bmj.j396.short

 

[Simon]

Doug Altman is a huge figure in the world of statistics, and one of the key players to raise doubts over the medical trials are reported.

A particular concern is that researchers should specify the primary and secondary outcomes that will be assessed, including the method and timing of measurement. When the trial’s findings are published, the main reported results should be for those predeclared outcomes and any differences should be highlighted and explained. Adherence to a prespecified plan avoids bias, gives credibility to the results, and avoids confusion for the reader. Conversely, when authors substitute other outcomes after the trial has started there must be concern that such changes were done with knowledge of the data. That casts doubt on the reliability and integrity of the results.

That's particularly true of PACE's recovery outcomes. Note they failed to report the predeclared recovery outcomes - let alone explaing differences - and they have now admitted their published findings were 'exploratory' ie after access to the data rather than the prespecified ones declared in the protocol.
Can patients with chronic fatigue syndrome really recover after graded exercise or cognitive behavioural therapy? A critical commentary and preliminary re-analysis of the PACE trial

 

[trishrhymes]

Doug Altman is a huge figure in the world of statistics, and one of the key players to raise doubts over the medical trials are reported.

Do you know whether he has looked specifically at PACE - or is aware of it?

 

[user9876]

Doug Altman is a huge figure in the world of statistics, and one of the key players to raise doubts over the medical trials are reported.



That's particularly true of PACE's recovery outcomes. Note they failed to report the predeclared recovery outcomes - let alone explaing differences - and they have now admitted their published findings were 'exploratory' ie after access to the data rather than the prespecified ones declared in the protocol.
Can patients with chronic fatigue syndrome really recover after graded exercise or cognitive behavioural therapy? A critical commentary and preliminary re-analysis of the PACE trial

Of course it wasn't just the recovery outcomes they switched it was their primary outcomes as well and the definition of harm. With PACE it was open label so even without seeing the data they would have a feel.

 

[Barry53]

concern that such changes were done with knowledge of the data

Exactly. Huge concerns. The authors might claim it was done before any analysis, but I would love to see them in a court of law, and state under oath it was done without any prior knowledge of the data, nor strong inkling of where the results were headed. Given all the evidence of how dishonestly they try to defend their claims ... what chance is there, (I mean - really!) that the PACE authors' switched outcomes for any other reason than knowing the originals would prove their prejudices to be unfounded.

 

[Sean]

The authors might claim it was done before any analysis, but I would love to see them in a court of law, and state under oath it was done without any prior knowledge of the data, nor strong inkling of where the results were headed

Yes, we have seen just how well their claims hold up in such a setting, which is not at all.

Not that it stops them from subsequently repeating the same claims.