4 SEPTEMBER 2019
By David Tuller, DrPH
As
Virology Blog has reported, the lead author of the revised version of Cochrane’s Risk of Bias tool,
published last week in BMJ, is a long-time Bristol University colleague of Professor Esther Crawley. In that capacity, he is a co-author of two high-profile studies that violated key principles of scientific investigation—the Lightning Process study, published by Archives of Disease in Childhood two years ago, and the 2011 school absence study published in BMJ Open.
I have reported at length on the problems with these studies
here and
here, and in many subsequent posts. In the Lightning Process study, the authors recruited more than half the sample before trial registration, swapped primary and secondary outcome measures after gathering data from these early participants, and failed to disclose these salient details in the published paper. The outcome swap allowed Professor Crawley, Professor Sterne and their colleagues to report positive rather than null findings for their official but revised primary outcome.
In July, after more than a year of investigation, Archives of Disease in Childhood, a BMJ journal, published a “correction” that acknowledged the methodological missteps. Yet the journal still allowed Professor Crawley, Professor Sterne and their colleagues to republish the biased findings from the initial paper.
Last week, Virology Blog sent
an open letter protesting this untenable decision to Dr Fiona Godlee, editorial director of BMJ. The letter was signed by 55 scientists, academics and clinicians from Harvard, Stanford, University College London, London School of Hygiene and Tropical Medicine, Berkeley, Columbia, Queen Mary University of London, etc., and expressed dismay at the journal’s “scientifically and ethically indefensible” actions. Some of the signers have subsequently made
their own calls for retraction of the Lightning Process study.
*****
Given this background, it is perhaps not surprising that Professor Sterne’s revised Risk of Bias tool makes it easier for unblinded studies to be identified as having a “low risk” of bias, as
the smart folks on the Science For ME forum have noted. Professor Sterne and his co-authors themselves make the point, in their discussion section.
Here’s what they write:
“We expect the refinements we have made to the RoB tool to lead to a greater proportion of trial results being assessed as at low risk of bias, because our algorithms map some circumstances to a low risk of bias when users of the previous tool would typically have assessed them to be at unclear (or even high) risk of bias. This potential difference in judgments in RoB 2 compared with the original tool is particularly the case for unblinded trials, where risk of bias in the effect of assignment to intervention due to deviations from intended interventions might be low despite many users of the original RoB tool assigning a high risk of bias in the corresponding domain..................
