• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Cracking letter from Prof. Racaniello to Dr Brown ed.in chief re: Dr Esther Crawley's LP paper.

Countrygirl

Senior Member
Messages
5,429
Location
UK
Here is a cracking letter from Prof. Racaniello and Dr David which I am enjoying with my morning coffee and which they sent to the BMJ's Dr Nick Brown, editor of Archives of Disease in Childhood on Monday concerning Dr Esther Crawley's Lightning Process study in children with ME. (I do believe she is on their editorial board too.)

Is that Esther's pidgeons I see flying home to roost?



Below is the letter Professor Vincent Racaniello sent to Dr Nick Brown, editor-in-chief of Archives of Disease in Childhood, on behalf of both of us. He sent it Monday morning in response to a letter from Dr Brown last Thursday in which he alerted us to the outcome of the examination of the Lightning Process study.

Dear Dr Brown—

Thank you for your letter. We are pleased that Archives of Disease in Childhood has published a correction and editor’s note confirming what Virology Blog documented in December, 2017: The investigators of the Lightning Process study violated BMJ policy on prospective trial registration, swapped outcome measures after gathering data from early trial participants, and failed to disclose these salient details in the published paper.

However, we are surprised you believe addressing these concerns is only a matter of “clarification” and that you have therefore allowed the study’s reported findings to stand. This decision appears to call into question previous statements from BMJ leaders about the critical role of prospective trial registration in protecting the integrity of the published literature. Examples include the following:

*In 2013, Dr Godlee testified before the House of Common’s Science and Technology Committee about BMJ’s efforts to implement a zero-tolerance policy for trials in which recruitment of participants preceded registration. “As far as we are aware, in the last two years we have not published any trial that has not been prospectively registered,” testified Dr Godlee–a claim she could not truthfully make today.

*In a 2017 article, Dr Elizabeth Loder, The BMJ’s head of research, discussed how the medical and research communities might render these trials “a thing of the past.” One solution, proposed Dr Loder, “might be to treat them as the research equivalent of medical ‘Never Events’…These are ‘particularly shocking medical errors (such as wrong-site surgery) that should never occur.’”

*Last year, Dr Trish Groves, BMJ Open’s former editor-in-chief, tweeted the following: “Prospective registration of #clinicaltrials isn’t red tape: it’s done for sound ethical and scientific reasons.”

When it comes to the Lightning Process study, you seem to regard the registration issue as a “red tape” mix-up that can be resolved through a simple “clarification while the findings themselves remain intact. In this case, BMJ has apparently abandoned the “sound ethical and scientific reasons” behind its principled position on trial registration and adopted a more expedient or flexible approach. In the editor’s note, you declare that non-adherence to the prospective registration requirement does not warrant retraction–but you offer no rationale for this relaxed interpretation of long-standing BMJ policy.

Your current stance is not compatible with the tough, no-nonsense views articulated by Dr Godlee, Dr Loder and Dr Groves. As Dr Loder noted in her 2017 article, “Trustworthy clinical trials = prospectively registered trials that faithfully report their prespecified outcomes.” In a 2018 BMJ Open article, Dr Loder and colleagues described prospective registration as “an important safeguard” against selective reporting of results. The investigators of the Lightning Process study not only failed to register their trial prospectively but also failed to “faithfully report” their prespecified outcomes.

These investigators swapped primary and secondary outcomes after collecting data from early participants, who were initially recruited for a feasibility study and then included in the full trial sample. For the feasibility study, school attendance at six months was designated as the primary outcome, and self-reported physical function at six months as a secondary outcome. At the end of the full trial, school attendance at six months yielded null results—but this measure had been demoted to a secondary outcome. The new primary outcome, self-reported physical function at six months, had positive results.

In a clinical trial, it is problematic to identify the primary outcome after collecting data from more than half the participants. Whatever the intentions of the investigators, their undisclosed outcome-swapping allowed them to present positive rather than null results for what they reported as their primary outcome. It is irrelevant whether or not they analyzed outcomes data from the feasibility study before deciding to swap measures for the full trial. In open-label trials relying on self-reported measures, like the Lightning Process study, trends in outcomes are generally apparent to investigators long before the data are formally scrutinized.

In the editor’s note, you discuss the journal’s extensive and lengthy review of the multiple issues raised about the study. With regards to the outcome-swapping, you explain that the journal’s review involved “seeking assurance from the authors that the change in primary outcome was not influenced by (positive) findings in the feasibility phase.”

This statement is striking. Individuals subject to potential bias are often not considered impartial and authoritative arbiters of whether this potential bias has influenced their decision-making. Perhaps you and Dr Godlee believe it is reasonable and appropriate to take such “assurance from the authors” at face value while ignoring the self-evident impact of their methodological choices on the reported results. We do not.

Beyond the truism that humans tend to be blind to their own biases, it is worth noting that these investigators withheld important information about trial registration and outcome-swapping in their public version of events. In a significant departure from reality, the published paper portrayed the trial as if it were prospectively registered, with pre-designated primary and secondary outcomes. This sort of misrepresentation does not inspire confidence in any subsequent assurances of impartiality.

Archives of Disease in Childhood would not have published this paper in the first place except for two major failures: 1) The investigators’ failure to provide editors with an accurate account of how they conducted the trial; and 2) The journal’s failure to detect and address unacceptable deficiencies, despite its obligation to subject the paper to rigorous processes of peer review and editorial oversight.

Given the undisputed facts, BMJ’s decision to publish a “corrected” version of this study that re-affirms the original findings is scientifically and ethically indefensible. It is also potentially harmful to the health and well-being of children.

Because of public interest, this exchange will be posted on Virology Blog and forwarded to others engaged with the issue.

Best—

Vincent Racaniello, Ph.D. | Higgins Professor
Department of Microbiology & Immunology
Columbia University, New York

David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley
 
Last edited: