Misrepresentation of Randomized Controlled Trials in Press Releases & News Coverage: A Cohort Study

[urbantravels]

What's needed, of course, are scientific journalists capable of detecting the spin and explaining it to a lay readership. To do that, the journalist would need the scientific background to actually read and understand the paper itself, or at least find an objective expert source who could provide a close reading. Assessing study design and methods, being aware of the existing literature on the subject and any relevant political factors, having the ability to judge whether the data actually supports the conclusions (and to spot key omissions in the data)...well, that's an impossibly tall order.

There are some very good science journalists out there, but not many. I don't really blame the bad ones for being bad; outlets that don't care to devote the resources to paying the good ones for good work get, instead, a person who regurgitates a press release. Or else they just print the press release without alteration, which is by far the simplest and cheapest thing to do.

 

[alex3619]

All of this is why I proposed a global media release centre for neuroimmune diseases. I will be discussing this more several blogs from now. Bye, Alex

 

[Jarod]

All of this is why I proposed a global media release centre for neuroimmune diseases. I will be discussing this more several blogs from now. Bye, Alex

Aloha

I see one potential pitfall with having big centralized organizations for controlling everything. Maybe a centralized power structure can be taken over by the .01% and used contrary to the best interests of the 99.9%?

For instance; It's seems pretty easy for a few rich and powerful people to take over one centralized agency, but what about taking over 10,000 local agencies?

Ron Paul proposed eliminating the federal department of education in the United States. (Might sound strange to eliminate a department of education....) However, (I think) his goal was to leave the decisions in control of the localities instead of imposing regulations from one central education agency. In essence, let the people most familiar with the neighborhoods decide how they would educate their kids, what books to use, and how to structure the program.

I think Ron Paul's basic premise was to prevent a few people from controlling the whole process for the interest of the 0.1%.

Who knows maybe his de-centralized approach would be more effective if applied somehow to global media....?

 

[alex3619]

Hi Jarod, we have a decentralized approach now, its failing. The big issue with proper implementation of things like a global media centre is the checks and balances that go into its formation. Get that wrong and it can indeed be manipulated. Much of this disinformation that is out there is because the checks and balances that are established in society are failing. Part of the reason for that is the existing checks and balances do not apply to new organizations and processes, and for old organizations they reflect society decades or centuries ago. The systems of checks and balances are not set up for the modern world.

The other problem is that I propose a media centre as a counter to existing PR machines that are churning out disinformation - propaganda. Something has to be done. This is not the entire solution, but it might be part of one. One key issue I hope to push is freedom of dissent. Free speech is intrinsic to proper information management. I will have more to say on that in the next several blogs. I will also be covering a major scandal involving editorial bias in scientific publishing - but its not new, just overlooked.

If something could be done to reverse the decline in investigative journalism this would also help, but that is probably outside of our reach.

Bye, Alex

 

[alex3619]

A huge chunk of the problem we face is that the psych crowd are telling the political and economic elites (and the 'moral' thugs) what they want to hear, so the psychs get lots of funding and political protection.

Hi Sean, this is almost a definition of Zombie Science. Bye, Alex

 

[Jarod]

Hey Alex. Not sure how it works really. Haven't thought about it much. I just like to talk/think about various ideas/concepts.

Couple of snipetts from the SMC and Welcome trust:

Science Media Centre

The Science Media Centre is first and foremost a press office for science when science hits the headlines. We provide journalists with what they need in the form and time-frame they need it when science is in the news - whether this be accurate information, a scientist to interview or a feature article..

SMC covers a broad range of subject matter, as one can see from a quick read of the press release titles.

Anyways. I guess if one doesnt like what they get at SMC, they can go to Welcome Trust "SPIN" science policy in the news. Love that name. More SPIN?

Produced by the Wellcome Trust’s Strategic Planning and Policy Unit, SPIN provides rapid access to concise digests of articles relating to science policy and is issued as a weekly email newsletter every Friday.

Just my two cents worth.

 

[biophile]

Two more studies supporting the common existence of spin in medical research:

-

Impact of Spin in the Abstracts of Articles Reporting Results of Randomized Controlled Trials in the Field of Cancer: The SPIIN Randomized Controlled Trial.

Boutron I, Altman DG, Hopewell S, Vera-Badillo F, Tannock I, Ravaud P.

J Clin Oncol. 2014 Nov 17. pii: JCO.2014.56.7503. [Epub ahead of print]

PURPOSE: We aimed to assess the impact of spin (ie, reporting to convince readers that the beneficial effect of the experimental treatment is greater than shown by the results) on the interpretation of results of abstracts of randomized controlled trials (RCTs) in the field of cancer.

METHODS: We performed a two-arm, parallel-group RCT. We selected a sample of published RCTs with statistically nonsignificant primary outcome and with spin in the abstract conclusion. Two versions of these abstracts were used-the original with spin and a rewritten version without spin. Participants were clinician corresponding authors of articles reporting RCTs, investigators of trials, and reviewers of French national grants. The primary outcome was clinicians' interpretation of the beneficial effect of the experimental treatment (0 to 10 scale). Participants were blinded to study hypothesis.

RESULTS: Three hundred clinicians were randomly assigned using a Web-based system; 150 clinicians assessed an abstract with spin and 150 assessed an abstract without spin. For abstracts with spin, the experimental treatment was rated as being more beneficial (mean difference, 0.71; 95% CI, 0.07 to 1.35; P = .030), the trial was rated as being less rigorous (mean difference, -0.59; 95% CI, -1.13 to 0.05; P = .034), and clinicians were more interested in reading the full-text article (mean difference, 0.77; 95% CI, 0.08 to 1.47; P = .029). There was no statistically significant difference in the clinicians' rating of the importance of the study or the need to run another trial.

CONCLUSION: Spin in abstracts can have an impact on clinicians' interpretation of the trial results.

PMID: 25403215

http://www.ncbi.nlm.nih.gov/pubmed/25403215

-

"Spin" in wound care research: the reporting and interpretation of randomized controlled trials with statistically non-significant primary outcome results or unspecified primary outcomes.

Lockyer S, Hodgson R, Dumville JC1, Cullum N.

Trials. 2013 Nov 6;14:371. doi: 10.1186/1745-6215-14-371.

BACKGROUND: Spin in the reporting of randomized controlled trials, where authors report research in a way that potentially misrepresents results and mislead readers, has been demonstrated in the broader medical literature. We investigated spin in wound care trials with (a) no statistically significant result for the primary outcome and (b) no clearly specified primary outcome.

METHODS: We searched the Cochrane Wounds Group Specialised Register of Trials for randomized controlled trials (RCTs). Eligible studies were: Parallel-group RCTs of interventions for foot, leg or pressure ulcers published in 2004 to 2009 (inclusive) with either a clearly identified primary outcome for which there was a statistically non-significant result (Cohort A) or studies that had no clear primary outcome (Cohort B).We extracted general study details. For both Cohorts A and B we then assessed for the presence of spin. For Cohort A we used a pre-defined process to assess reports for spin. For Cohort B we aimed to assess spin by recording the number of positive treatment effect claims made. We also compared the number of statistically significant and non-significant results reported in the main text and the abstract looking specifically for spin in the form of selective outcome reporting.

RESULTS: Of the 71 eligible studies, 28 were eligible for Cohort A; of these, 71% (20/28) contained spin. Cohort B contained 43 studies; of these, 86% (37/43) had abstracts that claimed a favorable treatment claim. Whilst 74% (32/43) of main text results in Cohort B included at least one statistically non-significant result, this was not reflected in the abstract where only 28% contained (12/43) at least one statistically non-significant result.

CONCLUSIONS: Spin is a frequent phenomenon in reports of RCTs of wound treatments. Studies without statistically significant results for the primary outcome used spin in 71% of cases. Furthermore, 33% (43/132) of reports of wound RCTs did not specify a primary outcome and there was evidence of spin and selective outcome reporting in the abstracts of these. Readers should be wary of only reading the abstracts of reports of RCTs of wound treatments since they are frequently misleading regarding treatment effects.

PMID: 24195770

http://www.ncbi.nlm.nih.gov/pmc/articles/pmid/24195770