statistical nonsense and spin

[Mithriel]

This is a study into papers which spin results which are not statistically significant to seem to be meaningful for the results the authors would like.

http://jama.ama-assn.org/cgi/content/short/303/20/2058

Conclusion In this representative sample of RCTs published in 2006 with statistically nonsignificant primary outcomes, the reporting and interpretation of findings was frequently inconsistent with the results.

The failure of the peer review process is not confined to ME/CFS.

Mithriel

 

[Dolphin]

Thanks Mithriel.
I think the FINE Trial would qualify: http://www.bmj.com/cgi/content/abstract/340/apr22_3/c1777

 

[dean]

great post.....

What is most interesting to me is the "spin" component of many scientific papers in the service of self interest. The first step I take in looking at any scientific paper is to ask myself, "How did they get it wrong?" In other words the first step in good science is good critique....

"To identify the nature and frequency of distorted presentation or "spin" (ie, specific reporting strategies, whatever their motive, to highlight that the experimental treatment is beneficial, despite a statistically nonsignificant difference for the primary outcome, or to distract the reader from statistically nonsignificant results"

 

[Angela Kennedy]

I'd be interested in how they define "spin" (their own 'definition' is not quite enough here).

The reason I say this is because with this article we're getting into the realms of social science and discourse analysis (analysising language and rhetoric/polemic). I'm interested to see their methodology.

 

[alex3619]

Hi Mithriel

I have been saying something similar for years, it was really nice to read this piece when it came out. Look at my current signature.

The very first psychobabble paper I read had results that could be interpreted as the exact opposite of their claims, but I no longer recall the name of the paper. I don't think they understand statistics (not that I do), nor their reviewers, and I wonder if this misrepresentation is deliberate or the result of people not getting expert statistical advice when writing papers.

Bye
Alex

This is a study into papers which spin results which are not statistically significant to seem to be meaningful for the results the authors would like.

http://jama.ama-assn.org/cgi/content/short/303/20/2058



The failure of the peer review process is not confined to ME/CFS.

Mithriel

 

[Snow Leopard]

More on this topic:

"Medical statistics; Why much medical research is wrong"
http://www.economist.com/science/displaystory.cfm?story_id=8733754

"Sifting the evidence---what's wrong with significance tests?"
http://www.bmj.com/cgi/content/extract/322/7280/226

"Contradicted and Initially Stronger Effects in Highly Cited Clinical Research"
http://jama.ama-assn.org/cgi/content/abstract/294/2/218

"Why Most Published Research Findings Are False"
http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124

 

[dean]

From an old professor

"Epidemiology is the science of critique"