statistical nonsense and spin

Mithriel

Senior Member
Messages
688
Likes
819
Location
Scotland
This is a study into papers which spin results which are not statistically significant to seem to be meaningful for the results the authors would like.

http://jama.ama-assn.org/cgi/content/short/303/20/2058

Conclusion In this representative sample of RCTs published in 2006 with statistically nonsignificant primary outcomes, the reporting and interpretation of findings was frequently inconsistent with the results.
The failure of the peer review process is not confined to ME/CFS.

Mithriel
 
Messages
87
Likes
13
great post.....

What is most interesting to me is the "spin" component of many scientific papers in the service of self interest. The first step I take in looking at any scientific paper is to ask myself, "How did they get it wrong?" In other words the first step in good science is good critique....

"To identify the nature and frequency of distorted presentation or "spin" (ie, specific reporting strategies, whatever their motive, to highlight that the experimental treatment is beneficial, despite a statistically nonsignificant difference for the primary outcome, or to distract the reader from statistically nonsignificant results"
 
Messages
1,026
Likes
146
Location
Essex, UK
I'd be interested in how they define "spin" (their own 'definition' is not quite enough here).

The reason I say this is because with this article we're getting into the realms of social science and discourse analysis (analysising language and rhetoric/polemic). I'm interested to see their methodology.
 

alex3619

Senior Member
Messages
13,798
Likes
37,580
Location
Logan, Queensland, Australia
Hi Mithriel

I have been saying something similar for years, it was really nice to read this piece when it came out. Look at my current signature.

The very first psychobabble paper I read had results that could be interpreted as the exact opposite of their claims, but I no longer recall the name of the paper. I don't think they understand statistics (not that I do), nor their reviewers, and I wonder if this misrepresentation is deliberate or the result of people not getting expert statistical advice when writing papers.

Bye
Alex

This is a study into papers which spin results which are not statistically significant to seem to be meaningful for the results the authors would like.

http://jama.ama-assn.org/cgi/content/short/303/20/2058



The failure of the peer review process is not confined to ME/CFS.

Mithriel
 
Messages
5,885
Likes
12,656
Location
South Australia
More on this topic:

"Medical statistics; Why much medical research is wrong"
http://www.economist.com/science/displaystory.cfm?story_id=8733754

"Sifting the evidence---what's wrong with significance tests?"
http://www.bmj.com/cgi/content/extract/322/7280/226

"Contradicted and Initially Stronger Effects in Highly Cited Clinical Research"
http://jama.ama-assn.org/cgi/content/abstract/294/2/218

"Why Most Published Research Findings Are False"
http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124