I just saw the influential and renegade psychologist, James C. Coyne re-tweet a message linking to this paper:
Free at: http://www.tc.umn.edu/~pemeehl/144WhySummaries.pdf
It looks interesting, but I don't have the time to read it at the moment, but will be interested to read what anyone says who does read it.
"Why summaries of research on psychological theories are often uninterpretable"
Meehl, Paul E.
Psychological Reports, Vol 66(1), Feb 1990, 195-244. doi: 10.2466/PR0.66.1.195-244
Abstract
Summary.-Null hypothesis testing of correlational predictions from weak substantive theories in soft psychology is subject to the influence of ten obfuscating factors whose effects are usually (1) sizeable, (2) opposed, (3) variable, and (4) unknown. The net episternic effect of these ten obfuscating influences is that the usual research literature review is well-nigh uninterpretable. Major changes in graduate education, conduct of research, and editorial policy are proposed.
Free at: http://www.tc.umn.edu/~pemeehl/144WhySummaries.pdf
It looks interesting, but I don't have the time to read it at the moment, but will be interested to read what anyone says who does read it.