Science Alert: Two more classic psychology studies just failed the reproducibility test

[AndyPR]

For years now, researchers have been warning about a reproducibility crisis in science - the realisation that a lot of seminal papers, particularly in psychology, don't actually hold up when scientists take the time to try to reproduce the results.

Now, two more key papers in the psychology have failed the reproducibility test, serving as an important reminder that many of the scientific 'facts' we've come to believe aren't necessarily as steadfast as we thought.

To be fair, just because findings can't be reproduced, it doesn't automatically mean they're wrong. Replication is an important part of the scientific method that helps us nut out what's really going on - it could be that the new researchers did something differently, or that the trend is more subtle than originally thought.

But the problem is that, for decades now, the importance of replicating results has been largely overlooked, with researchers usually choosing to chase a 'new' discovery rather than fact-checking an old one - thanks to the pressure to publish exciting and novel findings in order to secure jobs.

http://www.sciencealert.com/two-more-classic-psychology-studies-just-failed-the-reproducibility-test

Not quite sure I agree with this opinion though, although, to be fair, my knowledge of psychological studies is more than somewhat limited.

"It shows how much effort and attention has gone towards improving the accuracy of the knowledge produced," John Ioannidis, a Stanford University researcher who led a 2005 reproducibility study, told Olivia Goldhill at Quartz.

"Psychology is a discipline that has always been very strong methodologically and was at the forefront at describing various biases and better methods. Now they are again taking the lead in improving their replication record."

 

[Simon]

But the problem is that, for decades now, the importance of replicating results has been largely overlooked, with researchers usually choosing to chase a 'new' discovery rather than fact-checking an old one - thanks to the pressure to publish exciting and novel findings in order to secure jobs.

In a nutshell.

 

[Gijs]

First of all i would like to see an objective scientific definition of what psychology is. If everything is materialistic than the 'mind' doesn't exist. Conclusion: psychology as science is nonsens.

 

[Glycon]

Both studies in question (the 1988 one on facial expressions and the 1998 one on "ego depletion") relate to issues of concern for ME/CFS sufferers. Recent discussion of Chalter et al. on emotional expressiveness elsewhere on these forums testifies to that.

 

[Gijs]

What?! NO!!!

What you should say instead is: If everything is materialistic, then mind is ALSO materialistic. It's not like any serious empirical psychologist ASSUMES that mind is an immaterial soul or something!

Psychology is a real science. The fact that many psychologists overstep its proper boundaries doesn't change that fact.

Give me a scienetifical definition of psychology? What is the psycho or mind, Where is the exact location? Look at Karl Popper it is pseudoscience. Psychotherapy etc.. is not falsifiable.

 

[Gijs]

Pace studies have no objective endpoints or observations. Totally subjective, no science
Research in psychodynamic psychotherapy is simply impossible because the entity it deals with, the psyche, does not lend itself to scientific methodology.(Karasu) If you can show me where the psyche is on an MRI then we can talk more.

 

[wdb]

Psychology is a discipline that has always been very strong methodologically

hahahaa yes with all those therapies that are impractical to blind, difficult to placebo match and outcome measures that can only be subjectively assessed, really methodologically strong!

 

[Large Donner]

Pace studies have no objective endpoints or observations. Totally subjective, no science
Research in psychodynamic psychotherapy is simply impossible because the entity it deals with, the psyche, does not lend itself to scientific methodology.(Karasu) If you can show me where the psyche is on an MRI then we can talk more.

Logic.

***********************************************************************************************************************************************

Dr. T. Byram Karasu:

The author proposes the practice of spiritual psychotherapy, which transcends but does not preclude traditional modalities or strategies of treatment. The terms soul and spirit are distinguished as different transpersonal abstractions, yet are inextricably linked. The former aims at revealing the mystery of relatedness and intimacy in everyday life, the latter at finding the divine in universal life. For the spiritual therapist, these concepts are applied to a therapeutic context of care and compassion--which means love and belief beyond oneself. More specifically, the way to soulfulness requires Love of Others, Love of Work, and Love of Belonging, whereas the way to spirituality requires Belief in the Sacred, Belief in Unity, and Belief in Transformation. By cultivating a soulful and spiritual existence, thus conducting one's clinical practice on the basis of these six tenets of transcendence, the therapist can guide the patient to reach his or her own authentic self.

Bollocks.

 

[Large Donner]

Either you're being sarcastic, or you don't know what the word "logic" means. The post you quoted is literally a nonsensical non sequitur.

Are you using the same dictionary as the PACE authors. They had trouble understanding definitions of improvement and recovery.

I notice you didn't challenge the definition of bollocks though.

 

[Large Donner]

Large Donner said: Are you using the same dictionary as the PACE authors. They had trouble sticking to definitions of improvement and recovery.

I notice you didn't challenge the definition of bollocks though.

I swear I'm not exaggerating when I say that I literally have no idea what you're talking about.

I think that herein lies the problem.

 

[alex3619]

To be fair, just because findings can't be reproduced, it doesn't automatically mean they're wrong. Replication is an important part of the scientific method that helps us nut out what's really going on - it could be that the new researchers did something differently, or that the trend is more subtle than originally thought.

We do need to keep this in mind. Failure to replicate creates doubt, its not proof of more than that. Indeed this is where so many detractors of the XMRV research got it wrong. It takes a well designed study to demonstrate real failure, as happened with XMRV, but what failure to replicate shows is a need for such a study.

 

[Glycon]

Anyway, back on topic... There is a genuine practical problem faced by modern Big Science when it comes to the question of reproducibility. Namely, the cost. Many research projects are so expensive and so time-consuming, that it is not always clear that attempting to reproduce them is worth it, EVEN IF we leave any extra-scientific concerns aside (e.g., "vested interests").

For example, suppose we have $10 million to spend on biomedical research. We could spend this money investigating something that hasn't been investigated before ("choosing to chase a 'new' discovery"). Alternatively, we could spend it reproducing someone else's research. If we choose to do the latter, then we either corroborate the original conclusions, or we don't. If we do, then we learn SOMETHING, but not as much as we could have learned had we pursued original research. Moreover, if it's something worth replicating in the first place, then our prior credence that it is largely correct should be pretty high.

This is less of a problem for psychology (or sociology) than it is for the likes of CERND and NASA, so insofar as there is a ceiling to how expensive and time-consuming methodologically sound studies in psychology can get, certain advantage over cutting-edge particle physics and astrophysics is built into them. It's up to the scientists to make use of this advantage, however...

 

[Sushi]

Please keep this thread on topic. Several off-topic posts have been deleted.