Researchers overturn landmark study on the replicability of psychological science

A.B.

Senior Member
Messages
3,780
Likes
23,086
The funny thing is that replicability doesn't even mean the results are true. It may just mean that the same biases leading to the false appearance of a result are present.

I recall one study where a placebo was as good as asthma medication if one only looked at subjective measures. Only looking at objective measures revealed that the placebo actually made no difference. Studies of psychological interventions rarely look at objective measures. When they do, they may simply ignore inconvenient null results on objective measures (PACE being a good example)

Yes, we cannot compare psychotherapy to placebo, but that is no excuse for not even trying to minimize bias, because there are other ways as well.

It's a junk science factory.
 

Gijs

Senior Member
Messages
678
Likes
1,401
Where is the psyche? In what place? Can someone show me on mri? The psyche does not exist objectively. Conclusion: psychological illnesses are nonsense. We are our brains.
 
Messages
5,902
Likes
12,705
Location
South Australia
When we compared the replication rates of the endorsed and unendorsed protocols, we discovered that the endorsed protocols were nearly four times as likely to produce a successful replication (59.7%) as were the unendorsed protocols (15.4%).

This strongly suggests that the infidelities did not just introduce random error but instead biased the replication studies toward failure. If OSC had limited their analyses to endorsed studies, they would have found that 59.7% [95% confidence interval (CI): 47.5%, 70.9%] were replicated successfully.

In fact, we estimate that if all the replication studies had been high enough in fidelity to earn the endorsement of the original authors, then the rate of successful replication would have been 58.6% (95% CI: 47.0%, 69.5%) when controlling for relevant covariates.

Remarkably, the CIs of these estimates actually overlap the 65.5% replication rate that one would expect if every one of the original studies had reported a true effect.

Although that seems rather unlikely, OSC’s data clearly provide no evidence for a “replication crisis” in psychological science.
60%? Gee, I'm so impressed! Crisis over!
 
Last edited:

*GG*

senior member
Messages
6,381
Likes
5,089
Location
Concord, NH
When we compared the replication rates of the endorsed and unendorsed protocols, we discovered that the endorsed protocols were nearly four times as likely to produce a successful replication (59.7%) as were the unendorsed protocols (15.4%).

This strongly suggests that the infidelities did not just introduce random error but instead biased the replication studies toward failure. If OSC had limited their analyses to endorsed studies, they would have found that 59.7% [95% confidence interval (CI): 47.5%, 70.9%] were replicated successfully.

In fact, we estimate that if all the replication studies had been high enough in fidelity to earn the endorsement of the original authors, then the rate of successful replication would have been 58.6% (95% CI: 47.0%, 69.5%) when controlling for relevant covariates.

Remarkably, the CIs of these estimates actually overlap the 65.5% replication rate that one would expect if every one of the original studies had reported a true effect.

Although that seems rather unlikely, OSC’s data clearly provide no evidence for a “replication crisis” in psychological science.

60%? Gee, I'm so impressed! Crisis over!
Breaking this up, to much freaking text in a large block!

GG