• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

"When does peer review make no damn sense?" (Andrew Gelman)

Dolphin

Senior Member
Messages
17,567
http://andrewgelman.com/2016/02/01/peer-review-make-no-damn-sense/

When does peer review make no damn sense?
Posted by Andrew on 1 February 2016, 10:55 am

Disclaimer: This post is not peer reviewed in the traditional sense of being vetted for publication by three people with backgrounds similar to mine. Instead, thousands of commenters, many of whom are not my peers—in the useful sense that, not being my peers, your perspectives are different from mine, and you might catch big conceptual errors or omissions that I never even noticed—have the opportunity to point out errors and gaps in my reasoning, to ask questions, and to draw out various implications of what I wrote. Not “peer reviewed”; actually peer reviewed and more; better than peer reviewed.

Peer review has its place. But peer reviewers have blind spots. If you want to really review a paper, you need peer reviewers who can tell you if you’re missing something within the literature—and you need outside reviewers who can rescue you from groupthink.

Peer review is subject to groupthink, and peer review is subject to incentives to publishing things that the reviewers are already working on.

I think that patients (and other family members/similar) can add new and useful perspectives that can be missed by reviewers.

http://www.stat.columbia.edu/~gelman/
Andrew Gelman is a professor of statistics and political science and director of the Applied Statistics Center at Columbia University.
 

whodathunkit

Senior Member
Messages
1,160
Peer-review is also subject to cronyism. And commercialism (pay-to-publish or pay for positive/problem-free review).

And more disturbingly these days, "popular narrative"-support-ism (support of a popular narrative regardless of whether it's scientifically supportable).

Peer-review seems subject also to general stupid-ism, in that a lot of what gets published in ostensibly reputable publications apparently isn't reviewed by the brightest lights on the tree.

That said, there is no fool-proof system. We all know that reviewers spam "cattle call" review systems like Yelp and Amazon to either make the thing being reviewed seem much worse or much better than it really is. Perhaps a blend of the two systems would be best: let it pass actual peer review, then put it in the open to be reviewed by others.
 
Last edited: