New Atmosphere, New Vision: Gibson and Whittemore Kick Off Invest in ME Conference 2016
Mark Berry reports on Dr. Gibson's introduction and Dr. Whittemore's keynote speech, at the 11th Invest in ME International ME Conference in London.
Discuss the article on the Forums.

Sense about science document on What is peer review

Discussion in 'Information and Resources' started by Snowdrop, May 16, 2017.

  1. Snowdrop

    Snowdrop Rebel without a biscuit

    Messages:
    2,896
    Likes:
    10,089
    PhoenixDown and barbc56 like this.
  2. wdb

    wdb Senior Member

    Messages:
    1,377
    Likes:
    2,956
    London
    The trouble with peer review is it's often a total circle-jerk, you have psychologists peer reviewing psychology papers, homeopaths reviewing homeopathy papers, etc, journals may even ask you to suggest suitable reviewers for your own submissions. Most crackpot researchers have enough like-minded crackpot peers to rubber-stamp all the research they like. I'd like to see it made much more independent, like non-peer review, papers are reviewed by researchers well outside of your circle who specialise sound methodology and analysis practices.


    Also sense about science: http://senseaboutscience.org/activities/2012-john-maddox-prize/
     
    Webdog, PhoenixDown, slysaint and 7 others like this.
  3. alex3619

    alex3619 Senior Member

    Messages:
    12,479
    Likes:
    35,007
    Logan, Queensland, Australia
    I completely agree. There is a growing need for reviewers to be independent and well versed in sound methodology and analysis. Review circles may be even more damaging than citation circles.
     
    MEMum, Manganus and barbc56 like this.
  4. Snowdrop

    Snowdrop Rebel without a biscuit

    Messages:
    2,896
    Likes:
    10,089
    Agree. In posting this I was hoping to point out that the establishment treats the idea of peer review as laid out akin to stone tablets from on high.

    Clearly the peer review process is not up to standard or fit for purpose since it allows things like the PACE trial to be seen as gods gift to peer reviewed research.

    There are some waves being made I think (around the edges) about peer review. Clearly needs a reformation.
     
    MEMum, SamanthaJ, barbc56 and 2 others like this.
  5. trishrhymes

    trishrhymes Moved to s4me.info

    Messages:
    2,153
    Likes:
    17,870
    Maybe the science journals need to employ experts at peer review to do the job full time, properly paid and given time to study the raw data and protocols properly. This would necessarily drastically cut the number of papers getting published. Which would be a good thing.

    Maybe there could also be a different sort of online repository for any papers people want to put out, with attached raw data, data analysis, protocols etc inviting post publication open reviews.

    And a separate open repository for the pilot studies and too small to be valid studies including most PhD papers to be summarised.

    The whole academic sausage factory, publish or perish system seems to be extremely dysfunctional, especially in psychology / psychiatry where there seems to be little understanding of scientific method and a crisis of replicability.

    Edit: I guess I should really read the Sense about science document before commenting...
     
    MEMum, Sancar, Manganus and 5 others like this.
  6. Snowdrop

    Snowdrop Rebel without a biscuit

    Messages:
    2,896
    Likes:
    10,089
    Yes, I expect there are a number of ways to come at the problem and fix it.

    I suspect though that any new protocols may attract their own problems too.

    In this particular case I think looking outside to see if there is somewhere else that already deals well with the problem of peer review is a good starting place and then applying new measures on a trial basis one at a time in order to assess their effectiveness.
     
    barbc56, JohnM and BurnA like this.
  7. wdb

    wdb Senior Member

    Messages:
    1,377
    Likes:
    2,956
    London
    Post-publication peer review is an interesting proposal to address some of the problems, it could definitely add some transparency and enable a broader range of opinions on the research.
     
    MEMum, Valentijn, barbc56 and 3 others like this.
  8. Snowdrop

    Snowdrop Rebel without a biscuit

    Messages:
    2,896
    Likes:
    10,089
    Will tweet that when I go on twitter.
     
    barbc56 likes this.
  9. chipmunk1

    chipmunk1 Senior Member

    Messages:
    764
    Likes:
    2,818
    Sense about Science seems more like a lobbying group with S. Wessely on board. Run by PR people.

    The reason why they are promoting peer review as reliable is because the process is broke and unreliable in reality.

    They want you to follow scientific publications like a religion. (Infallibility)

    If the priests say something is true you must follow it. If you do not believe what Priest Review says you are irrational or you do not understand Science(God).

    Not sure what the agenda is but they seem to be closely related to the Science Media Centre, a PR agency pretending to be scientists while making sure that industry tobacco science is getting the media exposure it deserves. A thought police for journalists.
    Both of course against the view that ME is a real illness.
     
    Last edited: May 17, 2017
    alex3619 and trishrhymes like this.
  10. trishrhymes

    trishrhymes Moved to s4me.info

    Messages:
    2,153
    Likes:
    17,870
    Edit: Sarcasm alert:

    On their logic, both the PACE trial and the Caroline Wiltshire et al. recent paper demolishing it are published in peer reviewed journals, so are both true. I wonder how they rationalise that one.

    Oh, yes, I remember, they get their chums to write rude and offensive peer reviews of her next paper...
    So that's all right then. Nothing to see here folks, move along.
     
    Last edited: May 17, 2017
    MEMum, Large Donner and barbc56 like this.
  11. barbc56

    barbc56 Senior Member

    Messages:
    3,652
    Likes:
    5,006
    Very interesting topic but certainly confusing as far as the intent of Sense about Science..The award for 2015 is Edzard Ernst, a debunker of pseudoscience. I think he's a credible scientist. However, I'm not sure what his stance is on me/cfs.

    The winners seem to be "survivors of personal criticism", which is irritating and maybe even irrelevant when it comes to good science.

    http://senseaboutscience.org/activities/2015-john-madd
     
  12. trishrhymes

    trishrhymes Moved to s4me.info

    Messages:
    2,153
    Likes:
    17,870
    Edit: Sarcasm alert!

    Ah, yes, who could forget the award of the Maddox prize to that brave seeker after truth S Wessely. :vomit::vomit::vomit::vomit::vomit::vomit::vomit::vomit::vomit:

    I'm just waiting for the day the equally heroic E Crawley :alien: gets nominated. Her recent talk seems to indicate that she just knows in her dear little beating heart that she deserves it.

    Edit: Do I need to put a sarcasm alert on my posts? Probably not.

    Later edit: Done, thanks for the reminder, @alex3619
     
    Last edited: May 17, 2017
    MEMum, wdb and barbc56 like this.
  13. alex3619

    alex3619 Senior Member

    Messages:
    12,479
    Likes:
    35,007
    Logan, Queensland, Australia
    I often do, largely because it may cause problems for any of us with deep brain fog.
     
    MEMum, Manganus and trishrhymes like this.
  14. SamanthaJ

    SamanthaJ Senior Member

    Messages:
    219
    Likes:
    1,460
    Last edited: May 17, 2017
    emsho, Orla and trishrhymes like this.
  15. trishrhymes

    trishrhymes Moved to s4me.info

    Messages:
    2,153
    Likes:
    17,870
    MEMum likes this.
  16. wdb

    wdb Senior Member

    Messages:
    1,377
    Likes:
    2,956
    London
    Orla, alex3619, Snow Leopard and 3 others like this.
  17. Barry53

    Barry53 Senior Member

    Messages:
    2,277
    Likes:
    13,703
    UK
    This is where it can all go so horribly wrong ... right at the start in fact. If the journal is biased, then their choice of reviewers is going to be biased. And aside from that, if the reviewers are confined to others from the same field and who also publish, we are heading into the realms of heavy selection bias of reviewers.

    I can understand good reasons why you must include reviewers from the same field - of course you must. But there are also many things common across science, and a good sprinkling of other stakeholders is usually crucial. In engineering peer reviews there will often be a wide spread of expertise amongst reviewers, to catch those issues the "usual experts" might miss.
     
    MEMum, Aroa, Snowdrop and 4 others like this.
  18. alex3619

    alex3619 Senior Member

    Messages:
    12,479
    Likes:
    35,007
    Logan, Queensland, Australia
    What I call a review circle. A cozy little family.
     
    MEMum, Aroa, barbc56 and 1 other person like this.
  19. RogerBlack

    RogerBlack Senior Member

    Messages:
    883
    Likes:
    2,860
    Open review can at least bring this into the open.
    Especially open review where outsiders can comment on the reviews.
    Otherwise (to be charitable) reviewers can make bald statements which are in fact incorrect, and pull the wool over editors who are not skilled in the field.
     
    Barry53 likes this.
  20. user9876

    user9876 Senior Member

    Messages:
    2,583
    Likes:
    18,182
    moderated with many others
    Also reviewers can give their friends and fellow believers an easy time.
     
    Barry53, aaron_c, MEMum and 1 other person like this.

See more popular forum discussions.

Share This Page