• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

"Science is harmed by hype. How to live for 969 years" (updated version of a 2006 blog post)

Dolphin

Senior Member
Messages
17,567
Just on problems in scientific research in general. Links to a nice spoof.

Science is harmed by hype. How to live for 969 years.
February 2nd, 2006

The New York Times (17 January 2006) published a beautiful spoof that illustrates only too clearly some of the bad practices that have developed in real science (as well as in quackery). It shows that competition, when taken to excess, leads to dishonesty

continues at: http://www.dcscience.net/?p=156
.
 

Simon

Senior Member
Messages
3,789
Location
Monmouth, UK
Just on problems in scientific research in general. Links to a nice spoof.
Science is harmed by hype. How to live for 969 years.
Thanks, it's a good read, eg:
It also shows the only-too-frequent failure of peer review to detect problems.

The future lies on publication on the web, with post-publication peer review. It has been shown by sites like PubPeer that anonymous post-publication review can work very well indeed.
What has suprised me is the extent to which the top journals, in their bid to find eyecatching findings, publish poor studies too (alongside more great stuff than lesser jourals). At least one top scientist is taking a stand:
Recent Nobel-prizewinner Randy Schekman has helped with his recent declaration that "his lab will no longer send papers to Nature, Cell and Science as they distort scientific process"

This shows that the problem is by no means confined to psychology or CFS research. The PubPeer link above is about a biomedical paper on the detection of disease biomarkers where the data presented imply the detection of single molecules of biomarkers. Even if this were the case, numerous posters point out there should be far more noise in the data: if you have multiple samples and an average of one biomarker molecule per sample, some will actually have 0, others 2 or 3 etc. Yet the data didn't show such noise, and the Nature editor contacted about this didn't seem to want to discuss the problem seriously. I know that sort of problem has come up in published CFS research too.

Perhaps the biggest difference is that in CFS it's left almost entirely to patients to challenge poor research (XMRV excepted), as biophile notes on another thread:
These problems go almost entirely unchallenged by professionals involved with CFS, leaving patients and carers to do it, ...

Talking about "vigilantes" in Neuroskeptic's post, i.e. people scrutinizing research papers because the authorities have failed or seem incapable of doing so, that is what many people scrutinizing CFS research could be described as.
I know scienitists who say that in their fields strong criticism is routine from fellow-professionals. While this can descend into people gaming the system (eg reviewer's unreasonably blocking a competitor's paper) I think it's a much healthier state of affairs.

Thanks. This from the Peter Lawrence paper [pdf above] sums up the dilemma faced by many scientists forced to chase publication over discovery:
It has become vital to get papers into high impact-factor journals; just one such paper can change the prospects of a postdoc from nonexistent to substantial (because of the weight put on such papers by grant-awarding bodies). Two or three such papers can make the difference between unemployment and tenure. These facts have cut a swathe through scientific thinking like a forest fire, turning our thoughts and efforts away from scientific problems and solutions, and towards the process of submission, reviewing and publication.