"6 Reasons You Can't Trust Science Anymore" Aug 13 (lists some problems, don't think title is great)

[Dolphin]

(August 13, 2015)
http://www.cracked.com/article_22712_6-ways-modern-science-has-turned-into-giant-scam.html

Headings:

#6. Negative Results Are Ignored [i.e. not published (often)]

#5. Scientists Don't Have To Show Their Work

#4. You Have To Pay To Get Published

#3. It's All About Profit [publishers' fees, etc.]

#2. No One Can Share Their Work

#1. Predatory Companies Publish Sham Science

http://www.cracked.com/article_22712_6-ways-modern-science-has-turned-into-giant-scam.html

 

[whodathunkit]

Interesting article! Although I would argue that #6 ought to be #1, given that it's the primary reason that people are not taking science and scientists as seriously as they once did. It's a big credibility problem when you only publish stuff that suits your preferred narrative. It's being done in all areas, from food stuffs to climate change. Sadly, the trend seems to be rampant. "He who controls the narrative controls the universe." Or something like that.

 

[WillowJ]

A link I followed from there tells how a science journalist (PhD in molecular biology of bacteria) organized a study specifically designed to show how badly-designed studies get picked up by the press and reported as fact, when they may in fact be "noise".
http://io9.com/i-fooled-millions-into-thinking-chocolate-helps-weight-1707251800

My colleagues and I recruited actual human subjects in Germany. We ran an actual clinical trial, with subjects randomly assigned to different diet regimes. And the statistically significant benefits of chocolate that we reported are based on the actual data. It was, in fact, a fairly typical study for the field of diet research. Which is to say: It was terrible science. The results are meaningless, and the health claims that the media blasted out to millions of people around the world are utterly unfounded.

Although published in a vanity journal, it was pretty easy to make all the headlines.

particularly telling:

Not a single reporter seems to have contacted an outside researcher. None is quoted.

These publications, though many command large audiences, are not exactly paragons of journalistic virtue....

But even the supposedly rigorous outlets that picked the study up failed to spot the holes.

Shape magazine’s reporting on our study—turn to page 128 in the June issue—employed the services of a fact-checker, but it was just as lackadaisical.

All the checker did was run a couple of sentences by me for accuracy and check the spelling of my name. The coverage went so far as to specify the appropriate cocoa content for weight-loss-inducing chocolate (81 percent) and even mentioned two specific brands (“available in grocery stores and at amazon.com”).

The only problem with the diet science beat is that it’s science. You have to know how to read a scientific paper—and actually bother to do it.

For far too long, the people who cover this beat [I think he could mean nutrition/diet studies particularly, but I suspect it applies to other areas as well] have treated it like gossip, echoing whatever they find in press releases. Hopefully our little experiment will make reporters and readers alike more skeptical.

There was one glint of hope in this tragicomedy. While the reporters just regurgitated our “findings,” many readers were thoughtful and skeptical. In the online comments, they posed questions that the reporters should have asked.

 

[Woolie]

1 reason you can't trust articles about how you can't trust Science any more:

* They see all these problems as somehow new evils, unique to the terrible modern times we live in.

None of them are all that novel, except perhaps paying for publishing, but even that's been going on since the 70's. .... But then how can you write an "evils of our modern age" narrative if you acknowledge that?

I know, I'm missing the point. Just hate those "evil of our modern age" stories!

 

[Woolie]

Okay, on a more serious note, I was thinking about biases in science. Here are some more:

1. Publication bias. That's the one about non-significant findings not getting published. This bias also influences replications: Studies that are replications are less likely to get published than "novel" studies, so this means work is not getting properly replicated... it can take years to discover a finding is dodgy!

2. Citation bias: the tendency to cite articles that support a prevailing view over those that cast doubt on it

3. Outcome reporting bias: the tendency for researchers to report outcomes that yield significant effects and not report on those that yield non-significant different.

4. Failure to correct for multiple comparisons. So for example, a study of CFS patients gets them to complete 5 questionnaires, measuring 16 different psychological constructs (not unusual), and compares results to some control group. Guess what the chances are of getting at least 1 significant difference? Almost 100%!

And here are a few bias that in Psychology that affect the way the actual participants respond:

5. Response bias: a collection of biases that affect how people complete self-report questionnaires. For example, they may be biased to respond in ways that are socially desirable, or in ways that please the researcher

6. Recall bias: if you ask someone about their past when they are depressed, they are more likely to recall negative than positive events. Same goes for people who are severely ill.