Part B: Churnalism and Dogmatic Verificationism
Churnalism
When a study is reported the science journalists get to report on it and then fairly and accurately reflect on the paper's importance, right? Or at least thats what is supposed to happen. For a general discussion of churnalism you might like to watch this.
The media has started to question its own capacity to present accurate news in recent years. The practice of churnalism is the reprinting of news stories, especially press releases, without appropriate fact checking or analysis.
According to Martin Robbins there are two major problems with churnalism in science reporting. The first is that press releases become unpaid advertising: they are not thoughtful reviews that get published but uncritical promotion of the ideas. The second issue is that it is undermining science journalism. The journalists who investigate closely are no longer necessary.
In 2009 there was a US survey of health care journalists. 28% cited newsroom cutbacks as a challenge, whicle other issues including lack of depth in coverage were also noted. Inexperience for health care journalists was rated a concern by 16% of responders, and undue influence ranked at 12%. 53% say the amount of time they have to research a story has declined. Two of the most common topics that get the worst coverage are disparities in health (though its not unclear if this is about health, or health care, or both) and health care politics. Just over two-thirds are unhappy with the direction that such coverage is going in their respective organizations.
There is now a site for checking press releases to see how much it is churned in the media: http://www.churnalism.com
The modern media age is a real problem for traditional news outlets. Revenue is down and the complexity of issues to be covered ... well, its not getting simpler. In a bid to cut costs the fourth estate is becoming yet another victim of excessive economic rationalism.
Schwitzer discusses the effect of this on public health care. One thing that is a concern is that editors of health care journalists often don't know enough on the topic. One of many possible interventions that he discusses is that the public should make sure that news organizations know when they are unhappy with news coverage.
Press releases and media responses are known to have the potential to severely distort the science. Timmer recounts the issues with human genome research in the ENCODE project. The results were reported as : "These data enabled us to assign biochemical functions for 80 percent of the genome." The problem is this was wrong. One of the fallacies that is promoted is that because some junk DNA is found to have a use, then most or all of it must have a use. This might be proven correct in time, but is not supported by the current science. What is supported is that the percentage of junk DNA, that is DNA that appears to have no useful function, is declining. Much of this junk DNA originally had a biochemical function, but that function is now lost. Ambiguous terms and imprecision in general can easily lead to reporters mis-reporting findings. The ENCODE project reported on how much of the DNA was transcribed into RNA, but could not determine if that RNA was useful. The media ignored this.
So what did the ENCODE researchers do? They redefined the term "functional". However the new definition does not differentiate between regulatory DNA and junk DNA. Both are "functional". When terms are redefined the media often misses the significance of the change. Timmer notes that even a journal like Nature misreported this: "Far from being junk, the vast majority of our DNA participates in at least one biochemical event in at least one cell type." Nature was not alone in this.
Timmer goes on to say that the ENCODE team has some responsibility for the mistake: they should have ensured that the press release was accurate. The press release officers should also have confirmed accuracy. Of course I might add that the journalists who reported on it should also have checked it. Thats three failures in a row in order to get errors like this to press.
"Unfortunately, this is a case where scientists themselves get these details wrong very often, and their mistakes have been magnified by the press releases and coverage that has resulted. That makes it much more likely that any future coverage of these topics will repeat the past errors." Timmer
Similar issues to these are noteworthy as occuring in coverage of the PACE trial which I will blog about in time.
I was hoping to discuss ideas about how to improve quality of science publishing, but that will have to wait until another blog as I want this blog out by Christmas.
Dogmatic Verificationism Revisited
Dogmatic verificationism, a single minded attempt to support an hypothesis or model without careful regard for testing for accuracy is extremely problematic under conditions like we have today with zombie science and churnalism. It might be considered an extreme form of confirmation bias. The only real counter to dogmatic verificationism is critical analysis from scientific peers, the wider scientific audience, the media and other interested parties. It must be conducted in an open fashion, where even raw data and fine details of experimental design are available to other scientists. Open publishing is a step toward this.
When this is not happening, when there is no opposition, dogmatic verificationism can promote the view that no opposition means they must be right. This can apply generally, but also within specific fields. For example there are not many cautionary tales coming from within the psychosomatic medical literature. Those who read only within this field might therefore draw the conclusion that nearly all the research and researchers support these views.
Zombie science means a dogmatic untested view can be pushed far beyond what is warranted. Churnalism means that press releases and other commentaries can be received without sufficient scrutiny, further rubber stamping the view. The gatekeepers of science who are supposed to adequately scrutinize and review material presented for publication are increasingly failing in the goal of publishing quality research.
This is a systemic failure in the balance of power - its an unchecked system of processes that can let unreliable science sound very appealing. Balance of power is the subject of my next blog: A Pen or a Sword?
Churnalism
When a study is reported the science journalists get to report on it and then fairly and accurately reflect on the paper's importance, right? Or at least thats what is supposed to happen. For a general discussion of churnalism you might like to watch this.
The media has started to question its own capacity to present accurate news in recent years. The practice of churnalism is the reprinting of news stories, especially press releases, without appropriate fact checking or analysis.
According to Martin Robbins there are two major problems with churnalism in science reporting. The first is that press releases become unpaid advertising: they are not thoughtful reviews that get published but uncritical promotion of the ideas. The second issue is that it is undermining science journalism. The journalists who investigate closely are no longer necessary.
In 2009 there was a US survey of health care journalists. 28% cited newsroom cutbacks as a challenge, whicle other issues including lack of depth in coverage were also noted. Inexperience for health care journalists was rated a concern by 16% of responders, and undue influence ranked at 12%. 53% say the amount of time they have to research a story has declined. Two of the most common topics that get the worst coverage are disparities in health (though its not unclear if this is about health, or health care, or both) and health care politics. Just over two-thirds are unhappy with the direction that such coverage is going in their respective organizations.
There is now a site for checking press releases to see how much it is churned in the media: http://www.churnalism.com
The modern media age is a real problem for traditional news outlets. Revenue is down and the complexity of issues to be covered ... well, its not getting simpler. In a bid to cut costs the fourth estate is becoming yet another victim of excessive economic rationalism.
Schwitzer discusses the effect of this on public health care. One thing that is a concern is that editors of health care journalists often don't know enough on the topic. One of many possible interventions that he discusses is that the public should make sure that news organizations know when they are unhappy with news coverage.
Press releases and media responses are known to have the potential to severely distort the science. Timmer recounts the issues with human genome research in the ENCODE project. The results were reported as : "These data enabled us to assign biochemical functions for 80 percent of the genome." The problem is this was wrong. One of the fallacies that is promoted is that because some junk DNA is found to have a use, then most or all of it must have a use. This might be proven correct in time, but is not supported by the current science. What is supported is that the percentage of junk DNA, that is DNA that appears to have no useful function, is declining. Much of this junk DNA originally had a biochemical function, but that function is now lost. Ambiguous terms and imprecision in general can easily lead to reporters mis-reporting findings. The ENCODE project reported on how much of the DNA was transcribed into RNA, but could not determine if that RNA was useful. The media ignored this.
So what did the ENCODE researchers do? They redefined the term "functional". However the new definition does not differentiate between regulatory DNA and junk DNA. Both are "functional". When terms are redefined the media often misses the significance of the change. Timmer notes that even a journal like Nature misreported this: "Far from being junk, the vast majority of our DNA participates in at least one biochemical event in at least one cell type." Nature was not alone in this.
Timmer goes on to say that the ENCODE team has some responsibility for the mistake: they should have ensured that the press release was accurate. The press release officers should also have confirmed accuracy. Of course I might add that the journalists who reported on it should also have checked it. Thats three failures in a row in order to get errors like this to press.
"Unfortunately, this is a case where scientists themselves get these details wrong very often, and their mistakes have been magnified by the press releases and coverage that has resulted. That makes it much more likely that any future coverage of these topics will repeat the past errors." Timmer
Similar issues to these are noteworthy as occuring in coverage of the PACE trial which I will blog about in time.
I was hoping to discuss ideas about how to improve quality of science publishing, but that will have to wait until another blog as I want this blog out by Christmas.
Dogmatic Verificationism Revisited
Dogmatic verificationism, a single minded attempt to support an hypothesis or model without careful regard for testing for accuracy is extremely problematic under conditions like we have today with zombie science and churnalism. It might be considered an extreme form of confirmation bias. The only real counter to dogmatic verificationism is critical analysis from scientific peers, the wider scientific audience, the media and other interested parties. It must be conducted in an open fashion, where even raw data and fine details of experimental design are available to other scientists. Open publishing is a step toward this.
When this is not happening, when there is no opposition, dogmatic verificationism can promote the view that no opposition means they must be right. This can apply generally, but also within specific fields. For example there are not many cautionary tales coming from within the psychosomatic medical literature. Those who read only within this field might therefore draw the conclusion that nearly all the research and researchers support these views.
Zombie science means a dogmatic untested view can be pushed far beyond what is warranted. Churnalism means that press releases and other commentaries can be received without sufficient scrutiny, further rubber stamping the view. The gatekeepers of science who are supposed to adequately scrutinize and review material presented for publication are increasingly failing in the goal of publishing quality research.
This is a systemic failure in the balance of power - its an unchecked system of processes that can let unreliable science sound very appealing. Balance of power is the subject of my next blog: A Pen or a Sword?
