• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Coyne: Does anyone read the classic studies they cite?

Messages
13,774
Full paper: http://www.ehps.net/ehp/issues/2007/v9iss3_August2007/EHP_Aug07_Coyne&Palmer.pdf

I'm sure I've seen others discuss this on here, but cannot find any thread about it.

It focuses on claims that psychological therapies increase life-span for cancer patients, but it also makes general points relevant to CFS.

I was going to quote relevant sections, but it's only four pages long, and is almost all relevant. These are the five main points, and I quoted the final three in full, as they seem particularly relevant.



1. Primary sources, even classics, often go unread.



2. Critical appraisal skills and the ability to apply
basic standards for interpreting clinical trials are
in short supply in psychology.


3. Findings that are in sync with cultural beliefs and
values can take on a life of their own, and
dethroning these findings does not make one
popular.


In the case of the claims made by Spiegel and his
colleagues, as well as later commentators, the idea that
patients should view their illness as a personal
responsibility to be overcome through the hard work of
psychotherapy appealed to strongly held values,
particularly in North American culture. Of course, the
study ought to have shown that patients can extend
their lives. Didn’t we know that already, even if there
had not yet been a study? Skeptics risk being seen as
rejecting what we already know and as undermining the
coping efforts of patients.


4.
Numerous groups had a vested interest in the
results of studies being seen as having positive
outcomes.


We often think of “conflict of interest” as more a
circumscribed issue than it most likely is in practice.
Beliefs are shaped by needs as much as evidence. As
Lesperance and Frasure-Smith (1999) pointed out
“Prevention of mortality has always been one of the
most important factors in determining the allocation
of funding for research and clinical activities.”
Findings that psychotherapy prolongs the lives of
cancer patients is extremely useful, even vital for
advancing the claims of diverse groups, ranging from
researchers seeking funding for
psychoneuroimmunology studies to promoters of the
virtues of mind control and positive thinking, most
recently seen in the huge popularity of Rhonda
Byrne’s 2006 book, The Secret. Those who see a
benefit for the credibility of their own claims are
going to have a stake in promoting and protecting the
claim that psychotherapy promotes survival.


5. A persistent champion can play a key role in
promoting the value of an intervention in the face
of contrary evidence.


Spiegel and his colleagues repeated claims that the
original study had shown that psychotherapy
prolongs life over two dozen times in journal articles,
as well as in numerous presentations to lay and
professional audiences, and even on national
television. As was discovered by Bernard Fox and
others, critics were excoriated (cf. Goodwin et al.,
1999). Moreover, one might have assumed that a
consistent pattern of failed attempted replications
would have caused a reevaluation of the original
study. However, champions of the original study
countered these new results by reinterpreting other
studies as positive and of equivalent value (Spiegel &
Giese-Davis, 2003), despite these studies not being
designed to test whether psychotherapy improved
survival and also having confounded psychosocial
intervention with improved medical care (Coyne et
al., 2007). There was a distinct bracket creep in what
was considered relevant evidence, allowing portrayal
of the overall subsequent literature as being mixed,
rather than more uniformly negative.


This is the conclusion:


What larger lessons are to
be learned? First, we need to read original sources. We
encourage prospective authors to read carefully the
studies they cite, even when there is near unanimity in
secondary sources about the nature of findings being
reported. Second, we encourage scholars to acquire and
apply the critical skills needed to appraise the claims
they find in published articles. These skills are sorely
needed, and critical application of them can be an
important contribution to the literature. But yes, if you
take on the task of challenging entrenched, but
erroneous, claims you must be prepared to take some
heat.
 

SOC

Senior Member
Messages
7,849
First, we need to read original sources. We
encourage prospective authors to read carefully the
studies they cite, even when there is near unanimity in
secondary sources about the nature of findings being
reported.

What!? They don't actually read the papers they cite? :eek: I'd flunk an undergraduate for that. What has happened to scientific research? :( Or is this just the psychotherapy researchers who don't understand the basics of scientific research?
 
Messages
13,774
What!? They don't actually read the papers they cite? :eek: I'd flunk an undergraduate for that. What has happened to scientific research? :( Or is this just the psychotherapy researchers who don't understand the basics of scientific research?

One of the real surprises for me on reading more CFS research carefully, and checking citations, was to realise how often cited papers do not support the point they're supposed to. It's something that other people here constantly draw attention to as well.
 

Shell

Senior Member
Messages
477
Location
England
Reading research papers has been a real eye-opener for me. So many published papers are so badly written. I get particularly naffed off with assertions made that have no references to back them up (usually, I think because there are no references to back them up). I can't help thinking those assertions can then be referenced in other papers because they were published in the first place.
Some of the papers I've read I would not have accepted from a nursing assistant doing an NVQ 2 let alone someone supposedly qualified!
 

Dolphin

Senior Member
Messages
17,567
Re:

3. Findings that are in sync with cultural beliefs and values can take on a life of their own, and dethroning these findings does not make one popular.

In the case of the claims made by Spiegel and his colleagues, as well as later commentators, the idea that patients should view their illness as a personal responsibility to be overcome through the hard work of psychotherapy appealed to strongly held values, particularly in North American culture. Of course, the study ought to have shown that patients can extend their lives. Didn’t we know that already, even if there had not yet been a study? Skeptics risk being seen as rejecting what we already know and as undermining the coping efforts of patients.
I think something similar may happen in ME/CFS with the appeal to medical and health professionals of rehabilitative therapies (graded exercise therapy, graded activity-oriented CBT, etc.).
 
Last edited:

taniaaust1

Senior Member
Messages
13,054
Location
Sth Australia
One of the real surprises for me on reading more CFS research carefully, and checking citations, was to realise how often cited papers do not support the point they're supposed to. It's something that other people here constantly draw attention to as well.

That's an issue with the CFS article at wikipedia (well at least was, I assume things there hadnt changed). Things written in good support of the things we know (which made that feel a touch better to us).. were backed by links which werent even to do with what they were supposed to support (not surprisely bad info links put onto info we'd support).

Hence I think there is purposely deception going on there too about things and not always the case that the one who put the links are naive about what they were linking too or just a mistake (thou it could often be the case).
 

biophile

Places I'd rather be.
Messages
8,977
As we have seen with the PACE researchers, White and Chalder at least, sometimes researchers do not even bother to read their own published papers properly before citing them incorrectly to support exaggerated claims.

Coyne said:
If you take on the task of challenging entrenched, but erroneous, claims you must be prepared to take some heat.

Ain't that the truth!
 
Last edited:

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
Blog by James Coyne from Sep 2013. (I can't see it posted anywhere on the forum.)

"Seven studies claimed to show psychotherapy extends survival of cancer patients did not"

http://jcoynester.wordpress.com/201...-extends-survival-of-cancer-patients-did-not/

I like Coyne's articles.

Here is an important bit:
When adequately in place, peer review should ensure that claims are reconciled with the available literature, not only for professionals, but for consumers, in this case, patients. Authors are certainly free to express their opinions, but should place them within the ongoing debate within the scientific community.

I wish more psychiatrists paid attention to this.