• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Researcher allegiance in psychotherapy outcome research: An overview of reviews (Munder et al. 2013)

Dolphin

Senior Member
Messages
17,567
Research psychologist, James C. Coyne on Twitter highlighted this as:
Meta-analysis suggest that the researcher allegiance effects substantial and robust in psychotherapy research.


Researcher allegiance in psychotherapy outcome research: An overview of reviews

Clinical Psychology Review

Volume 33, Issue 4, June 2013, Pages 501–511

Thomas Mundera, b, , ,
Oliver Brütscha,
Rainer Leonhartc,
Heike Gergera,
Jürgen Bartha

a Institute of Social and Preventive Medicine, University of Bern, Switzerland
b Institute of Psychology, University of Kassel, Germany
c Institute of Psychology, University of Freiburg, Germany

http://dx.doi.org/10.1016/j.cpr.2013.02.002,
Abstract*
Researcher allegiance (RA) is widely discussed as a risk of bias in psychotherapy outcome research.


The relevance attached to RA bias is related to meta-analyses demonstrating an association of RA with treatment effects. However, recent meta-analyses have yielded mixed results.


To provide more clarity on the magnitude and robustness of the RA-outcome association this article reports on a meta-meta-analysis summarizing all available meta-analytic estimates of the RA-outcome association.


Random-effects methods were used. Primary study overlap was controlled. Thirty meta-analyses were included.


The mean RA-outcome association was r = .262 (p = .002, I2 = 28.98%), corresponding to a moderate effect size.


The RA-outcome association was robust across several moderating variables including characteristics of treatment, population, and the type of RA assessment.


Allegiance towards the RA bias hypothesis moderated the RA-outcome association.


The findings of this meta-meta-analysis suggest that the RA-outcome association is substantial and robust.


Implications for psychotherapy outcome research are discussed.
_______________________________
Highlights

► An overview of reviews on researcher allegiance and outcome was conducted.
► A substantial association of researcher allegiance and outcome was found.
► Moderator analyses supported the robustness of the allegiance-outcome association.
► Researcher allegiance can be considered a risk of bias in psychotherapy outcome research.

Keywords

Researcher allegiance;
Psychotherapy outcome research;
Overview of reviews


*I gave each sentence its own paragraph
 

Simon

Senior Member
Messages
3,789
Location
Monmouth, UK
Research psychologist, James C. Coyne on Twitter highlighted this

"The mean RA-outcome association was r = .262 (p = .002, I2 = 28.98%), corresponding to a moderate effect size."
Which is particularly interesting as the typical effect size in such therapies is also 'moderate'.

I'm not sure if this means the RA bias is big enough to account for most of the observed outcome effect size in trials, or just that, on average, RA bias has a moderate effect on the reported moderate outcome :)
 

Dolphin

Senior Member
Messages
17,567
Which is particularly interesting as the typical effect size in such therapies is also 'moderate'.

I'm not sure if this means the RA bias is big enough to account for most of the observed outcome effect size in trials, or just that, on average, RA bias has a moderate effect on the reported moderate outcome :)
I'm afraid you're just going to have to do a PhD in statistics ;) (alternatively they didn't give enough info).
 
Messages
13,774
Thanks. I've seen a few things about this recently, and they always remind me of the poor and spun results we see from BPS CFS studies run by those with a clear 'allegiance'.
 

Dolphin

Senior Member
Messages
17,567
Which is particularly interesting as the typical effect size in such therapies is also 'moderate'.

I'm not sure if this means the RA bias is big enough to account for most of the observed outcome effect size in trials, or just that, on average, RA bias has a moderate effect on the reported moderate outcome :)
Yesterday, James C. Coyne tweeted (relating to a blog on this paper):

MT @benmeg Association between therapy allegiance and outcome appears stronger than effects of specific therapy. http://bit.ly/114Wa8W
 

Dolphin

Senior Member
Messages
17,567
Blog on this paper which tries to explain the area a bit*
I believe in CBT and my research shows it works! Therapy allegiance in psychotherapy research

http://criticalscience.com/researcher-allegiance-psychotherapy-research-bias.html

*although then talking about the possible bias for the allegiance bias hypothesis in some allegiance bias researchers (but not others) as well as a discussion of a "meta-meta-analysis" may undo the explanation and leave some confused by the end!?
 
Messages
13,774
*although then talking about the possible bias for the allegiance bias hypothesis in some allegiance bias researchers (but not others) as well as a discussion of a "meta-meta-analysis" may undo the explanation and leave some confused by the end!?

I wonder what the allegiance of those doing the meta-meta-analysis was?

My take home: unless the research shows a consistent and large affect for an intervention, don't bother with it in the real world as it's probably not worth the as yet undetected side-effects. People need less medical advice, and there should be more accountability for making money from giving poor and misleading medical advice.
 

Dolphin

Senior Member
Messages
17,567
I wonder what the allegiance of those doing the meta-meta-analysis was?
Fair point.

My take home: unless the research shows a consistent and large affect for an intervention, don't bother with it in the real world as it's probably not worth the as yet undetected side-effects. People need less medical advice, and there should be more accountability for making money from giving poor and misleading medical advice.
I'm not sure one can read all that from the study(s), although it may well be true.

Aside: this is one of the main reasons I have a soft spot for infectious theories. If you treat infections in such situations, it seems like it might increase the chance it is a useful therapy (over knocking out abnormalities in other models for the illness).
 
Messages
13,774
Fair point.

I feel like I'm looking at one of those M C Esher infinite perspective things!

I'm not sure one can read all that from the study(s), although it may well be true.

No, not from just this one study - but I've come to realise that I used to have a quite foolish faith in expertise and science (even while thinking I was a cynical sceptic), and that quite a lot of research in different areas is badly done and misleading. Ah well.