Esther12
Senior Member
- Messages
- 13,774
I have these open, but am not going to be able to read them today, so thought I'd post them up. v Only the last one links to the full paper, and I've not yet searched to see if the others are available for free on-line somewhere.
They could all be rubbish - I was reading something which assumed that these findings were all quite uncontroversial... but lots of false claims as seen as uncontroversially true by some.
One on therapist's tendency to over-estimate their own efficacy:
http://www.ncbi.nlm.nih.gov/pubmed/22662416
Some on the tendency of researchers to find that the therapy they like most is most effective in RCTs they run, even when others find different results:
http://www.sciencedirect.com/science/article/pii/S0272735813000275#gr1
http://www.ncbi.nlm.nih.gov/pubmed/22946981
https://psychology.sas.upenn.edu/system/files/Leykin CPSP 2009 Allegiance.pdf
They could all be rubbish - I was reading something which assumed that these findings were all quite uncontroversial... but lots of false claims as seen as uncontroversially true by some.
One on therapist's tendency to over-estimate their own efficacy:
An investigation of self-assessment bias in mental health providers.
Walfish S, McAlister B, O'Donnell P, Lambert MJ.
Source
Department of Psychology, Brigham Young University, 272 Taylor Building, Provo, UT 84602, USA.
Abstract
Previous research has consistently found self-assessment bias (an overly positive assessment of personal performance) to be present in a wide variety of work situations. The present investigation extended this area of research with a multi-disciplinary sample of mental health professionals. Respondents were asked to: (a) compare their own overall clinical skills and performance to others in their profession, and (b) indicate the percentage of their clients who improved, remained the same, or deteriorated as a result of treatment with them. Results indicated that 25% of mental health professionals viewed their skill to be at the 90th percentile when compared to their peers, and none viewed themselves as below average. Further, when compared to the published literature, clinicians tended to overestimate their rates of client improvement and underestimate their rates of client deterioration. The implications of this self-assessment bias for improvement of psychotherapy outcomes are discussed.
http://www.ncbi.nlm.nih.gov/pubmed/22662416
Some on the tendency of researchers to find that the therapy they like most is most effective in RCTs they run, even when others find different results:
Researcher allegiance in psychotherapy outcome research: An overview of reviews
- a Institute of Social and Preventive Medicine, University of Bern, Switzerland
- b Institute of Psychology, University of Kassel, Germany
- c Institute of Psychology, University of Freiburg, Germany
Abstract
Researcher allegiance (RA) is widely discussed as a risk of bias in psychotherapy outcome research. The relevance attached to RA bias is related to meta-analyses demonstrating an association of RA with treatment effects. However, recent meta-analyses have yielded mixed results. To provide more clarity on the magnitude and robustness of the RA-outcome association this article reports on a meta-meta-analysis summarizing all available meta-analytic estimates of the RA-outcome association. Random-effects methods were used. Primary study overlap was controlled. Thirty meta-analyses were included. The mean RA-outcome association was r = .262 (p = .002, I2 = 28.98%), corresponding to a moderate effect size. The RA-outcome association was robust across several moderating variables including characteristics of treatment, population, and the type of RA assessment. Allegiance towards the RA bias hypothesis moderated the RA-outcome association. The findings of this meta-meta-analysis suggest that the RA-outcome association is substantial and robust. Implications for psychotherapy outcome research are discussed.
http://www.sciencedirect.com/science/article/pii/S0272735813000275#gr1
Is the allegiance effect an epiphenomenon of true efficacy differences between treatments? a meta-analysis.
Munder T, Flückiger C, Gerger H, Wampold BE, Barth J.
Source
Institute of Social and Preventive Medicine, University of Bern, Switzerland. tmunder@ispm.unibe.ch
Abstract
Many meta-analyses of comparative outcome studies found a substantial association of researcher allegiance (RA) and relative treatment effects. Therefore, RA is regarded as a biasing factor in comparative outcome research (RA bias hypothesis). However, the RA bias hypothesis has been criticized as causality might be reversed. That is, RA might be a reflection of true efficacy differences between treatments (true efficacy hypothesis). Consequently, the RA-outcome association would not be indicative of bias but an epiphenomenon of true efficacy differences. This meta-analysis tested the validity of the true efficacy hypothesis. This was done by controlling the RA-outcome association for true efficacy differences by restricting analysis to direct comparisons of treatments with equivalent efficacy. We included direct comparisons of different versions of trauma-focused therapy (TFT) in the treatment of posttraumatic stress disorder (PTSD). RA was measured from the research reports. Relative effect sizes for symptoms of PTSD were calculated. Random effects meta-regression was conducted. Twenty-nine comparisons of TFTs from 20 studies were identified. Initial heterogeneity among relative effect sizes was low. RA was a significant predictor of outcome and explained 12% of the variance in outcomes. The true efficacy hypothesis predicted the RA-outcome association to be zero; however, a substantial association was found. Thus, this study does not support the true efficacy hypothesis. Given findings from psychotherapy research and other fields that support a biasing influence of researcher preferences, RA should be regarded as a causal factor and conceptualized as a threat to the validity of conclusions from comparative outcome studies.
(c) 2012 APA, all rights reserved.
http://www.ncbi.nlm.nih.gov/pubmed/22946981
© 2009 American Psychological Association. Published by Wiley Periodicals, Inc., on behalf of the American Psychological Associ
ation.
All rights reserved. For permissions, please email: journalsrights@oxon.blackwellpublishing.com
54
Blackwell Publishing Inc
Malden, USA
CPSP
Clinical Psychology: Science and Practice
0969-5893
© 2009 American Psychological Association. Published by Blackwell Publishing on behalf of the American Psychological Associatio
n. All rights reserved. For permission, please email: journalsrights@oxon.blackwellpublishing.com
XXX
Original Article
ALLEGIANCE IN PSYCHOTHERAPY OUTCOME RESEARCH • LEYKIN & DERUBEIS
CLINICAL PSYCHOLOGY: SCIENCE AND PRACTICE • V16 N1, MARCH 2009
Allegiance in Psychotherapy Outcome Research: Separating
Association From Bias
Yan Leykin, University of California, San Francisco
Robert J. DeRubeis, University of Pennsylvania
Concern about the contamination of psychotherapy
outcome studies by “allegiance bias”—distortion of
findings because of investigators’ preferences—has led
to the proposal that findings to date should not be used
to make inferences about the relative efficacies of
psychotherapies. It has also been proposed that results
from all such studies should be adjusted to cancel the
presumed distorting effects of allegiances. We argue
that although much effort has been devoted towards
establishing the existence of statistical associations between
allegiances and outcomes, the causal implication—that
investigators’ allegiances influence results—has gone
virtually untested. We present a new vocabulary with
the aim of sharpening the allegiance discourse, and we
propose that research strategies markedly different from
the ones used to date are needed to address some of
the more serious limitations of allegiance bias research.
https://psychology.sas.upenn.edu/system/files/Leykin CPSP 2009 Allegiance.pdf