Conflicts of interest and spin in reviews of psychological therapies: a systematic review

[Bob]

Conflicts of interest and spin in reviews of psychological therapies: a systematic review.
Lieb K, von der Osten-Sacken J, Stoffers-Winterling J, Reiss N, Barth J.
BMJ Open 2016;6:e010606
26 April 2016
doi:10.1136/bmjopen-2015-010606

http://dx.doi.org/10.1136/bmjopen-2015-010606
http://bmjopen.bmj.com/content/6/4/e010606

Abstract
Objective To explore conflicts of interest (COI) and their reporting in systematic reviews of psychological therapies, and to evaluate spin in the conclusions of the reviews.

Methods MEDLINE and PsycINFO databases were searched for systematic reviews published between 2010 and 2013 that assessed effects of psychological therapies for anxiety, depressive or personality disorders, and included at least one randomised controlled trial. Required COI disclosure by journal, disclosed COI by review authors, and the inclusion of own primary studies by review authors were extracted. Researcher allegiance, that is, that researchers concluded favourably about the interventions they have studied, as well as spin, that is, differences between results and conclusions of the reviews, were rated by 2 independent raters.

Results 936 references were retrieved, 95 reviews fulfilled eligibility criteria. 59 compared psychological therapies with other forms of psychological therapies, and 36 psychological therapies with pharmacological interventions. Financial, non-financial, and personal COI were disclosed in 22, 4 and 1 review, respectively. 2 of 86 own primary studies of review authors included in 34 reviews were disclosed by review authors. In 15 of the reviews, authors showed an allegiance effect to the evaluated psychological therapy that was never disclosed. Spin in review conclusions was found in 27 of 95 reviews. Reviews with a conclusion in favour of psychological therapies (vs pharmacological interventions) were at high risk for a spin in conclusions (OR=8.31 (1.41 to 49.05)). Spin was related in trend to the inclusion of own primary studies in the systematic review (OR=2.08 (CI 0.83 to 5.18) p=0.11) and researcher allegiance (OR=2.63 (0.84 to 8.16) p=0.16).

Conclusions Non-financial COI, especially the inclusion of own primary studies into reviews and researcher allegiance, are frequently seen in systematic reviews of psychological therapies and need more transparency and better management.

 

[Esther12]

I remember thinking that Larun would count as having an ideological COI according to their criteria (along with just about every other author of the Cochrane IPD Exercise protocol). I don't really know how much this matters.

 

[Dolphin]

With respect to outcome research of psychological therapies, researcher allegiance constitutes an important non-financial COI. Allegiance covers the belief of a researcher in the superiority of a treatment.16 ,17 Allegiance may be due to a special training in one specific psychological therapy, the involvement in previous efficacy research about this psychological therapy or the involvement in development of aetiological models via basic research.18–20

 

[Dolphin]

All disclosed COI were extracted: financial COI (honoraria, eg, for consulting, lectures, scientific articles, training courses, or money for research projects), non-financial COI (eg, researcher allegiance to a psychological therapy, special qualification in a psychological therapy, enthusiasm for a psychological therapy in scientific publications, lectures and research, or inclusion of own primary studies in reviews), and personal COI (eg, employee or private relationship to an employee of a company—regularly addressed as relationships to pharmaceutical companies).

 

[Dolphin]

Researcher allegiance was rated in 73 of the 86 included primary studies since 13 reviews did not compare psychological therapies to other treatments and were therefore excluded.

Researcher allegiance was defined to be present if the author (1) recommended the respective psychological therapy over another therapy and was (2) either involved in the development of the respective psychological therapy, or (3) was involved in research of/development of the aetiological model of the psychological therapy.

 

[Dolphin]

Assessment of spin in review conclusions
To assess spin in review conclusions, we evaluated whether the conclusion of the review as expressed in the abstract or the discussion section was inconsistent or consistent with the empirical results described in the results section of the review. If the conclusion was consistent with the empirical results, the review was considered as showing no spin. If it was inconsistent, the review was rated as showing spin.

 

[Dolphin]

Authors in 25 of 95 reviews (26.3%) made COI statements as follows: own study included in the review (n=2), research activities in relation to one psychological therapy (n=2), research support (n=18), author has served as consultant (n=4), served as speaker on congresses (n=1), get honoraria (n=5), have holdings (n=2), have patents (n=1), served as a trainer for a psychological therapy (n=1), being influenced as employer (n=1). In other words, financial, non-financial and personal COI were disclosed in 22 reviews (23.1%), 4 reviews (4.2%), and 1 review (1%), respectively.

 

[Dolphin]

Although psychologists may mostly judge themselves as free of financial COI, however, researcher allegiance, as well as the inclusion of own studies into a review (which we both rated as non-financial COI), may well lead to financial gains indirectly.15 Since psychologists who develop new psychological treatments are often the ones who distribute and train other psychologists in those therapies, the demonstration of effectiveness of a specific psychotherapy in a review may potentially lead to high financial incentives. The promotion of the respective therapy might be easier, and the number of trained psychotherapists with high course fee increases. Showing the effectiveness of a treatment can also be an important step for patents and for the implementation in treatment guidelines.

 

[Dolphin]

The fact that researchers developing and evaluating the effectiveness of psychological therapies are mostly allied with a specific psychotherapy (eg, cognitive–behavioural therapy or psychoanalysis), makes the issue of COI in psychology therapy research very complex and much more complicated than in pharmacological research. Psychotherapy researchers who realise that the effect of the therapy to which they are allied is less beneficial than another therapy cannot easily switch their research programme to another therapy (since they have often been trained in that therapy for many years), in contrast with a researcher addressing pharmacotherapy who can more easily change his or her research agenda to another drug if a drug proves to be less effective than previously thought.

Also if they are not a doctor, they can't switch to pharmacotherapy.
Same applies with other non-medical healthcare workers.

 

[Dolphin]

Similar to non-financial COI, also personal COI were very seldom disclosed (only in one review). This is probably due to the common definition of personal COI meaning any relationship to a person working in a pharmaceutical company. This, of course, is a less relevant COI for psychotherapist assessing treatment effects of psychological therapies. However, psychotherapists, especially the ones who develop new therapies, are very often personally involved in institutes promoting the distribution and training of new psychological therapies. Such personal COI may indirectly lead to considerable financial gains.

 

[Dolphin]

Both reviews with inclusion of own primary studies and reviews with researcher allegiance showed more often a spin (statistical trend). Since researcher allegiance has been shown to be significantly related to outcome of psychological therapies,21 authors should be transparent in disclosing their own psychotherapeutic training background and the inclusion of own outcome studies in systematic reviews to make an assessment of COI and allegiance easier. The allegiance indicators of our study might be an initial step for such a statement (development of treatment or basic research on the aetiological model for a specific treatment).

 

[Dolphin]

Strategies to mitigate biases may include the detection and removal of spin at the editorial stage, using independent authors and reviewers interpreting the findings of meta-analyses, the rejection of systematic reviews that demonstrate selective citation biases, and providing free access to all data of systematic reviews to ensure that systematic reviews can be more easily replicated.

 

[Esther12]

Both reviews with inclusion of own primary studies and reviews with researcher allegiance showed more often a spin (statistical trend).

My memory of this paper was that it was questionable whether they found evidence of an effect, and maybe they were spinning a bit in favour of their own ideological views?

 

[Dolphin]

My memory of this paper was that it was questionable whether they found evidence of an effect, and maybe they were spinning a bit in favour of their own ideological views?

Yes, although the results for statistical tests were all (as I recall) in the one direction, I think only one reached statistical significance.

 

[Dolphin]

Here is how the reviewers reported their own competing interests

Competing interests JB was involved in the development of indicators of allegiance. KL, NR, JS-W and JB are psychotherapists trained in CBT, KL and NR also in schematherapy. NR and JB received money from institutes providing training in schematherapy and CBT within the last 3 years. JS-W and KL are coauthors on two reviews included into the study (Gibbon et al, 2010 and Stoffers et al, 2012; see online supplementary table S3), and NR and KL are coauthors on one primary study (Reiss et al, 2014; see online supplementary table S6) included in one of the reviews.

 

[Esther12]

Yes, although the results for statistical tests were all (as I recall) in the one direction, I think only one reached statistical significance.

Thanks. And thanks for pulling out all the above too.

 

[AndyPR]

@Keith Geraghty also shared this paper on Twitter.

 

[alex3619]

They should come here and read about PACE.