Meta-analyses everywhere, but not a critical thought…
What is accepted as necessary for drug trials is routinely ignored for trials of psychotherapy treatments and meta-analyses integrating their results. Investigator allegiance has been identified as one of the strongest predictors of outcomes, regardless of the treatment that is being evaluated. But this does not get translated into enforcement of disclosures of conflict of interests by authors or by readers having heightened skepticism.
Yet, we routinely accept claims of superiority of psychological treatments made by those who profit from such advertisements. Meta-analyses allow them to make even bigger claims than single trials. And journals accept such claims and pass them on without conflict of interest statements. We seldom see any protesting letters to the editor. Must we conclude that no one is bothered enough to write?
Meta-analyses of psychological treatments with undisclosed conflicts of interest are endemic. We already know investigator allegiance is a better predictor of the outcome of the trial than whatever is being tested. But this embarrassment is explained away in terms of investigator’s enthusiasm for their treatment. More likely, results are spurious or inflated or spun. There is high risk of bias associated with investigators having a dog in the fight. And a great potential for throwing the match with flexible rules of data selection, analysis, interpretation (DS, A, and I). And then there is hypothesizing after results are known (
HARKING).
It is scandalous enough that the investigators can promote their psychological products by doing their own clinical trials, but they can go further, they can do meta-analysis. After all, everybody knows that a meta-analysis is a higher form of evidence, a stronger claim, than an individual clinical trial.
Because meta-analyses are considered the highest form of evidence for interventions, they provided an important opportunity for promoters to brand their treatments as evidence-supported. Such a branding is potentially worth millions in terms of winning contracts from governments for dissemination and implementation, consulting, training, and sale of materials associated with the treatment. Readers need to be informed of potential conflict of interest of the authors of meta-analyses in order to make independent evaluations of claims.
And the requirement of disclosure ought to apply to reviewers who act as gatekeepers for what gets into the journals. Haven’t thought of that? I think you soon will be. Read on.
