Interesting. James Coyne is one of a few voices who is calling for higher quality and reporting standards in psychotherapeutic/behavioural research. This has enormous relevance to studies of CBT/GET interventions in ME/CFS.
Here are some of his comments from another recent blog:
http://blogs.plos.org/mindthebrain/2014/06/10/salvaging-psychotherapy-research-manifesto/
".... As it now stands, the psychotherapy literature does not provide a dependable guide to policy makers, clinicians, and consumers attempting to assess the relative costs and benefits of choosing a particular therapy over others. If such stakeholders uncritically depend upon the psychotherapy literature to evaluate the evidence-supported status of treatments, they will be confused or misled.
Psychotherapy research is scandalously bad.
"Many RCTs are underpowered, yet consistently obtain positive results by redefining the primary outcomes after results are known. The typical RCT is a small, methodologically flawed study
conducted by investigators with strong allegiances to one of the treatments being evaluated. Which treatment is preferred by investigators is a better predictor of the outcome of the trial than the specific treatment being evaluated.
"Many positive findings are created by spinning a combination of confirmatory bias, flexible rules of design, data analysis and reporting and significance chasing. Many studies considered positive, including those that become highly cited, are basically null trials for which results for the primary outcome are ignored, and post-hoc analysis of secondary outcomes and subgroup analyses are emphasized.
"The bulk of psychotherapy RCTs involve
comparisons between a single active treatment and an inactive or neutral control group such as wait list, no treatment, or “routine care” which is typically left undefined but in which
exposure to treatment of adequate quality and intensity is not assured. At best these studies can tell us whether a treatment is better than doing nothing at all or than patients expecting treatment because they have enrolled in a trial and not getting it
(nocebo).
Designations of Treatments as Evidence-Supported
"There are
low thresholds for professional groups such as the
American Psychological Association Division 12 or governmental organizations such as the
US Substance Abuse and Mental Health Services Administration (SAMHSA) declaring treatments to be “evidence-supported.” Seldom are any treatments deemed ineffective or harmful by these groups."
"
Departures from preregistered protocols in published reports of RCTs are common, and there is little checking of discrepancies in abstracts from results that were actually obtained or promised in preregistration by authors. There is inconsistent and incomplete adherence to these requirements. There is little likelihood that noncompliant authors will held accountable and a high incentive to report positive findings.."
Does that remind anyone of a particularly high profile behavioural intervention study of ME/CFS?
