2013 British Association for Behavioural & Cognitive Psychotherapies (BABCP) conference abstracts

[Esther12]

A bit OT, but relevant to the discussion imo:

Psychol Med. 2013 Aug 12:1-14. [Epub ahead of print]
Mechanisms of change underlying the efficacy of cognitive behaviour therapy for chronic fatigue syndrome in a specialist clinic: a mediation analysis.

Stahl D, Rimes KA, Chalder T.
Source

Department of Biostatistics, Institute of Psychiatry, King's College London, UK.
Abstract

BACKGROUND:

Several randomized controlled trials (RCTs) have shown that cognitive behavioural psychotherapy (CBT) is an efficacious treatment for chronic fatigue syndrome (CFS). However, little is known about the mechanisms by which the treatment has its effect. The aim of this study was to investigate potential mechanisms of change underlying the efficacy of CBT for CFS. We applied path analysis and introduce novel model comparison approaches to assess a theoretical CBT model that suggests that fearful cognitions will mediate the relationship between avoidance behaviour and illness outcomes (fatigue and social adjustment). Method Data from 389 patients with CFS who received CBT in a specialist service in the UK were collected at baseline, at discharge from treatment, and at 3-, 6- and 12-month follow-ups. Path analyses were used to assess possible mediating effects. Model selection using information criteria was used to compare support for competing mediational models.
RESULTS:

Path analyses were consistent with the hypothesized model in which fear avoidance beliefs at the 3-month follow-up partially mediate the relationship between avoidance behaviour at discharge and fatigue and social adjustment respectively at 6 months.
CONCLUSIONS:

The results strengthen the validity of a theoretical model of CBT by confirming the role of cognitive and behavioural factors in CFS.

Looked like full text was available from: http://www.ncbi.nlm.nih.gov/pubmed/23931831

I've not read it, so wasn't sure if it was worth a new thread.

 

[Simon]

Another post on this from me, because I think this paper - or the non-appearance of a full paper - is very significant.

Looks like this abstract has appeared at 4 conferences: the first 3 appear to be the most rigorous, and all basically conclude "nothing doing":
- Clincal Methodology conf
- 33rd Annual Conference of the International Society for Clinical Biostatistics
- Ghent uni symposium on "Causal mediation Analysis"

While the altogether less, er, 'technical' "2013 British Association for Behavioural & Cognitive Psychotherapies (BABCP)" makes it sound like a big deal, and ignores the serious methodology issues raised in the first 3 abstracts.

A key problem is working out cause and effect: do attitudes change because the patient had improved, or are they the cause? The presumed 'mediators' changed during the therapy but not after - but the same things is pretty-well true of the outcomes too, with almost all the gains made by 6 months. So changes in attitudes could reflect lower levels of fatigue, not vice versa. I think the IV methods in the original abstracts is trying to probe this problem - and found only very weak mediation effects.

Also, when you only have a modest effects to look at in the first place (ignoring self-report issues), and when the 'control' improvement is bigger than the incremental gain from therapy, then mediation analysis is never likely to find very much. A partial-mediator of a smallish effect is going to be lost in the noise.

It's pretty tough to sustain a BPS theory when your analysis of your flagship study can't find the evidence that CBT/GET changing beliefs and behaviours is responsible for any gains. Especially not when you set up the study in such a way that you should be able to find such evidence, if it was really there.

 

[Gijs]

This study suggest causality i.e. that CVS is a form of an anxiety disorder. This study is not objective and must be exluded as ''evidence bases science'.

 

[user9876]

Another post on this from me, because I think this paper - or the non-appearance of a full paper - is very significant.

Looks like this abstract has appeared at 4 conferences: the first 3 appear to be the most rigorous, and all basically conclude "nothing doing":
- Clincal Methodology conf
- 33rd Annual Conference of the International Society for Clinical Biostatistics
- Ghent uni symposium on "Causal mediation Analysis"

While the altogether less, er, 'technical' "2013 British Association for Behavioural & Cognitive Psychotherapies (BABCP)" makes it sound like a big deal, and ignores the serious methodology issues raised in the first 3 abstracts.

A key problem is working out cause and effect: do attitudes change because the patient had improved, or are they the cause? The presumed 'mediators' changed during the therapy but not after - but the same things is pretty-well true of the outcomes too, with almost all the gains made by 6 months. So changes in attitudes could reflect lower levels of fatigue, not vice versa. I think the IV methods in the original abstracts is trying to probe this problem - and found only very weak mediation effects.

Another issue is due to the lack of a prepublished statistical analysis plan. How long have they been trawling through different techniques to find the results they wanted.

I did once read a psych paper (on decision making not a BPS one) where they explicitly trawled through many techniques till they found one giving significant results.

 

[Simon]

Another issue is due to the lack of a prepublished statistical analysis plan. How long have they been trawling through different techniques to find the results they wanted.

I did once read a psych paper (on decision making not a BPS one) where they explicitly trawled through many techniques till they found one giving significant results.

They haven't even post-published the statistical analysis plan, despite saying they were going to do so.

Amazing that paper should be so explicit about it's dodgy practice, I know it has been suggested that this is a widespread but unreported research practice.

Not sure how much trawling they did in practice on mediation; the BJK method they used initially is one of the standard ones, I think, while using IVs seems to be more sophisticated (hence its presentation at specialist conferences) - and gave a worse result.

 

[biophile]

The statistical analysis plan has since been published:

http://www.trialsjournal.com/content/14/1/386

No mention of mediators, moderators, predictors (aside cost-analyses), or processes.

So we could be in for some more post-hoc dubiousness?

 

[alex3619]

And if all else fails, redefine your terms so that success now has a definition that includes failure.

 

[Esther12]

It looked like this hadn't been bumped since the Lancet Psych PACE mediators paper came out, so I thought I'd let people see this again, in case it was of any further interest.