Goldsmith et al piece on PACE Trial inc. link to free txt (help sought to explain it)

[Snow Leopard]

Well her name isn't on the PACE protocol paper, so...

 

[oceanblue]

I notice Kimberly Goldsmith is the second name on the Lancet PACE Trial paper suggesting a major role. I wonder how much she was involved with and/or agreed with the changes made, definition of normal functioning, etc.

As the first-named statistician, I assume she did agree to (if not initiate) the changes made, including the post-hoc definition of normal.

 

[oceanblue]

Thanks, Dolphin, you're supreme at retrieving research.

Thanks to Marco too:

When I was a psych undergrad we covered quite a bit on questionnaire development, validation, reliability etc and one of the problems in designing them is to try to avoid issues such as social desirability bias, acquiesence bias (where people always agree with a statement regardless of the content) and even lying (perhaps again relating to social desirability).

One way around it is to include items in the questionnaire that set out to detect these biases rather than pertaining directly to the subject under study (e.g. "I have never lied in my life" in clearly a false statement and anyone agreeing with it is likely to also give false answers to other questions).

Getting back to PACE its a slightly different matter as they are not being asked to respond to statements but to self rate their levels of fatigue and physical function. There is no easy means here to include 'lie scale' or other items to measure any social desirability bias so if such bias was to be discounted from the results some other means would have to be found.

Perhaps if each individual's rating of their therapist and the trial therapy arm had been recorded these could be modelled as mediating variables but I doubt they have.

Which again leads us back to the only way to discount these potential biases when dealing with cognitive therapies and subjective outcome measures is to use objective activity measures.

Interesting point about the problems of measuring the kind of self-report bias that may have taken place in PACE; it seems slightly different from the normal examples of social desirability (eg false answers to Qs like 'do you have a sexually transmitted disease'?).

There's a problem with fatigue since it's purely a subjective symptom with no objective measure available, so as far as I can tell self-report is the only option. Even if they had therapist rating or therapeutic alliance data for every patient it would be hard to tell if that contributed to self-report bias as there are 2 possibilities:
1. The therapy works. It works better with a stronger patient/therapist relationship
2. The Therapy doesn't work, but patients rate themselves better where there is a stronger patient/therapist relationship due to self-report bias.
I don't know how you distinguish between the two possibilities (it may even be a bit of both).

I agree there should have been greater use of objective activity measures and it's a real shame they dropped the original plan to use actometers for measuring outcomes. But it's worth remembering the actometers are by no means perfect measures of activity since they measure acceleration, not force (so, for instance, can't tell the difference between climbing a flight of stairs and walking across the living room, while it's a pretty obvious different to a patient). They are very accurate for walking/running on level ground but are much less accurate in 'free-living' humans. Though of course they don't suffer from self-report bias. It may be several measures are needed eg 6MWT, actometers and SF36; ideally someone would do testing first to see which was most accurate at measuring change (AFAIK this has never been done). Maybe such measurements could also establish which would be best as the primary outcome measure.

Overall, I guess I'm saying it's a complex situation, but PACE didn't really address the complex issues at all.

 

[Sean]

All these (well known) problems with Social Desirability Bias, etc, just reinforce how critically important it is to use reliable independent objective measures, and use them appropriately (ie also measure for post-exertional features).

 

[floydguy]

All these (well known) problems with Social Desirability Bias, etc, just reinforce how critically important it is to use reliable independent objective measures, and use them appropriately (ie also measure for post-exertional features).

What reliable objective measures do you suggest for post exertional features?

 

[Dolphin]

I agree there should have been greater use of objective activity measures and it's a real shame they dropped the original plan to use actometers for measuring outcomes. But it's worth remembering the actometers are by no means perfect measures of activity since they measure acceleration, not force (so, for instance, can't tell the difference between climbing a flight of stairs and walking across the living room, while it's a pretty obvious different to a patient). They are very accurate for walking/running on level ground but are much less accurate in 'free-living' humans. Though of course they don't suffer from self-report bias. It may be several measures are needed eg 6MWT, actometers and SF36; ideally someone would do testing first to see which was most accurate at measuring change (AFAIK this has never been done). Maybe such measurements could also establish which would be best as the primary outcome measure.

(Repeating points I've said before, but it is another thread) There are different sorts of motion sensing devices. Some work in three-dimensions which might get around the problem you mention.

Also, while on an individual level it might not be as good as double-labelled water (say), I would think in aggregate form (and (arithmetic) means are aggregates) actometers would still be preferable to SF-36 PF (say).

 

[Sean]

What reliable objective measures do you suggest for post exertional features?

Repeating an objective or relatively objective test (actometer, 6 minute-walk-test, oxygen uptake, gene expression, neuro-cognitive capacity, etc) 24 hours or so later, and then maybe again 3-4 days after that. Not perfect, but a major methodological improvement on just using non-repeated, subjective self-report for post-exertional features, which is what PACE did.

And yes, oceanblue, actometers are not perfect. Nobody is arguing that they are the final word, just that they are substantially more objective than self-report based measures, and should be routinely used, especially when baseline data already exists. They bring a little more honesty about results to the table. Besides, controlling for the significant use of stairs in a patient's daily life shouldn't be that hard, and could provide some useful info.

I agree there needs to be a constellation of measures, including employment levels, and various subjective measures. But ultimately if the more objective measures don't show a good result, then all the glowing subjective self-reporting adds up to diddly squat. Which is really the main problem with results from the psych based studies minimal correlation between subjective and objective outcomes, specifically, unduly favourable subjective reports with no objective support.

 

[oceanblue]

(Repeating points I've said before, but it is another thread) There are different sorts of motion sensing devices. Some work in three-dimensions which might get around the problem you mention.

Also, while on an individual level it might not be as good as double-labelled water (say), I would think in aggregate form (and (arithmetic) means are aggregates) actometers would still be preferable to SF-36 PF (say).

My point is that there isn't any good data to show which is best*, particularly for measuring change after therapy, and I suspect a composite measure may be required. Adding 3D over 2D wouldn't necessarily help, unfortunately, if it still measures acceleration rather than load. Aggregate form should help, but not if actometers have systematic errors and again there is no data. The manufacturers of these devices seem to content themselves with tests in unrealistic lab-based scenarios, where they are indeed highly accurate.

And we still have the problem of fatigue, where there are no objective measures available.

* what would be good - and PACE would have been an ideal opportunity to do this given their budget - would be to use double labelled water vs SF36, actometers, and any other plausible measures both before and after therapy in a sub-sample of patients. Then see which most accurately reflects the double labelled water figures. Without data we remain in the dark as to the most reliable measures of activity. However, the discrepancy between PACE self-rated items and the 6MWT - and the low 6MWT distances compared to data on healthy subjects - do show that SF36 are highly suspect as the sole primary outcome measure..

 

[Dolphin]

Perhaps this should get its own thread?

From:

2013 British Association for Behavioural & Cognitive Psychotherapies (BABCP) conference abstracts
http://www.babcpconference.com/programme/Abstract Book_2013.pdf

Mediation effects in the PACE trial of complex treatments for chronic fatigue syndrome

Trudie Chalder, King's College London, Kim Goldsmith, King's College London; Peter White, Queen Mary's London; Michael Sharpe, Oxford University

Background:

We have previously shown that both cognitive behaviour therapy (CBT) and graded exercise therapy (GET) are superior to adaptive pacing therapy (APT) and specialist medical care (SMC) in reducing fatigue and physical functioning in people with chronic fatigue syndrome (White et al 2011).

The aim of this study was to investigate potential mechanisms of change underlying the efficacy of these treatments

Method :

We examined a number of cognitive and behavioural mediators such as fearful cognitions, avoidance behaviour and walking.

Mediation was assessed using Baron-Judd-Kenny, or BJK methods fitting a series of regression models.

Results:

Cognitive and behavioural mediating variables generally showed similar patterns, with the majority of change in the mediators occurring during the treatment phase.

There was no change in the mediators between the end of treatment at 24 weeks and follow up at 52 weeks.

Beliefs had the largest mediated effect on both fatigue and physical functioning for both CBT and GET.

However the effect of these mediators on outcomes in GET was stronger than for CBT.

Conclusion:

Both CBT and GET were mediated primarily by beliefs.

Both CBT and GET should target specific beliefs through behaviour change in order to change fatigue and disability.

 

[Dolphin]

Another abstract:

http://www.da.ugent.be/cvs/pages/en/final_program.pdf

Causal Mediation Analysis.

Symposium Organized by the Center for Statistics

Ghent University

Ghent January 28 and 29, 2013

Kimberley Goldsmith (Kings College London, UK)

Exploration of instrumental variable methods for estimation of causal mediation effects in the PACE trial of complex treatments for chronic fatigue syndrome

Exploration of instrumental variable methods for estimation of causal mediation effects in the PACE trial of complex treatments for chronic fatigue syndrome

Kimberley Goldsmith, T. Chalder, P. White, M. Sharpe, A. Pickles

Institute of Psychiatry, King's College London

Background:

Chronic fatigue syndrome (CFS) is characterised by chronic disabling fatigue.

The PACE trial compared four treatments for CFS and found cognitive behaviour therapy (CBT) and graded exercise therapy (GET) to be more effective in improving physical function and fatigue than two other treatments.

It is of interest to study whether the effects of CBT and GET are mediated through cognitive measures such as fear avoidance and activity avoidance.

The traditional Baron, Judd and Kenny (BJK) methods for studying mediation do not account for unmeasured confounders and so may provide incorrect mediation effects; instrumental variable methods (IV) from economics can address this problem.

Aim:

to explore mediation in PACE using BJK and IV estimates.

Methods:

BJK and IV methods were applied using linear regression models.

IV methods require instrumental variables - variables not in the postulated mediation model.

Several interaction terms between baseline variables and treatment were assessed as instruments using the R2 change between models with main effects only and with the interaction term.

Different IV estimators were compared. Collective instrument strength was assessed using recommended measures.

Results:

Tests of instrument strength indicated these were weak (ie. poor predictors of the mediator).

The IV estimators were different in magnitude and less precise than the BJK estimators.

The relative precision of different IV estimators varied 10-18%.

There is scope for modelling a common effect of the mediators across different treatments.

Conclusions:

Interaction term IVs in PACE were found to be weak.

Combining trial arms may allow for more efficient analysis

 

[Dolphin]

Another abstract - looks similar to last one

http://kivik.no/ISCB/wordpress/wp-content/uploads/2012/08/iscb33_2012_abstractbook_web.pdf

33rd Annual Conference of the International Society for Clinical Biostatistics

19-23 August 2012 – Bergen, Norway

Kimberley Goldsmith:

Exploration of instrumental variable methods for estimation of causal mediation effects in the PACE trial of complex treatments for chronic fatigue syndrome

P3 Causal inference

P3.1

Exploration of instrumental variable methods for estimation of causal mediation effects in the PACE trial of complex treatments for chronic fatigue syndrome

Kimberley Goldsmith

Trudie Chalder

Peter White

Michael Sharpe

Andrew Pickles

Institute of Psychiatry, King's College London, London, UK,

Wolfson Institute of Preventive Medicine, Bart's and the London School of Medicine, Queen Mary University of London, London, UK,

University Department of Psychiatry, University of Oxford, Oxford, UK

Background

Chronic fatigue syndrome (CFS) is characterised by chronic disabling fatigue.

The PACE trial compared four treatments for CFS and found cognitive behaviour therapy (plus specialist medical care, CBT+SMC) and graded exercise therapy (GET+SMC) to be more effective than adaptive pacing therapy (APT+SMC) and SMC alone in improving physical function and fatigue.

Estimates of causal mediation effects are of interest, for example, fear avoidance and activity avoidance as mediators of the effect of CBT and GET respectively.

Traditional Baron, Judd and Kenny (BJK) methods can be subject to bias; instrumental variable methods (IV) can address this problem.

The aims were to explore causal analyses using IVs in PACE and to compare IV and BJK estimates.

Methods

BJK methods were applied using ordinary least squares regressions.

IV methods were applied by assessing several baseline variables in interaction terms with treatment arm.

Instrument strength was assessed using the R2 change between models with main effects only and with the interaction term.

Different IV estimators were compared.

Collective instrument strength was assessed using an F test and partial R2.

Results

The IVs were weak, with small R2 changes.

The IV-derived estimators were different in magnitude and less precise than the BJK estimators.

The relative precision of different IV estimators varied 10-18%.

There is scope for modelling a common effect of mediators on outcomes across trial arms.

Conclusions

Potential IVs for the study of PACE treatment mechanisms can be found, however, these were weak.

Combining trial arms may allow for more efficient IV analysis.

 

[Esther12]

Looks like they're doing more digging in to the data to try to find 'positive' results to publish. If the evidence really supported their views, they could just release the data as it was laid out in the protocol, and let the evidence speak for itself.

It would be funny if the Goldsmith data transformed into the papers above:

Objectives

Background

Chronic fatigue syndrome (CFS) is characterised by chronic disabling fatigue. The PACE trial compared four treatments for CFS and found that for therapies added to specialist medical care (SMC), cognitive behaviour therapy (CBT) and graded exercise therapy (GET) were more effective than adaptive pacing therapy (APT) and SMC alone in improving physical function and fatigue. What are the mechanisms of these treatments? CBT and GET may affect outcomes through thought processes and behaviours (mediators). Traditional Baron, Judd and Kenny (BJK) methods for estimating mediation effects can be subject to bias; instrumental variable methods (IV) can address this problem. The aims were:
To explore potential IVs for causal analysis of mediation in PACE.
To compare IV estimates to those obtained using BJK methods, which are unbiased only under restrictive assumptions such as no unmeasured confounding.
Methods

Two treatment arms were compared at a time. BJK methods were applied using three ordinary least squares (OLS) regression models. IV methods were applied by compiling a list of baseline variables that could act as IVs in interaction terms with treatment arm and then assessing these using OLS with the mid-treatment measurement of the putative mediator as the outcome. Instrument strength was assessed using the R2 change between models with main effects only and with the interaction term. Two-stages least squares regression (2SLS) was used to estimate effects in the presence of IVs. Collective instrument strength was assessed using an F test and partial R2.
Results

The IVs were weak, with a maximum R2 change of 0.03. The five strongest IVs were therefore used in the 2SLS in each case. There was modest mediation of CBT and GET effects (approximately 20% of the total effect). The IV-derived estimators were somewhat different in magnitude than the BJK estimators and were less precise. There is scope for modelling a common effect of mediators on outcomes across trial arms.
Conclusions

There was evidence for modest mediation of CBT and GET effects. Potential IVs for the study of PACE treatment mechanisms can be found, however, these were weak. Combining trial arms may allow for more efficient analysis using IVs.

 

[Dolphin]

The abstracts on mediation are also discussed in this thread: http://forums.phoenixrising.me/inde...hotherapies-babcp-conference-abstracts.24483/

 

[Dolphin]

K Goldsmith did her PhD on the PACE Trial data.

https://kclpure.kcl.ac.uk/portal/en...me(413e5fb0-03b9-40bc-b993-0465b1bcbdee).html

FWIW:

Instrumental variable and longitudinal structural equation modelling methods for causal mediation: the PACE trial of treatments for chronic fatigue syndrome
Doctoral Thesis › Doctor of Philosophy

Kimberley Goldsmith

• Biostatistics

Background

Understanding complex psychological treatment mechanisms is important in order to refine and improve treatment. Mechanistic theories can be evaluated using mediation analysis methods. The Pacing, Graded Activity, and Cognitive Behaviour Therapy: A Randomised Evaluation (PACE) trial studied complex therapies for the treatment of chronic fatigue syndrome. The aim of the project was to study different mediation analysis methods using PACE trial data, and to make trial design recommendations based upon the findings.

Methods

PACE trial data were described using summary statistics and correlation analyses. Mediation estimates were derived using: the product of coefficients approach, instrumental variable (IV) methods with randomisation by baseline variables interactions as IVs, and dual process longitudinal structural equation models (SEM). Monte Carlo simulation studies were done to further explore the behaviour of IV estimators and to examine aspects of the SEM.

Results

Cognitive and behavioural measures were mediators of the cognitive behavioural and graded exercise therapies in PACE. Results were robust when accounting for correlated measurement error and different SEM structures. Randomisation by baseline IVs were weak, giving imprecise and sometimes extreme estimates, leaving their utility unclear. A flexible version of a latent change SEM with contemporaneous mediation effects and contemporaneous correlated measurement errors was the most appropriate longitudinal model.

Conclusions

IV methods using interaction IVs are unlikely to be useful; designs with randomised IV might be more suitable. Longitudinal SEM for mediation in clinical trials seems a promising approach. Mediation estimates from SEM were generally robust when allowing for correlated measurement error and for different model classes. Mediation analysis in trials should be longitudinal and should consider the number and timing of measures at the design stage. Using appropriate methods for studying mediation in trials will help clarify treatment mechanisms of action and allow for their refinement, which would maximize the information gained from trials and benefit patients.

Original language English
Awarding Institution

• King's College London

Supervisor(s)/Advisor

• Pickles, Andrew, Supervisor
• Chalder, Trudie, Supervisor

Date of Award 2014