Robert says:
December 19, 2015 at 8:09 am
Hi Andrew,
Thanks for an interesting blog. Interesting to hear your views on mediation analyses. It seems to me that there is plenty of room for guess-work, false assumptions, error and statistical noise in mediation analyses, perhaps especially in the case of analyses that use self-report questionnaires (that may be vulnerable to response bias, therapist allegiance and other biases).
A brief note about the particular Secondary Mediation Analysis in question…
Chalder et al. [1] used the “single mediation model” for their methodology, which is explained in detail in a book by MacKinnon [2]. Explaining the methodology MacKinnon says a temporal separation between variables must be observed (i.e. changes in mediating variable must occur before changes in the mediated variable) for a mediation effect to be empirically and robustly established.
Chalder et al. were working to this model and acknowledged that they failed to establish a temporal or causal relationship between variables, and therefore did not empirically establish a mediation effect: “Given the pattern of change in the mediators was similar to the pattern of change in the outcomes it is possible that the variables were affecting each other reciprocally”.
However, despite the lack of empirical evidence to support a mediation effect, the investigators concluded that they had established mediation effects, e.g: “Our main finding was that fear avoidance beliefs were the strongest mediator for both CBT and GET.”
The study’s conclusion relied upon an assumption that the investigators’ favoured hypothetical model of illness for ME/CFS has a robust empirical evidence base and is applicable to this study. The hypothesis is based upon the idea that symptoms and disability in ME/CFS are perpetuated by false illness beliefs, fear, and an avoidance of activity.
However, the prestigious National Academy of Medicine (formerly known as the Institute of Medicine) recently released a comprehensive report [3] into ME/CFS that rejected such a hypothetical model of illness, and unambiguously concluded that ME/CFS does not have a psychological or cognitive-behavioural basis, but is an organic illness that requires biomedical research.
Chalder et al. discussed the possibility that more frequent measurements may have potentially demonstrated a temporal separation between the variables, and therefore a mediation effect. However, this raises the possibility of whether changes in the primary outcome variables (self-report physical function and fatigue) may, in fact, have occurred before changes in the presumed mediator variables. Such an outcome would entirely contradict the investigators’ premature conclusions. According to MacKinnon [2] and Wiedermann et al. [4], unexpected outcomes should not be ruled out.
Chalder et al. concluded that symptoms and physical impairment, in ME/CFS patients, are mediated by activity avoidance and other factors. However, from a common sense point of view, this seems like rather a convoluted conclusion, and it seems more likely that symptoms would be the direct cause of activity avoidance in any illness. And physical impairment is identical to activity avoidance. To conclude that activity avoidance causes fatigue (rather than fatigue being a direct cause of activity avoidance), is similar to concluding that a person has flu because they’ve taken a day off work, rather than the obvious conclusion that they’ve taken a day off work because they have flu.
In the case of fatigue, it seems reasonable to consider the possibility that, as the symptom of fatigue fluctuates, patients may intuitively or rationally adapt their activity levels according to what is comfortable and safe. i.e. patients reduce activity levels because they are fatigued. The investigators have concluded that patients are fatigued because they have reduced activity levels.
Perhaps patients’ perspectives and insights would help clarify the issues but, unfortunately, patients were not consulted for this study.
References:
1. Chalder T, Goldsmith KA, White PD, Sharpe M, Pickles AR. Rehabilitative therapies for chronic fatigue syndrome: a secondary mediation analysis of the PACE trial. Lancet Psychiatry 2015; 2: 141–52.
2. MacKinnon DP. Introduction to Statistical Mediation Analysis. Taylor and Francis: New York 2008.
3.
http://iom.nationalacademies.org/Reports/2015/ME-CFS.aspx
4. Wiedermann W, von Eye A. Direction of Effects in Mediation Analysis. Psychol Methods 2015; 20: 221-44.
