Critical letters for PACE mediation paper and non-response response from PACE.

[Dolphin]

Note the authors' reply was written by five people, not just Trudie Chalder:

Trudie Chalder, Kimberley A Goldsmith, Peter D White, Michael Sharpe, Andrew R Pickles

My guess is Peter White was heavily involved in the wording.

 

[medfeb]

Yes, I agree this is the central problem. The Oxford criteria for recruitment are also an issue but if the trial had been robustly designed at least the conclusions would have been valid for people falling under the Oxford criteria.

I agree with the range of concerns raised with study design and execution. And I think its important to expose those issues.

But it seems to me that Sarah Darwins' and Twisk's points are key. Even if the study were perfectly executed, Oxford is a meaningless grouping of conditions based on chronic fatigue for which there is no medical explanation. But there is no scientific rationale or proof that all people with chronic fatigue for which there is not a medical explanation are suffering from the same condition which can rationally be studied and treated as the same thing. The underlying theoretical model and its psychogenic foundation is flawed.

So if that's true, then I'd think the conclusions drawn from an Oxford study can't be valid because you have no way of knowing which specific group of patients and conditions they should be applied to.

Chalder's claim in her response that it worked "irrespective of how we defined the illness" ignores Twisk's point that they used non-standard "CFS" and "ME" criteria.

 

[Jonathan Edwards]

But it seems to me that Sarah Darwins' and Twisk's points are key. Even if the study were perfectly executed, Oxford is a meaningless grouping of conditions based on chronic fatigue for which there is no medical explanation. But there is no scientific rationale or proof that all people with chronic fatigue for which there is not a medical explanation are suffering from the same condition which can rationally be studied and treated as the same thing. The underlying theoretical model and its psychogenic foundation is flawed.

So if that's true, then I'd think the conclusions drawn from an Oxford study can't be valid because you have no way of knowing which specific group of patients and conditions they should be applied to.

I sympathise with the idea but I don't think it will stand up against counterattack. There is nothing meaningless about grouping patients by chronic fatigue unexplained by other major diagnoses. Medicine very often works like that. High blood pressure unexplained by other major disease is called essential hypertension and there is a vast validated literature on its management. Different patients respond to different treatments, just as they may do for fatigue, but we know that certain drugs have a likelihood of being helpful.

You are making the suggestion, I think, that there is a narrower definition that has a better theoretical base, but the sad fact is that there probably isn't. Even within ME as defined by the most strict criteria there may be several different diseases all looking rather similar. And I do not think the Oxford criteria imply any theory - it is just a practical clinical grouping - although I may be wrong there.

So it must remain that if a study is well designed and reliable evidence emerges about response to treatment in people fitting Oxford criteria that is entirely valid, just as the evidence for using enalapril for hypertension is valid. This is not a scientific validity, but an evidence based validity. The two things are different and in clinical practice it is the evidence base that always has to take precedent over science, which can only guide trial design.

Where there may be a specific problem is if a treatment helps some people in a group but makes others worse. If the chances of making people worse because you do not distinguish them adequately is high then the treatment cannot be recommended. There would be an argument here for saying that conclusions about GET on Oxford criteria would be inappropriate for a narrower grouping that required PEM, for instance. That may well apply to PACE but the issue is not about PACE anyway because the results are not even solid enough to worry if they apply to narrower groups - we do not even know they apply to the Oxford group.

So criticism of Oxford criteria can be countered by criticism of any other criteria on the same grounds. But criticism of the use of subjective endpoints in an unblinded study is checkmate.

 

[seaturtle]

Omission of data weakens the case for causal mediation in the PACE Trial
Simon McGrath
http://www.thelancet.com/journals/lanpsy/article/PIIS2215-0366(15)00085-1/fulltext

This study demonstrated association between putative mediators and outcomes, but not the causal link central to establishing mediation. For example, patients feeling less fatigued could lead to a reduction in beliefs classed as fearful or aversive. In this instance a primary outcome would be the mediator of the belief, a scenario that the report did not explore.

Although the authors acknowledged that simultaneous change in their study's outcomes means mediators and outcomes might affect each other reciprocally, the abstract and conclusions unambiguously assert mediation.1 Claims of causal mediation, and the linked assertion that this strengthens the theoretical treatment model, are premature.

Methods and outcome reporting in the PACE trial–Author's reply
Trudie Chalder, Kimberley A Goldsmith, Peter D White, Michael Sharpe, Andrew R Pickles
http://www.thelancet.com/journals/lanpsy/article/PIIS2215-0366(15)00114-5/fulltext

McGrath reiterates the limitations of this mediator analysis, which we have already acknowledged in the Article.2 We intend to further explore multiple mediator effects and the potential effects of outcomes on mediators in future papers.

So, despite an enormous amount of hyperbole, it seems that, in fact, no mediation effect was observed for fear-avoidance.

And the authors' conclusion, that fear-avoidance was a mediating factor for CBT/GET, was simply an invented conclusion? Is that acceptable in a scientific journal these days?

This seems to be a continuation of a theme. i.e. you don't like your data, so distract the public with outlandish conclusions which aren't based on the evidence.

 

[medfeb]

Thank you for the response, Professor Edwards.

High blood pressure unexplained by other major disease is called essential hypertension and there is a vast validated literature on its management. Different patients respond to different treatments, just as they may do for fatigue, but we know that certain drugs have a likelihood of being helpful.

Good point. I can see how that could be a productive approach when there is a well-defined and measurable biomarker for selecting patients and evaluating treatments as with essential hypertension. Even if you don't know what's causing it, you know which patients have "it."

For me, what's problematic here is the use of an ill-defined, common symptom like medically unexplained chronic fatigue as the sole criteria while failing to require hallmark criteria like PEM. So what is it really that is being studied? It feels like a diagnosis of the leftovers. But not all leftovers belong in the same stew.

And I do not think the Oxford criteria imply any theory - it is just a practical clinical grouping - although I may be wrong there.

Agree. I didn't intend to imply that Oxford implies a specific theory although on reflection, the exclusion of all medically explained causes of fatigue would appear to favor a psychological or behavioral explanation. It certainly has been conveniently embraced by those who advocate psychogenic models like the biopsychosocial model or the idea of functional somatic disorder and MUS.

 

[Jonathan Edwards]

For me, what's problematic here is the use of an ill-defined, common symptom like medically unexplained chronic fatigue as the sole criteria while failing to require hallmark criteria like PEM. So what is it really that is being studied? It feels like a diagnosis of the leftovers. But not all leftovers belong in the same stew.

Being devil's advocate, what is being studied is unexplained fatigue. PEM may be a hallmark of the group of people with PEM, but so far we do not have any evidence that PEM is a reliable guide to a group that responds to a specific treatment or even has a common causal origin. It seems a good bet, but it may turn out to be unhelpful. Perhaps unrefreshing sleep is the best hallmark of some particular process. Who knows? Once we thought there was a disease called diabetes but now we know that there are two quite different diseases that commonly give the same 'hallmark' features of sugar in the urine, peeing a lot, thirst and hunger (pity about the weight loss in type 2). One might think that the usefulness of sulphasalazine would depend on whether one had rheumatoid arthritis or spondarthritis, because they have quite different causes. But in fact sulphasalazine seems to work for knees and wrists and hands and not for the spine or heel - it does not seem to care what disease it is treating. Practical evidence takes precedent.

But I am only arguing this in the way that a lawyer would in a patent case - to stick to the killer argument and give the opposition no room to shift position. Good trials take precise criteria to improve the chances of getting a useful result.

 

[lansbergen]

Perhaps unrefreshing sleep is the best hallmark of some particular process.

That would rule me out now but I am not cured.

 

[sarah darwins]

But criticism of the use of subjective endpoints in an unblinded study is checkmate.

Point well made and taken, Jonathan. And I'm going to try to remember “subjective endpoints in an unblinded study” for later use, if you don’t mind!

I’m torn, reading this thread, between the part of me that wants to be objective and analytical in the hope that reason and good sense will drive out bad science, and the part of me that wants to run (very slowly) to the barricades and scream bloody revolution. I suspect that mention of PACE (and the mediation analysis) may have this effect on a lot of sufferers.

 

[medfeb]

Being devil's advocate, what is being studied is unexplained fatigue. PEM may be a hallmark of the group of people with PEM, but so far we do not have any evidence that PEM is a reliable guide to a group that responds to a specific treatment or even has a common causal origin. It seems a good bet, but it may turn out to be unhelpful. Perhaps unrefreshing sleep is the best hallmark of some particular process. Who knows?

I agree that what is being studied is medically unexplained fatigue. But the question is whether that is scientifically meaningful and appropriate - is medically unexplained fatigue an effective research construct or appropriate clinical entity?

Fatigue is the most common complaint so the construct relies on a common symptom plus the current state of medical knowledge to differentiate cases from non-cases. Based on the Oxford (or Fukuda and Empirical) inclusion and exclusion criteria, it would appear that a variety of different conditions - depression, deconditioning, any unspecified cause of chronic fatigue, plus unfortunately ME - can be stuffed under that one tent. But where's the formal scientific justification to support the assumption that all those conditions are related in some way and could reasonably be expected to have similar underlying pathologies or similar responses to treatment?

I do agree with you that we don't have evidence yet that PEM is the best guide for treatment or evidence that there is the same pathology or cause. Within that group of patients, I expect there will be subtypes and differential responses to treatment, although in my son's one-off case, improved tolerance to exertion has been a solid indicator of improvement while on Rituxan.

But patient reports, multiple definitions back to Ramsay plus now the IOM report have defined post-exertional reaction to exertion as a hallmark of the disease along with other hallmarks. To me, its scientifically ludicrous to try to study that population of patients by throwing them into what I believe is an indefensible "unexplained fatigue" construct.

But I am only arguing this in the way that a lawyer would in a patent case - to stick to the killer argument and give the opposition no room to shift position. Good trials take precise criteria to improve the chances of getting a useful result

You are right, of course. There's a killer argument about the flawed methodology in the PACE trial and given the size of that trial and its impact in Europe and also in the U.S., that needs to be pushed hard. And there may be good reason in the context of the politics in the U.K. to focus on that argument. But the argument is by necessity one trial or study at a time. The questionable construct of unexplained fatigue goes across many studies and to me, gets at the underlying flawed foundation of this body of evidence

 

[Jonathan Edwards]

I agree that what is being studied is medically unexplained fatigue. But the question is whether that is scientifically meaningful and appropriate .....


But where's the formal scientific justification to support the assumption that all those conditions are related in some way and could reasonably be expected to have similar underlying pathologies or similar responses to treatment?

... And there may be good reason in the context of the politics in the U.K. to focus on that argument. But the argument is by necessity one trial or study at a time. The questionable construct of unexplained fatigue goes across many studies and to me, gets at the underlying flawed foundation of this body of evidence

Yes, I agree, all good points.

The only problem is that there is no such thing as 'formal scientific justification'. Bad theories abound but nobody is allowed to deny research on the basis that they think a theory is bad. My theory that rituximab would help RA was considered bad by almost everyone - until we tried it. Much worse theories about TH1 and Th2 cells are very popular but I am not allowed to stop people trying them too!

 

[seaturtle]

Those providing cognitive behavioural therapy (CBT) and graded exercise therapy (GET) encouraged gradual increases in activity (mostly walking for GET), whereas those providing adaptive pacing therapy (APT), a novel therapy created specifically for the trial, were told that an important consideration in APT is the 70% rule: never going beyond 70% of a person's perceived energy limit.

That made me think... I wonder if the outcomes for APT were roughly 70% of those for CBT/GET in the PACE trial?

 

[seaturtle]

Posting these for easy reference...

The Lancet Psychiatry.
Volume 2 - p281-362 e6-e13 - Apr 2015.
Index:
http://www.thelancet.com/journals/lanpsy/issue/vol2no4/PIIS2215-0366(15)X0004-6


George Faulkner:
http://www.thelancet.com/journals/lanpsy/article/PIIS2215-0366(15)00089-9/fulltext

Simon McGrath:
http://www.thelancet.com/journals/lanpsy/article/PIIS2215-0366(15)00085-1/fulltext

Frank Twisk:
http://www.thelancet.com/journals/lanpsy/article/PIIS2215-0366(15)00044-9/fulltext

Robert Courtney:
http://www.thelancet.com/journals/lanpsy/article/PIIS2215-0366(15)00054-1/fulltext

Sean B M Kirby:
http://www.thelancet.com/journals/lanpsy/article/PIIS2215-0366(15)00110-8/fulltext

Authors' Reply.
Trudie Chalder, Kimberley A Goldsmith, Peter D White, Michael Sharpe, Andrew R Pickles:
http://www.thelancet.com/journals/lanpsy/article/PIIS2215-0366(15)00114-5/fulltext

 

[seaturtle]

If anyone hasn't read through these letters yet, I recommend reading them...

One letter (Simon McGrath) effectively trashes the conclusions of PACE trial mediation analysis, and points out that the investigators didn't actually demonstrate a mediation effect, and that their conclusions were not based on the evidence. (i.e. the conclusions were fabricated and based on wishful thinking rather than robust evidence - This is my own interpretation of McGrath's letter.)

Another letter (Robert Courtney) highlights that the deconditioning theory of CFS has effectively been debunked by the PACE authors, in the mediation analysis, and that the fear-avoidance theory was not supported by the PACE outcomes either.

Frank Twist points out that "fear avoidance in patients with post-exertional malaise is a reasonable and learned response, serving as a rational defence mechanism to avoid long-lasting relapses".

George Faulkner discusses the significance of objective measures in a non-blinded medical trial, and comments that "this latest report might merely be providing information about how response bias can change patient self-report measures."

And Sean Kirby asks that the authors "release the results for all of the outcomes as laid out in the published trial protocol, so that other researchers, and clinicians and patients, can decide for themselves" the value of the interventions.


The PACE authors acknowledge the limitations of the analysis as discussed by McGrath (i.e. that their conclusions were not supported by the evidence), and they then go on to say: "Most correspondents make criticisms of the trial as a whole [...] which might reflect the apparent campaign to bring the robust findings of the trial into question."

So they've engaged in a thorough and proper scientific discussion there! Not! Their response seems more like an ad hominem attack on the motives of the correspondents, rather than scientific discourse. They make ad hominen comments and then presume to judge the correspondents? I'm surprised that the Lancet published such a comment. Oh, wait, it's the Lancet.

 

[Esther12]

So they've engaged in a thorough and proper scientific discussion there! Not! Their response seems more like an ad hominem attack on the motives of the correspondents, rather than scientific discourse. They make ad hominen comments and then presume to judge the correspondents? I'm surprised that the Lancet published such a comment. Oh, wait, it's the Lancet.

The Lancet Psych twitter feed has just retweeted this about itself:

@Kentjrchess
@GeorgiaBelam @TheLancetPsych a breath of fresh air to have another journal dedicated to psychiatry;that too with additional focus on debate

https://twitter.com/Kentjrchess/status/583705263523422208

LOL.

 

[A.B.]

The problem with the PACE trial and similar research isn't so much the definition of CFS but the poor quality of research and dishonesty by researchers. And that this is being tolerated in the media and medical community.

One gets the impression that this field is ripe with charlatans who protect each other.

There is nothing wrong per se with studying patients that self report fatigue. It is wrong to equate this to CFS or ME when all other definitions require other symptoms. It's also wrong to extrapolate from a subset to the entire set. A child can understand that these things are a mistake - why can't these "researchers"?

 

[Sean]

One gets the impression that this field is ripe with charlatans who protect each other.

"Political language... ...is designed to make lies sound truthful and murder respectable, and to give an appearance of solidity to pure wind."
G. Orwell

 

[Dolphin]

(In case anyone missed it)
The journalist, David Tuller DrPH, has today posted a substantial piece on the PACE Trial:

TRIAL BY ERROR: The Troubling Case of the PACE Chronic Fatigue Syndrome Study
http://www.virology.ws/2015/10/21/trial-by-error-i/

There's an introduction and summary at the start if you don't want to take on the whole thing.

It's being discussed in this PR thread:
http://forums.phoenixrising.me/inde...he-pace-chronic-fatigue-syndrome-study.40664/

ME Network have also posted their own summary piece:
http://www.meaction.net/2015/10/21/david-tuller-tears-apart-pace-trial/