Controversy over exercise therapy for CFS: key lessons for clinicians & academics (Mitchell, 2017)

[Dolphin]

http://apt.rcpsych.org/content/23/3/145?rss=1

Round the Corner


Controversy over exercise therapy for chronic fatigue syndrome: key lessons for clinicians and academics
COMMENTARY ON… COCHRANE CORNER
Alex J. Mitchell
BJPsych Advances May 2017, 23 (3) 145-148; DOI: 10.1192/apt.bp.116.016261

• 
  

SUMMARY
Chronic fatigue syndrome (CFS) is syndrome of unremitting fatigue of at least 6 months’ duration that causes significant disability. Exercise therapy has a proven track record in medicine and could be effective for some patients with CFS. An updated Cochrane review of eight studies appeared to suggest that exercise helps fatigue symptoms, but with only a small probability of recovery and/or improvement in daily function. Provisional data on acceptability suggest that most patients are willing to participate. However, one key study (PACE), which was well powered and influential in the Cochrane review, has been met with considerable controversy owing to lack of clarity on outcomes. Following release of the PACE study primary data, re-analysis suggested smaller effect sizes than initially reported.

• ©2017 The Royal College of Psychiatrists

 

[RogerBlack]

I wrote a reply to this, we'll see if it's going to be published.
In short - any even statistically significant improvement may be clinically irrelevant and that small clinical improvement has to be set against poorly measured harms, which are likely to be systematically underreported.

 

[Esther12]

I wrote a reply to this, we'll see if it's going to be published.
In short - any even statistically significant improvement may be clinically irrelevant and that small clinical improvement has to be set against poorly measured harms, which are likely to be systematically underreported.

Have you sent it already? That was quick! Or did you know this article had been out for a while?

 

[RogerBlack]

Have you sent it already? That was quick! Or did you know this article had been out for a while?

Last night.

 

[user9876]

I wrote a reply to this, we'll see if it's going to be published.
In short - any even statistically significant improvement may be clinically irrelevant and that small clinical improvement has to be set against poorly measured harms, which are likely to be systematically underreported.

I think it is worse than that. The measures they are looking at are subjective measures and highly likely to be biased in an open label trial. So any results they talk about are simply too error prone to have any meaning.

 

[Chrisb]

Do psychiatrists have a tin ear, or is it considered witty to call a feature in their comic "Round the Corner?" Do they think that they score extra points if they get away with it?

In their favour it might be regarded as an honest representation of the attitude which some have for their patients.

 

[Mohawk1995]

Can't they just put the notion that CBT and GET can effectively treat ME/CFS to bed! Could exercise and progressive functional restoration be beneficial in someone who has received effective clinical treatment for ME/CFS? I definitely think so, but as an effective primary treatment, NO! If it was, there would be far more people going around singing its praises much like what happens when someone receives antivenom for a deadly snake bite and survives.

This coming from a Physical Therapist who believes specific types of exercises can be effective treatment for many conditions. I am a big proponent of activity and exercise, but not in this way.

I am glad they are at least acknowledging there are flaws in the data.

 

[deleder2k]

What the F:

Almost all of the studies reported comorbid depression, affecting between 18% (Wearden 2010) and 39% (Jason 2007) of participants, and this is likely to have in uenced treatment response – indeed, up to 54% of patients were taking antidepressants.

 

[RogerBlack]

I think it is worse than that. The measures they are looking at are subjective measures and highly likely to be biased in an open label trial. So any results they talk about are simply too error prone to have any meaning.

Yes, also mentioned.
But, even the most optimistic claims of PACE, the recoveries weren't clinically what any reasonable person would take to be 'recovered', meaning the very real question of bias is almost irrelevant.

 

[Barry53]

Do psychiatrists have a tin ear, or is it considered witty to call a feature in their comic "Round the Corner?" Do they think that they score extra points if they get away with it?

In their favour it might be regarded as an honest representation of the attitude which some have for their patients.

Oh ... I missed that! It is actually just a bit sick, and unless I'm missing something, not really funny.

 

[Barry53]

Yes, also mentioned.
But, even the most optimistic claims of PACE, the recoveries weren't clinically what any reasonable person would take to be 'recovered', meaning the very real question of bias is almost irrelevant.

But being as we are dealing with some highly influential, highly unreasonable people, who still try and cling to the very small numbers, the unblinded-study-with-subjective-outcomes absurdity completely wipes out those small numbers.

I would love to know if there is some formal statistical technique for quantifying/qualifying the effect of combining an unblinded study with subjective outcomes. And then apply that to the tiny numbers of PACE, and prove that those numbers are really zero!

 

[alex3619]

And then apply that to the tiny numbers of PACE, and prove that those numbers are really zero!

Kieth Laws did a reanalysis of the published long term recovery data, just what was published in their paper, and showed patients were better off with SMC than CBT or GET. However this used the highly flawed criteria from PACE, if we had full data we could probably prove patients were much worse off with CBT or GET.

 

[Esther12]

Edit - Dolphin pointed this out:

Small correction: this isn't the main British Journal of Psychiatry:

Thanks - that's actually quite a big correction, and makes me feel much less positive about it.

Considering it was in the British Journal of Psychiatry, that comment was not as bad as I expected. I found the tone irritating when I first read it, and still a bit did on a re-read, but less so.

Really, the British Journal of Psychiatry should have published a piece saying "oh-oh, our President has been promoting quackery, and then smearing the patients who tried to point this out, and none of us noticed until some American academics took a look at the evidence for us". Submitting a piece like that might require more bravery than British academics are known for. I don't know how much leeway we should give UK psychiatrists for the dire situation of UK academic psychiatry, but I felt a bit more sympathetic on a second reading. It almost read like it was hinting that there was serious trouble here, without wanting to come out and say it.

Some notes, but I've not gone back to the Cochrane review to fact checking all the details, so this is just from memory and I may have missed stuff.

The main annoying thing was that it really failed to address problems with nonblinded trials relying on self-report outcomes properly, particularly important when, as part of trial, patients are given 'empowering' models of illness and positive claims about treatment efficacy. There was this paragraph, but throughout most of the article it was written as if changes in self-report outcomes reflected real changes in health and ignored the key reason for concern from patients:

It is important to remember that releasing
individual patient data does not correct any prior
methodological flaws – it simply opens the data up
for transparent re-interpretation. For example, in
this case it is also alleged that the investigators
(perhaps inadvertently) influenced participants’
self-reports with indiscriminate encouragement
in newsletters sent out during the trial. It is also
alleged that the investigators switched their own
scoring methods mid-trial.

PS: Not just alleged, they state that they did!

Other annoying things:

Good the issue of objective outcomes was mentioned, but fails to mention PACE's objectively measured fitness data, which showed GET was not associated with an improvment in fitness:

Few studies measured objective aerobic capacity
such as maximal oxygen consumption (VO2
max), although White et al (2011) did ask people
to undertake a 6-minute walking test in order to
examine real-world effects.

Given the problems with it, I think it was good PACE attracted controversy, rather than being mindlessly accepted in the way that it was by so many of the thickos in British medicine.

Unfortunately, the PACE study has since attracted
huge controversy. Patient groups have long been
critical of the CFS concept (as well as CFS trials
in general), but the criticism of the PACE trial
came from both patients and professionals. The
matter could have been easily resolved if the
original authors had issued a prompt correction
or released suitably anonymised primary data.

Also, the professionals criticising PACE have recognised that it was patients who led the way - the leadership role of patients does not exactly come across here, and generally I didn't like that way patient's concerns were written about, especially as I reckon half the members of this forum could tear apart some of this guys claims in a debate. Good to have problems with data sharing mentioned, but that could not have easily resolved the problems with the trial's design, and he does recognise this later on, in a section I already quoted:

It is important to remember that releasing
individual patient data does not correct any prior
methodological flaws – it simply opens the data up
for transparent re-interpretation.

Only some of PACE's data has been released (hence the new expression of concern at PLoS):

Under court order, the PACE study’s authors
finally were required to release their raw data in
September 2016 and it is now publicly available

-It largely fails to criticise the Larun review, which is odd given the problems recognised by the author about PACE and the presentation of results, yet dismissed by Larun even when these issues were raised with her in submitted comments. The comments from Courtney and Kindlon do a good job of showing the problems with the review, and Larun's responses are embarassing.

Good things:

-Some little important details were clarified that often are not. eg the limited nature of Specialist Medical Care in PACE: "In the PACE study, the control arm received specialist medical care alone (effectively, treatment as usual)".

-Mentioned the lack of things like actometer data, and say this data would be useful.

This paragraph gets the figures right, and is a fair summary of things. Although it would have been good to get an explanation of the problems with the earlier recovery critieria:

Independent re-analysis examined data for
recovery at the end of the trial and findings were
also disappointing (Matthees 2016). The recovery
rates using a priori thresholds were as follows: 3.1%
for specialist medical care alone, 6.8% for CBT,
4.4% for GET and 1.9% for adaptive pacing therapy,
with no significant differences between groups.
The PACE authors themselves maintained that
CBT and GET were associated with significantly
increased recovery rates of 22% at 52-week follow-
up, compared with only 8% for adaptive pacing
therapy and 7% for specialist medical care alone
(White 2013). Both reports were different from
the editorial claims that appeared in the BMJ at
the time of initial publication of the PACE study,
which suggested that 28–30% of patients recover
using CBT and GET (Knoop 2011).

He gets this right and clear:

Long-term follow-up at 2.5 years found that
any differences apparent between treatment arms
at 52 weeks were lost as adaptive pacing and
specialist medical care caught up with CBT and
GET (Sharpe 2015).

Here's the conclusion:

Conclusions
At face value the overall findings are that exercise
therapy is somewhat effective for CFS, particularly
when compared with treatment as usual, in that it
reduces symptoms at the end of therapy, possibly
with some sustained benefits. The hidden detail
is, as usual, rather more complicated. Exercise
therapy is probably the most effective of the
modalities studied in terms of daily function, as
measured by a walking test, but results are so
poor that, despite being statistically significant,
they are no cause for celebration. Recovery rates
are similarly disappointing. Independent re-
analysis of the PACE data found that only about
3% recover with standard medical care, which
tells us that standard medical care is not working
adequately for patients with CFS and we need to
re-examine why it is so ineffective. Only about
4–7% of patients recover in active treatment over
3–6 months, which is a significant improvement in
terms of relative risk, but not in terms of absolute
risk change.

Beyond the raw results this controversy has a
number of critical lessons. First and foremost, it
is imperative for researchers to publish studies in
the most open and transparent manner possible.
This may include responding to requests for
methodological clarification, requests for re-
analysis and even requests for primary data.
In some online journals there is actually a
requirement to release such data on request. It
is remarkable that it is still not normal practice
for researchers to reveal or copublish their actual
anonymised raw data. A second lesson is that
clinicians and researchers should work more
closely with patients in both study design and
study interpretation. Clinicians and academics
may not have the same views on what is and is not
acceptable therapy for patients. The third lesson
is that, to promote acceptability, psychosocial
treatments should be integrated into medical care.
In practical terms this means that patients should
be offered these options as an optional add-on
while in medical care, not as a way of discharging
patients perceived as difficult into a mental health
service. One major reason for low parity of esteem
is that physical concerns are overlooked in patients
with mental health complications. May patients
with CFS need psychological support and, where
necessary, mental health input, but not at the
expense of thorough medical care.

Overall: step in the right direction, and I now feel a confused mix of anger and gratitude, but we've got a long way to go.

 

[Barry53]

Overall: step in the right direction, and I now feel a confused mix of anger and gratitude, but we've got a long way to go.

I think it is part and parcel of the snowball effect - the truth is now at the beginning of being on a roll, and is on the cusp of attaining its critical mass point. What would be the chances of this being published without all the campaigning that has gone before it? What are the chances of more and more like, and better than, this with continued campaigning?

I am especially pleased to see this ...

Itis remarkable that it is still not normal practice for researchers to reveal or copublish their actual anonymised raw data.

I have been saying in posts along the way, that releasing of trial results should actually mean releasing a complete package, comprising results along with the full (anonymised) trial data; not just cherry-picked, but the whole data set.

No data? No publication of results! Results do not deserve to be published if not released alongside the dataset for enabling open unbiased peer review. And by unbiased, I include the avoidance of data subset bias.

In fact the notion of "releasing trial results" is something of a misnomer as it currently stands - it is only ever really a "releasing of one interpretation of trial results". Which is why the enabling of open peer review is so important.

 

[Snow Leopard]

The bias of unblinded studies without objective measures of disability are always going to be a blind spot for these people, because this blind spot is central to their field - psychiatrists believe thoughts and cognitions, namely how we feel about symptoms are more important than objective disability caused by those symptoms. Apart from this blind spot, I am glad to see reasonable criticism of the malpractise by the PACE trial authors - even if he did not tell the complete story.

 

[A.B.]

It's a blind spot because if they acknowledged it, they would put themselves out of business.

 

[adreno]

Really, the British Journal of Psychiatry should have published a piece saying "oh-oh, our President has been promoting quackery, and then smearing the patients who tried to point this out, and none of us noticed until some American academics took a look at the evidence for us".

That's not going to happen straight out, but when one from their own ranks starts to tentatively criticize the top brass that is a good thing, and a step in the right direction, as you say.

 

[Valentijn]

Considering it was in the British Journal of Psychiatry, that comment was not as bad as I expected. I found the tone irritating when I first read it, and still a bit did on a re-read, but less so.

Based on the definition given of CFS and the largely uncritical acceptance of the primary treatments for it being CBT and GET, the author appears to be deeply involved in the field of psychobabble. That's nothing new, unfortunately

But what makes this piece exceptional is that even a committed psychobabbler is questioning the methodology of PACE and the trial authors' policy of data secrecy. The PACE authors are essentially under siege from all sides now, including from within their own stronghold of psychobabble.

 

[Yogi]

Really, the British Journal of Psychiatry should have published a piece saying "oh-oh, our President has been promoting quackery, and then smearing the patients who tried to point this out, and none of us noticed until some American academics took a look at the evidence for us". Submitting a piece like that might require more bravery than British academics are known for. I don't know how much leeway we should give UK psychiatrists for the dire situation of UK academic psychiatry, but I felt a bit more sympathetic on a second reading. It almost read like it was hinting that there was serious trouble here, without wanting to come out and say it.

This is the journal of RPsych so finally after 10 years after being told about the scandal they have made one small step towards whispering it.

Don't worry under the tab for the quack you refer, Professor Sir Lord Regius Wessely, there is the topic of Malingering by patients (i.e. fraud)

So they won't deal with the biggest medical and scientific and insurance fraud possibly in history perpetrated by those right under their noses. But they deal with the issue of malingering fraudulent patients (with ME patients making the bulk of those being accused).

They have articles on patients presumably with false illness beliefs and MUS malingering and how to call out such patients without actually referring to as such.

http://www.rcpsych.ac.uk/default.aspx?page=17609

http://apt.rcpsych.org/content/23/2/115

 

[Yogi]

Even when directed to release data by the Information Commission for England, QMUL spent £200000 on legal fees in a meaningless appeal (Savage 2016). Under court order, the PACE study’s authors nally were required to release their raw data in September 2016 and it is now publicly available (https://sites.google.com/site/pacefoir/pace- ipd_foia-qmul-2014-f73.xlsx?attredirects=0). In an odd move, the day before the release, the PACE authors released a re-analysis of their own data (Goldsmith 2016).

For example, in this case it is also alleged that the investigators (perhaps inadvertently) influenced participants’ self-reports with indiscriminate encouragement in newsletters sent out during the trial. It is also alleged that the investigators switched their own scoring methods mid-trial.

"perhaps inadvertently" .

Imagine a CFS patient using that line in a court of law or with the DWP or insurer when one is caught out.