Controversy over exercise therapy for chronic fatigue syndrome: continuing the debate

[Dolphin]

Mitchell says: ‘First and foremost, it is imperative for researchers to publish studies in the most open and transparent manner possible’. In fact almost all our papers were published with open access, and we have responded to scientific queries and criticisms appropriately and repeatedly in papers cited here, in journal correspondence, and in over 100 frequently asked questions available on the trial website (www.wolfson.qmul.ac.uk/currentprojects/ pace-trial). We have also shared data when ethically possible (White 2016).

The frequently asked questions were often ones they wanted to answer themselves rather than questions others wanted answered.

 

[Dolphin]

We suggest that the most obvious lesson from our experience of the PACE trial is that science can sometimes provide answers that are not popular with everyone (Lancet 2011; Hawkes 2011; Wessely 2015; Sharpe 2016). However, such answers should stand or fall by independent replication, not by unreasonable criticism and demands for retraction.

A £5 million trial can't easily be replicated by critics.

Replication is not the only way trials can be evaluated. The methodology and reporting can also be challenged.

Certainly not all of the criticism of the PACE trial was unreasonable.

Demands for retraction can have a place. The way they talk it makes out that there is no place for calls for retraction in science and medicine.

 

[Dolphin]

That said, our recovery rates were similar to those found in previous studies (22% recovered after CBT and GET) (Sharpe 2017).

From:

Carolyn Wilshire, Tom Kindlon & Simon McGrath (2017) PACE trial claims of recovery are not justified by the data: a rejoinder to Sharpe, Chalder, Johnson, Goldsmith and White (2017), Fatigue: Biomedicine, Health & Behavior, 5:1, 62-67, DOI: 10.1080/21641846.2017.1299358

Putting aside the issue of whether these studies are indeed representative of the literature, there are serious problems with this reasoning. First, most of the cited studies defined recovery differently to PACE, and if we make even a reasonable attempt to equate the definitions, then the convergence amongst recovery rates does not look quite so good.2

2. For example, the definition of recovery used in the studies by Deale et al. [8] and Knoop et al. [9] were a lot stricter than the revised criteria used in PACE. If the Deale et al. recovery criteria are applied to the PACE data, for example (it is possible to use three of the four criteria), the PACE recovery rates fall to a maximum of 9% for CBT, which is very different from the 24% for CBT cited in Deale et al.

 

[Dolphin]

Here I find that 27% of those who received GET and similarly 27% of those who received CBT no longer met Oxford criteria at 52 weeks (i.e. the two groups showed exactly the same rate of recovery). However, combining this metric with data for patient-rated change (a clinical global impression of change score at 52 weeks of much improved or greater) as well as data for those who were no longer fatigued (judging by a score of 3 or lower using binary scoring on the Chalder Fatigue Questionnaire at 52 weeks), I find that 12.5% who received GET v. 10.8% who received CBT were recovered. Overall, these results suggest to me that GET was probably more effective than CBT, but only on some measures. Ultimately, only a small proportion who received either (or indeed any) treatment were fully recovered at 52 weeks.

I don't like this definition of recovery. The Chalder fatigue scale is not well anchored. I don't find it that trustworthy especially after behavioural therapies.

Also, why should "much better" as well as "very much better" be counted as recovered. If you were recovered from CFS why would you not say you were "very much better". Especially given that all participants in the trial were below certain thresholds so weren't at the very top end of functioning before the trial.

 

[Dolphin]

I like the last paragraph from Mitchell:

The last point I want to make is that my call for open data reflects a movement that has been gathering momentum for several years and that will hopefully reduce misunderstanding between researchers, journal editors, patients and the public, who often wish to comment on medical research. The AllTrials campaign launched in 2013 called for a clinical trials register together with a published summary of trial protocols (www.alltrials.net/ about/). In the same year, the BMJ launched the Open Data Campaign to encourage sharing of primary research data and announced that it will no longer publish any trial where the authors do not commit to making the relevant anonymised patientlevel data available on reasonable request (www. bmj.com/open-data). The International Committee of Medical Journal Editors (ICMJE) believes that there is an ethical obligation to responsibly share data generated in interventional clinical trials because trial participants have put themselves at risk, and it hopes to create an environment in which the sharing of de-identified individual participant data no later than 6 months after publication becomes the norm (Taichman 2016). Facilities to allow researchers to upload their data are available. Some have objected, arguing for a much longer delay before publication of trial data and financial compensation from those using their data who were not involved in the trial (International Consortium of Investigators for Fairness in Trial Data Sharing 2016). The ICMJE accepts the quality standard of trial registration (ideally with data) in any registry listed in the WHO International Clinical Trials Registry Platform (ICTRP) or in ClinicalTrials. gov. It endorses these registries because they meet several criteria. They are accessible to the public at no charge, open to all prospective registrants, managed by a not-for-profit organisation, have a mechanism to ensure the validity of the registration data, and are electronically searchable. I thank Chalder et al for the opportunity to raise these points again. It is important that all clinicians and academics involved in mental health research are aware of the Open Data Campaign as well as the AllTrials campaign, because in my opinion there is no reason for mental health research to be treated any differently from any other area of medical research.

 

[Large Donner]

The Wilshire re-ananlysis simply makes different assumptions, using more stringent thresholds to determine recovery.

Yes it uses your own thresholds.

That said, our recovery rates were similar to those found in previous studies (22% recovered after CBT and GET)

So basically once you had an idea of the rates you changed the protocols for recovery and drew the line "coincidentally" to match another paper to show the same 22% rates.

Doesn't this prove both of those papers equally flawed if the PACE trial paper was so pointless one could be ill enough to enter the trial and declared recovered at the same time?

 

[Large Donner]

We suggest that the most obvious lesson from our experience of the PACE trial is that science can sometimes provide answers that are not popular with everyone (Lancet 2011; Hawkes 2011; Wessely 2015; Sharpe 2016).

Yes your answers are very unpopular with scientists.

However, such answers should stand or fall by independent replication, not by unreasonable criticism and demands for retraction.

Only a fucking monkey would replicate the PACE trial.

 

[Sean]

I dunno. The monkeys I have met have more sense and self-respect than that.

 

[user9876]

A £5 million trial can't easily be replicated by critics.

Replication is not the only way trials can be evaluated. The methodology and reporting can also be challenged.

Certainly not all of the criticism of the PACE trial was unreasonable.

Demands for retraction can have a place. The way they talk it makes out that there is no place for calls for retraction in science and medicine.

I the form PACE takes there would be no point of a replication. Redoing the trial properly would be necessary with proper objective measures and proper control groups.

I think if someone were to replicate PACE with their methodology they would get similar results as the results are a feature of the methodology not the interventions.

 

[user9876]

Yes your answers are very unpopular with scientists..

Its not the answers that are unpopular with scientist but the method for getting them which means that the answers are unreliable.

 

[Jonathan Edwards]

That was a bit frustrating. I felt like White & co left themselves open to a strong response with their comment, which seemed less carefully phrased than their usual work. The Mitchell reply was pretty weak though.

At least it seems there is some debate in response to efforts from members. But Mitchell, being a psychiatrist, is essentially accepting the validity of this type of study and just dancing around some details. I strongly suspect he is happy to conduct similarly weak trials himself. Both sides completely miss the point - that the Cochrane review that purports to analyse a set of controlled trials does not, because none of them are controlled in a valid sense. The psychiatrists are sticking their heads in the sand. In the meantime they are all involved in a major growth industry of selling unvalidated talking therapies on the grounds that they 'save money'.

 

[Barry53]

... they're so ethically void that they think dishonesty is okay if you just imply it instead of lying outright.

I think people who are so ethically void (I like that phrase) just don't comprehend it's dishonest to imply untruths, and it is only dishonest if you explicitly state them. They are wired up so differently.

 

[Large Donner]

I think people who are so ethically void (I like that phrase) just don't comprehend it's dishonest to imly untruths, and it is only dishonest if you explicitly state them. They are wired up so differently.

I just dont understand how they can make a career out of other peoples misery, at some point they must understand that they impede scientific development, cause us to have no access to medical treatment and force us into poverty, housing issues and stigma via having to fight for benefits that are often no awarded.

It doesn't just affect the patient it causes poverty down the whole family including the children of the patient due to household income, borrowing potential for mortgages, landlord prejudice for rental properties etc and impeded family life.

I just couldn't do that as job I don't get it. I guess its just to difficult to step of the gravy train, they would have to face the financial wilderness we do and all the problems that brings on top of being sick.

 

[Large Donner]

At least it seems there is some debate in response to efforts from members. But Mitchell, being a psychiatrist, is essentially accepting the validity of this type of study and just dancing around some details.

@Jonathan Edwards

In all your time in medicine what was the general consensus with other physicians on psychiatry as a whole. It just seems so glaringly obvious that if medicine claims to be aimed at obtaining objective evidence before it can make any claims how can you have a field "within medicine" that just gets a pass on such things?

 

[Esther12]

At least it seems there is some debate in response to efforts from members. But Mitchell, being a psychiatrist, is essentially accepting the validity of this type of study and just dancing around some details. I strongly suspect he is happy to conduct similarly weak trials himself. Both sides completely miss the point - that the Cochrane review that purports to analyse a set of controlled trials does not, because none of them are controlled in a valid sense. The psychiatrists are sticking their heads in the sand. In the meantime they are all involved in a major growth industry of selling unvalidated talking therapies on the grounds that they 'save money'.

I'm a bit concerned that, f I was new to the debate and read that exchange, I'd come away thinking that the PACE response was pretty solid and that there wasn't too much reason for concern.

I guess that people can be pretty cautious though, so even a whiff of real controversy (rather than just unreasonable patients making a fuss) might still be enough to mean people avoid recommending CBT/GET as evidence based treatments.

 

[Jonathan Edwards]

@Jonathan Edwards
In all your time in medicine what was the general consensus with other physicians on psychiatry as a whole. It just seems so glaringly obvious that if medicine claims to be aimed at obtaining objective evidence before it can make any claims how can you have a field "within medicine" that just gets a pass on such things?

In my time in medicine I did not think much about how rigorous psychiatrists were. I had no real contact with academic psychiatry. I was pretty surprised when I started looking in to PACE and trials for CBT in general, not so much that poor trials were being done but that there was clearly no concern on the peer review front.

We do not always need objective evidence. We need reliable evidence. There are reliable ways to assess if a teaming is good for pain, for instance, and pain is subjective. But this is a subtle distinction - I agree that psychiatry seems to be getting a pass on reliability. It is something I am tempted to take up with the Cochrane administration in due course.

 

[Jonathan Edwards]

I'm a bit concerned that, f I was new to the debate and read that exchange, I'd come away thinking that the PACE response was pretty solid and that there wasn't too much reason for concern.

I guess that people can be pretty cautious though, so even a whiff of real controversy (rather than just unreasonable patients making a fuss) might still be enough to mean people avoid recommending CBT/GET as evidence based treatments.

I fear that Mitchell accepts that CBT and GET work and thinks that what has been identified is an issue of detail. It is very likely that if were to face up to the real problem he would have to put a line through at least half of his 300 publications.

 

[trishrhymes]

It is something I am tempted to take up with the Cochrane administration in due course.

Please please do!

If you could get them to confront the problem of unblinded trials with subjective primary outcome measures and make a policy decision not to include any such trials in Cochrane reviews that would be amazing.

The trouble I see with this is that this would mean they could not include any of the trials of CBT and other therapies, since by the nature of the treatment they are unblinded, and by the nature of the conditions usually treated (anxiety and depression), there are no objective things to measure.

If you could at least get them to exclude trials of psychological treatments for conditions such as ME and the other physical conditions dumped in the MUS dustbin where there are objective outcome measures possible, but none have been used, that would be a huge step forward.

This would mean, for example, reanalysing the effectiveness of the PACE trial using only the 6 minute walk test (taking into account that lots didn't do it), step test and employment data, and excluding all subjective measures. It might also force the likes of Esther Crawley to start using actometers. Still doesn't get around all the other problems with PACE like whether the participants actually had ME and whether they complied with treatments though. I don't really think PACE can be rescued and provide any meaningful data, apart from the fact that despite their best efforts to get the result they wanted, they inadvertently managed to show their treatments make no long term difference.

You point out that part of the problem lies in peer review. The problem is, of course, that the BPS cabal all work together to approve each others trials and review each others papers. And funding bodies are in their pockets too. In a sense Cochrane is the summit of a stinking pile of bad science right up from the foundation of bad trial design. Until that is sorted out, nothing will change.

Sorry, you know all this already. Just letting off steam!

 

[Large Donner]

In my time in medicine I did not think much about how rigorous psychiatrists were. I had no real contact with academic psychiatry. I was pretty surprised when I started looking in to PACE and trials for CBT in general, not so much that poor trials were being done but that there was clearly no concern on the peer review front.

Did you ever find yourself in a situation wherby a patient came to you with symptoms that at first fell within rheumatology but you couldn't find an explanation for them and ever thought to refer them to a psychiatrist or was this something that you saw lots of other physicians doing?

Where you ever encouraged to do such things by means of the hierarchy or "training" you were given?

I can understand that working in one field you wouldn't necessarily give thought to how rigorous psychiatry was but isn't there some suspicion of psychiatry as a whole within the rest of medicine, surely there must be some given how we see it to be operating now.

 

[Barry53]

I just dont understand how they can make a career out of other peoples misery

The thing I eventually got to understand a good few years ago now ... is that you never are going to understand, and you don't really want to. When we "try to understand" another person, we try to put ourselves in their shoes, empathise with them, tune our own emotions and thinking in with theirs. But to do that we make a fundamental assumption about the other person and we do it unconsciously - that their moral code is essentially the same as our own, no matter how different they may be in other ways. When the other person's morality is by and large as you expect, then your efforts to understand them at least have a fighting chance. But if the other person's behaviour is instead driven by self-serving, cause-and-effect manipulations of others, where even the "good" they do proves to usually be only when it benefits them - that sort of person is impossible to really understand in terms that you understand, and you have to accept that. I think of them more as an alien species, just happening to look like people.

These days I can get my head round them to a degree as an academic exercise, and have no wish to try understand them any deeper than that.

On a slightly philosophical note, and assuming that human evolution leads to a merging of character traits into the population, that suggests that as you go back in time human character traits would perhaps have been more diverse and granular. Maybe long way back there were complete clans of people like this, who maybe merged in over time.