PACE Trial and PACE Trial Protocol

[oceanblue]

Assuming charities/advocacy organisations firmly believe that ME/CFS has an organic etiology, why didn't they just state something along the lines of :

Thanks for showing that the modest results of CBT and GET for some ME/CFS patients are smaller than those found with similar approaches with MS (plus many other diseases I've previously referenced) thereby confirming that neither faulty illness beliefs nor deconditioning play any major or specific role in ME/CFS.

I read a lot of papers trying to back up exactly that point, mainly for MS and Rheumatoid Arthritis. The problem is that the evidence is all over the place with some studies showing great results and some little gain from CBT/GET - and mostly the research is of poor quality*. That recent MS study is a perfect example: only 26 patients and no control group (results are for pre/post treatment). So while I suspect the point is true, I couldn't find robust evidence one way or another for the effectiveness of CBT/GET in such illnesses. If anyone knows different, please let me know.

*it was quite an eye-opener to see just how much rubbish research is out there across a range of illnesses: CFS is by no means unique.

 

[Esther12]

We are all aware of how the PACE results were spun and reported in the press yet the response of ME/CFS charities/advocates has been either to report the results as presented and then appeal to patient surveys to counter the over inflated claims or to get into arcance arguments over case definitions and scoring systems which would mean little to the casual observer (I'm as guilty as anyone of the latter).

This may have been discussed before in this thread and this letter may not be the only exception but there is ample scope to use the PACE results to make some very simple points that suggest a very different interpretation to what has been reported.

Assuming charities/advocacy organisations firmly believe that ME/CFS has an organic etiology, why didn't they just state something along the lines of :

Thanks for showing that the modest results of CBT and GET for some ME/CFS patients are smaller than those found with similar approaches with MS (plus many other diseases I've previously referenced) thereby confirming that neither faulty illness beliefs nor deconditioning play any major or specific role in ME/CFS:

Thanks also for showing that following a year of therapy, ME/CFS patients were still functioning physically at a similar level to 85 year olds (6MWT) and below or close to the lowest 16% of the population (SF36 - PF mean minus 1SD, a level which the PACE authors defined as 'normal').

Clearly, this illness model contributes little to our understanding of ME/CFS and the associated therapies offer only marginal benefits to this very disabled population.



I know the flaws go much deeper than this but these are stark and easily understood.

Yeah... but... the charities had to prepare responses without having the paper. It did allow others to act as if the response was 'anti-science', and maybe no immediate response at all would have been preferable, but they were in a difficult situation.

They were right not to trust the interpretation of results presented by Chalder and White, but were poorly placed to respond full to it.

 

[Marco]

Yeah... but... the charities had to prepare responses without having the paper. It did allow others to act as if the response was 'anti-science', and maybe no immediate response at all would have been preferable, but they were in a difficult situation.

They were right not to trust the interpretation of results presented by Chalder and White, but were poorly placed to respond full to it.

I agree Esther but it isn't too late to add this interpretation to any mention they make of PACE specifically or CBT/GET generally or when commenting on government provided information etc.

 

[Esther12]

I agree Esther but it isn't too late to add this interpretation to any mention they make of PACE specifically or CBT/GET generally or when commenting on government provided information etc.

Yeah - sorry. I took 'response' to mean 'initial response'. Really though, it's only the initial response that got reported in the media.

 

[Dolphin]

"Methodological Inconsistencies in the PACE trial for ME/CFS" by Tate Mitchell

"Methodological Inconsistencies in the PACE trial for ME/CFS" by Tate Mitchell

http://www.mediafire.com/file/58xlwu12afj903x/PACE trial critique Dec. 02, 2011.docx

(or if that link doesn't work, see a plain text version (no graphs) at: https://listserv.nodak.edu/cgi-bin/wa.exe?A2=ind1112a&L=co-cure&T=0&P=1152

Haven't read this yet but did read a draft which I thought looked good.

 

[Marco]

"Methodological Inconsistencies in the PACE trial for ME/CFS" by Tate Mitchell

http://www.mediafire.com/file/58xlwu12afj903x/PACE trial critique Dec. 02, 2011.docx

(or if that link doesn't work, see a plain text version (no graphs) at: https://listserv.nodak.edu/cgi-bin/wa.exe?A2=ind1112a&L=co-cure&T=0&P=1152


Haven't read this yet but did read a draft which I thought looked good.

Excellent

I wonder would Fiona Godlee (BMJ editor) care to comment given that she has made her position so clear on adhering to published protocols?

 

[Bob]

Here's a new critique of the PACE Trial, by Prof Malcolm Hooper, which highlights and explains a few of the statistical flaws in the published Lancet paper...

It also includes some interesting background info about how Hooper's critique was rejected for publication in the magazine of the Royal Statistical Society, Significance.

It's quite dense reading, but I think it throws some devastating punches, esp re the 'normal range', use of the term 'moderate improvements', and the 'six minute walking distance test' results.

It's published on Invest in ME's website...

Webpage:
http://www.investinme.org/Article435 Statistics and ME.htm?forumid=331851

PDF version:
http://www.investinme.org/Documents/PACE Trial/Statistics and ME.pdf

 

[Dolphin]

Here's a new critique of the PACE Trial, by Prof Malcolm Hooper, which highlights and explains a few of the statistical flaws ...

It also includes some interesting background info about how it was rejected for publication in the magazine of the Royal Statistical Society, Significance.

It's quite dense reading, but I think it throws some devastating punches, esp re the 'normal range', 'moderate improvements', and the 'six minute walking distance test'.

It's published on Invest in ME's website...

Webpage:
http://www.investinme.org/Article435 Statistics and ME.htm?forumid=331851

PDF version:
http://www.investinme.org/Documents/PACE Trial/Statistics and ME.pdf

Pity it wasn't published. Interesting that they got a request.

 

[Sean]

Thank you Tate Mitchell.



This graph seems to be missing the critical definition of the green and purple colouring.

Purple = baseline
Green = therapeutic gain



If I may be so bold and suggest a handful of small clarifications in a passage from Section VI, so it reads:

In the PACE Trial, actometers were used at baseline, but were not used outcome. The only objective outcome measure reported was the six-minute walk test (6MWT), which only increased for CBT participants by 21m to 354m, a change that was actually slightly smaller than that of the (SMC) control group.



Additional issues that add to the PACE problems include:

1) not using a test - retest protocol, especially for any objective measure(s)

2) the reliability of the way they used the 6MWT (did they account properly for the learning effect?)

3) what exactly was "specialist" about the Standard Specialist Medical Care they used, beyond the standard general practitioner (primary care physician) level of medical care?

4) why any supposed therapeutic gains tapered off by about the 12th week of the 52 week trial, with no further gains being made for the remainder of the trial?

 

[biophile]

Good suggestion and questions, Sean.

Have only skim read the new documents from Hooper and Mitchell respectively, but so far so good.

Judging from the invitation that Hooper received, the editor seemed aware of Hooper's critical views and knew what to expect. Hooper's introduction was clever:

Imagine the following fictitious scenario: a high impact medical journal publishes the results of a clinical trial described as a large RCT of Eutensia, a new drug for refractory hypertension; the results are impressive, with 30% of those given Eutensia reported as having normal blood pressure at the end of the trial. However, the trial investigators had redefined the normal range of blood pressure such that it was possible for a person receiving Eutensia to leave the trial with higher blood pressure than before treatment but still be counted as having blood pressure within the normal range.

 

[Purple]

In the PACE Trial, actometers were used at baseline, but were not used outcome. The only objective outcome measure reported was the six-minute walk test (6MWT), which only increased for CBT participants by 21m to 354m, a change that was actually slightly smaller than that of the (SMC) control group.

My understanding is (please correct me if I am wrong) that the SMC group is not a control group - a control group would be a 'no intervention' group, i.e. no treatment at all. So this is not a controlled trial, just a randomised trial. (Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial.)

This is what Prof Hooper says about this in the second last paragraph of the document linked by Bob:
...the international ME community wonders why the PACE Trial is being hailed as a gold standard study which demonstrated the efficacy of CBT and GET for ME/CFS patients (although the Protocol refers to it as an RCT, The Lancet paper at no point describes PACE as a controlled trial, yet it was described in the press release as the highest grade of clinical evidence and as extremely rigorous (and) carefully conducted).

also from the document:
In a radio interview, one of the Investigators stated candidly: What this trial isnt able to answer is how better are these treatments than really not having very much treatment at all

Webpage:
http://www.investinme.org/Article435 Statistics and ME.htm?forumid=331851

PDF version:
http://www.investinme.org/Documents/PACE Trial/Statistics and ME.pdf

 

[biophile]

The 2007 PACE trial protocol refers to itself as a randomized controlled trial. Dropping of the word "controlled" in the 2011 Lancet paper may have been to save space in the title, although it is interesting that they do not refer to it elsewhere as a controlled trial. Perhaps because the SMC-alone group received more sessions of SMC than the adjunctive therapy groups and therefore was not adequately controlled? A trial does not have to blinded or placebo-controlled to be regarded as a RCT, but obviously response bias can be a problem too in an unblinded trial aimed at altering patients perceptions and behaviours then asking them to fill out subjective questionnaires. I tentatively view the SMC-alone group as a de facto control group, but yes there should have been a 5th group that received nothing at all just like in the real world.

 

[oceanblue]

This, from Statistics on ME, was new to me:

The Protocol set specific benchmarks against which findings were to be judged;
additionally, in 2006 the Chief Principal Investigator assured the Multicentre Regional
Ethics Committee that a categorical positive outcome would be an SF-36 score of at
least 75, saying that this would [reassert] a ten-point score gap between entry criterion
and positive outcome, and that it would bring the PACE trial into line with the FINE
trial, an MRC funded trial for CFS/ME and the sister study to PACE (14).

When in April 2010 the FINE (Fatigue Intervention by Nurses Evaluation) Trial reported,
the difference between intervention and comparison groups at the primary outcome
point was not statistically significant, so it is notable that when the PACE report was
published in February 2011, these same benchmarks of a positive outcome had been
dropped.

14. Letter dated 9th February 2006 sent by Professor Peter White to Mrs Anne
McCullough, Administrator, West Midlands Multi-centre Research Ethics Committee

Nice.

Though of course the PACE authors would fall back on their guff about continuous outcome measures being more appropriate (even though they add in the categorical post-hoc normal measure, while claining there was "insufficient space" to report on the categorical recovery item specified in the protocol).

 

[oceanblue]

Control group or not?

I've long been puzzled by this. The authors studiously avoid refererring to a control group, yet I agree with Biophile that the SMC group is a control of sorts - it's what the study compares treatments against, which is exactly the way any other trial would use a control group.

Ideally, I think they should have used an attention-placebo control eg relaxation therapy, as other trials have done. This would also have helped to control for any self-report bias resulting from a strong relationship between therapist and patient.

 

[Bob]

I've long been puzzled by this. The authors studiously avoid refererring to a control group, yet I agree with Biophile that the SMC group is a control of sorts - it's what the study compares treatments against, which is exactly the way any other trial would use a control group.

Ideally, I think they should have used an attention-placebo control eg relaxation therapy, as other trials have done. This would also have helped to control for any self-report bias resulting from a strong relationship between therapist and patient.

My thoughts about this are that the authors wish to downplay and obscure the significance of the SMC being a control group, as they have done persistently in their media work by only quoting the "30%" figure (The "30%" improvement was as a result of GET+SMC or CBT+SMC, not GET or CBT alone, which was only about 15%.)

In the protocol(s), they clearly set out that the SMC was to be used as a control group, and also that each of the groups was to be used as a control group for all the other groups to be compared with. So the authors also used the SMC+CBT and SMC+GET groups as control groups to compare the SMC+APT group against (i.e. they have compared the APT group to the CBT group to highlight how ineffective their form of 'pacing' is.)

Here are some helpful quotes from the protocol(s):

"SSMC is also the trial comparison intervention against
which the three supplementary therapies will be judged."

"This trial compares the efficacy of the additional therapies when added to
specialist medical care against specialist medical care alone."

"There is therefore an urgent need to: (a) compare the supplementary
therapies of both CBT and GET with both APT and SSMC alone, seeking evidence of both
benefit and harm (b) compare supplementary APT against SSMC alone and (c) compare
the supplementary therapies of APT, CBT and GET in order to clarify differential predictors
and mechanisms of change."

This shows that the intention was for it to be an RCT:

"Long title of trial:
A randomised controlled trial of adaptive pacing, cognitive behaviour therapy, and graded
exercise, as supplements to standardised specialist medical care versus standardised
specialist medical care alone for patients with the chronic fatigue syndrome / myalgic
encephalomyelitis or encephalopathy"



One very interesting, and telling, section from the protocol is this:

"4.4.2 Differential outcomes
Because CBT and GET are based on a graded exposure to activity or exercise, they may
preferentially reduce disability, whilst APT, being based on the theory that one must stay
within the limits of a finite amount of "energy", may reduce symptoms, but at the expense
of not reducing disability. By measuring both symptoms and disability as our primary
outcomes, we will be able to test this secondary hypothesis."

Notice that their hypothesis was demonstrated to not carry any weight because while CBT minimally improved subjective fatigue, it failed to improve physical function or disability. This clearly took them by surprise, and went against all their expectations, and they conveniently failed to highlight this failure of CBT in the final paper.

 

[Esther12]

"Methodological Inconsistencies in the PACE trial for ME/CFS" by Tate Mitchell

http://www.mediafire.com/file/58xlwu12afj903x/PACE trial critique Dec. 02, 2011.docx

(or if that link doesn't work, see a plain text version (no graphs) at: https://listserv.nodak.edu/cgi-bin/wa.exe?A2=ind1112a&L=co-cure&T=0&P=1152

Haven't read this yet but did read a draft which I thought looked good.

This looks really good.

lol at how long this nicely condensed summary is though. No wonder it's hard to explain the problems to anyone.

Including quotes with the references is a really good idea too - it makes it a lot easier to casually look in to references, especially for those who struggle to access academic papers. I'm going to start doing this.

 

[urbantravels]

I wonder if the Hooper paper could be submitted for PLosOne. I'm not clear on whether it would be considered "primary research" or a "systematic review" by their standards, but it seems like something worth looking into...?

http://www.plosone.org/static/guidelines.action

 

[biophile]

I wonder if the Hooper paper could be submitted for PLosOne. I'm not clear on whether it would be considered "primary research" or a "systematic review" by their standards, but it seems like something worth looking into...? http://www.plosone.org/static/guidelines.action

Good idea. One wonders if the 2011 Lancet paper on the PACE Trial itself would meet PLoS ONE standards? (emphasis added): ...

To be accepted for publication in PLoS ONE, research articles must satisfy the following criteria:

1. The study presents the results of primary scientific research.
2. Results reported have not been published elsewhere.
3. Experiments, statistics, and other analyses are performed to a high technical standard and are described in sufficient detail.
4. Conclusions are presented in an appropriate fashion and are supported by the data.
5. The article is presented in an intelligible fashion and is written in standard English.
6. The research meets all applicable standards for the ethics of experimentation and research integrity.
7. The article adheres to appropriate reporting guidelines and community standards for data availability.

 

[Mark]

Doesn't seem to fit the first requirement though:

"1. The study presents the results of primary scientific research."

 

[Dolphin]

There are other journals out there. One that isn't PubMed-listed but is open access after one issue is the Bulletin of the IACFS/ME: http://iacfsme.org/BulletinArchives/tabid/309/Default.aspx .