PACE Trial and PACE Trial Protocol

[OverTheHills]

Don't read this post if you want an optimistic spin

I am not feeling so optimistic about this.

Remember the review of the NICE guidelines on ME (which define the range of treatments and tests allowed for use by doctors, or rather the lack of same) is currently on hold until the results of the PACE trial are released.

The relevant NICE committee issued a view that although there wasn't any new evidence in the last few years to warrant changing the guidelines(!) it would be good to wait until PACE results were released.

Regardless of the damp squib results I suspect the Wessleyites will now try to further entrench CBT/GET in the NICE guidelines (compulsion? on threat of benefit withdrawal? expansion of CBT/GET funding?) and try to railroad this in before the scientific community has had time to reflect on and repudiate the PACE results.

Remember the NICE guideline people are a cabal, and as ever Wessley has control of the publicity machine. MEA's rebuttal, Prof Hooper's letters and complaints do not get traction with the press or Govt .

I think we are in dangerous times.

OTH

 

[Snow Leopard]

Cort, have you fully read the article, they state:

In another post-hoc analysis, we compared the proportions of participants who had scores of both primary outcomes within the normal range at 52 weeks. This range was defined as less than the mean plus 1 SD scores of adult attendees to UK general practice of 142 (+46) for fatigue (score of 18 or less) and equal to or above the mean minus 1 SD scores of the UK working age population of 84 (–24) for physical function (score of 60 or more). 32,33

They then go on to claim that 16% of APT, 30% of CBT, 28% of GET, 15% SMC patients returned to the normal range.

Of course claiming 60% on the SF-36 physical subscale as normal is a cruel joke.

edit- has anyone read the editorial?

 

[Sean]

The main reference they use to define CFS in this paper seems to be:

Prins JB, van der Meer JW, Bleijenberg G. Chronic fatigue syndrome. Lancet 2006; 367: 346-355.


I think we can see where the PACE authors are coming from, and heading to.

 

[oceanblue]

They then go on to claim that 16% of APT, 30% of CBT, 28% of GET, 15% SMC patients returned to the normal range.

Of course claiming 60 on the SF-36 physical subscale as normal is a cruel joke.

Bonkers: with a score of 65 you are deemed ill enough to enter the trial - it's a recruitment criterion.

 

[oceanblue]

I thought Tuller blew it in his rather small article. These were my comments to the NYTimes

Excellent response, Cort. I think this study shows that the biopsychosocial model is in flames.

 

[Enid]

Thanks Cort and all for your thoughts and analysis on this disappointment. Looks as though the Psychos hanging on by fingertips or anything else to stay in the "game"

 

[Snow Leopard]

Bonkers: with a score of 65 you are deemed ill enough to enter the trial - it's a recruitment criterion.

I didn't even notice that. You're right. And it's completely bizarre.

 

[Angela Kennedy]

I am not feeling so optimistic about this.

Remember the review of the NICE guidelines on ME (which define the range of treatments and tests allowed for use by doctors, or rather the lack of same) is currently on hold until the results of the PACE trial are released.

The relevant NICE committee issued a view that although there wasn't any new evidence in the last few years to warrant changing the guidelines(!) it would be good to wait until PACE results were released.

Regardless of the damp squib results I suspect the Wessleyites will now try to further entrench CBT/GET in the NICE guidelines (compulsion? on threat of benefit withdrawal? expansion of CBT/GET funding?) and try to railroad this in before the scientific community has had time to reflect on and repudiate the PACE results.

Remember the NICE guideline people are a cabal, and as ever Wessley has control of the publicity machine. MEA's rebuttal, Prof Hooper's letters and complaints do not get traction with the press or Govt .

I think we are in dangerous times.

OTH

This community has been in 'dangerous times' for a very long time. People do need to be thinking about ways of mitigating any damage done by the inaccurate spin placed upon these results certainly. But the research itself is quite easy to critique. It has massive flaws and a dodgy history.

 

[oceanblue]

I didn't even notice that. You're right. And it's completely bizarre.

And this is how the protocol said they would measure recovery:

4. "Recovery" will be defined by meeting all four of the
following criteria: (i) a Chalder Fatigue Questionnaire
score of 3 or less [27], (ii) SF-36 physical Function score
of 85 or above [47,48], (iii) a CGI score of 1 [45], and (iv)
the participant no longer meets Oxford criteria for CFS
[2], CDC criteria for CFS [1] or the London criteria for ME
[40]. (see top of page 12 of protocol doc)

odd that.

edit: sorry, Dolphin posted that recovery protocol stuff well before me.

 

[Esther12]

Thanks for pulling it out again oceanblue. It's funny stuff.

The media response has been pretty depressing.

I've still not read the whole paper. I think I might wait for more analysis from those who have been following this closely, and then dive in myself later.

 

[Dolphin]

Other changes to the protocol

I haven't had time to go through it but they have changed other things.

They changed the primary outcome measures quite a bit. That is quite serious. Drug companies/similar wouldn't get away with that, I think.

They have also changed the definition of an adverse outcome:
Protocol:

An adverse outcome will be considered to have occurred if the physical function score of the SF-36 [28] has dropped by 20 points from the previous measurement.

So has to happen once.

To:

Serious deterioration in health was
defi ned as any of the following outcomes: a short form-36
physical function score decrease of 20 or more between
baseline and any two consecutive assessment interviews

so has to happen twice.

If we could collate such changes it'd be good.

 

[Dolphin]

Cort, have you fully read the article, they state:



They then go on to claim that 16% of APT, 30% of CBT, 28% of GET, 15% SMC patients returned to the normal range.

Of course claiming 60% on the SF-36 physical subscale as normal is a cruel joke.

You are right, it is a joke.
Knoop et al. used this strange way to define recovery and full recovery in their paper (which Peter White was a co-signatory on).

Also, they have been selective in the data they picked. Most data would not say that 1 SD from the norm would bring you down to 60 on the SF-36. It certainly wouldn't for a healthy population.

As oceanblue points out, there is now overlap with their entry criteria. Also, they used different SF-36 thresholds in their protocol but changed them.
One isn't supposed to release the protocol of how one is going to analyse the data and then change it.

 

[Dolphin]

Clock hasn't started on 2 week deadline

Nothing is straightforward in the ME world!

When the PACE Trial was published, it was an online first article - it hasnt appeared in the print edition.

They presumably have to allow their print readers time to write in.
So I believe that the clock hasnt started ticking on replying within two weeks.

 

[Bob]

Many thanks to everyone for analysing this study...
I haven't got the brain capacity to look at it today.

They then go on to claim that 16% of APT, 30% of CBT, 28% of GET, 15% SMC patients returned to the normal range.

Of course claiming 60% on the SF-36 physical subscale as normal is a cruel joke.

Bonkers: with a score of 65 you are deemed ill enough to enter the trial - it's a recruitment criterion.

Apologies, my brain's not working today... Does this mean that those who are considered to have reached 'normal functioning' in this study, are actually far from normal functioning?


Sean... thanks for your inspired post... i totally agree with you.

Cort... Thanks for your response to the NYT... Great stuff.


I just want to make an short observation...
In the UK, our NHS ME 'specialist services' offer only group sessions of CBT/GET as a 'treatment' for ME.
I embarked on my local NHS course a year or two ago... it was a 10 week group course but I left after a couple of weeks because I found the course inappropriate, unhelpful and irritating.

If the very best spin that the authors can put on this study is that only 4 out of 10 of the patients on these NHS courses will be helped, moderately at best, and that it's a pointless or harmful exercise for 60% of the patients, then how will they be able to continue to justify putting these patients through the course?

I agree with what Cort and Sean have said... That psychological interventions have now had their day.

And once we've picked apart the data from this study, then i think and hope that CBT and GET will be further trashed.



Action for ME, the UK's largest (I think?) patient charity has done member surveys over the years which persistently showed that their members were being harmed by CBT and GET... Action for ME didn't know what to make of these results as they thought the results did not correspond to the 'scientific evidence'. Well, now they have their evidence, as provided by White and Wessely, and Action for ME can start taking their members seriously at long last!

Here are the AfME survey results, and on the surface, they seem to correspond closely to the PACE trial results, although it seems apparent that the real PACE trial results are somewhat less successful than the gloss that's been painted on them:

Action for ME 2008 member survey
http://www.afme.org.uk/res/img/resources/Survey Summary Report 2008.pdf
GET:
helpful 45% no change 21% made worse 34%
(helpful = 45% not helpful/harmful = 55%)


For me personally, any results from any study that only measures fatigue (which is just one symptom of ME) would be pretty irrelevant because, for me, cognitive impairment has always been the thing that's had the worst affect on my life since I became ill, so any study that only investigates fatigue is unhelpful to me. If I had no fatigue, but the same level of cognitive impairment, then I think that the subjective quality of my life wouldn't be much better than it is now.


One other point to make... The NYT article says:

The authors note that the goal of cognitive behavioral therapy, the type of psychotherapy tested in the study, is to change the psychological factors “assumed to be responsible for perpetuation of the participant’s symptoms and disability.”
http://www.nytimes.com/2011/02/18/health/research/18fatigue.html?_r=1&ref=science

Well, as we all pointed our early on... The UK government shouldn't have wasted millions of pounds/dollars on a study that was based on such a faulty premise... Since when did 'psychological factors' cause, or perpetuate, neurological diseases? (And even the UK government accepts that ME/CFS is a neurological disease!)

(i know you know all this already, but i needed to say it anyway!)

 

[Doogle]

40% were taking antidepressants at baseline. Is this normal?

Yes, that number leapt out at me too. The only other study to quote usage I could easily find (36.3%) was Use of medications by people with chronic fatigue syndrome and healthy persons: a population-based study of fatiguing illness in Georgia that used the Reeves criteria. We know from Jason that the Reeves criteria selects large numbers of people with simple depression.

 

[Mark]

Although this paper and the furious spin that will accompany it will be a serious PITA to deal with, the other side of that coin is that this is their best shot, but all they could manage was a "moderate" improvement in a limited number of subjective measures. (And let's wait and see the actual numbers say. I think it is reasonable to assume that the available abstract gives the most favourable possible spin on it they can get away with. If they are only able to claim a moderate improvement, and if their spin (and methodology) is their usual standard, then I will bet a closer look at the numbers shows the actual outcomes to be less than even merely moderate.)

They almost totally dominate the politics and public relations in the field, get the lion's share of the funding, had almost total control over the study, and had first crack at spinning the results, at framing the debate. It is the most favourable possible set of conditions for them and their model, and so presumably this is the best they can do with it. But they have not delivered any substantial real-world outcome. And that is before we factor in the serious methodological problems with this study and its underlying model. Frankly, it is pathetic, especially given how extravagant and assertive the claims about the effectiveness of this approach has been over the years.

Definitely no killer blow in there for them.

I would suggest that this is their high point. They will not be able to do any better than this. It is all down hill from here.

I can't thank you enough for this post Sean, your positive spin gives me a crumb of comfort, something to cling on to in what may be our darkest hour, if it is indeed the high-water mark for the psychs.

I will wait and see how the new 1.5m UK pot is allocated next, later this year, to see whether there really will now be any attempt at genuine research into ME or CFS. But frankly I'm still not optimistic that they can't think of any further psycho lines of inquiry to pursue. We shall see - the allocation of that MRC pot will be the watershed for me; if that goes the wrong way I might start to take more of an 'any means necessary' line about things...

But for the here and now, the impact of this, culturally, in the UK especially, will be devastating. Here are the press headlines I see on Google News right now - these headlines are all that will matter for the vast majority, of course, who won't even see beyond that to the article, let alone to the comments, the study itself, or god help us, drilling down to get to the truth: this is the only message that the vast majority of our friends and family and society will get, in order (according to GoogleNews) of the most popular stories on the web:

http://news.google.co.uk/?ar=1298042567

"Talking and Exercise Could Cure ME" - Sky News
(Cure! Even though the study itself finds the opposite)

"Study finds therapy and exercise best for ME" - The 'Guardian'
(the most inexcusable rag of all: the ultimate betrayal: the "Guardian" (of whom, nowadays?) lies through its teeth again: "best" here presumably meaning the top two 'treatments' out of a comparison of therapy and exercise!)

"CBT and exercise the prescription for CFS" - Health Insurance and Protection Magazine
(THE prescription. Say the insurers...)

"Counselling and exercise appear best for ME" - Netdoctor
(best? out of which options?...)

"Brain and body training treats ME, UK study says" - BBC News
(Treats?)

"Exercise and therapy can help ME sufferers, study claims" - Daily Telegraph
(OMG! The Telegraph is the first headline to do as much as to just say "help" and add the word "claims" to put any kind of realistic spin on the story. The first decent headline - the only decent headline - and the first headline that isn't an outright lie - and it comes from the Telegraph. What goes on? When did they turn the world upside down?)

"CBT and exercise challenge 'no cure' for ME" - Nursing Times
(WOW! A decent and honest headline with a positive spin! Thank you Nursing Times!)

"Trial offers hope for ME sufferers" - The Press Association
(Way to kick the disabled in the teeth, Press Association - thanks for that)

"Got ME? Just get out and exercise, say scientists" - The Independent
(Really? Is that really what the 'scientists' (sic, and sick) said?)

"Pushing limits can help chronic fatigue patients" - Reuters
(A prize here for the most dangerous headline of all?)

"ME/CFS: Pacing yourself isn't the answer" - WebMD.Boots.com
(Picking up on the ancillary bonus of attacking the pacing theory by using a fake pacing strategy)

And on and on it goes...lies, lies, and more lies, in a massive and co-ordinated global strike against the chronically ill...all of it more sinister and damaging and dangerous and outright untrue than even the study itself.

Scientifically, maybe yes, maybe this really is their highpoint. But in the real world, in the here and now, in the UK especially, they have set us up for years of sadistic abuse of the most neglected disabled people in society.

Desperate times call for desperate measures. How on earth can we respond to this situation? I'm sorry, but I'm losing it right now. This obscene bullshit is just killing me.

 

[Enid]

What exactly were they studying anyway their "CFS also known as ME" carefully excluding neurological symptoms (WHO class) makes it meaningless. This will be taken up by many here (not all publicly known yet and over many years). Press distortions just typical.

 

[Esther12]

Thanks for the collection of headlines Mark. It might be useful to have them in one place.

One point re the apparent opposition to 'pacing'... when patients have been through CBT/GET, what are they then meant to do, other than PACE? For the 40% who gained no benefit from GET/CBT, wouldn't they have been better off 'pacing'?

(It's a bit difficult to talk about this because it seems the the PACE trial version of 'pacing' is itself something of a chore, combined with being told you'll never get better... but for most people 'pacing' is a simple and common sense way of trying to make the most of limited energy... is this not what they would recommend for patients who can gain no (further) benefit from CBT/GET?)

(I thought I had a simple and worthwhile point here, but it's come out slightly garbled. Curses!!)

 

[Dolphin]

Criticism of recovery-type definition in PACE Trial (SF-36 part)

The authors changed some of the thresholds they said they were going to use (see protocol paper: http://www.biomedcentral.com/1471-2377/7/6)

In the end, the highest level they presented data on was scoring 60 or more in the SF-36 physical functioning scale.
They said this was within 1 S.D. of the norm for the working population.
It isn't for most of the populations I have seen and does not represent being healthy.

Basically for the following 10 questions, you score 10 points if one says:
"No, Not Limited At All" to a question and 5 points if one says: "Yes, Limited A Little".

So one could (say) put "Yes, Limited A Lot" to 2 of the questions and "Yes, Limited A Little" to 4 more and still satisfy their recovery-type definition.

Also, a lot of the people in the PACE Trial wouldn't be working - some of the people who scored 60+, might not reach the threshold if they were working at the same time.

Tom

4. The following items are about activities you might do during a typical day. Does your health now limit you in these activities? If so, how much?

ACTIVITIES
Yes, Limited A Lot

Yes, Limited A Little

No, Not Limited At All

a. Vigorous activities, such as running, lifting heavy objects, participating in strenuous sports b. Moderate activities, such as moving a table, pushing a vacuum cleaner, bowling, or playing golf c. Lifting or carrying groceries d. Climbing several flights of stairs
e. Climbing one flight of stairs
f. Bending, kneeling, or stooping
g. Walking more than a mile
h. Walking several blocks
i. Walking one block
j. Bathing or dressing yourself

 

[oceanblue]

The PACE trial team have been very naughty boys and girls

They changed the primary outcome measures quite a bit. That is quite serious. Drug companies/similar wouldn't get away with that, I think.

Yes, extraordinary that they should make such a big change without even justifying it, or listing it under 'trial limitations' in the Discussion section. Apparently the 'per protocol' analyses are in the Web appendix but this isn't open access and I'm trying to get a copy of it.

So, to recap on how they moved the goal posts:

Primary outcomes were fatigue and physical function as measured by the Chalder Fatigue Scale and Sf-36 PF sub scale respectively - these haven't changed. What has changed is how 'improvement' is defined, and there are no prizes for guessing in which direction the change works.

Fatigue

Protocol says:

The 11 item Chalder Fatigue Questionnaire measures the
severity of symptomatic fatigue [27], and has been the
most frequently used measure of fatigue in most previous
trials of these interventions. We will use the 0,0,1,1 item
scores to allow a possible score of between 0 and 11. A
positive outcome will be a 50% reduction in fatigue score,
or a score of 3 or less, this threshold having been previously
shown to indicate normal fatigue [27].

So, with a mean baseline fatigue score of approx 28 a 50% reduction would require a typical improvement of 14 points to count as a positive outcome.

The paper switches to likert (0,1,2,3) scoring - I have no problem with this - but now a reduction of 2 points is required for all participants to count as improvement.

To clarify: the protocol says a 14 point improvement in fatigue score is required, the pubished paper now says 2 will do.

The justification for these changes is

A clinically useful diff erence between the means of
the primary outcomes was defined as 05 of the SD of
these measures at baseline,31 equating to 2 points for
Chalder fatigue questionnaire and 8 points for short form-36.

Seems to be based on ref 31 but I think they need to spell out the reason for this major change.

Physical Function

The protocol says

The SF-36 physical function sub-scale [29] measures physical
function, and has often been used as a primary outcome
measure in trials of CBT and GET. We will count a
score of 75 (out of a maximum of 100) or more, or a 50%
increase from baseline in SF-36 sub-scale score as a positive
outcome. A score of 70 is about one standard deviation
below the mean score (about 85, depending on the
study) for the UK adult population [51,52].

The average baseline SF-36 PF score is around 38 so typically an improvement of 19 points is required by the protocol.

The paper says an improvement of 8 points for each participant will do.

The scale only scores in 5 point intervals so that would be a 20 point improvement required, on average, according to the protocol, versus 10 points required in the paper.

Finally, the protocol was simply going to look at the proportion of improvers in each therapy group and use that as the measure of success (they expected 60% of the CBT group to improve this much!). The paper just looks at the average score in each group.

Of course, there may be a sound rationale for these changes (other than "shit, our trial is going to bomb, let's change how we define success") but if there is, they need to spell it out and explain exactly why they changed from thier carefully considered protocol. As I understand it, the CONSORT guidelines exist precisiely to stop this sort of Gerrymandering.