New attempt to avoid releasing data on 'recovery' from PACE

Esther12

Senior Member
Messages
13,774
Likes
28,359
I expect that most of you will be aware of how the minor changes in questionnaire scores found under PACE have been used to justify all sorts of claims about a third of patients being cured, or CBT and GET leading to a recovery rate of 30-40%. Much of this was based upon a redefinition of 'normal' levels of fatigue and disability which allowed very ill patients to be classed as 'back to normal' (even if they were sicker than when they'd started the trial) - this was also all measured by questionnaire scores in an unblinded RCT which gave patients receiving CBT and GET far more time with therapists than those in the control group (appx 16 hrs vs 3hrs?) . The PACE researchers have always been quiet about how this was spun to mean recovered, and let others cite their work for these nonsense claims, while refusing to release the figures for the number of patients which reached the criteria for 'recovery' which was laid out in the trials published protocol.

[Edited to insert link]: http://www.whatdotheyknow.com/request/pace_trial_recovery_rates_and_po

It now seems that they have come up with with a new criteria for 'remission'. We do not know what is required for a patient to be classed as being in remission (it certainly wasn't laid out in their protocol!), but it looks like it will again be small changes in questionnaire scores. In a recent presentation, Peter White has provided figures on rates of 'remission', and it looks like more patients ended up in 'remission' than received a clinically significant benefit from treatment. It looks like this is something else which could be spun for claims of 'recovery' and 'cure'.

Page 13 of this:

[Edit: It seems that the presentation has now been deleted. A graph from it was copied below though, and I saved a copy if anyone wants me to put it on-line somewhere edit2 @maxwhd put it online here: http://twitdoc.com/view.asp?id=1481...d&doc=235586809&key=key-zpr4xbwX8WrWvSnyyEtC]:

http://ifdm2012.rsm.ac.uk/downloads/presentations/peter-white.pdf

If a pharmaceutical company were behaving in this way, it would be recognised as unacceptable. Because it's psychiatric researchers and CFS, everyone knows that the only reason patients would want outcomes released in the manner laid out in the published protocol is that they're anti-psychiatry!

The PACE protocol is here: http://www.biomedcentral.com/1471-2377/7/6

4. "Recovery" will be defined by meeting all four of the following criteria: (i) a Chalder Fatigue Questionnaire score of 3 or less [27], (ii) SF-36 physical Function score of 85 or above [47,48], (iii) a CGI score of 1 [45], and (iv) the participant no longer meets Oxford criteria for CFS [2], CDC criteria for CFS [1] or the London criteria for ME [40].
It should be recognised that this still is not a terribly tight criteria for 'recovery', and leaves room for those classed as 'recovered' to have significant ongoing problems which were not present prior to falling ill, but an SF-36 PF score of 85 is much closer to the average (for healthy working age adults) of 95 than the score of 60 which has been claimed to indicate recovery previously). The old criteria again relies upon questionnaire scores, when we have seen how psychosocial interventions which encourage positive thinking or an increased sense of control can improve questionnaire scores without improving objective measures of activity levels or neuropsychiatric performance, but as actometers were dropped as outcome measures after it had been shown in another RCT that CBT was ineffective in allowing patients to increase activity levels, the only external measures of capacity we have are the 6 minute walking test and employment levels (employment levels showed no significant difference between groups, 6mwt showed slight improvement for graded exercise group, no difference for the others - CBT+SMC actually doing slightly worse than SMC alone). The original criteria is still far better than the re-definitions that they have come up with recently, and importantly, it is the one that they laid out in their own protocol - it should be the one that they are held to.
 
Last edited:

Graham

Senior Moment
Messages
5,188
Likes
13,225
Location
Sussex, UK
Oh just look at the concluding slide! Translated in my terms it says "Now that we have proved that GET and CBT are pretty useless, we are now going to start spreading the idea that people with ME would be cured if only they, their doctors, and their employers had a better attitude". But there is an interesting statistic on page 22 that 14% of CFS/ME claimants returned to work. I'll bet that is riddled with fiddles.
 

Bob

Senior Member
Messages
16,455
Likes
34,055
Location
England (south coast)
But there is an interesting statistic on page 22 that 14% of CFS/ME claimants returned to work. I'll bet that is riddled with fiddles.
Yes, but I'm not sure if that figure is related to treatment with CBT/GET. It might be the general figure for all CFS patients, with or without CBT/GET. It isn't clear what it means.

The PACE Trial's cost benefit analysis paper clearly stated that there was no significant increase in employment after treatment with CBT/GET.
 

Bob

Senior Member
Messages
16,455
Likes
34,055
Location
England (south coast)
The number of patients who 'responded' to treatment in the PACE Trial after treatment with CBT/GET (using their own figures, and deeply questionable methodology), was only 13%.

White's presentation suggests that he is claiming that about 15% of PACE patients experienced 'remission', after treatment with CBT/GET, when compared to the SMC control group.


But least White seems to be admitting that CBT/GET do not get CFS patients back into work:

"CBT & GET have ... only modest effects in helping RTW..." [RTW = return to work.]

By 'modest', he means "not statistically significant", as clearly stated in the PACE Trial's cost benefit analysis paper.

Considering that PACE was part funded by the DWP, I think this admission will be helpful for us. It means that the DWP have no reason for force us to undertake CBT/GET.
 

biophile

Places I'd rather be.
Messages
8,977
Likes
16,581
Here is the relevant slide on "remission" (p13/28 http://ifdm2012.rsm.ac.uk/downloads/presentations/peter-white.pdf):

remission.gif


I have a feeling that this will be a major case of statistically "crossing the border". The deceptive "normal range" in the 2011 Lancet paper gives me cause for concern about remission too.

According to the response to a recent FOI request, PACE will not be publishing data using the original definition of "recovery", but have changed this as well:
With regards to the recovery rates: the criteria thresholds for measuring recovery in the Trial were changed in the light of more detailed consideration of previous published studies (making the Trial’s analyses either consistent with these studies or more stringent) and in the light of newly published work (on the normal range of fatigue in the U.K. population). These changes were made before analysing any data. A paper that includes all analyses on recovery is currently under review by a peer-reviewed journal. This was submitted for publication two months ago and we expect to know whether this has been accepted for publication by the end of this year. If accepted, we would expect this to be published in the first three months of 2013.

http://www.whatdotheyknow.com/request/pace_trial_recovery_rates_and_po
It is possible that new definitions for remission and recovery will both be presented.
 

Esther12

Senior Member
Messages
13,774
Likes
28,359
To me it seems a bit ambiguous what White is saying about getting people back to work. He doesn't seem to make it clear that CBT/GET did not lead to increases in the ammount of work people could do. It's also a bit rich him claiming "Rehabilitation based treatments are moderately helpful, but are not aimed to help RTW". It's been stated a number of times that they are aiming for recovery: that would seem likely to involve return to work for some patients... or is this the strange new conception of recovery again.

I've seen a couple of examples of White spinning this study in presentations to others too, failing to mention that there was only a significant difference between the groups at six months, not twelve, and that at six months the intervention group had a much higher rate of return on questionnaires:

http://www.kcl.ac.uk/content/1/c6/01/47/68/EBVRCT.pdf

A randomised controlled trial of a psycho-educational
intervention to aid recovery in infectious mononucleosis

Bridget Candy, Trudie Chalder, Anthony J Cleare,
Simon Wessely, Matthew Hotopf*
Here's video of one of White's presentations : http://www.scivee.tv/node/6895

Also this presentation made before a parliamentary committee features a slide which claims:

Post-IM fatigue

70 % of GPs only advice is to rest
Inactivity most replicated predictor of
prolonged fatigue
Educational intervention, based on graded
return to activity, halved the incidence of
prolonged fatigue
Link

When they're claiming that their cognitive-behavioral interventions halve incidence, that inspires faith that they know what they doing. If he was honest about the fact that there was no statistical difference in incidence at 12 months, I do not think that those listening would have been so interested in his other claims, or in having him do more research.

I wouldn't trust any claim made about CFS at a presentation, when I could not get my hands on the raw data. Especially not from White, Chalder, etc.
 

biophile

Places I'd rather be.
Messages
8,977
Likes
16,581
But least White is admitting that CBT/GET do not get CFS patients back into work:

"CBT & GET have ... only modest effects in helping RTW..." [RTW = return to work.]

By 'modest', he means "not statistically significant", as clearly stated in the PACE Trial's cost benefit analysis paper.
That statement was for FSS in general, but still, notice how he tries to imply that CBT/GET returns CFS patients back to work, even though the evidence presented does not even support that.
 

Bob

Senior Member
Messages
16,455
Likes
34,055
Location
England (south coast)
To me it seems a bit ambiguous what White is saying about getting people back to work. He doesn't seem to make it clear that CBT/GET did not lead to increases in the ammount of work people could do.
Yes, sorry, that was my own interpretation of his slides (based on my knowledge of the facts).
I agree with you Esther, his text is ambiguous.

I was interpreting this sentence:

"CBT & GET have moderate efficacy but only modest effects in helping RTW after FSS"

Here, he actually means that CBT/GET are moderately effective as treatments (his interpretation), and not statistically helpful at getting people back to work (if he is referring to the PACE data).
 

Mark

Senior Member
Messages
5,238
Likes
6,200
Location
Sofa, UK
Unless they give access to the raw data, they can construct whatever graphs, barcharts and slides they like; their claim to scientific validity is no more than an appeal to authority and an expectation that people should simply trust them and their analysis. In my view it's all scientifically meaningless without open access to the data.
 

Esther12

Senior Member
Messages
13,774
Likes
28,359
Unless they give access to the raw data, they can construct whatever graphs, barcharts and slides they like; their claim to scientific validity is no more than an appeal to authority and an expectation that people should simply trust them and their analysis. In my view it's all scientifically meaningless without open access to the data.
They are part of the scientific process - they're publishing in peer reviewed journals. To many people, that's all that matters.

We need to really hope that they come across some reviewers who push them to release more data, or present results in the manner laid out in their protocol. Peer review is meant to be helpful - although in the case of PACE it seemed that the reviewers had never even read the protocol, and certainly didn't check the data cited to support the claim that an SF36 PF score of just 60 was 'normal' for those of working age (or seem concerned that at the start of the trial a score of 65 and under had been taken to indicate severe and disabling fatigue as part of the inclusion criteria).

Even if we get lucky, and they end up with a reviewer who thinks patients deserve an accurate understanding of the efficacy of treatments being offered, they can still just ignore their requests though. One of the follow up papers to FINE had some really critical and sensible comments from a reviewer that were pretty much ignored.
 

alex3619

Senior Member
Messages
13,810
Likes
37,687
Location
Logan, Queensland, Australia
Given misdiagnosis in CFS in the UK is as high as 70%, and more typically around 50%, a 14% RTW is a joke. Every single case who returned to work might have been misdiagnosed in the first place. Its worse than that though, because Oxford diagnosed CFS is the most heterogenous definition in clinical practice. With all the diagnostic bias one should require much stronger RTW results before drawing more than tentative conclusions.

I still question the existence of FSS. Its a hypothetical disease group. Sure these people are sick - but with what? FSS is, in my current view, always a failure to diagnose.
 

Esther12

Senior Member
Messages
13,774
Likes
28,359
According to the response to a recent FOI request, PACE will not be publishing data using the original definition of "recovery", but have changed this as well:

With regards to the recovery rates: the criteria thresholds for measuring recovery in the Trial were changed in the light of more detailed consideration of previous published studies (making the Trial’s analyses either consistent with these studies or more stringent) and in the light of newly published work (on the normal range of fatigue in the U.K. population). These changes were made before analysing any data. A paper that includes all analyses on recovery is currently under review by a peer-reviewed journal. This was submitted for publication two months ago and we expect to know whether this has been accepted for publication by the end of this year. If accepted, we would expect this to be published in the first three months of 2013.

http://www.whatdotheyknow.com/request/pace_trial_recovery_rates_and_po
It is possible that new definitions for remission and recovery will both be presented.
It's also interesting that they're now turning down FOI request for this data because they do not have it and will never publish it, whereas they turned down a previous request for the data on 'recovery' as defined in the protocol because "this information is exempt under Section 22 of the FOIA 2000 on the basis that the authors intend to publish these in the future."

http://www.meassociation.org.uk/wp-content/uploads/2011/06/FOI+from+Queen+Mary.pdf
 

Dolphin

Senior Member
Messages
17,559
Likes
28,261
But there is an interesting statistic on page 22 that 14% of CFS/ME claimants returned to work. I'll bet that is riddled with fiddles.
One thing one wants to know is how many people with CFS/ME at that time who were in work, dropped out. And how many were working full-time long-term. With the Belgian clinics, some people who weren't working started work - but this was balanced by a greater number who were working stopping working (and there were more people not working than working at the start so that group had "an advantage")
 

Dolphin

Senior Member
Messages
17,559
Likes
28,261
Oceanblue did some guestimates on recovery as originally defined, in these two posts:

http://forums.phoenixrising.me/inde...d-pace-trial-protocol.3928/page-35#post-88041
and
http://forums.phoenixrising.me/inde...d-pace-trial-protocol.3928/page-35#post-88043

Later posts came up with guestimates of: CBT and GET: 9%, SMC 4%. These figures are a lot lower than the "remission" figures given which presumably is a much broader definition (perhaps simply not meeting the criteria to enter the trial?)
 

Dolphin

Senior Member
Messages
17,559
Likes
28,261
Oceanblue did some guestimates on recovery as originally defined, in these two posts:

http://forums.phoenixrising.me/inde...d-pace-trial-protocol.3928/page-35#post-88041
and
http://forums.phoenixrising.me/inde...d-pace-trial-protocol.3928/page-35#post-88043

Later posts came up with guestimates of: CBT and GET: 9%, SMC 4%. These figures are a lot lower than the "remission" figures given which presumably is a much broader definition (perhaps simply not meeting the criteria to enter the trial?)
If one just used SF-36 PF >=85 alone, assuming normality it would be CBT: 13.4% and GET: 15.2% i.e. a lot less than the 22% or so remission figure in the PDW slide.
 

Simon

Senior Member
Messages
3,788
Likes
14,822
Location
Monmouth, UK
Though of course we don't know to what extent the scores are normally distributed, which was the assumption behind those calculations. Still, I'm willing to bet 'remission' will be defined by SF36 <85 - any takers?
 

Bob

Senior Member
Messages
16,455
Likes
34,055
Location
England (south coast)
Though of course we don't know to what extent the scores are normally distributed, which was the assumption behind those calculations. Still, I'm willing to bet 'remission' will be defined by SF36 <85 - any takers?
Purely a guess, but I'd be surprised if they used 85. I would guess 75.
 

Dolphin

Senior Member
Messages
17,559
Likes
28,261
Purely a guess, but I'd be surprised if they used 85. I would guess 75.
It really depends if they use CFQ also. If they do, I think the SF-36 PF threshold would be easier again. I'm wondering whether not satisfying both of the entry criteria is what they are counting?
 

Simon

Senior Member
Messages
3,788
Likes
14,822
Location
Monmouth, UK
I'm wondering whether not satisfying both of the entry criteria is what they are counting?
I like to think you say that in jest, but you may be right. I guess the logic would be that if you don't meet the entry criteria you don't have CFS and so must be in 'remission'.
 

Dolphin

Senior Member
Messages
17,559
Likes
28,261
I like to think you say that in jest, but you may be right. I guess the logic would be that if you don't meet the entry criteria you don't have CFS and so must be in 'remission'.
Presuming it is the case, it is something that I'm going to let mull around in my brain and with luck, perhaps I will see something published that criticises this before the paper comes out. I hope others will also. Whatever the threshold used, I can't see it being much different from the entry criteria, so it's not going to be a good definition.

Also, I remember somebody pointing out that in one of the minutes of the PACE Trial meetings, they wanted a reasonable gap between positive outcome and the entry criteria - if anyone has that, it would be useful to have the reason given.