"News" 8 Sep 2016: PACE trial team analyse main outcome measures according to the original protocol

[Esther12]

Are we expected to believe that no analysis of the data was conducted according to the original protocols before they were changed, and that such analyses were not subsequently available to management when authorising expenditure on the legal costs, if they requested it?

Why change the protocols if you do not know the outcomes which they produce?

Everything they did made it easier for them to claim success. They didn't need to have these figures to know that.

I wonder if these results were released in the way that they were in order to try to irritate patients, and get a response that they could use against the PACE data release. Having improvement rates drop by a third, and then insisting that this supports what you said all along is pretty infuriating. Sharpe promoting them on twitter using #PACEtrial was sure to attract attention, and the limitations of twitter makes it difficult for patients to present their concerns in a nuanced manner.

 

[BruceInOz]

Ummm p= 0.20 - when is that ever used for statistical significance. Usual is p=0.05

I think White et al are just admitting that it is not statistically significant because the p value is too large.

 

[Sam Carter]

A sight contradiction there; I doubt a therapy can be moderately effective if it's not a statistically significant outcome. CBT failed to significantly improve physical function (number of improvers) in these new analyses.

To continue @Bob 's riff, we can see from the Lancet / Lancet Psychiatry reports in 2011 and 2015 that "6-min walking distances ... were no different after CBT compared with APT and SMC" and also "[t]here was little evidence of differences in outcomes between the randomised treatment groups at long-term follow-up."

This makes CBT a tough sell, because apart from a small and transient improvement in subjectively reported fatigue, the published data indicate that six months of CBT is no better than usual medical care alone.

 

[Dolphin]

The 11 item Chalder Fatigue Questionnaire measures the severity of symptomatic fatigue [27], and has been the most frequently used measure of fatigue in most previous trials of these interventions. We will use the 0,0,1,1 item scores to allow a possible score of between 0 and 11. A positive outcome will be a 50% reduction in fatigue score, or a score of 3 or less, this threshold having been previously shown to indicate normal fatigue [27].

As entry for the trial requires a bimodal score of 6 or more, if you score 3 or less, you will automatically have a 50% reduction. So the criterion about a score of 3 or less never needs to be invoked i.e. the same percentage who have a 50% reduction in fatigue score will satisfy these criteria.

 

[Dolphin]

The SF-36 physical function sub-scale [29] measures physical function, and has often been used as a primary outcome measure in trials of CBT and GET. We will count a score of 75 (out of a maximum of 100) or more, or a 50% increase from baseline in SF-36 sub-scale score as a positive outcome. A score of 70 is about one standard deviation below the mean score (about 85, depending on the study) for the UK adult population [51, 52].

More people satisfied these criteria (in the new analyses)

Physical functioning
APT: 64/159 (40%)
CBT: 79/161 (49%)
GET: 97/160 (61%)
SMC: 70/160 (44%)

Than were in the normal range (from the recovery paper) (SF-36 >=60)

Physical function
Within SF-36-PF normal range
APT: 35% (53/153)
CBT: 52% (77/148)
GET: 53% (81/154)
SMC: 41% (62/152)

That is to say, the people who satisfied the (above) criteria included some people who scored less than 60. For example if your baseline score is 30, getting to 45 or more would satisfy these criteria.

 

[Simon]

Tidier version of the original graph. "Per protocol" are the results released by the authors yesterday, over five years after the original paper. At the very least, when they moved the goal posts in the Lancet paper, they should have presented the results according to the original protocol too.

And of course, in their new analysis they failed to give the comparison with the Lancet results.

 

[Denise]

Many thanks to all who are assessing what these new documents from White et al mean. Shoutouts to @Dolphin, @Simon, @TomKindlon among others.

 

[trishrhymes]

That is to say, the people who satisfied the (above) criteria included some people who scored less than 60. For example if your baseline score is 30, getting to 45 or more would satisfy these criteria.

I'd just spotted the same thing, Dolphin. Only I'd forgotten the 'improved' criteria included the 50% increase as an alternative to the 75 boundary.

I was going nuts trying to fathom how more people could score more than 75 than scored more than 60. Thanks for clarifying.

And you're right, it means 'improved' could include people who went from, say 20 to 30, ie were still very ill, as well as the anomaly we all know about of going from 65 entry criteria to 60+ at the end. What a load of nonsense! How dare they boast about such pathetic figures.

 

[worldbackwards]

Everything they did made it easier for them to claim success. They didn't need to have these figures to know that.

I wonder if these results were released in the way that they were in order to try to irritate patients, and get a response that they could use against the PACE data release. Having improvement rates drop by a third, and then insisting that this supports what you said all along is pretty infuriating. Sharpe promoting them on twitter using #PACEtrial was sure to attract attention, and the limitations of twitter makes it difficult for patients to present their concerns in a nuanced manner.

It's tempting but I don't buy it. Sharpe knows how to troll us properly, he did it with the LTFU paper: Hold a big press conference, lie about the results to lots of high profile journalists, declare CFS curable, pretend he's not taking an ideological line whilst inferring that he is with every word.

The ICO and the Tribunal remained unimpressed with their innocent face over the fury they caused and, interestingly enough, we behaved pretty well all things considered - most of the whining was restricted to James Coyne's increasingly unhelpful contributions. It didn't help them and this is far short of that. They can no longer appeal on those terms anyway and it would be sure to anger the Tribunal if they tried that on again.

All of which makes me think that, short of Chalder suddenly calling a press conference, waving the results above her head Neville Chamberlain stylee, and telling us about "a cure for Chronic Fatigue in our time", this is all just damage limitation, part of a subtle game for sure but one where they are well and truly on the back foot.

Indeed, they may have used the hashtag (knowing we'd find it anyway, as if we wouldn't), but they haven't gone out of their way to tell anyone else. They are now being forced to do things they never dreamed of or wanted to do. It means something, but if it's trolling then they're very much off of their game.

 

[Bob]

CBT failed to significantly improve physical function (number of improvers) in these new analyses.

To continue @Bob 's riff, we can see from the Lancet / Lancet Psychiatry reports in 2011 and 2015 that "6-min walking distances ... were no different after CBT compared with APT and SMC" and also "[t]here was little evidence of differences in outcomes between the randomised treatment groups at long-term follow-up."

This makes CBT a tough sell, because apart from a small and transient improvement in subjectively reported fatigue, the published data indicate that six months of CBT is no better than usual medical care alone.

Great point, Sam! So, with the new data, CBT failed to improve individual-level self-report physical function, the 6MWDT test, fitness, employment, all long-term outcomes, and all other objective measures. It's not looking great for CBT! (Note, for group-level analysis there was still a modest average improvement for physical function with the new data.)

 

[Simon]

A sight contradiction there; I doubt a therapy can be moderately effective if it's not a statistically significant outcome. CBT failed to significantly improve physical function (number of improvers) in these new analyses.

Though to be fair, the physical function gain in CBT wasn't a clinically useful difference in the Lancet paper either. Again, hard to claim moderately successful (but worse in new analysis)

 

[worldbackwards]

Again, hard to claim moderately successful

But it's never stopped them before.

 

[Large Donner]

In this case, I'd guess that their narrative will be something like: "we had good reasons to change the recovery criteria (subtext: these were complex statistical reasons that these irrational, crazy and vexatious militant patients couldn't possibly understand) but we've acquiesced to their unreasonable demands and released our results assessed in line with the original criteria. The figures may not be as impressive as those results reported initially but they're still very positive. So shut up."

I am willing to put a hundred quid on the Lancet publishing that statement Verbatim.

 

[worldbackwards]

If the PACE authors are now grieving for their data, does this represent a move from denial onto bargaining?

 

[A.B.]

QMUL is not appealing.

www.qmul.ac.uk/media/news/items/smd/181216.html

 

[aimossy]

Boom!

 

[worldbackwards]

Let the fun commence!

 

[Chrisb]

QMUL is not appealing

It never has been.

 

[AR68]

QMUL is not appealing.

www.qmul.ac.uk/media/news/items/smd/181216.html

I didn't think they would appeal but at the risk of being a party pooper, they've not actually said that they won't be appealing (although reading between lines it's highly unlikely that they will take this any further) but the phrase "certain PACE trial data" sticks in my mind.

Given that they've played hardball over this (and Sharpe's tweet last night), watch them like a hawk. I know people will, and rightly so.

 

[aimossy]

This news has it's own thread here:
http://forums.phoenixrising.me/index.php?threads/the-pace-data-have-been-released-9-sept-2016.46707/