"News" 8 Sep 2016: PACE trial team analyse main outcome measures according to the original protocol

[user9876]

Has anyone noticed that in the Lancet paper the percent improved uses the the number of survey responses at 52 weeks as the denominator in the percent calculation while in the latest offering they use the number randomised. I'm no statistician so I don't know which is more correct but the Lancet numbers are inflated by a few percent compared to using number randomised.

I had noticed that. In their new analysis they are applying a regression analysis to do something which I assume is to deal with bias in different centers but it could also be used to fill in missing data in which case the numbers they are using would be correct.

I have issues with not putting the equations used and the lack of clarity on what they are doing but I guess this reflects state of the art?

 

[Stewart]

Exactly. This is Damage Control 101. That is, you get your spin out there ahead of the release of any embarrassing information in order to try and control the narrative.

Yep, that's exactly what I thought. It looks like they're hoping to muddy the waters - when the data is released and criticisms of the trial intensify they'll direct the media to this analysis and *insist* that it proves that CBT and GET are effective, even when you use the original protocol. Most journalists won't know who to believe and will end up giving equal weight to both sides in the mistaken belief that this is what 'objective journalism' should look like...

It's still a bit of a risky strategy, given that the wide divergence in the CBT/GET effectiveness figures from the Lancet paper is so striking and plays right into their critics' claim that they 'cooked the books' - but in the circumstances what else can they do?

I wonder if they've hired any damage control consultants? (most likely at taxpayers' expense)

I almost hope they have. Any damage control consultant worth their salt would be quietly telling Gaskell et al that if the worst comes to the worst, QMUL needs to be ready to throw Peter White under the bus.

 

[worldbackwards]

Exactly. This is Damage Control 101. That is, you get your spin out there ahead of the release of any embarrassing information in order to try and control the narrative.

But these results are dire. It's hard to imagine anyone coming away from this with anything other than the impression that these treatments are ineffective for the vast majority of patients. If this is the best they can do, they surely must be in deep trouble.

 

[A.B.]

But these results are dire. It's hard to imagine anyone coming away from this with anything other than the impression that these treatments are ineffective for the vast majority of patients. If this is the best they can do, they surely must be in deep trouble.

I think the average person will probably look at these results and conclude that CBT and GET are of little help but better than nothing.

We still need an independent analysis, with all the problems summarized, and published in a journal, and a retraction of PACE. The misleading information needs to be branded as misleading and removed from textbooks, websites, etc.

 

[Art Vandelay]

Yep, that's exactly what I thought. It looks like they're hoping to muddy the waters - when the data is released and criticisms of the trial intensify they'll direct the media to this analysis and *insist* that it proves that CBT and GET are effective, even when you use the original protocol. Most journalists won't know who to believe and will end up giving equal weight to both sides in the mistaken belief that this is what 'objective journalism' should look like...

Spot on. It reminds me of the tactic Horton used to avoid publishing the letter from Virology in the Lancet recently. He claimed that Tuller's arguments had been already addressed and the letter added nothing new to the debate. The fact that the PACE authors haven't substantially addressed any of the criticisms (they've merely responded with obfuscation and blather) is beside the point. Claiming that they have 'already dealt with the criticism' is a convenient excuse not to get involved in any debate of substance because they know that their position is untenable.

I almost hope they have. Any damage control consultant worth their salt would be quietly telling Gaskell et al that if the worst comes to the worst, QMUL needs to be ready to throw Peter White under the bus.

YES! Fingers crossed

 

[Cheshire]

I think the average person will probably look at these results and conclude that CBT and GET are of little help but better than nothing.

We still need an independent analysis, with all the problems summarized, and published in a journal, and a retraction of PACE. The misleading information needs to be branded as misleading and removed from textbooks, websites, etc.

There's still their claim that they made only minor changes to the protocol.
Minor changes don't X3 your improvement rates.
At best, it's "grossly exaggerated", to quote some tribunal report....
I think even a busy journalist can get that.

 

[sarah darwins]

Journalists may not be very interested in the arguments per se, but they could become interested in the public money spent on the back of this pitiful and disingenuous evidence base.

 

[lilpink]

Ooooohhhhh!

Someone is very sensitive today.

What has he got to worry about?. I thought all this spin and putting out this "scientific paper" on a website would have lessened the pressure. Perhaps not.

With all the blocking going on he seems very thin skinned all of a sudden.

https://twitter.com/i/web/status/773998771420160001

Badge of honour

 

[Chrisb]

Presumably this "new" analysis was produced with an innocent expression intended to be interpreted as "if only we had known".

Does it make any sense to accept the line, which seems to be being pushed, that this is new analysis, or, if it is, that other more potentially damaging analysis was not readily available?

Presumably there would have been managerial or administrative oversight of the FOI process-whatever the departmental preference for "carte Blanche". It might be expected that there would have been internal requests for details of what was in the files, what the likely damage would be in the event of a loss, and whether some nuanced arrangement might be made to mitigate the potential damage.

Are we expected to believe that no analysis of the data was conducted according to the original protocols before they were changed, and that such analyses were not subsequently available to management when authorising expenditure on the legal costs, if they requested it?

Why change the protocols if you do not know the outcomes which they produce?

 

[Yogi]

Peter White's service claims:



https://www.elft.nhs.uk/service/308/Chronic-Fatique-Service/

Retailers who sell something like chocolate, crisps, etc who advertise routinely 60 g but the product upon investigation contains 20g would be prohibited by the Advertising Standard Agency. If they repeatedly did this the Trading Standards or Office of Fair Trading would then take them to court and prosecute them and if they repeatedly did this they would be shut down.

Here is the famous Subway case over a 11.5 inch sub which was 1/2 inch smaller than Footlong
https://www.theguardian.com/us-news...s-its-footlong-sandwiches-will-be-a-foot-long

Subway customers can finally rest assured that their “Footlong” sandwiches will be as long as promised, after the chain agreed that the bread it uses will be at least 12 inches long.
The undertaking is part of the settlement of a class-action suit filed against the fast food chain after an Australian teenager in 2013 posted an image of his sandwich on Facebook that was only 11 inches long.
The image garnered international media attention, including an investigation by the New York Post, which found that four out of seven “Footlongs” it purchased in New York “measured only 11 or 11.5 inches.”
A judge had given preliminary approval in October to a settlement between Subway’s parent company and lawyers for the plaintiffs. Final approval was granted on 25 February.
As part of the settlement,Subway agreed to ensure that for at least four years its bread will be at least 12 inches long.

Someone explain to me why the law is a deterrent over something trivial like sandwiches and other consumer goods and services but not for medical research publications and healthcare & treatment which can injury and harm someone?

It would be good to produce a summary of all the false claims being made and where made and the evidence against these @Dolphin .

They can then be referred to the Advertising Standard Agency.

https://www.asa.org.uk/Consumers/How-to-complain.aspx

It seemed that the government regulate private entities over advertising harshly but seemingly allow the government depts such as NHS to get away with murder. However I have now checked and the ASA can investigate complaints about NHS as well!!!

Textbooks would not be covered. However magazines and newspapers are covered by IPSO.
https://www.ipso.co.uk/make-a-complaint/

 

[worldbackwards]

I don't think I've ever really felt hopeful before. But I do now.

 

[Deepwater]

Just another thought. I seem to recall that there was a problem with the way the FINE trial handled the high proportion of patients who did not complete the course - they were simply classed as not having taken part though most probably dropped out because the treatment was making them worse. It sounds as though the forthcoming MAGENTA study may be set to do likewise.
So what was done in PACE? They record a drop-out rate but it was very low. Does it include absolutely everybody who began the trial and did not complete, or not (for instance, were people who dropped out before a certain point scrubbed from the record)? Were the results of the drop outs factored into the totals? Sorry if I'm asking what everyone else knows, but I wouldn't put it past this crowd to have a definition of dropping out that isn't quite what it seems to be, and there was that odd thing with two new treatment centres having to be added to the trial at a late stage in order to make up numbers, and the consequent delay in publication of the trial results.
Sorry for any and all inaccuracies - am writing all this from ME memory.

 

[Bob]

Interpretation
All three of these outcomes are very similar to those reported in the main PACE results paper (White et al, 2011); physical functioning and fatigue improved significantly more with CBT and GET when compared to APT and SMC. One difference was found however; in this analysis with CBT compared to SMC, the difference for physical functioning was no longer statistically significant at p = 0.20.

In summary, these results support our initial interpretation that “CBT and GET can safely be added to SMC to moderately improve outcomes for chronic fatigue syndrome, but APT is not an effective addition.” (White et al, 2011).

A sight contradiction there; I doubt a therapy can be moderately effective if it's not a statistically significant outcome. CBT failed to significantly improve physical function (number of improvers) in these new analyses.

 

[Gijs]

If we take the publication Prins e.a. 2001 The Lancet and look at the natural course group of CFS patiënts after 14 months (table 4), we'll see that 23% improved (Karnofsky) and 32% (Self- rated). So the effect of White 20% CBT improvement is meaningless!!! Maybe Tom can look into this.

 

[worldbackwards]

A sight contradiction there; I doubt a therapy can be moderately effective if it's not a statistically significant outcome. CBT failed to significantly improve physical function (number of improvers) in these new analyses.

The new numbers don't fit the preconceived outcome, but the authors steam ahead anyway and say they it's all fine. Depressing how many times this happens and no one stops to question it.

 

[Keela Too]

Ummm p= 0.20 - when is that ever used for statistical significance. Usual is p=0.05
So were they using p=0.20 for other "significant" observations????

(Haven't read that in context - but quote above given by @Bob seems weird)

 

[Mrs Sowester]

So their post hoc manipulation inflated the results threefold? 300% ??!! Wow!

 

[Valentijn]

But these results are dire. It's hard to imagine anyone coming away from this with anything other than the impression that these treatments are ineffective for the vast majority of patients.

I wouldn't underestimate the media's (and doctors') capacity to read no further than the abstract. It's what happened with PACE in the first place ... they just trusted the authors' summary, and never bothered looking at the methodology, much less thinking about it.

If the Science Media Centre is involved, they'll give the authors' spin some extra spin, and probably end up making the new results sound like they show even more improvement than the post-hoc analysis in the original PACE papers did

 

[Bob]

Ummm p= 0.20 - when is that ever used for statistical significance. Usual is p=0.05
So were they using p=0.20 for other "significant" observations????

(Haven't read that in context - but quote above given by @Bob seems weird)

It's a good question but the other p values are decent values. It's only this single outcome that had a p value of 0.20. You can see the full list in their table 2.

 

[JaimeS]

he natural course group of CFS patiënts after 14 months (table 4), we'll see that 23% improved (Karnofsky) and 32% (Self- rated). So the effect of White 20% CBT improvement is meaningless!!!

This was always one of my questions. The SMC group looks fishy to me here. I think that if you ask any PWME if they're doing better than four months ago, several will say yes. Certainly some here would class their illness as relapsing-remitting, so it's to be expected, even if four months from now they feel worse.

Is this really only 10% as PACE authors are saying, or is it more like about 20% like Prins et al. are saying? Was there something special about the folk in the care as usual group? How were they being treated by their doctors and what was going on?

[Edit: NM, it's pretty useless to compare the ppl here with a cohort dxed by Oxford...]

-J