Welcome to Phoenix Rising!
Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of, and finding treatments for, complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia, long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.
To become a member, simply click the Register button at the top right.
I don't think the investigators have claimed they changed the protocol "in order to inflate the small differences between the treatments". I'm guessing he meant he thinks that was their real reason so I'm guessing a few words were missing.The PACE Trial investigators claimed that they changed the Protocol because the data ‘would be hard to interpret’ and in order to inflate the small differences between the treatments.
Just to point out that the score of 75 wasn't the only way to satisfy the criteria in the primary outcome: another way was improving by 50% of more from one's baseline score.The PACE Trial Protocol[2] Primary Outcome Measure criteria for ‘Positive Outcome’ required a score of 75 or more on the SF36PF (Short Form 36 Physical Function subscale), ‘Recovery’ required a score of 85.
There were statistically significant differences.The investigator’s explanation in the FAQ shows that changes were made for the purpose of inflating results that were undetectable with the published Primary Outcome Measures because there were no significant differences.
I'm not sure if the specific thresholds used in the primary outcomes were based on existing research.When the Protocol was designed, the researchers analysed previous research, including their own, into GET and CBT for ME/CFS. This informed their choice of the Outcome Measures.
The Protocol states: “We have chosen 15 sessions for all supplementary treatments on the basis of the previous trials of CBT and GET [18,23-26], as well as extensive clinical experience.” And: “The existing evidence does not allow precise estimates of improvement with the trial treatments. However the available data suggests that at one year follow up, 50 to 63% of participants with CFS/ME had a positive outcome, by intention to treat, in the three RCTs of rehabilitative CBT [18,25,26], with 69% improved after an educational rehabilitation that closely resembled CBT [43]. This compares to 18 and 63% improved in the two RCTs of GET [23,24], and 47% improvement in a clinical audit of GET [56].”
Therefore the Primary Outcome Measures in the Protocol were based on existing research, including that done by the Principal Investigators themselves and on their “extensive clinical experience”. Why would these experienced physicians and researchers believe it was necessary to lower their own target thresholds by 30% to 50% in order to detect a treatment effect? How did they know that the treatments had not reached the Protocol thresholds that they had so authoritatively defined?
I'm not convinced it is true to say that there had previously been no treatment effects "below the threshold of significance". The analyses released last week show significant differences with their original primary outcome measures.Changing the Primary Outcome Measures in order to exaggerate the small differences between the treatment arms, created the appearance of treatment effects that had been below the threshold of significance. It also created disturbing anomalies.
This is something people will be able to investigate if or when Alem releases the data.By happy chance or design, it seems that the threshold chosen for the SF36PF of ‘Normal Range’ was positioned in the exact spot – a score of 60, to make just enough participants ‘normal range’ for the PACE Trial to be able to claim a treatment effect. Had the threshold been set at 65, it appears that a substantial number of those currently classed ‘normal range’ would have been eliminated. Had the threshold been set at 55, it would have included substantially more of the Comparison Group (SMC, purple on the chart below). It further appears, that the authoritatively defined Protocol threshold of 75 for ‘Positive Outcome’, might have included only a few participants. It would be surprising if more than a handful reached the Protocol ‘Recovery’ threshold.
Not sure one can say the last sentence. The % of overall improvers for CBT and GET was twice rate for SMC.Please note that the researchers stated, “We propose that a clinically important difference would be between 2 and 3 times the improvement rate of SSMC”. If the researchers’ expected outcomes for their favoured treatments had been reached; the average scores of GET and CBT at 52 weeks should have been somewhere between 65 and 80. Therefore there was no “clinically important difference” between the comparison group and treatments.
I'm not sure that the last sentence is justified. The threshold of 75 was only one part of the primary outcomes; one could also satisfy the criterion by improving by at least 50%. Also the average score wouldn't need to be 75 for a therapy to be useful.In view of the above, one might take the view that the PACE investigator’s complaint that the original Protocol Primary Outcome Measures, “did not answer our main questions regarding comparative efficacy”, has some validity. It does not. The key word is: ‘efficacy’. Based on their own research and “extensive clinical experience”, the researchers set the Primary Outcome thresholds to detect “clinically important differences”. To that end they defined ‘Positive Outcome’ and ‘Recovery’ in order to measure ‘efficacy’. Failure of the treatments to reach those authoritative thresholds showed the absence of efficacy, making comparisons meaningless.