AndyPR
Senior Member
- Messages
- 2,516
The link is wrong somehow, this should work http://healthinsightuk.org/2016/10/...cting-medical-advice-and-what-to-do-about-it/
Welcome to Phoenix Rising!
Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of, and finding treatments for, complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia, long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.
To become a member, simply click the Register button at the top right.
I have a comment waiting on moderation on the Health Insight article, I chose to weight it towards MEGA and how the patient reaction is very much informed by the levels of distrust that have built up. The Hippocratic Post requires you to be a medical or healthcare professional to register and comment.Please read and leave comments on the Hippocratic Post and this latest article as he said it is read by doctors and he is trying to get published in mainstream UK media.
I’m grateful to the indefatigable Dr Zoe Harcombe, a hugely well-informed nutritionist with a clear grip on bio-medical statistics, for this explanation. She cheerfully admitted she didn’t understand the workings of this particular McGuffin, but that didn’t matter. She knew what its underlying assumptions were and what it was designed to do.
I'm discussing with a MD about PACE and the reanalysis after the data was made public.
He asks: "Given that the reanalysis is a more correct measure for the true effect of CBT, how do we know that the original protocol way of measuring is more accurate?"
What did they really change, and what does it mean practically that a patient is considered an improver, vs an improver in the original protocol?
I haven't read all about PACE in detail, but I think I remember that an "improver" could really be very sick, and that the difference between improvers and non-improvers are very small. Change of SF36 score? Also some very considered improved even though they got worse, isn't that true? Does anyone have some facts to back this up?
What did they really change, and what does it mean practically that a patient is considered an improver, vs an improver in the original protocol?
There is more.For example, the changes created an overlap between trial eligibility criteria for severe disabling fatigue, and the new “normal range”. Trial participants could consequently be classified as recovered without clinically significant improvements to self-reported physical function or fatigue, and in some cases without any improvement whatsoever on these outcome measures. Approximately 13% of participants at baseline simultaneously met the trial eligibility criteria for ‘significant disability’ and the revised recovery criteria for normal self-reported physical function.
According to the protocol, "improvement" (not "recovery") required a bimodal-scored Chalder Fatigue Scale (CFQ) of 3 or less (out of 11, recruitment required 6 or higher) or a 50% reduction. For the SF36 Physical Functioning subscale, scores had to be at least 75 (out of 100, 65 or lower required for recruitment) or a 50% increase.What did they really change, and what does it mean practically that a patient is considered an improver, vs an improver in the original protocol?
I think this take is absolutely right.I think it's important to remember that this propoganda is not really aimed at us. They don't expect to be able to win patients over. It is aimed at the general public, at colleagues, at NICE and at funders (these must be our target audience too). The word from the BPS lot will be that "we offered an olive branch even though we were not at fault, we've tried to engage and unite the community but unfortunately a small vocal minority are just very self destructive
It is questionable whether all changes to the published trial protocol were approved:
While the investigators stated that the recovery criteria was changed before the analysis was conducted [1] and therefore should be regarded as “pre-specified” [2], there is evidence suggesting that it was changed well after the investigators were already familiar with the distribution of scores for each component of the recovery criteria, including access to summary statistics and histograms:
a) Before appearing as part of the revised recovery criteria, the controversial post-hoc ‘normal range’ for fatigue and physical function appears to be first introduced during the peer review stage [3] of the 2011 Lancet publication on the main trial results (well after the authors were unblinded to trial data). [4]
b) The statistical analysis plan outlines the final planned or pre-specified analyses that were approved prior to database lock and unblinding of trial data, but does not mention changes to the recovery criteria. [5] It describes the summary data they had access to early on when preparing the first paper on the main results. It discusses the possibility of future post-hoc changes and exploratory analyses, including those based on trial data.
c) Changes to the primary outcomes were approved by an oversight body (e.g. Trial Steering Committee) [5] but there was no confirmation that the revised recovery criteria was similarly approved, not in the paper itself [1] or the investigators’ response to David Tuller when the issue of changes to the recovery criteria was raised. [4] Note that statements in the PACE trial literature about approval of changes typically refers to the main changes to the primary outcomes, not specifically to the revised recovery criteria. When referring to the overlap between the normal range for fatigue and physical function, with the trial eligibility criteria for severe disabling fatigue, Bruce Levin (Professor of Biostatistics, Columbia University) has stated: “I find it nearly inconceivable that a trial’s data monitoring committee would have approved such a protocol problem if they were aware of it.”. [6]
d) The original primary outcome i.e. positive outcome was the proportion of individuals meeting a threshold, but the new primary outcome was group-level averages; in other words the new primary outcome was not a direct replacement of the original. The closest (but much less stringent) equivalent to the original primary outcome is the secondary outcome for individual-level “clinically useful difference” (CUD). But this was also described in the Lancet publication as a post-hoc analysis [4] and not mentioned in the statistical analysis plan either [5] (group-level CUD is mentioned and approved) so it remains unclear if it was explicitly approved by the Trial Steering Committee.
The PACE investigators have not reported all pre-specified outcomes and have not provided adequate details about the timing of the changes; this is contrary to modern expectations of researchers (e.g. see AllTrials.net and compare-trials.org). A FOIA request to clarify this issue and make sure a possible post-hoc or exploratory analysis was not being misleadingly described as a pre-specified, was dismissed as part of a “vexatious” campaign. [7] The PACE investigators stated that the trial would be conducted in accordance to the Declaration of Helsinki, however, the refusal to clarify the timing of changes to the protocol is equivalent to withholding information about methods, which is against the declaration. [8]
The changes to the protocol were (generally) poorly justified, and the change to the threshold for normal physical function was demonstrably erroneous. The stated justification for this drastic change, incorrectly asserted that about half the general working age population score under 85, but it is actually 17.6%, so the change was unjustified. Note that 92.3% of the working age English population who do not report chronic disabilities score 85 to 100, and 61.4% score the maximum of 100, with scores under 80 being extreme outliers. [9,10] Again, PACE investigators claim that attempts to raise these issues and have them corrected are simply part of a “vexatious” campaign. [11,12]
1. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3776285
2. http://www.virology.ws/2015/10/30/pace-trial-investigators-respond-to-david-tuller
3. http://www.meactionuk.org.uk/whitereply.htm
4. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3065633
5. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4226009
6. http://www.virology.ws/2015/10/21/trial-by-error-i/
7. https://www.whatdotheyknow.com/request/timing_of_changes_to_pace_trial
8. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4396123
9. http://link.springer.com/article/10.1007/s11136-014-0819-0
10. http://www.bmj.com/content/350/bmj.h227/rr-16
11. https://ico.org.uk/media/action-weve-taken/decision-notices/2015/1043579/fs_50558352.pdf
12. http://www.thelancet.com/journals/lanpsy/article/PIIS2215-0366(15)00114-5/fulltext