• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

PACE trial authors continue to ignore their own null effect - JHP by Mark Vink

RogerBlack

Senior Member
Messages
902
A similarly trivially obvious work to the Tuller paper published yesterday. It's just a pity the obviousness is not clear to everyone.
Perhaps it needs an "Explain like I'm five" section.
 

Dolphin

Senior Member
Messages
17,567
I'm grateful for all the work that Mark Vink has put into writing this paper and other papers he has written. It is a real heroic effort given the severity of his illness. Also I admire him for writing in a language which is not his first language.

I think he makes a lot of good points in his papers but sometimes he makes points I disagree with and/or think are wrong and I'm going to highlight some regarding this paper now.
 

Dolphin

Senior Member
Messages
17,567
Trial Participants received on average 16 (cognitive behavior therapy (CBT)) and 17 sessions (graded exercise therapy (GET); including 3 sessions of specialist medical care (SMC)) yet in the SMC group that was only 5 (White et al., 2011).
Small points about this:
The CBT group had a median (not average) of 14 sessions of CBT +3 sessions of specialist medical care
The GET group had a median (not average) of 13 sessions of GET +3 sessions of specialist medical care
 

Dolphin

Senior Member
Messages
17,567
Chronic disabling fatigue is the only criterion of the Oxford criteria used by PACE. Only 56 percent of participants in the trial had ME according to the London criteria which do require the aforementioned main characteristic and 47 percent of participants had a comorbid depression or anxiety disorder (White et al., 2011). CBT is the most effective treatment for both (Lilienfeld, 2014). Therefore, improvement in PACE could simply be due to improvement in these comorbid psychiatric disorders.
The last sentence could theoretically be true but I don't think is in the PACE trial given the data we have. The people who satisfied ME criteria were almost exactly the same as all those who didn't have a psychiatric disorder (they made up 97% of this group) yet the SF-36 physical functioning and Chalder Fatigue Scale scores for the ME group were similar to the Oxford criteria group.
 

Dolphin

Senior Member
Messages
17,567
For example, the scoring of the Chalder Fatigue Questionnaire, one of its two (subjective) primary outcomes, was changed from Bimodal to Likert (White et al., 2011), which is more sensitive to change yet due to the ceiling effect is still too insensitive to deterioration (Stouten, 2005). After these changes, a Likert fatigue score of 18 or more (out of 33) was required to qualify for the trial,
The last part is incorrect. They did not change the entry criteria regarding the Likert fatigue scale. It was always six or more on the bimodal scale.
 

Dolphin

Senior Member
Messages
17,567
A binary or bimodal fatigue score of 3 equates to a Likert score of 6–9
No, it equates to a Likert score of 6-17

and a bimodal score of 4 to Likert 8–12.
No, it equates to a Likert score of 8-19

The consequence of this is that participants with a Likert score of 10–18 (inclusive) were severely fatigued and recovered at the same time.
Not really. To enter the trial you needed a bimodal score of six which translates to a minimum of 12 on the Likert scale.
 

Dolphin

Senior Member
Messages
17,567
Healthy sedentary controls from another trial by the PACE lead principal investigator, with the same mean age of 38 years as in the PACE trial, had a physicalfunctioning score of 100 (and a bimodal fatigue score of 0; White et al., 2004). Additionally, in Deale et al. (2001; which included one of the principal investigators of the PACE trial), a physical-functioning “cutoff score of 83 was used” for participants with a mean age similar to Pace, “as it represents the ability to carry out moderate activities” which does not represent recovery in healthy 38-year-olds. A score of 70 represents significant reductions in physical functioning (Reeves et al., 2005), and a score of 65 or less represents an “abnormal level of physical function” according to PACE (White et al., 2013) and severe disability according to the literature (Stulemeijer et al., 2005). Participants with a score of 60–65 (inclusive) were thus considered to be ill enough to participate, to have abnormal level of physical functioning yet also to be recovered and severely disabled.
I agree with all this
 

Dolphin

Senior Member
Messages
17,567
The extensive number of changes made to the recovery criteria during the trial broadened the definition of recovery, rendering it less accurate, and inflated the percentage of participants classed as recovered approximately four-fold (Wilshire et al., 2017). However, during a properly conducted (unblinded) trial, alterations should only be made by an independent trial steering committee without access to the data and only for compelling reasons as trial investigators may have impressions of the results to date, which might influence them (Evans, 2007) because outcome switching may, as Goldacre says, “exaggerate results (or even yield an outright false positive, showing a treatment to be superior when in reality it’s not).” The wrong conclusion in medicine is “not a matter of academic sophistry—it causes avoidable suffering” (Belluz, 2015).
 

Dolphin

Senior Member
Messages
17,567
The PACE trial authors point out that scores within the normal population ranges for fatigue and physical function were not sufficient to be considered as recovered and that participants also had to meet additional criteria (White et al., 2017). But participants (85/640 (13.3%)) who are already classed as recovered on one or both primary outcomes the moment they entered the trial, before receiving any treatment and without a change to their medical situation (Vink, 2017) should be excluded as a patient cannot be both (partially) recovered and ill enough to participate in a trial.

Moreover, the trial defined recovery partially on the basis of patients rating their overall health as “much better” or “very much better” (White et al., 2017) which reflects improvement but not (full) recovery.
I agree with all this
 
Messages
2,391
Location
UK
A similarly trivially obvious work to the Tuller paper published yesterday. It's just a pity the obviousness is not clear to everyone.
Perhaps it needs an "Explain like I'm five" section.
Many of the people who "the obviousness is not clear to" are very normal decent people who don't have medical science pouring out of their ears, yet are the very people who need enlightening and convincing, if public opinion is ever going to change.
 

Dolphin

Senior Member
Messages
17,567
Safety of CBT and GET In support of the conclusion that CBT and GET are safe, reference is made to a survey by Action for ME (White et al., 2017). Mention is not made, however, of another 2011 survey by the same charity which found that 60.2 percent of people with ME reported that GET had made their condition worse. In 44.1 percent of ME patients, GET had actually made it “much worse or very much worse” (Action for ME, 2011). The charity’s chief executive officer (CEO), Sir Peter Spencer, reacting to the survey said, “Our findings are disturbing. They show… that GET in particular makes them worse” (Action for ME, 2011).
Yes indeed. I'm attaching the survey results. See table 6.
 

Attachments

  • full analysis ORIGINAL COPY.pdf
    163.3 KB · Views: 3

Jonathan Edwards

"Gibberish"
Messages
5,256
I'm grateful for all the work that Mark Vink has put into writing this paper and other papers he has written. It is a real heroic effort given the severity of his illness. Also I admire him for writing in a language which is not his first language.

I think he makes a lot of good points in his papers but sometimes he makes points I disagree with and/or think are wrong and I'm going to highlight some regarding this paper now.

I think it is important and useful that this sort of friendly critique goes on amongst those who see PACE as valueless. It does not seem to occur within the PACE fraternity! I agree with your general analysis and salute your attention to detail.

What I think all this may demonstrate is that understanding how to design valid trials, or how to recognise flaws in those designed by others or oneself is not straightforward. My own experience certainly suggests that. When designing a trial you can spend two days in a meeting thinking you have perfected the design only to realise the second afternoon that you have sunk the whole structure by introducing a flaw. You need people with a wide angle view and you need people who focus on detail.

The difference with PACE, though, is that it is bit like one of those newspaper puzzles where you have to find 17 different words you can make out of another word. It is not that you cannot find words that sink the ship, it is just quite hard to get all 17! It really would serve very well as an extended exam question in how to to do a trial.
 
Messages
2,158
The last sentence could theoretically be true but I don't think is in the PACE trial given the data we have. The people who satisfied ME criteria were almost exactly the same as all those who didn't have a psychiatric disorder (they made up 97% of this group) yet the SF-36 physical functioning and Chalder Fatigue Scale scores for the ME group were similar to the Oxford criteria group.

I'm not sure I understand your point here. Do we have any analysis of the difference in outcomes for patients with comorbid depression compared with those without in each treatment group? l don't recall seeing any.
 

Dolphin

Senior Member
Messages
17,567
Chronic disabling fatigue is the only criterion of the Oxford criteria used by PACE. Only 56 percent of participants in the trial had ME according to the London criteria which do require the aforementioned main characteristic and 47 percent of participants had a comorbid depression or anxiety disorder (White et al., 2011). CBT is the most effective treatment for both (Lilienfeld, 2014). Therefore, improvement in PACE could simply be due to improvement in these comorbid psychiatric disorders.
The last sentence could theoretically be true but I don't think is in the PACE trial given the data we have. The people who satisfied ME criteria were almost exactly the same as all those who didn't have a psychiatric disorder (they made up 97% of this group) yet the SF-36 physical functioning and Chalder Fatigue Scale scores for the ME group were similar to the Oxford criteria group.

I'm not sure I understand your point here. Do we have any analysis of the difference in outcomes for patients with comorbid depression compared with those without in each treatment group? l don't recall seeing any.
I'm not sure what I can say except repeat it in a little more detail.

Here are the figures that showed that 97% of those who satisfied the Oxford criteria in this study who did not have a psychiatric disorder satisfied to ME criteria: https://listserv.nodak.edu/cgi-bin/...CO-CURE&P=R2764&I=-3&d=No+Match;Match;Matches

Thus the results for those who don't have a psychiatric disorder would be very similar to those who satisfied the London ME criteria in the study. Those who satisfied the London ME criteria were found to have similar results overall to the full Oxford criteria group. This then suggests that those who did not satisfy the London criteria also had similar results to the full Oxford criteria group. Those who did not satisfy the London criteria are almost exactly the same as those who have a psychiatric disorder in this study. Thus it would appear that those who had a psychiatric disorder had similar results as those who did not have a psychiatric disorder.
 
Messages
2,158
Thanks, @Dolphin for the extra information. I hadn't come across before the idea that because the London criteria exclude psych disorder, therefore the majority of the non London ones did have a psych disorder.

So, if I understand the figures correctly, of the 640 participants:

300 have 'any psychiatric disorder' ( some of whom may have ME including PEM under a different definition.

329 satisfy London criteria, ie no psych disorder

11 have no psych disorder and do not satisfy London criteria

I remember the claim that they analysed according to the different definitions and found no significant difference. I can't remember whether they actually gave figures for this. Given their propensity for twisting the truth, I'd want to see the figures before I believed this statement.
 

Dolphin

Senior Member
Messages
17,567
The individual participant data of the PACE trial showed that up to 82.2 and 79.8 percent of ME patients in the trial might have been negatively affected by CBT and GET, respectively (Vink, 2017),
This is theoretically true. Another alternative is that 0% of those with ME might have been negatively affected by CBT and GET. There is quite a range of possibilities.

One can see the data he is referring to and my comments on it here:
http://forums.phoenixrising.me/inde...bt-and-get-are-ineffective.48764/#post-803812