PACE trial authors continue to ignore their own null effect - JHP by Mark Vink

[Dolphin]

Thanks, @Dolphin for the extra information. I hadn't come across before the idea that because the London criteria exclude psych disorder, therefore the majority of the non London ones did have a psych disorder.

So, if I understand the figures correctly, of the 640 participants:

300 have 'any psychiatric disorder' ( some of whom may have ME including PEM under a different definition.

329 satisfy London criteria, ie no psych disorder

11 have no psych disorder and do not satisfy London criteria

I remember the claim that they analysed according to the different definitions and found no significant difference. I can't remember whether they actually gave figures for this. Given their propensity for twisting the truth, I'd want to see the figures before I believed this statement.

 

[Dolphin]

and in a trial by a leading proponent of the biopsychosocial model, 40 percent of ME patients reported deterioration of their health after GET (Moss-Morris et al., 2005).

Moss-Morris R, Sharon C, Tobin R, et al. (2005) A randomized controlled graded exercise trial for chronic fatigue syndrome: Outcomes and mechanisms of change. Journal of Health Psychology 10: 245–259.

This would be a noteworthy finding if it was true but I can see anything like it in the paper. Can anyone else?

 

[Dolphin]

Moreover, a review of the Belgium Government’s CFS Centers showed that treatment with CBT and GET did not change the physical capacity of patients yet caused an increase in unemployment rate and also of patients needing sickness benefits (Stordeur et al., 2008).

Good this is mentioned in the literature. It hasn't been mentioned much before I think.

 

[Dolphin]

Assessment of the individual participant data also showed that in the objective 6-minute walk test, only 3.7 percent (CBT) and 6.3 percent (GET) were objective improvers as defined by the same improvement of 50% or more, as used by the PACE trial itself, to classify someone as an improver (Vink, 2017). After removing the SMC effect, no participant improved objectively with CBT and only 1.3 percent (2/160) with GET yet 5 percent with SMC (Vink, 2017). This might indicate that CBT and GET impede the naturally occurring recovery process in ME/CFS.

Here is the data he is referring to from another paper of his. I wouldn't feel comfortable quoting the last sentence:

 

[Dolphin]

The trial classified 22 percent as recovered after CBT and GET, but assessment of the individual participant data found that none achieved the physical functioning, together with the fatigue scores, of the healthy sedentary controls from another trial by the PACE’s lead principal investigator or achieved Kennedy’s definition of recovery, where symptoms are eliminated and patients return to premorbid levels of functioning, which is the general public’s understanding of the meaning of recovery (Vink, 2017). Therefore, CBT and GET do not lead to actual recovery

I think this is referring to this paragraph in another paper by this author:

Table 3 also shows that if the PACE trial had used Kennedy‘s definition of recovery [34] - which reflects actual recovery and was made measurable by this analysis using two of the outcomes of the PACE trial itself - only 3.7% and 2.5% of patients in the CBT and GET group, respectively, would have achieved a normal physical functioning score for a healthy 38-yearold. If the SMC effect would have been deducted, the rates would be 1.2% and 0%. In addition, only two patients possibly achieved the normal score for the 6-minute walk test as well. It is impossible to be more precise as the released individual participant data were made anonymous to the point that the sex of the participant was removed. If these two participants were women, (one in the CBT group and one in the APT group, with scores of 631 m and 610 m, respectively) then both would be classed as recovered as these are considered normal scores. If the two participants were men, then none of the patients in the trial would be classed as recovered because they didn‘t reach 659 m, the normal score for men [37]. (If we had also used the individual Chalder Fatigue Questionnaire scores, the other subjective primary outcome of the trial itself, then regardless of the sex of that participant in the CBT group, that participant would not be classified as recovered because its Likert and Bimodal Chalder Fatigue Questionnaire scores were 19 and 8, respectively [22]. And according to the PACE trial protocol, a Bimodal score of 3 or less (out of a maximum of 11) was needed to be classified as recovered and during the trial this was changed to a Likert score of 18 or less (out of a maximum of 33), as found by the independent review of the PACE trial [5]. And the normal score for a healthy sedentary control from the other trial by the lead principal investigator of the PACE trial, as mentioned before, was a Bimodal score of 0 [23], which equates to a Likert score of 0 or 1).

However I think the reference to 19 and 8 is incorrect. The person in the CBT group who obtained a walking distance of 631 m had scores of 19 and 8 at baseline but their scores on completion were one and zero.

 

[Dolphin]

Despite receiving treatment deemed to be “effective” (White et al., 2017), and stating that at long-term follow-up “the benefits of CBT and GET were maintained some 2 years after treatment” (White et al., 2017), patients in all four treatment groups remained ill enough to re-enter the trial based on both subjective primary outcomes (Vink, 2017; White et al., 2011). There was no significant improvement on any of the trial’s objective measures, such as numbers returned to work or levels of fitness.

Given the was a statistically significant improvement on the main six minute walking test for graded exercise therapy, I would be uncomfortable quote in the last sentence.

 

[Dolphin]

The number of patients claiming state sick pay and disability benefits increased and the number of patients in receipt of income protection or private pensions in the CBT and GET groups actually doubled (Vink, 2016).

True, but the results for the specialist medical care-only group were not good either.

 

[Dolphin]

The actometer, an objective and reliable measure of activity, was not used at the end of the trial as it was deemed “too great a burden for patients.” Even though the device is light and small and patients had consented to its use (Vink, 2016). Patients by this stage had also apparently completed effective treatment (White et al., 2017) and 22 percent of those in the CBT and GET groups had recovered (White et al., 2013). Therefore, using the actometer at the end of the trial should have been easier and less of a burden than at the beginning.

The last sentence seems reasonable, all right.

 

[Dolphin]

The aforementioned study by the lead principal investigator of PACE, published in 2004, the year before the trial started, “found that exercise induced a sustained elevation in the concentration of TNF-α [a pro-inflammatory cytokine], which was still present three days later, and this only occurred in CFS patients” (White et al., 2004). In Rheumatoid Arthritis “fatigue…is due to TNF alpha. If you take away the TNF there is no fatigue” (Edwards, 2016). Why cytokines, including tumor necrosis factor- alpha (TNF-α; measured after exercise testing), were not a primary outcome is unclear as Kerr et al. (2003) demonstrated that recovery from ME/CFS led to normalization of cytokine levels.

Have other studies found the same thing about TNF-α? Cytokine results seem to be a bit all over the place in general in CFS but I can't remember what has been found in exercise studies.

 

[Dolphin]

Acknowledgements

I’m the author of the 2016 PACE trial review, for which I was nominated for the John Maddox Prize for Standing up for Science. I’m also the author of the 2017 assessment of the individual participant data of the PACE trial.

Does anyone know whether it would be hard to get nominated for this prize? If a single lay individual can nominate you, being nominated may not be that much of an achievement.

 

[Snow Leopard]

Thanks Dolphin. It seems the prepublication peer-review of this paper wasn't quite up to par...

 

[Barry53]

Thus it would appear that those who had a psychiatric disorder had similar results as those who did not have a psychiatric disorder.

Those results being based on people's perceptions, rather than objective outcomes. Presumably the still-unreleased PACE data holds information that would allow analysis of the 6-mwt with respect to having a psychiatric disorder or not. Also whether "any psychiatric disorder" for a given participant included a depressive component or not (71% did if I read the PACE paper's Table 1 right).

 

[trishrhymes]

Given the was a statistically significant improvement on the main six minute walking test for graded exercise therapy, I would be uncomfortable quote in the last sentence.

I think the 6 minute walk test significance is unsound because so many did not do the second walk, so any conclusions are nonsense. On that basis, I think he is right to say there is no significant improvement, or at least no valid evidence of a between group difference on the walk test.

Does anyone know whether it would be hard to get nominated for this prize? If a single lay individual can nominate you, being nominated may not be that much of an achievement.

This seems an unkind and unnecessary comment. I looked it up. The person nominating someone has to write a letter including biography and details of why they are nominating them and is has to be seconded by someone else willing to give information too. And presumably the nomination has to be deemed appropriate by the person or people receiving it and publishing the list of nominees. I think that's a pretty impressive achievement, especially given Mark Vink's severe ME.

OK there are some minor errors in this paper, if you dig hard enough, but the overall message is accurate and clear and I am very grateful to Mark Vink for putting in the huge effort to write it.

I agree with @Jonathan Edwards that it's good that errors of fact are pointed out and am grateful to @Dolphin also for using detailed knowledge of the PACE fiasco to take a forensic look at the detail and help clarify for me a couple of points I didn't understand.

But let us not forget the the motes in our side's eye is vastly overshadowed by the forests in the eye of the PACE authors, to misuse a biblical metaphor.

 

[Dolphin]

And presumably the nomination has to be deemed appropriate by the person or people receiving it and publishing the list of nominees.

Is anyone aware of such a list that has been published?

 

[Dolphin]

This seems an unkind and unnecessary comment.

I also find your comment unkind. I was simply asking a question and explaining why I was asking to increase the chance I'd get the info. I think it was a reasonable question to ask.

 

[trishrhymes]

I also find your comment unkind. I was simply asking a question and explaining why I was asking to increase the chance I'd get the info. I think it was a reasonable question to ask.

I apologise, my comment was not meant to be unkind. I guess I was trying to convey my own personal reaction to your phrase 'might not be much of an achievement' when you were simply seeking information. It felt unkind to me in this particular context, given that Mark Vink is a member of PR and very ill.

 

[Dolphin]

Also whether "any psychiatric disorder" for a given participant included a depressive component or not (71% did if I read the PACE paper's Table 1 right).

Note that the figure I posted also gives the results for the depression only group. Their results for CBT and GET look a little worse than the average. So I don't think for these two measures one can claim the results are down to people having depression which responded to CBT or GET.

 

[Barry53]

Given the was a statistically significant improvement on the main six minute walking test for graded exercise therapy, I would be uncomfortable quote in the last sentence.

What is the threshold for determining "statistically significant"? (Genuine non-sarky question, I don't know the answer). And does the GET improvement not have to be taken relative to the non-intervention SMC figure, meaning 1.3%? If so that sounds to me like statistically insignificant?

 

[A.B.]

Note that the figure I posted also gives the results for the depression only group. Their results for CBT and GET look a little worse than the average. So I don't think for these two measures one can claim the results are down to people having depression which responded to CBT or GET.

So this often mentioned hypothesis (that the effect is due to people with depression responding to CBT/GET) isn't supported by the data.

It's a shame that the PACE authors are so incompetent and can't design a trial that gives answers to any question.

 

[Esther12]

I apologise, my comment was not meant to be unkind. I guess I was trying to convey my own personal reaction to your phrase 'might not be much of an achievement' when you were simply seeking information. It felt unkind to me in this particular context, given that Mark Vink is a member of PR and very ill.

The other thing is that Vink might not realise that being nomintated for the prize involves no more than one person submitting a nomination. It's easy for people to get confused about these sorts of things, and always worth having possible confusions discussed.

I've been worried that promoting this nomination might end up seeming a bit silly to those who know how the nomination process works. I hadn't pointed this out, partly because I didn't want to have other people think I was being unkind, and I didn't feel it was important enough to get into a dispute about.

It's harmful for the community if we self-censor and avoid discussing these sorts of issues, when doing so could lead to more effective campaigning and writing in the future. I think that we should be doing all we can to encourage criticism of our own work from within the community, so that we can stengthen our work, and make it more effective for persuading those outside of the patient community. The PACE lot have clearly not been open to criticism of their work for a number of years, and this has played a role in them producing the junk they have.

Thanks a lot to Dolphin for going through and highlighting possible issues with this paper, and also for posting about it here.