PACE Trial and PACE Trial Protocol

[Esther12]

Kind of amusing that NICE don't think PACE showed anything worthwhile.

 

[biophile]

Since the 2007 NICE guidelines for CFS were published, evidence had come to light that CBT and GET do not increase employment hours and do not increase total activity levels, contrary to common allusions.

These findings come at no surprise to patients and carers who live in the real world, but such facts should be part of any guideline which purports to outline how patients should be managed with CBT/GET.

 

[Graham]

I think PACE used the London criteria, but to be honest, it is so irrelevant I haven't bothered comparing it with the others, otherwise we will all be debating how many psychiatrists in the Wessley camp can dance on the head of a pin.

 

[MeSci]

I think PACE used the London criteria, but to be honest, it is so irrelevant I haven't bothered comparing it with the others, otherwise we will all be debating how many psychiatrists in the Wessley camp can dance on the head of a pin.

I prefer the question of how many pins one can stick in the head of a psychoquack!

 

[Graham]

Well, I'd put more faith in voodoo that psychology for sorting out my ME. At least as a zombie I'd get about more.

 

[biophile]

It has been mentioned a few times on this forum that James Coyne is a renegade psychologist who critiques the flaws in CBT and other psychological research. He has recently been ripping into the CBT trial for schizophrenia, which was published in the Lancet and hyped uncritically, reminiscent of the PACE results being published there 3 years ago.

Coyne is at least cursorily aware that potential issues exist with the PACE Trial.
https://twitter.com/CoyneoftheRealm/status/407067890224230400

But if anyone has wondered why Coyne has not been all over the PACE Trial like a seagull at a picnic (or garbage dump), besides the usual such as being too busy or simply just not being interested, here is another possible reason:

"I know Simon Wessely and know that he has had death threats over his interpretation of a trial of CBT chronic fatigue."

http://jcoynester.wordpress.com/2014/02/12/my-response-to-the-hacking-of-this-blog-site

 

[Valentijn]

But if anyone has wondered why Coyne has not been all over the PACE Trial like a seagull at a picnic (or garbage dump), besides the usual such as being too busy or simply just not being interested, here is another possible reason:

"I know Simon Wessely and know that he has had death threats over his interpretation of a trial of CBT chronic fatigue."

Fascinating. Someone gets a death threat, and suddenly their bullshit theories cannot be politely and scientifically disputed. I guess that explains his I'm-the-victim media campaign regarding such threats.

 

[Roy S]

That's a good find. The other sentence regarding Wessely seems to indicate that he doesn't understand the issues.

"I know Simon Wessely and know that he has had death threats over his interpretation of a trial of CBT chronic fatigue. I do not think he wants to step back into the fray late judgments about a trial of CBT for unmedicated schizophrenics."

Gish Gallop is a good term that he uses.
http://rationalwiki.org/wiki/Gish_Gallop

 

[Min]

There is no evidence that even one person with myalgic encephalomyelitis has ever threatened a researcher. It is deliberately harmful propaganda being used to denigrate us. The only person ever convicted did not have M.E. or even chronic fatigue.

 

[biophile]

That's a good find. The other sentence regarding Wessely seems to indicate that he doesn't understand the issues.

"I know Simon Wessely and know that he has had death threats over his interpretation of a trial of CBT chronic fatigue. I do not think he wants to step back into the fray late judgments about a trial of CBT for unmedicated schizophrenics."

I am not sure who you are referring to, Coyne or Wessely? Just to be clear: Wessely is mentioned because, during a debate between Coyne and the "hacker" of his blog, the "hacker" apparently suggested using Wessely as a referee:

[edit: the debate involves a trial of CBT for schizophrenia]

The Lancet article authors never specified that this was a simple effect size. Furthermore, it is not at all clear how the authors calculated this particularly because at the end of follow-up less than half of the patients remained in what was already a small sample to start. Assumptions of nonrandom loss to follow-up no longer apply and so this is a misleading and even bogus effect size. I will say more about that in future blogs.

But the quotes suggest otherwise, and you know that’s true. You have shot out in ignorance and acted in a highly non-collegiate way. I suppose $500 is a lot of money. Perhaps you need to appoint someone else to judge whether you need to cough up? I suggest Simon Wessely. Why don’t you tweet him?

On the issue of abuse ...

There is no evidence that even one person with myalgic encephalomyelitis has ever threatened a researcher. It is deliberately harmful propaganda being used to denigrate us. The only person ever convicted did not have M.E. or even chronic fatigue.

Considering how common internet abuse is these days even for unimportant issues, I tend to think that a degree of abuse towards a few CFS researchers has plausibly occurred over heated issues. However, you are correct that it has been used successfully to tarnish the general impression of their critics, but no solid evidence has been produced, just unverifiable references to the police or authorities, despite the lack of arrests for such supposedly-common criminal offenses.

The Nigel Hawkes BMJ article, IIRC commissioned by the Science Media Center for what they later described as part of their organized campaign against abusers, does mention some disturbing examples though.

I also vaguely remember one researcher coming to PR to describe some abuse they allegedly received. However, after interpreting the slightest criticism or failure to respect their supposed scientific authority as "harassment", it unfortunately made me question the severity and nature of the other alleged abuses.

 

[Roy S]

I am not sure who you are referring to, Coyne or Wessely?
quote]
Sorry, I meant Coyne doesn't seem to understand the issues.

 

[biophile]

The other sentence regarding Wessely seems to indicate that he doesn't understand the issues.

Sorry, I meant Coyne doesn't seem to understand the issues.

I thought it would be the first sentence rather than the second sentence which indicates Coyne doesn't understand the issues surrounding PACE and ME/CFS, because he seemingly refers to it as a CBT trial for "chronic fatigue". Although on the other hand, PACE only required the Oxford criteria, which all other CFS definitions regard as idiopathic chronic fatigue.

The second sentence makes sense from the point of view of bringing in Wessely to referee an argument about "a trial of CBT for unmedicated schizophrenics", since that is what Coyne's blogpost was all about.

 

[Graham]

I can't get my head around this idea that abuse and threats over the internet has put professionals off. What about all those professionals who face real physical threats regularly, such as medical staff at A&E, social workers, teachers, …? Is it just that they see it as a vocation, whereas these folk just aren't that committed to their cause? Or is it simply that a larger salary reduces a person's dedication?

 

[Valentijn]

Is it just that they see it as a vocation, whereas these folk just aren't that committed to their cause? Or is it simply that a larger salary reduces a person's dedication?

Option 3: They're just looking for yet another excuse to vilify anyone who disagrees with them. Despite Wessely's histronics, I don't think we've lost any useful researchers or clinicians due to purported abuse. Even Wessely keeps coming back

 

[Graham]

Despite Wessely's histronics, I don't think we've lost any useful researchers or clinicians due to purported abuse. Even Wessely keeps coming back

Is that the solution then to getting better researchers?

 

[Art Vandelay]

There is no evidence that even one person with myalgic encephalomyelitis has ever threatened a researcher. It is deliberately harmful propaganda being used to denigrate us. The only person ever convicted did not have M.E. or even chronic fatigue.

Precisely. If real death threats had been made, they would have been reported to the police and action would have been taken. Alleging "death threats" is a tactic used by vested interests to shut down debate (it has been used in Australia in recent years on other issues).

 

[Dolphin]

MEA chairman Neil Riley unpicks a central thread in the PACE Trial
http://forums.phoenixrising.me/inde...cks-a-central-thread-in-the-pace-trial.28552/

 

[Tom Kindlon]

I haven't followed it closely but there has been a bit of a furore among some with how a trial of Cognitive therapy for another condition, Schizophrenia, has been reported in the Lancet. This article links to a lot of the information:

The pitfalls of pre-registration: The Morrison et al CBT paper
http://persuasivemark.blogspot.be/2014/03/the-pitfalls-of-pre-registration.html

 

[Esther12]

From Lancet PACE:

A clinically useful difference between the means of
the primary outcomes was defined as 0·5 of the SD of
these measures at baseline,31

I was just about to look into reference 31, when I found that Dolphin already had (back on page 32!).
I thought I'd repost, as I think that this was one of the two papers that they cited to justify changing their primary outcomes from those laid out in the trial's protocol:

I can't remember has this been highlighted or not:
One of their references is:

Guyatt GH, Osaba D, Wu AW, et al. Methods to explain the clinical
significance of health status measures. Mayo Clinic Proceedings
2002; 77: 371–83.

Free at: http://www.mayoclinicproceedings.com/content/77/4/371.long

The proportion of patients achieving a particular benefit,
be it a small, moderate, or large difference, is therefore
much more relevant than a mean difference from the
clinician’s point of view and less likely to mislead. To
calculate the proportion who achieve a MID, one must
consider not only the difference between groups in those
who achieve that improvement but also the difference between
groups in those who deteriorate by the same amount. (we weren't given that proportion (i.e. those who got worse) in the paper when they told us how many went up by 8 points on the SF-36 PF and/or by two points on the Chalder fatigue scale)
One must therefore classify patients as improved, unchanged, or deteriorated. In a parallel group trial, the subsequent calculation is not altogether straightforward, and 1
approach involves assumptions about the joint distribution
of responses in the 2 groups.13 Statisticians are developing
alternative approaches to this problem, several of which are
likely to prove reasonable.58 What is not reasonable is
simply to present mean values without taking the second
step that is necessary for clinicians to interpret clinical trial
results effectively.​

Distribution-based methods have, in general, 2 fundamental
limitations. First, estimates of variability will differ
from study to study. For instance, if one chooses the between-
patient standard deviation, one has to confront its
dependence on the heterogeneity of the population under
study. If a trial enrolls an extremely heterogeneous population,
an important effect may be small in terms of the
between-person standard deviation and thus judged trivial.
The same effect size, in a trial that enrolls an extremely
homogeneous population, may be large in terms of the
between-person standard deviation, and thus judged extremely
important. The true impact of the change remains
the same, but the interpretation differs radically.

There are at least 2 ways to deal with this problem. One
is to choose the variability from a particular population,
such as the standard deviation of a measure when applied to
the general population at a point in time, and always refer to that same measure of variability. The second is to choose
the standard error of measurement (which we will discuss
subsequently), which is theoretically sample independent.

[..]

BETWEEN-PERSON STANDARD DEVIATION UNITS
The most widely used distribution-based method to date is
the between-person standard deviation. The group from
which this is drawn is typically the control group of a
particular study at baseline or the pooled standard deviation
of the treatment and control groups at baseline. As we have
mentioned herein, an alternative is to choose the standard
deviation for a sample of the general population or some
particular population of special interest, rather than the
population of the particular treatment study under consideration.
An advantage of this approach is that it has been
applied widely in areas of investigation other than QOL.
In case people forget, they chose the baseline standard deviations rather than population standard deviations.

The text is no longer available at that link.

 

[Esther12]

The other paper that they cited to justify their changes in primary outcome measures:

PACE Lancet:

We used continuous scores for primary outcomes to
allow a more straightforward interpretation of the individual
outcomes, instead of the originally planned
composite measures (50% change or meeting a
threshold score).10,30 We prorated primary outcomes
scales only when there were at most two items per scale
missing (nine participants for Chalder fatigue questionnaire
and 11 for short form-36). Prorating involved
calculating the mean value of the item scores present
and replacing the missing values with that score.
We summarised continuous variables with mean (SD)
or median (IQR) and categorical variables with
frequencies and proportions. Diff erentiation of
treatment compared independent ratings of therapy
sessions with actual treat ment. We calculated the interrater
reliability (κ and 95% CI) between the two
assessors. We used Kruskal-Wallis tests for comparisons
of therapy received, therapeutic alliance, and manual
adherence. We compared categorical variables with
Fisher’s exact test.

30: http://www.ncbi.nlm.nih.gov/pubmed/19455540

This paper got a few bits of correspondence, and I meant to look into it more in 2011, but I don't think we ever got copies of them.