PACE Trial and PACE Trial Protocol

[Esther12]

My point was just that maybe K&B didn't check the PF thresholds (because they were in a hurry), saw 30% were normal and went with that as 'recovery'. the alternative is that they knew exactly what they were doing and just lied outright, but I don't think that's so likely because it's almost inevitably going to end with a correction, as happens here. I think they just erroneously assumed PACE had used the same critieria as the original Knoop study.

I had assumed it was an honest mistake too...

but these people use the SF-36 PF figures in their own work. And only really patients will know about the correction - it's not like it got the sort of coverage in the media that claims about curative CBT/GET did. If it was a deliberate lie, it was one that worked out well for the promotion of their view of CFS and it's treatments.

I don't know how little to think of them.

Also - if we start talking like CFS researchers are being intentionally dishonest, then we will be dismissed as conspiracy nuts even if we're right. Maybe it's best to try to go on assuming they're honest but stupid.

 

[Snow Leopard]

Who writes an editorial without reading the paper properly? It's either confirmation bias after misreading of the paper, or a deliberate stretch of the facts. Neither fills you with confidence of what they have to say.

 

[Sean]

Who writes an editorial without reading the paper properly? It's either confirmation bias after misreading of the paper, or a deliberate stretch of the facts. Neither fills you with confidence of what they have to say.

You are being too polite.

I am getting very tired of this endless flood of 'mistakes' and 'misinterpretations' from these guys. Especially with a study as supposedly important as this one.

It is arguable that a handful of these mistakes/misinterpretations are just that. But not 20 years worth, all of them conveniently biased in their favour. If that is not a major warning sign about them and their work, I don't know what is.

There are no excuses for this shit from the editorial writers. If they can't even get basic stuff like this right, then the whole editorial should be considered suspect and withdrawn, and apologised for. And a serious independent enquiry launched into why it happened at all in the first place.

 

[Dolphin]

PACE Trial is mentioned in the BMJ this week

The BMJ has four commentaries/editorials on ME/CFS this week. The full text of all of them is at: http://www.meassociation.org.uk/?p=6711

The PACE Trial is mentioned in all four of them.

People can discuss these commentaries in general elsewhere e.g. http://forums.phoenixrising.me/show...s-painting-PWME-as-threatening-psych-patients

 

[oceanblue]

There's a full list of responses here

I particularly liked this (excerpt) from:
Andrew J. Kewley, Student
Flinders University

...
One of the major problems of clinical trials in chronic diseases is that
outcomes are poorly chosen, usually to make the results look more
impressive, along with a high prevalence of potential biases [4]. In trials
of Cognitive Behavioural Therapy (CBT) or GET (Graded Exercise Therapy),
there is an over emphasis on short term improvements on self report
questionnaires and little emphasis on recovery as defined by activity levels
and employment status at long term follow-ups.

Patients in general are sceptical of improvements on self report
questionnaires when there are no improvements in activity levels. Patients
aren't interested in short term cognitive changes such as increases in self
efficacy, without a corresponding reduction in disability. They understand
that such questionnaires are subject to a variety of biases, including
response bias and social desirability effects.

Now a double blind RCT will effectively control for many of these biases,
but this is not possible in the case of CBT or GET. This is why clinically
significant outcomes must be chosen to have long term an objective
component, such as hours spent in employment and study, or actometer
measurements.

Unfortunately, when activity levels in CBT trials are measured, no
improvement is found [5]. Likewise, in an audit of the Belgian
Interdisciplinary rehabilitation programme for patients with CFS (n=593),
there was no improvement in activity levels at the conclusion of treatment
[6]. The therapy component systematically included CBT and GET, with
sessions spread over 6 to 12 months [6]. Employment status decreased at the
end of the therapy and the 6 month follow-up, however there was an increase
in study and volunteer work, so the overall level of physical capacity as
reported by the study did not change [6].
...

[4] Vollenweider, Boyd, Puhan, 2011. "High prevalence of potential biases
threatens the interpretation of trials in patients with chronic disease"
http://www.biomedcentral.com/1741-7015/9/73

[5] Wiborg et al. 2010 "How does cognitive behaviour therapy reduce fatigue
in patients with chronic fatigue syndrome? The role of physical activity."
http://www.ncbi.nlm.nih.gov/pubmed/20047707

[6] KCE Reports 88A Chronisch Vermoeidheidssyndroom: diagnose, behandeling
en zorgorganisatie, 2008. (Parts are in English)
http://www.kce.fgov.be/Download.aspx?ID=1222

Competing interests: Student potentially interested in doing CFS research.

Most of this has been covered on this thread before but he makes the points very well. However, I don't think I'd seen the point that the impossibility of blinded RCTs make objective outcomes even more important made like that before, and I think it makes a very powerful argument.

 

[Bob]

Professor Malcolm Hoopers further concerns about the PACE Trial

A new letter from Malcolm Hooper sent to The Lancet.

"Professor Malcolm Hoopers further concerns about the PACE Trial article published in The Lancet"

24th June 2011

The full text can be found at both of these links:

http://www.investinme.org/Article42...rticle published in Lancet.htm?forumid=331851

http://www.meactionuk.org.uk/Normal-fatigue.htm


This letter focuses on the 'Normal Range' related issues.

 

[Jenny]

In the July issue of The Psychologist there's a response to the letter by Oceanblue and myself which they published in June. I've tried to attach the pdf of the letters section but can't as the file is too large.

It's entitled 'As useless as a chocolate teapot' and it's by Kristina Downing-Orr. She agrees with what we said, elaborates on the physical abnormalities, and argues that although some mild cases benefit from CBT, CBT and GET are:

'as useless as a chocolate teapot. In fact they cause more distress, because clients are usually blamed and then abandoned. To treat CFS effectively, the body first needs medical or nutritional intervention and only after this stabilisation should any fitness regime be implemented......Today's medical orthodoxies are tomorrow's abuses.'

Jenny

 

[Jenny]

In the July issue of The Psychologist there's a response to the letter by Oceanblue and myself which they published in June. I've tried to attach the pdf of the letters section but can't as the file is too large.

It's entitled 'As useless as a chocolate teapot' and it's by Kristina Downing-Orr. She agrees with what we said, elaborates on the physical abnormalities, and argues that although some mild cases benefit from CBT, CBT and GET are:

'as useless as a chocolate teapot. In fact they cause more distress, because clients are usually blamed and then abandoned. To treat CFS effectively, the body first needs medical or nutritional intervention and only after this stabilisation should any fitness regime be implemented......Today's medical orthodoxies are tomorrow's abuses.'

Jenny

Dolphin has kindly reduced the size of the pdf so here it is attached.

 

[biophile]

Thanks everyone for keeping this thread alive with new information and perspectives.

I'm glad that Hooper has tackled the flaws in how the fatigue scale has been (ab)used in the PACE trial, it is just as bad as the physical functioning issue.

Snow Leopard wrote:

Who writes an editorial without reading the paper properly? It's either confirmation bias after misreading of the paper, or a deliberate stretch of the facts. Neither fills you with confidence of what they have to say.

Indeed.

Sean wrote:

You are being too polite.

I am getting very tired of this endless flood of 'mistakes' and 'misinterpretations' from these guys. Especially with a study as supposedly important as this one.

It is arguable that a handful of these mistakes/misinterpretations are just that. But not 20 years worth, all of them conveniently biased in their favour. If that is not a major warning sign about them and their work, I don't know what is.

There are no excuses for this shit from the editorial writers. If they can't even get basic stuff like this right, then the whole editorial should be considered suspect and withdrawn, and apologised for. And a serious independent enquiry launched into why it happened at all in the first place.

The whole ordeal does seem too suspicious to attribute solely to sloppiness or incompetence.

 

[Dolphin]

Minimum Important Difference

I've just posted a little thread on this also at:
http://forums.phoenixrising.me/show...h-international-consensus&p=189843#post189843

Basically the values used in the PACE Trial (2 on the Chalder Fatigue Questionnaire (CFQ) and 8 on the SF-36 PF subscale) that were post-hoc analyses look small:

Low back pain isn't the same condition of course.

Anyway, if one was to use the 30% of baseline values (mean SF-36 PF: 38.025, mean CFQ: 28.175), the values would have been:

SF-36 PF: 11.4075 (vs. 8)
CFQ: 8.4525 (vs. 2)

As one of the letters said, using the SD might not be reliable as they used a sample that was restricted to certain values.
It would be like testing a weight loss program where one said all the participants had to be between 10 stone/140lb/63.5kg and 12 stone/168lb/76Kg. The standard deviation would be small so if somebody lost 3lb/1.5Kg, that might come out as a success when really one probably needs a bigger value as too much of that change could have been due to chance and it didn't really prove it was a good program.

 

[anciendaze]

Perhaps our problem is just that we are misinterpreting what they said. For example, based on associated puffery in the press, I think we might find the following equivalence:

"rigorously controlled study" = "we wrote the conclusions first" :innocent1:

 

[Graham]

How else can you ensure rigorous control on a study if you don't write the conclusions first? If you aren't careful the results could make the last 20 years of your announcements look stupid! The real skill lies in getting paid vast sums for it.

Whoops - my carefully neutral and non-judgemental mask just slipped a bit. You won't tell anyone, will you?

 

[Dolphin]

Alastair M. Santhouse BMJ RR on PACE Trial

If anyone felt so inclined, it might be good to have a response or two up to Alastair Santhouse:
http://www.bmj.com/content/342/bmj.d3780.extract

It won't make it into the BMJ (they only accept letters within 5 days of publication) so it is probably not worth a big effort. Up to people themselves.

Research into CFS/ME

Alastair M. Santhouse, Consultant in Psychological Medicine
South London and Maudsley NHS Foundation Trust, SE5

Re: Dangers of research into chronic fatigue syndrome. Hawkes
342:doi:10.1136/bmj.d3780

There is an obvious paradox in wanting more research in CFS/ME, but then
rejecting research that one does not like, with claims that it is
unscientific. "Science" is the application of the scientific method, and the
sound rebuttals by both the MRC and Lancet to allegations that the PACE
trial was in some way improper should be proof enough. That cognitive
behavioural therapy (CBT) and graded exercise therapy (GET) have been shown
by the PACE trial to be safe whilst moderately improving outcomes for CFS/ME
would ordinarily be a cause of considerable satisfaction to both doctors and
patients. Certainly it would be so in any other branch of medicine. Entry
criteria to the trial were broad, but subgroup analysis on more narrowly
defined CFS/ME still produced the same results. Whilst these treatments do
not resolve the problem for everyone, undoubtedly they do benefit some. It
is precisely for these reasons that we have trials, in which treatments and
outcomes can be standardised. By all means let us have research into other
aspects of this condition, but let us also acknowledge the good intentions
of researchers and doctors involved in all aspects of this field.

Competing interests: Specialist medical care doctor, PACE trial; Member of
guideline development group for National Institute of Health and Clinical
Excellence (NICE) CFS/ME guidelines

Read all rapid responses published for this article
Submit rapid response
Published 4 July 2011

 

[Dolphin]

I have just been made aware of the following ongoing Freedom of Information request regarding the PACE Trial data: http://www.whatdotheyknow.com/request/request_for_raw_data_from_the_pa#outgoing-127757

I thought it might be better to start a separate thread on this in case anyone was interested. It's at: http://forums.phoenixrising.me/show...rial-data-Freedom-of-Information-request-(MRC)

That thread has now been updated.

 

[Graham]

If anyone felt so inclined, it might be good to have a response or two up to Alastair Santhouse:
http://www.bmj.com/content/342/bmj.d3780.extract

It won't make it into the BMJ (they only accept letters within 5 days of publication) so it is probably not worth a big effort. Up to people themselves.

I did a quick reply and it has been added. Not up to your standards of course, but at least it was a snap back. It could have been more cutting, but I thought I would just aim at creating a few doubts as to whether they had really looked at the data properly.

"As a mathematician with ME, I would like to point out that I do not reject the PACE study. I do however reject its conclusions, and that is very different.

It has taken me many hours of analysis to come to that conclusion: the data is complex. However an NNT of 7 or 8 for a heterogeneous group suggests that a generalised conclusion is inappropriate, particularly when there is no reliable quantitative assessment to back up the results."

Do I now join the esteemed band of "people with published responses"? And do I get a brownie point?

 

[Dolphin]

I did a quick reply and it has been added. Not up to your standards of course, but at least it was a snap back. It could have been more cutting, but I thought I would just aim at creating a few doubts as to whether they had really looked at the data properly.

"As a mathematician with ME, I would like to point out that I do not reject the PACE study. I do however reject its conclusions, and that is very different.

It has taken me many hours of analysis to come to that conclusion: the data is complex. However an NNT of 7 or 8 for a heterogeneous group suggests that a generalised conclusion is inappropriate, particularly when there is no reliable quantitative assessment to back up the results."

:Sign Good Job:

Do I now join the esteemed band of "people with published responses"? And do I get a brownie point?

I'm not sure what the context of "published responses" is you are referring to: although the BMJ make a bit of a fuss about e-letters, and they are useful, I think generally what would be considered a published response is something that is in a print edition and the like - the sort of thing that would show up in: http://www.ncbi.nlm.nih.gov/pubmed/ (or a few other databases).

But I think practice helps with letter writing - I had written a lot of e-letters before I submitted a letter for publication.

 

[Graham]

Thanks Dolphin. I know my place. Proper publication is another ball game altogether.

Thanks for the brownie! But brownie points are generally held to be points collected by members of the junior girl guides brigade for doing something good, although I am unlikely to refuse the cookie version. http://www.phrases.org.uk/meanings/brownie-points.html I spent a number of years teaching in an all-girls school (which was much tougher than an all-boys school), which is where I picked up what is, I believe, an American expression.

 

[Bob]

A humorous short animation on YouTube about the PACE Trial:
http://www.youtube.com/watch?v=-ILpjAMC2go

 

[biophile]

Can staunch CBT/GET proponents be "utterly impartial"?

I never understood why this particular editorial featured in the above YouTube video claimed that Hooper's criticism was "now available via Wikipedia". When I first had a look for it on Wikipedia (eg this article) I couldn't find it. When I had a look at today's current version of the above mentioned Wikipedia article, the previous criticism of the PACE trial using patient organisations websites as references has been replaced with a brief mention of the existence of Hooper's criticism etc, and referenced by, lo and behold, this same editorial as well as the BMJ article "Dangers of research into chronic fatigue syndrome":

Older text: Patient groups have criticized the trial for over-simplified and exaggerated conclusions, for using a flawed psychosocial illness model that ignores biological evidence, for testing a non-representative version of pacing, and because the results seriously conflict with their member surveys which show that pacing is effective and CBT or GET can cause deterioration in many patients who use the treatments.[32][33] The authors of the study suggest that bad advice to patients, such as being told to just go out and join a gym, could be responsible for bad experiences that some patients have with exercise therapy.[34]

Newer text: The trial generated a furious response from patient groups and campaigners. Professor Malcolm Hooper branded the results "unethical and unscientific" and submitted a 442 page response to the Medical Research Council and a shorter 43-page complaint to the Lancet. The MRC and the Lancet have considered the submissions and rejected them. The Lancet commented that there appears to be "an active campaign to discredit the research".[31][32]

I couldn't find Hooper's criticism anywhere on Wikipedia, AFAIK it has never been there and is still not "available" there as claimed by The Lancet. Am I missing something, or is it possible that editorials from The Lancet regarding ME/CFS are carrying on the fine tradition of sloppiness (remember the sloppy errors in first PACE editorial), with "Wikipedia" actually meaning "the internet"? Time for another "correction" to be issued?

The Lancet editorials on the subject are irritating. Implying that it is the critics who "made up their mind before the trial was published" while according to Horton on radio the PACE authors were "utterly impartial", is unbelievable for the following reasons:

The PACE authors are not casual independent observers as presented, they are notoriously staunch proponents of the psychosocial interpretation of ME/CFS, claim 25% cure rates from CBT/GET, and even designed the size of the PACE trial around an expected 60% response rate from CBT, 50% for GET, 25% for APT and 10% for SMC. The CBT and GET treatment manuals are more optimistic about recovery than the APT manuals. The Lancet claims it is unfair to accuse these PACE authors of potential conflicts of interest and "forming their opinion about the intended outcome before the trial began", but it is obvious what the authors expected from the trial and they also declared some of these potential conflicts of interest in The Lancet paper itself!

The PACE authors also oppose the pacing therapy that they supposedly tested "impartially" and modified it to the point where the ME/CFS community is universally saying it isn't pacing as they understand and practice it (even the patient organisation that "collaborated" with PACE to develop APT is saying it), meanwhile The Lancet states these authors "should be praised for their willingness to test competing ideas and interventions in a randomised trial." What they actually tested was a strawman, which doesn't deserve any praise.

When stating that the main authors (principle investigators) were "utterly impartial", Horton may have meant that they weren't involved in conducting the therapies themselves. But these same CBT/GET proponents designed the entire trial (ie not an independent endeavor) using methodology which has repeatedly been subject to criticism for being flawed, they then changed the previously established goalposts in a way which just happened to coincide with making the results look better than they would have otherwise been. Their reputations and possibly careers were at stake if the pet approach they have been promoting for 20+ years was deemed a failure. The health and livelihoods of patients are at stake here, there should be no mystery why the ME/CFS community is disputing flawed research that is hyped at their expense. Accusing patients/advocates of being inappropriately vested in the outcome while claiming the PACE authors were "utterly impartial" is irresponsible.

 

[anciendaze]

Newer text: The trial generated a furious response from patient groups and campaigners. Professor Malcolm Hooper branded the results "unethical and unscientific" and submitted a 442 page response to the Medical Research Council and a shorter 43-page complaint to the Lancet. The MRC and the Lancet have considered the submissions and rejected them. The Lancet commented that there appears to be "an active campaign to discredit the research".

Let's try for a simple objective correction.

Sample:

The hypothesis used as the basis for the PACE trial was that CFS patients are part of the general population without physical illness. The reference to SF-36 physical measures shows a distribution for scores in the general population with a mean of 85 and a mode of 95. This is not a normal distribution. Q.E.D.

Please inform readers that the statistical significance of results should be considered only as valid as the assertion 85 = 95.