TRIAL BY ERROR: The Troubling Case of the PACE Chronic Fatigue Syndrome Study

Daisymay

Senior Member
Messages
754
Likes
4,060
Excellent article by Dave Tuller on the PACE trial. Here's the summary and the link.It says there is a second half to come tomorrow.

http://www.virology.ws/2015/10/21/trial-by-error-i/

TRIAL BY ERROR: The Troubling Case of the PACE Chronic Fatigue Syndrome Study

21 OCTOBER 2015

By David Tuller, DrPH

SUMMARY

This examination of the PACE trial of chronic fatigue syndrome identified several major flaws:

*The study included a bizarre paradox: participants’ baseline scores for the two primary outcomes of physical function and fatigue could qualify them simultaneously as disabled enough to get into the trial but already “recovered” on those indicators–even before any treatment. In fact, 13 percent of the study sample was already “recovered” on one of these two measures at the start of the study.

*In the middle of the study, the PACE team published a newsletter for participants that included glowing testimonials from earlier trial subjects about how much the “therapy” and “treatment” helped them. The newsletter also included an article informing participants that the two interventions pioneered by the investigators and being tested for efficacy in the trial, graded exercise therapy and cognitive behavior therapy, had been recommended as treatments by a U.K. government committee “based on the best available evidence.” The newsletter article did not mention that a key PACE investigator was also serving on the U.K. government committee that endorsed the PACE therapies.

*The PACE team changed all the methods outlined in its protocol for assessing the primary outcomes of physical function and fatigue, but did not take necessary steps to demonstrate that the revised methods and findings were robust, such as including sensitivity analyses. The researchers also relaxed all four of the criteria outlined in the protocol for defining “recovery.” They have rejected requests from patients for the findings as originally promised in the protocol as “vexatious.”

*The PACE claims of successful treatment and “recovery” were based solely on subjective outcomes. All the objective measures from the trial—a walking test, a step test, and data on employment and the receipt of financial information—failed to provide any evidence to support such claims. Afterwards, the PACE authors dismissed their own main objective measures as non-objective, irrelevant, or unreliable.

*In seeking informed consent, the PACE authors violated their own protocol, which included an explicit commitment to tell prospective participants about any possible conflicts of interest. The main investigators have had longstanding financial and consulting ties with disability insurance companies, having advised them for years that cognitive behavior therapy and graded exercise therapy could get claimants off benefits and back to work. Yet prospective participants were not told about any insurance industry links and the information was not included on consent forms. The authors did include the information in the “conflicts of interest” sections of the published papers.

Top researchers who have reviewed the study say it is fraught with indefensible methodological problems. Here is a sampling of their comments:

Dr. Bruce Levin, Columbia University: “To let participants know that interventions have been selected by a government committee ‘based on the best available evidence’ strikes me as the height of clinical trial amateurism.”

Dr. Ronald Davis, Stanford University: “I’m shocked that the Lancet published it…The PACE study has so many flaws and there are so many questions you’d want to ask about it that I don’t understand how it got through any kind of peer review.”

Dr. Arthur Reingold, University of California, Berkeley: “Under the circumstances, an independent review of the trial conducted by experts not involved in the design or conduct of the study would seem to be very much in order.”

Dr. Jonathan Edwards, University College London: “It’s a mass of un-interpretability to me…All the issues with the trial are extremely worrying, making interpretation of the clinical significance of the findings more or less impossible.”

Dr. Leonard Jason, DePaul University: “The PACE authors should have reduced the kind of blatant methodological lapses that can impugn the credibility of the research, such as having overlapping recovery and entry/disability criteria.”
 

Sasha

Fine, thank you
Messages
17,863
Likes
34,236
Location
UK
This looks BRILLIANT. Thanks so much to David Tuller.

In the intro, he says:

David Tuller said:
Much of what I report here will not be news to the patient and advocacy communities, which have produced a voluminous online archive of critical commentary on the PACE trial. I could not have written this piece without the benefit of that research and the help of a few statistics-savvy sources who talked me through their complicated findings.


Congratulations to those expert patients who have been posting for years about the problems with PACE and shame on the medical establishment for not recognising that you were being better scientists than they were.

The PACE trial is a disgrace and has been a bane on the lives of ME patients in the UK. Very glad to see it getting the treatment it deserves.

Thanks, @Jonathan Edwards, for what you said to David Tuller for his piece:

David Tuller said:
Jonathan Edwards, a professor emeritus of connective tissue medicine from University College London, pioneered a novel rheumatoid arthritis treatment in a large clinical trial published in theNew England Journal of Medicine in 2004. For the last couple of years, he has been involved in organizing clinical trial research to test the same drug, rituximab, for chronic fatigue syndrome, which shares traits with rheumatoid arthritis and other autoimmune disorders.

When he first read the Lancet paper, Edwards was taken aback: Not only did the trial rely on subjective measures, but participants and therapists all knew which treatment was being administered, unlike in a double-blinded trial. This unblinded design made PACE particularly vulnerable to generating biased results, said Edwards in a phone interview, adding that the newsletter testimonials and other methodological flaws only made things worse.

“It’s a mass of un-interpretability to me,” said Edwards, who last year called the PACE results “valueless” in publicly posted comments. “Within the circle who are involved in this field, it seems there were a group who were prepared to all sing by the hymn sheet and agree that PACE was wonderful. But all the issues with the trial are extremely worrying, making interpretation of the clinical significance of the findings more or less impossible.”
I hope we can see the MRC and others sitting up and taking notice of what the PACE trial authors have done to their £5 million trial. I'd like to know if the Trial Steering Committee were aware of what has been done in their name.
 

user9876

Senior Member
Messages
4,556
Likes
18,068
I hope we can see the MRC and others sitting up and taking notice of what the PACE trial authors have done to their £5 million trial. I'd like to know if the Trial Steering Committee were aware of what has been done in their name.
It is really unclear as to who approved what changes to the protocol, when and what information was given. All we are allowed to know is what is published in the papers and their reasoning for changing things is largely wrong or at best would count as tinkering. A valid reason for changing a protocol would be a particular test required a given distribution which was not seen in the data. But this was not the case here.

Ideally those independent people on the steering committee would speak out and say 'yes we really did approve changes including allowing recovery to be defined below the trial entry criteria' or 'no we didn't approve that' or even 'we didn't realize that was the case'.
 
Messages
2,239
Likes
16,240
This is how the MEA reported the results of the PACE trial back in February 2011:

http://www.meassociation.org.uk/2011/02/me-association-press-statement-on-the-pace-trial-results/

People with more distant memories will recall that the MEA was opposed to this trial even taking place right from the very start and in 2004 organised a petition against it - see reference 18 in this review of the planning, implementation and pubication of the PACE trial from Peter White:

http://pb.rcpsych.org/content/39/1/24

Our most recent (2015) report of patient evidence relating to the safety, effectiveness and acceptability of CBT, GET and Pacing continues to demonstrate that CBT is often ineffective; that many people with ME/CFS are made worse as a result of GET; and that the most acceptable and effective form of activity/energy management is Pacing

Following publication of this latest MEA report on CBT, GET and Pacing report we have called on NICE to review their guideline on ME/CFS and remove GET from their list of recommended treatments for everyone with mild or moderate ME/CFS

MEA report on CBT, GET and Pacing:

http://www.meassociation.org.uk/how-you-can-help/fundraising-support/

Dr Charles Shepherd
Hon Medical Adviser, MEA
 

Esther12

Senior Member
Messages
13,774
Likes
28,359
@charles shepherd : I also remember Horton attacking you for your criticism of the trial, and saying you were letting down patients.

This bit seems so typical of the biopsychosocial lot:

(I did have a telephone conversation with Chalder in January of this year, organized as part of the media campaign for the Lancet Psychiatry paper published that month by the PACE team. In Chalder’s memory of the conversation, we talked at length about some of the major concerns examined here. In my memory, she mostly declined to talk about concerns related to the 2011 Lancet paper, pleading poor recall of the details.)
Confident and persuasive with those who don't know what they're talking about, forgetful and evasive when they come up against someone who has done some independent research.
 

snowathlete

Senior Member
Messages
5,374
Likes
14,600
Location
UK
Very good. Thanks to Dave Tuller and those involved in all the work that came before it.
It's a very good summary...and a great title too.

The first point alone is enough from a research-ethics point of view to reasonably expect the paper to be withdrawn.
 

Valentijn

Senior Member
Messages
15,786
Likes
45,671
And of course who would play @Tom Kindlon
It would have to be an Irish actor of course, so I'll nominate Cillian Murphy. He can even rock the Kindlon beard:



What would the final outcome be (withdrawal of the research?)
Court ordered psychotherapy for the psychobabblers, to address their inability to accept reality :D
 

Esther12

Senior Member
Messages
13,774
Likes
28,359
As I was reading it, (particularly the bit about Tom Kindlon), I could see the idea of a film script cutting between the different groups involved as the trial was planned, run and spun.
Colin Farrell?

I've just finished the piece (part one anyway). Great to have a piece which actually took the time to examine whether patient's concerns were valid or not, rather than just assuming we were all driven by prejudices about psychiatry or mental health interventions.

I wondered if maybe it tried to cover too much? There are so many complicated issues around PACE that trying to put them all in could be confusing for a lot of readers?

I can also understand why Tuller would want to put it all in - there's a lot of irritating stuff about PACE, it seems that he has realised this, and realised how poorly treated patient's concerns have been. It seems like he's the only journalist to take the time to actually do some research and look at the evidence for themselves, so he could easily feel that if he left it out, no-one else would cover it? We're very lucky he's been willing to do so much work, and take this on.

Hopefully this will encourage others to take the time to look at the evidence, rather than just trust those with titles.
 

user9876

Senior Member
Messages
4,556
Likes
18,068
I can also understand why Tuller would want to put it all in - there's a lot of irritating stuff about PACE, it seems that he has realised this, and realised how poorly treated patient's concerns have been. It seems like he's the only journalist to take the time to actually do some research and look at the evidence for themselves, so he could easily feel that if he left it out, no-one else would cover it? We're very lucky he's been willing to do so much work, and take this on.

Hopefully this will encourage others to take the time to look at the evidence, rather than just trust those with titles.
The problem is it requires quite a lot of research and background knowledge to do that type of research. I'm not sure there are many journalists set up to do that. Its great that Tuller has put the work in and by the sounds of when interviews were refused by White and friends he started last year. Its obviously taken a lot of effort.
 

Esther12

Senior Member
Messages
13,774
Likes
28,359
Also, thanks to the academics willing to go on the record with the concerns. I've started to forget that @Jonathan Edwards isn't just another patient here, but is only involved to try to help others. PACE is such a mess that it must be hard for anyone to drag themselves through it without a direct personal incentive.
 

Tom Kindlon

Senior Member
Messages
1,683
Likes
8,838
(Off-topic)

user9876 said:
As I was reading it, (particularly the bit about Tom Kindlon), I could see the idea of a film script cutting between the different groups involved as the trial was planned, run and spun.
Colin Farrell?
Colin Farrell actually grew up in Castleknock also, around 1km from where I grew up (and still live). I knew people down where he lived and I played football/soccer on lots of greens in Castleknock so knew hundreds of guys around my age. Unfortunately I can't remember him (though know people who knew him when he was young).