• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

PACE Trial Non-Starter Flaws - in a nutshell please

Messages
2,391
Location
UK
I would be interested to compile a list of the most blatant PACE "non-starter flaws" that have been identified ... but with a twist. The "headline" in a short'ish sentence, followed by a nice simple illustration so lay people - non-medical, non-technical - can easily see what the essential problem is ... or maybe people qualified to understand, but too little time or inclination to!

I've started off with two, to give the idea:-
  • Unblinded trial combined with subjective outcome measures.

    Akin to trialling which fresh carrots people think taste best after they have prepared and cooked them, but having to leave all the labels on so participants know if they are from a supermarket, local allotment society, organic farm, etc.
  • Very poor case definition criteria, inevitably including other than ME sufferers, yet it recommended treatments presumed safe/relevant for ME.

    Akin to researching the migratory routes of ospreys, but radio-tagging not just ospreys, but also other birds which likely migrate as per researchers’ pet theory. Then interpret and apply the results as if all the tagged birds were ospreys.
 

MEMum

Senior Member
Messages
440
Hi Barry
This is something I prepared earlier to hand out to friends, hope it helps.

· Broad Entry Criteria

Using the Oxford definition meant that anyone with longstanding fatigue could be included, not only those who have the additional symptoms specific to ME.

Some of those with persistent fatigue arising from different conditions, such as depression, may well be helped by GET/CBT, (Graded Exercise/Cognitive Behavioural Therapy).


· Altered definition of ‘recoveryduring the trial

A physical health questionnaire was used. The higher the number, the better someone’s physical ability. Initially, people with a score of 65 or less were eligible for the trial, and a score of 75 was set as the minimum for recovery at the end. However, by the time of publication, this recovery threshold had been reduced to 60.

Thus, there could be people who had entered the trial with a score of 65, deteriorated to a level of 60 who would be counted as supposedly having ‘recovered’. This is despite having a poorer physical function than 92% of the working age population.

One of the authors of the paper said at a press event that “twice as many people on GET/CBT got back to normal”. THIS DOES NOT SOUND LIKE NORMAL TO ME.


· Objective tests of physical health dropped

Most of these proposed tests were dropped/not reported on for a variety of dodgy reasons. They said they would use an actometer, a step test, income from work/benefits, whether participants had been able to return to work.

Results from the 6 minute walking test showed a small increase for the GET group. At<400 metres this was still way off the distance for a similar aged adult, who would average 644m in 6mins. People with ME were still unable to walk as far as a person with serious heart or lung disease.


· Conflict of interest

The fact that two of the lead investigators earned money from the insurance industry, relating to consultancy on payouts to people with disabling illnesses, was not disclosed to the participants.


· A newsletter promoting the benefits of GET/CBT was produced during the trial

GET/CBT was recommended as the only evidence-based treatment, with glowing patient testimonials.


· Wrong stats used

Analysis of mean/standard deviation is intended to be used with a normal, ie bell-shaped distribution. Distribution of physical ability of a country’s population will be skewed to the right. This is because the majority of working age people will have a high score- of 85-100%, with only a few, with very low scores.
 
Messages
2,391
Location
UK
That is incomprehensible to the average person. One could say that the trial was setup so that it was impossible to clearly distinguish between wishful thinking and real improvement in health.

Even worse, it was design to enhance wishful thinking in the CBT and GET groups.
Agreed, which is why the explanatory example is intended to follow each heading. But maybe an additional sentence to "translate" each heading. I want to still keep headings that tie in to the PACE-critique literature, so the correlation is not lost.
 

Jonathan Edwards

"Gibberish"
Messages
5,256
Agreed, which is why the explanatory example is intended to follow each heading. But maybe an additional sentence to "translate" each heading. I want to still keep headings that tie in to the PACE-critique literature, so the correlation is not lost.

I am afraid I am responsible for the incomprehensible bit. Trouble is, in technical terms, this is the basic issue. My attempt at a metaphor would be:

Akin to comparing reading glasses off the shelf from Tesco with rose tinted prescription glasses from an expensive optician and telling people in advance where the specs came from and that the rose tinting helps; then scoring the result by how rosy people say the world looks rather than by getting them to read an eye chart!
 

user9876

Senior Member
Messages
4,556
Unblinded trial combined with subjective outcome measures.

Akin to trialling which fresh carrots people think taste best after they have prepared and cooked them, but having to leave all the labels on so participants know if they are from a supermarket, local allotment society, organic farm, etc.

The way I think of it is to say that two of the trial arms said how ill are you; if you ignore your symptoms (and do things) you will get better and then asked how ill are you now. The others said how ill are you, doing this may help natural recovery; how ill are you. The first two had people say they were more slightly better than the second two. (unsurprisingly).
 
Messages
2,391
Location
UK
Perhaps move that to be the first bullet point as the other points emerge from this fact - and people usually remember the first and the last items in a list (unless there is something very emotive).
Part of my motivation here is that a good trial will constitute a whole suite of methodological requirements, each of them essential to the validity of the trial as a whole; failure of any one of these therefore rendering PACE a non-starter. The thing that seems to stand out with PACE, is that it didn't just suffer from a non-starter ... but was flooded with them! Any one should have killed it off, but even with loads it seems to lead a charmed life for so long.

At this moment I'm not necessarily trying to prioritise them, just note them.
 

Chezboo

NOT MY BOARD
Messages
55
You could deteriorate, become sicker and more disabled during the trial and still be counted as ‘recovered’. The authors described this as “a comprehensive and conservative definition of recovery”.


Harms were not adequately reported which resulted in them being fudged over and ignored. This is shocking enough in any trial and wouldn’t be tolerated in a drug trial. In a condition defined by exacerbation of symptoms and relapse from exertion, and where patient surveys and feedback suggests GET harms at least to 70% to 80% of people who undertake it, it is equally as shocking to fudge over the harms experienced.
 
Messages
2,158
On conflict of interest, a parallel might be using doctors in the pay of tobacco companies setting up a trial in such a way as to prove cigarettes are good for you, spending months persuading patients that any symptoms they get like coughing or shortness of breath are a sign that the lungs are healthy, then getting them to fill in questionnaires asking whether they think their symptoms are improving, without measuring lung function or doing chest X rays. And not telling patients that the doctors are being paid by the tobacco company.
 
Messages
2,158
On too broad a definition of MECFS, I think the parallel has already been used of doing a study of a drug or diet for diabetes, and when it works for some of the sample, claiming it a success.

This could be very dangerous for Type 1 diabetes, but work for some people with type 2.

Like exercise can help people with fatigue due to depression, but be dangerous for people with ME.

No study should be done on everyone with any type of diabetes all mixed in together, nor should one be done with all fatigue patients mixed together. The results are equally meaningless and potentially dangerous.
 

Jenny TipsforME

Senior Member
Messages
1,184
Location
Bristol
scoring the result by how rosy people say the world looks rather than by getting them to read an eye chart!

persuading patients that any symptoms they get like coughing or shortness of breath are a sign that the lungs are healthy, then getting them to fill in questionnaires asking whether they think their symptoms are improving, without measuring lung function or doing chest X rays
(perhaps a slight edit "not a sign that lungs are unhealthy "?)

These two are excellent analogies! I think there may be a better one than diabetes but not generating new ideas right now.

BTW I think @ScottTriGuy point about Primacy-Recency effect is relevant when a list is finalised. PACE proponents will know these things so we need to be up on these Persuasion and psychology tools to hope to be convincing. Clearly if it was down to the factual merits of a trial PACE wouldn't have even got published.
 

Jenny TipsforME

Senior Member
Messages
1,184
Location
Bristol
Also thinking of Persuasion techniques, starting with conflict of interest may set it up as a "the system is broken" scenario in readers' minds, which is helpful.

Rather than "this is their perspective and this is why we disagree", which works as an academic convention, but doesn't work in terms of being persuasive, because readers have to decide who is more credible. Even if they decide PACE proponents aren't necessarily more credible than us, the likelihood is they are left unsure about the whole thing, rather than being convinced by us.
 

Jenny TipsforME

Senior Member
Messages
1,184
Location
Bristol
On the too broad a definition one, pain I think could be a good alternative to Fatigue because it can be a major symptom in completely different conditions.

Perhaps something like investigating Tai Chi movement for treating broken legs, but including patients by whether or not they have leg pain, not differentiating whether it is muscle based or bone based. Then at the end saying Tai Chi improves broken legs.

This could be improved.
 

ScottTriGuy

Stop the harm. Start the research and treatment.
Messages
1,402
Location
Toronto, Canada
a parallel might be using doctors in the pay of tobacco companies setting up a trial in such a way as to prove cigarettes are good for you, spending months persuading patients that any symptoms they get like coughing or shortness of breath are a sign that the lungs are healthy, then getting them to fill in questionnaires asking whether they think their symptoms are improving, without measuring lung function or doing chest X rays. And not telling patients that the doctors are being paid by the tobacco company

Yes, an awesome analogy familiar to almost everyone - I think I'm going to use this to drive the point home to the lay person - I made a couple of tweaks, what do you think?

"a parallel might be using doctors in the pay of tobacco companies setting up a trial in such a way as to prove cigarettes are good for patients with lung cancer, spending months persuading patients that any symptoms they get like coughing or shortness of breath are a sign that the lungs are healthy, telling them that other patients are getting better with their cigarette therapy, and then getting them to fill in questionnaires asking whether they think their symptoms are improving, without measuring lung function or doing chest X rays, and then manipulating the data to prove cigarette therapy helps patients with lung cancer. All the while not telling patients that the doctors are being paid by the tobacco company, and have built their careers and reputations on the idea of cigarette therapy."
 
Messages
13,774
You could deteriorate, become sicker and more disabled during the trial and still be counted as ‘recovered’. The authors described this as “a comprehensive and conservative definition of recovery”.

You could report declines for the trial's two primary outcomes and be classed as recovered, but, for the Psych Med 2013 recovery critieria patients also had to rate themselves as 'much better' or 'very much better', and were not allowed to fulfil every aspect of the entry criteria for the trial.

The be decribed as 'recovered' in the Lancet commentary that accompanied the PACE paper in 2011 patients could report declines for everything.