May 12, 2017 Is International ME/CFS and FM Awareness Day
International ME/CFS and FM Awareness Day is May 12th, 2017. Jody Smith shares some information about upcoming events and ways you can be heard ...
Discuss the article on the Forums.

PACE Trial Non-Starter Flaws - in a nutshell please

Discussion in 'General ME/CFS Discussion' started by Barry53, Jun 16, 2017.

  1. Barry53

    Barry53 Senior Member

    Messages:
    2,276
    Likes:
    13,677
    UK
    I would be interested to compile a list of the most blatant PACE "non-starter flaws" that have been identified ... but with a twist. The "headline" in a short'ish sentence, followed by a nice simple illustration so lay people - non-medical, non-technical - can easily see what the essential problem is ... or maybe people qualified to understand, but too little time or inclination to!

    I've started off with two, to give the idea:-
    • Unblinded trial combined with subjective outcome measures.

      Akin to trialling which fresh carrots people think taste best after they have prepared and cooked them, but having to leave all the labels on so participants know if they are from a supermarket, local allotment society, organic farm, etc.
    • Very poor case definition criteria, inevitably including other than ME sufferers, yet it recommended treatments presumed safe/relevant for ME.

      Akin to researching the migratory routes of ospreys, but radio-tagging not just ospreys, but also other birds which likely migrate as per researchers’ pet theory. Then interpret and apply the results as if all the tagged birds were ospreys.
     
    SamanthaJ, AndyPR, Snowdrop and 3 others like this.
  2. A.B.

    A.B. Senior Member

    Messages:
    3,724
    Likes:
    23,023
    That is incomprehensible to the average person. One could say that the trial was setup so that it was impossible to clearly distinguish between wishful thinking and real improvement in health.

    Even worse, it was design to enhance wishful thinking in the CBT and GET groups.
     
    SamanthaJ, AndyPR, Valentijn and 4 others like this.
  3. MEMum

    MEMum Senior Member

    Messages:
    391
    Likes:
    2,016
    Hi Barry
    This is something I prepared earlier to hand out to friends, hope it helps.

    · Broad Entry Criteria

    Using the Oxford definition meant that anyone with longstanding fatigue could be included, not only those who have the additional symptoms specific to ME.

    Some of those with persistent fatigue arising from different conditions, such as depression, may well be helped by GET/CBT, (Graded Exercise/Cognitive Behavioural Therapy).


    · Altered definition of ‘recoveryduring the trial

    A physical health questionnaire was used. The higher the number, the better someone’s physical ability. Initially, people with a score of 65 or less were eligible for the trial, and a score of 75 was set as the minimum for recovery at the end. However, by the time of publication, this recovery threshold had been reduced to 60.

    Thus, there could be people who had entered the trial with a score of 65, deteriorated to a level of 60 who would be counted as supposedly having ‘recovered’. This is despite having a poorer physical function than 92% of the working age population.

    One of the authors of the paper said at a press event that “twice as many people on GET/CBT got back to normal”. THIS DOES NOT SOUND LIKE NORMAL TO ME.


    · Objective tests of physical health dropped

    Most of these proposed tests were dropped/not reported on for a variety of dodgy reasons. They said they would use an actometer, a step test, income from work/benefits, whether participants had been able to return to work.

    Results from the 6 minute walking test showed a small increase for the GET group. At<400 metres this was still way off the distance for a similar aged adult, who would average 644m in 6mins. People with ME were still unable to walk as far as a person with serious heart or lung disease.


    · Conflict of interest

    The fact that two of the lead investigators earned money from the insurance industry, relating to consultancy on payouts to people with disabling illnesses, was not disclosed to the participants.


    · A newsletter promoting the benefits of GET/CBT was produced during the trial

    GET/CBT was recommended as the only evidence-based treatment, with glowing patient testimonials.


    · Wrong stats used

    Analysis of mean/standard deviation is intended to be used with a normal, ie bell-shaped distribution. Distribution of physical ability of a country’s population will be skewed to the right. This is because the majority of working age people will have a high score- of 85-100%, with only a few, with very low scores.
     
    AndyPR, Jenny TipsforME, Sean and 2 others like this.
  4. Barry53

    Barry53 Senior Member

    Messages:
    2,276
    Likes:
    13,677
    UK
    Agreed, which is why the explanatory example is intended to follow each heading. But maybe an additional sentence to "translate" each heading. I want to still keep headings that tie in to the PACE-critique literature, so the correlation is not lost.
     
    MEMum and BurnA like this.
  5. Barry53

    Barry53 Senior Member

    Messages:
    2,276
    Likes:
    13,677
    UK
    Thanks :)
     
    MEMum likes this.
  6. ScottTriGuy

    ScottTriGuy Stop the harm. Start the research and treatment.

    Messages:
    1,074
    Likes:
    4,325
    Toronto, Canada
    Perhaps move that to be the first bullet point as the other points emerge from this fact - and people usually remember the first and the last items in a list (unless there is something very emotive).
     
  7. Jonathan Edwards

    Jonathan Edwards "Gibberish"

    Messages:
    5,211
    Likes:
    31,641
    I am afraid I am responsible for the incomprehensible bit. Trouble is, in technical terms, this is the basic issue. My attempt at a metaphor would be:

    Akin to comparing reading glasses off the shelf from Tesco with rose tinted prescription glasses from an expensive optician and telling people in advance where the specs came from and that the rose tinting helps; then scoring the result by how rosy people say the world looks rather than by getting them to read an eye chart!
     
    SamanthaJ, Forbin, Valentijn and 10 others like this.
  8. user9876

    user9876 Senior Member

    Messages:
    2,590
    Likes:
    18,258
    moderated with many others
    The way I think of it is to say that two of the trial arms said how ill are you; if you ignore your symptoms (and do things) you will get better and then asked how ill are you now. The others said how ill are you, doing this may help natural recovery; how ill are you. The first two had people say they were more slightly better than the second two. (unsurprisingly).
     
    MEMum likes this.
  9. Barry53

    Barry53 Senior Member

    Messages:
    2,276
    Likes:
    13,677
    UK
    Part of my motivation here is that a good trial will constitute a whole suite of methodological requirements, each of them essential to the validity of the trial as a whole; failure of any one of these therefore rendering PACE a non-starter. The thing that seems to stand out with PACE, is that it didn't just suffer from a non-starter ... but was flooded with them! Any one should have killed it off, but even with loads it seems to lead a charmed life for so long.

    At this moment I'm not necessarily trying to prioritise them, just note them.
     
    MEMum and ScottTriGuy like this.
  10. Chezboo

    Chezboo NOT MY BOARD

    Messages:
    55
    Likes:
    390
    You could deteriorate, become sicker and more disabled during the trial and still be counted as ‘recovered’. The authors described this as “a comprehensive and conservative definition of recovery”.


    Harms were not adequately reported which resulted in them being fudged over and ignored. This is shocking enough in any trial and wouldn’t be tolerated in a drug trial. In a condition defined by exacerbation of symptoms and relapse from exertion, and where patient surveys and feedback suggests GET harms at least to 70% to 80% of people who undertake it, it is equally as shocking to fudge over the harms experienced.
     
    MEMum, ScottTriGuy, Valentijn and 2 others like this.
  11. trishrhymes

    trishrhymes Save PR. Sack the President of the Board.

    Messages:
    2,155
    Likes:
    17,892
    On conflict of interest, a parallel might be using doctors in the pay of tobacco companies setting up a trial in such a way as to prove cigarettes are good for you, spending months persuading patients that any symptoms they get like coughing or shortness of breath are a sign that the lungs are healthy, then getting them to fill in questionnaires asking whether they think their symptoms are improving, without measuring lung function or doing chest X rays. And not telling patients that the doctors are being paid by the tobacco company.
     
    Barry53, MEMum, ScottTriGuy and 5 others like this.
  12. trishrhymes

    trishrhymes Save PR. Sack the President of the Board.

    Messages:
    2,155
    Likes:
    17,892
    On too broad a definition of MECFS, I think the parallel has already been used of doing a study of a drug or diet for diabetes, and when it works for some of the sample, claiming it a success.

    This could be very dangerous for Type 1 diabetes, but work for some people with type 2.

    Like exercise can help people with fatigue due to depression, but be dangerous for people with ME.

    No study should be done on everyone with any type of diabetes all mixed in together, nor should one be done with all fatigue patients mixed together. The results are equally meaningless and potentially dangerous.
     
  13. Jenny TipsforME

    Jenny TipsforME Senior Member

    Messages:
    1,119
    Likes:
    3,755
    Bristol
    (perhaps a slight edit "not a sign that lungs are unhealthy "?)

    These two are excellent analogies! I think there may be a better one than diabetes but not generating new ideas right now.

    BTW I think @ScottTriGuy point about Primacy-Recency effect is relevant when a list is finalised. PACE proponents will know these things so we need to be up on these Persuasion and psychology tools to hope to be convincing. Clearly if it was down to the factual merits of a trial PACE wouldn't have even got published.
     
    Barry53 and ScottTriGuy like this.
  14. Jenny TipsforME

    Jenny TipsforME Senior Member

    Messages:
    1,119
    Likes:
    3,755
    Bristol
    Also thinking of Persuasion techniques, starting with conflict of interest may set it up as a "the system is broken" scenario in readers' minds, which is helpful.

    Rather than "this is their perspective and this is why we disagree", which works as an academic convention, but doesn't work in terms of being persuasive, because readers have to decide who is more credible. Even if they decide PACE proponents aren't necessarily more credible than us, the likelihood is they are left unsure about the whole thing, rather than being convinced by us.
     
    SamanthaJ, Barry53 and ScottTriGuy like this.
  15. Jenny TipsforME

    Jenny TipsforME Senior Member

    Messages:
    1,119
    Likes:
    3,755
    Bristol
    On the too broad a definition one, pain I think could be a good alternative to Fatigue because it can be a major symptom in completely different conditions.

    Perhaps something like investigating Tai Chi movement for treating broken legs, but including patients by whether or not they have leg pain, not differentiating whether it is muscle based or bone based. Then at the end saying Tai Chi improves broken legs.

    This could be improved.
     
    Barry53 and ScottTriGuy like this.
  16. ScottTriGuy

    ScottTriGuy Stop the harm. Start the research and treatment.

    Messages:
    1,074
    Likes:
    4,325
    Toronto, Canada
    Yes, an awesome analogy familiar to almost everyone - I think I'm going to use this to drive the point home to the lay person - I made a couple of tweaks, what do you think?

    "a parallel might be using doctors in the pay of tobacco companies setting up a trial in such a way as to prove cigarettes are good for patients with lung cancer, spending months persuading patients that any symptoms they get like coughing or shortness of breath are a sign that the lungs are healthy, telling them that other patients are getting better with their cigarette therapy, and then getting them to fill in questionnaires asking whether they think their symptoms are improving, without measuring lung function or doing chest X rays, and then manipulating the data to prove cigarette therapy helps patients with lung cancer. All the while not telling patients that the doctors are being paid by the tobacco company, and have built their careers and reputations on the idea of cigarette therapy."
     
  17. Esther12

    Esther12

    Messages:
    8,454
    Likes:
    28,553
    You could report declines for the trial's two primary outcomes and be classed as recovered, but, for the Psych Med 2013 recovery critieria patients also had to rate themselves as 'much better' or 'very much better', and were not allowed to fulfil every aspect of the entry criteria for the trial.

    The be decribed as 'recovered' in the Lancet commentary that accompanied the PACE paper in 2011 patients could report declines for everything.
     

See more popular forum discussions.

Share This Page