Information Commissioner's Office orders release of PACE trial data

[Dolphin]

Yes, it was less GET than pacing done badly.

Can someone explain to me the difference in the treatments between the GET arm and the APT arm in the PACE study? If the GET is not typical GET, but pacing done badly, then what was APT? Pacing done well?

Why do they have to make up their own definitions of everything? It makes it impossible to figure out what's going on. If their GET is not what everyone understands as GET, and CBT is not the CBT the rest of the world uses, and SMC is not medical care, but no treatment for ME, and pacing is goodness knows what, how is the world supposed to interpret the results of the PACE trial?

Yeah, I know....

I think the GET in the GET manual is fairly typical GET. However, that doesn't mean that that is what was done in practice in the trial as the quotes I mention suggest:

For what it's worth, my physio told me that if I felt myself crashing due to something other than the GET, I should cut back to 50% and definately not keep going at the same rate. If I felt that the GET was too much, I cut back to a previous baseline that hadn't caused any side-effects.

"If your physio won't listen to you, the easiest thing to do is lie . For example, if 10 minutes of housework uses the same amount of energy as a 10 minute walk, then remove that amount from your set exercise and claim you just did the walking."

"Look up the term Active Pacing, I think it has been discussed here a few times. It was what I aimed for with my physio and we used things I wanted to do like housework to count towards my daily "exercise". She realised that these things use up energy too, and that energy had to be accounted for."

So the GET arm participant could have been doing some sort of pacing in practice.

Here's a table from one of the official PACE Trial manuals

 

[Dolphin]

...and had received millions of pounds of public to support the research in the first place...yet they cry poor on a few hundred pounds, if that!

In December 2013, they turned down a request for the positive outcomes (i.e. the original primary outcomes) saying:

For your information the appropriate limit is £450, calculated as the estimated cost of one person spending 18 hours in determining whether the information is held, then locating, retrieving and extracting the information. Section 12 of FOIA therefore makes provision for public authorities to refuse such requests.

The processes would include work of a statistician to perform the various programming and data file operations as well as the calculations to produce accurate data. Moreover, as there is no longer a statistician employed by the PACE trial, one would need to be recruited for this operation and trained.

https://www.whatdotheyknow.com/request/pace_trial_recovery_rates_and_po_2

Yet they have published papers since then and have more in the works. They said this in a letter written earlier this year (2015):

We intend to further explore multiple mediator effects and the potential effects of outcomes on mediators in future papers.

These are complicated issues requiring a statistician.
Hard to believe there was nobody who could have answered the query.

 

[SOC]

I think the GET in the GET manual is fairly typical GET. However, that doesn't mean that that is what was done in practice in the trial as the quotes I mention suggest:

So the GET arm participant could have been doing some sort of pacing in practice.

Here's a table from one of the official PACE Trial manuals
View attachment 13543

That looks like one hot mess with much too much overlap between treatment arms to clearly distinguish the effects of any specific treatment. Patients in the APT arm didn't increase activity when they felt they could? Of course they did. So they increased activity when they felt able. Patients in the GET arm only increased activity when they felt they could? It appears so. So they increased activity when they felt able. The difference? I'm not sure. Perhaps it was only in the psychological manipulation used in the different arms? Were the patients told different things about the consequences of exercise and/or increased activity? In that case they are not studying the effect of GET, as they reported, but the effect of psychological manipulation of patients. Which, I'm glad to say, doesn't seem to have worked, despite their glowing reports of efficacy.

This is SO far from gold-standard evidence-based research it's laughable. It's like the hyperbolic example used in a classroom to show what is NOT well-designed, high-quality research.

I know, I'm preaching to the choir....

 

[Sean]

Does that suggest there should be data for the first half? Or would they have been allowed to delete it?

If they have not deleted it (and I would be surprised if they had been allowed to do that), then there still should be baseline data for actometers.

 

[Dolphin]

Well there's another horrifying thought. If they had used the actimeters and found lack of improvement in fitness in ME patients, they might have accused them of not complying with the treatment, even if the patients had been diligently following protocol. Yet another way to mess with patients' minds.

Patient: "Really, doctor, I did everything you said. I increased exercise a tiny amount every week, even when it made me feel much worse. I want very much for this to work and make me more functional, but it isn't."
Therapist: "That has to be untrue because we see no improvement in fitness. Exercise must result in the improvement of fitness, therefore you are lying. You are not compliant, so I will have to report that GET didn't work for you because you didn't follow protocol."
Patient: "No, honestly, I did it exactly the way you wanted me to."
Therapist: "Objective measures don't lie."

The devil is in the assumptions and there are a lot of really bad assumptions in the BPS model and treatment protocols.

Somebody wrote this during the week on a list I'm on:

Very prosperous indeed for the researchers who have no doubt continued to earn substantial sums 'advising' insurance companies to ensure claimants are denied their payments -

* insurance claims cannot be considered further until 'optimal treatment' of CBT / GET is undertaken

* that if patients have already undertaken CBT / GET they have done the wrong sort of CBT / GET and they should undertake a further course of the 'right' sort of CBT / GET

* if the patients deteriorate it is their fault for not doing the CBT / GET properly - so their claims should still be refused

* it is not the insurers fault if the 'right sort' of CBT / GET is not available in the area in which the patient lives - an 'optimal treatment' has been identified and the claim can continue to be refused until the 'right sort' of CBT / GET is successfully completed.

* If the patient does manage to complete the 'right sort' of CBT / GET they will have recovered as evidenced by a SF-36 score of 60 (rigorously determined by the PI's in the highly respected PACE trial) and so the claim will not need to be paid

The DWP no doubt find this a highly attractive strategy for saving money - how tempted they must be........

 

[Sean]

Why do they have to make up their own definitions of everything? It makes it impossible to figure out what's going on.

Very convenient, isn't it. Use criteria, definitions, and thresholds that nobody else uses, but still claim that your work applies to the standard patient population.

At the very least it is sloppy and incompetent. At the very least.

 

[SOC]

Somebody wrote this during the week on a list I'm on:

This whole thing is just one giant horror story. Someday authors and filmmakers are going to have a field day with this. I wish it weren't true that I'll likely be dead by then.

 

[Dolphin]

Planned physical activity and not symptoms are used to determine what the participant does.

Participants are encouraged to see symptoms as temporary and reversible, as a result of their current physical weakness, and not as signs of progressive pathology.

So you have your exercise schedule for the week and you're supposed to stick to it. (I could probably find more quotes if necessary or maybe somebody else would). However, the big question is how many did. The researchers have never given information on activity logs. The only measure of compliance was turning up for your appointment (and turning up could include ringing in).

 

[SOC]

Very convenient, isn't it. Use criteria, definitions, and thresholds that nobody else uses, but still claim that your work is comparable to the standard patient population.

At the very least it is sloppy and incompetent. At the very least.

We studied the habits of domestic cats and their interactions with humans. They make excellent safe pets for adults and children over 6yo. Lions, tigers, and jaguars are cats, therefore they make excellent safe pets for adults and children over 6yo.

 

[SOC]

So you have your exercise schedule for the week and you're supposed to stick to it. (I could probably find more quotes if necessary or maybe somebody else would). However, the big question is how many did. The researchers have never given information on activity logs. The only measure of compliance was turning up for your appointment (and turning up could include ringing in).

So it sounds like what the research protocol says should be done, and what the researchers claim was done -- preplanned physical activity not symptoms control activity level -- is not necessarily the way the patients did GET, or even the way PACE therapists encouraged the patients to do it. Sloppy, sloppy, sloppy.

I retract my earlier statement. Hot mess doesn't begin to describe it. I don't think I have words to describe this so-called research project.

 

[ahimsa]

I'll give a heart to anyone who posts the I am Annie Gsampel meme on Twitter.

Hey, Snow Leopard, @Kati tweeted this yesterday:

Link: https://twitter.com/Katiissick/status/662787019324551168

I had not read this thread before now although I had seen the tweet by @Tom Kindlon quoting Graham's joke:

https://twitter.com/TomKindlon/status/662708340825284609

 

[Snow Leopard]

We studied the habits of domestic cats and their interactions with humans. They make excellent safe pets for adults and children over 6yo. Lions, tigers, and jaguars are cats, therefore they make excellent safe pets for adults and children over 6yo.

 

[wdb]

The biggest difference between GET and APT that I would attribute to the differing results was the expectation of improvement that was given to the participants.

Wessely himself knows very well this is an important factor.

Expectations can influence the outcomes, especially in psychological treatments, which is why so called patient preference trials, in which patients chose the intervention they prefer – give results that can be difficult to interpret, which indeed is an issue around the longer term outcomes of PACE after the end of the formal follow up (see the references below). Randomisation removes the worst of this problem, since patients by definition cannot select what they get. But if they still have higher or lower expectations of one treatment over another, it can still matter. And that did happen in the PACE trial itself. One therapy was rated beforehand by patients as being less likely to be helpful, but that treatment was CBT. In the event, CBT came out as one of the two treatments that did perform better. If it had been the other way round; that CBT had been favoured over the other three, then that would have been a problem. But as it is, CBT actually had a higher mountain to climb, not a smaller one, compared to the others. - Simon Wessely

He conveniently ignores any differences in expectation that may have occurred after the trial had started and influenced the outcomes.

Look in the participant manuals, this is what you get in the GET manual

What is Graded Exercise Therapy (GET)? GET is the use of regular, physical exercise (and consider exercise in its broadest sense, including lifting a finger or rolling in bed) to aid recovery from CFS/ME. GET works directly in partnership with your own individual level of current ability, and is directed by your own goals and objectives. In other words, the activity that you decide to work with is related to what you want to achieve. For example, if you would like to mange your home or garden better, GET helps to improve your strength and movement to allow you to do this. If you would love to be able to walk your children to school or get back to playing a sport you enjoy, GET helps you to gradually build up your strength and fitness to achieve this.

RESEARCH INTO GET In previous research studies, most people with CFS/ME felt either ‘much better’ or ‘very much better’ with GET. Exercise has been considered a useful strategy for many years in the rehabilitation of fibromyalgia, Multiple Sclerosis, and many other neurological conditions. Research has now shown that carefully graded exercise (Graded Exercise Therapy) can also be a very helpful therapy for CFS/ME. You may be aware that the Chief Medical Officer’s Report of 2002 recommended GET as one of the most effective therapy strategies currently known.

but in the APT manual there is nothing like that, the equivalent is statements like this.

The Adaptive Pacing Model of CFS/ME The concept of fixed limits
The basic underlying concept of adaptive pacing is that a person can adapt to
CFS/ME but that there is a limited amount that they can do to change it, other
than provide the right conditions for natural recovery. CFS/ME is regarded as
limiting your available energy. Exceeding the available energy causes an
exacerbation of fatigue and other symptoms often after a time lag.

Pacing as a helpful self-management strategy for
CFS/ME
The essence of pacing is that the person with CFS/ME uses self-management
of their level of activity in order to avoid exacerbations of symptoms and
disability (AfME 2002). The CFS/ME working group report (2002) described
adaptive pacing as follows: “Pacing is an energy management strategy in which
sufferers are encouraged to achieve an appropriate balance between rest and
activities. This usually involves living within physical and mental limitations imposed
by the illness and avoiding activities that exacerbate symptoms or interspersing
activity with planned rest. The aim is to prevent sufferers entering a vicious cycle of
over activity and setbacks, whilst assisting them to set realistic goals for increasing
their activity when appropriate”.

 

[MeSci]

The biggest difference between GET and APT that I would attribute to the differing results was the expectation of improvement that was given to the participants.

Wessely himself knows very well this is an important factor.

Expectations can influence the outcomes, especially in psychological treatments, which is why so called patient preference trials, in which patients chose the intervention they prefer – give results that can be difficult to interpret, which indeed is an issue around the longer term outcomes of PACE after the end of the formal follow up (see the references below). Randomisation removes the worst of this problem, since patients by definition cannot select what they get. But if they still have higher or lower expectations of one treatment over another, it can still matter. And that did happen in the PACE trial itself. One therapy was rated beforehand by patients as being less likely to be helpful, but that treatment was CBT. In the event, CBT came out as one of the two treatments that did perform better. If it had been the other way round; that CBT had been favoured over the other three, then that would have been a problem. But as it is, CBT actually had a higher mountain to climb, not a smaller one, compared to the others. - Simon Wessely

He conveniently ignores any differences in expectation that may have occurred after the trial had started and influenced the outcomes.

My tired brain may not be analysing this well, but I don't think that Wessely's claim is even entirely logical. It fails to take into account the degree to which common/general beliefs and fact can differ. If this difference is very large, it can overcome the fact that a patient does not expect a treatment to work, if it is actually an extremely effective/ineffective treatment, which will work/not work despite their strong belief that it wouldn't/would.

Many of us have experiences that surprise us and are not in line with our prior expectations. Yes - our prior expectations will have some effect, but they are not the only determinants. And the degree to which they determine outcomes will vary between people, just as some people are very susceptible to hypnosis and others are not.

Not sure if I'm explaining this well or even whether it makes sense! And I am not of course intending to suggest that CBT is an effective treatment for ME.

 

[soti]

Why do they have to make up their own definitions of everything? It makes it impossible to figure out what's going on.

That I don't have a problem with -- they are free to choose which interventions to study. I'd probably advise against using a term that is *wildly* different from how it's normally understood, but I'm not sure that's the case here.

I'm still struck by the fact that there is no reply to Jonathan Edwards' point about blinding and subjective measures. I suspect it has something to do with this idea that expectations are important. It's hard to avoid the conclusion that they were testing a placebo effect. I'm sure in their minds it makes sense, i.e. that the nature of the disorder means that placebo along with gradual exposure to activity IS the only treatment, because the disorder is essentially a phobia (at least, that's what they used to say, though now it seems they are walking that back, so now frankly I can't sort out what they actually think).

Does anyone know anything about how research on treatment of phobias is generally carried out? Is it usually done better than this? Certainly placebo research is.

 

[A.B.]

I'm still struck by the fact that there is no reply to Jonathan Edwards' point about blinding and subjective measures.

If they acknowledge this, the house of cards comes crashing down. From what I've seen, poor study design that quite intentionally leaves room for bias that can later be interpreted as treatment effect is the norm in the field of psychosomatic medicine.

 

[alex3619]

I'd probably advise against using a term that is *wildly* different from how it's normally understood, but I'm not sure that's the case here.

It is the case here. Comparing to conventional use, normal is wildly different. Normal can mean very disabled. So is recovery, which can also mean disabled. A patient might be simultaneously severely disabled, recovered and normal.

CBT is not what people expect. This is not regular CBT, its highly modified. We are debating here if GET is what people expect. Ditto for pacing. I still argue that what was delivered is wildly different from what most people would expect.

ME is not a recognized definition, though modified from one. CFS uses the Oxford definition and there is now a call to retire it.

Similarly there are issues with "rigorous" and "gold standard", both terms used to defend the PACE trial. There are more too, I think, but they escape me right now.

There is also a difference between how the terms are used in the papers, in a technical sense, and how they are used in a press release or media commentary. The authors have systematically failed to correct mistakes in the media. I have not sufficiently investigated press releases to make a comment regarding failure to correct them.

Some of the failures in the media are probably entirely due to the media getting it wrong though. CBT, in any form, is not positive thinking.

These issues are yet another reason why the raw data is so very important.

 

[Esther12]

I think that they made APT more controlling and limiting than it should be, in part, to help distinguish it from GET and CBT. There are still some people insisting that CFS patients should do less that they think is best for them and, despite it's flaws, I think that the results from PACE indicate that this is not useful. (It was never reasonable imo).

 

[MeSci]

Some of the failures in the media are probably entirely due to the media getting it wrong though. CBT, in any form, is not positive thinking.

A lot of failures in the media are probably due to them getting their info from the Science Media Centre rather than having their own scientifically-literate journalists.

 

[soti]

Oh, yeah, "normal" is a doozy! That one is beyond the pale, and it wasn't what I meant - I meant their names for the treatment arms. I was particularly thinking about GET vs. pacing. (Of course there's another problem if the actual intervention didn't even match their own definition.)

"Normal," "gold standard", etc. are pure propaganda.