PACE Trial and PACE Trial Protocol

[Snow Leopard]

I just came across the following:
[/IMG]
(from http://www.anh-europe.org/news/edzard-ernst-on-the-self-promotion-trail )

Seems like it might be a good model for what happened in the PACE Trial

See also:

http://jpgmonline.com/article.asp?i...e=54;issue=3;spage=214;epage=216;aulast=Ernst
Journal of Postgraduate Medicine, Vol. 54, No. 3, July-September, 2008, pp. 214-216
A trial design that generates only ''positive'' results


http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2796.2006.01707.x/full
Efficacy or effectiveness?
E. Ernst,
M. H. Pittler

 

[Graham]

Thanks for those links - very interesting. I could make use of that!

 

[Valentijn]

I'm a bit uncertain about that, anciendaze. I never really knew where Heisenberg was coming from, and he was always too fast for me.

Harhar

 

[Dolphin]

See also:

http://jpgmonline.com/article.asp?i...e=54;issue=3;spage=214;epage=216;aulast=Ernst
Journal of Postgraduate Medicine, Vol. 54, No. 3, July-September, 2008, pp. 214-216
A trial design that generates only ''positive'' results


http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2796.2006.01707.x/full
Efficacy or effectiveness?
E. Ernst,
M. H. Pittler

Thanks Snow Leopard.
They both give the free full text. However, only one gives a free pdf; for the 2008, people can try this link instead: http://www.bioline.org.br/pdf?jp08073.

They discuss trials comparing treatments A+B vs treatment A (treatment A could be care as usual) and say the results in such circumstances may not be reliable.

This would be like comparing SMC + GET vs SMC alone or SMC + CBT vs SMC alone (which did happen).
However, the trial did also compare SMC + GET vs SMC + APT or SMC + CBT vs SMC + APT.

In relation to the trials of treatments A+B vs treatment A (SMC + GET vs SMC alone or SMC + CBT vs SMC alone), it says these can be problematic if objective measures aren't used/can be less problematic if objective measures are used.

It describes A+B vs treatment A as effectiveness trials; it compares these to efficacy trials (which might use the notation: A+B vs A+C). It says effectiveness trials tend not to use objective measures while efficacy trials tend to use them. Unfortunately, actometers (for example) weren't used in the PACE Trial.

 

[Guido den Broeder]

They discuss trials comparing treatments A+B vs treatment A (treatment A could be care as usual) and say the results in such circumstances may not be reliable.

Yes, that's what I said from the beginning. Also, the statistics they used are not suited for this situation, i.e. all the p-values are wrong.

 

[oceanblue]

In relation to the trials of treatments A+B vs treatment A (SMC + GET vs SMC alone or SMC + CBT vs SMC alone), it says these can be problematic if objective measures aren't used/can be less problematic if objective measures are used.

It describes A+B vs treatment A as effectiveness trials; it compares these to efficacy trials (which might use the notation: A+B vs A+C). It says effectiveness trials tend not to use objective measures while efficacy trials tend to use them. Unfortunately, actometers (for example) weren't used in the PACE Trial.

Very interesting. Thanks, Dolphin, and Snow Leopard.

Thought it was worth highlighting this section on the problems of A vs A+B:

The reason for the inability of the 'A + B versus B' design to generate negative results (under the above-stated three conditions) seems obvious: even in the absence of any specific therapeutic effect, the results of such studies would be positive due to nonspecific effects such as a placebo-effect, the additional care given to patients, the therapist-patient relationship or social desirability. A further contributor could be the disappointment experienced by patients of the control groups when not receiving the experimental treatments they may have hoped for.

 

[Dolphin]

Another study suggesting SF-36 PF may not be a good measure of activity

Kop WJ, Lyden A, Berlin AA, Ambrose K, Olsen C, Gracely RH, Williams DA, Clauw DJ. Ambulatory monitoring of physical activity and symptoms in fibromyalgia and chronic fatigue syndrome. Arthritis Rheum. 2005 Jan;52(1):296-303.

Free full text: http://dx.doi.org/10.1002/art.20779

The SF-36 Physical functioning subscale is often used in ME/CFS trials (incl. the PACE Trial). Thus, I thought the following was interesting:

Ambulatory activities and symptoms of patients
in relation to self-reported function. Ambulatory peak
activity levels had a positive correlation with better
self-reported physical function as determined by higher
SF-36 scores (physical role r=0.45, physical function
r=0.30; both P<0.05), but there was no significant
association between peak activity levels and selfreported
pain (r=0.21, P=0.22) or general health (r
0.22, P=0.35). Associations between average activity
levels and self-reported physical function were in the
same direction, but were statistically nonsignificant (P>
0.1).

(from Discussion)

Peak activity, but not average
activity, also has an association with self-reported measures
of physical function, suggesting that patients are
reporting their inability to engage in high-level activities
when completing such questionnaires.

In a condition like ME/CFS, the average activity is of much more interest and importance than the peak activity. Indeed, higher peak activities could even potentially leave some people open to "boom and busting".

It is interesting that in the PACE Trial, the APT + SMC group did worse than the SMC group alone on the SF-36 PF, a somewhat odd finding. This sort of finding would tally with that finding: people who had done pacing might be less inclined to do high intensity activities/more likely to avoid doing them.

Also, if one does a GET or CBT (involving walking) program, one might have periods of relatively high intensitiy e.g. during the exercise session. This would give a higher SF-36 PF score (one of primary outcome measures in PACE Trial) without an average increase in activity. The Kop study, I think, doesn't give us enough information as to how much it would explain of the SF-36 PF score it might explain.

 

[Graham]

The thing that bothers me about using the A+B versus A argument in the PACE trial is that, along GET+SMC and CBT+SMC, we also have APT+SMC. So we cannot use the argument that it was simply the extra attention that tipped the balance. Actually the whole thing was so badly designed, we can't use any evidence much either. But ... if APT was cautious, but CBT and GET were more positive in outlook (which I believe they were), it could be that the cautious nature of the APT offset the (small amount of) good that the extra attention gave. Remember, as the Ernst study said, extra attention alone can distort subjective accounts of how effective something is. The trouble is that all we can do is speculate.

But then, it really doesn't matter - the onus was on the authors of the PACE trial to prove that CBT and GET were effective, and they failed to do so. So we are entitled just to ignore APT as a comparison.

 

[Guido den Broeder]

The SF36 can perhaps generate reasonably unbiased one-time findings, but it cannot reliably measure differences between before and after.

That is because the patients remember what they said before treatment, and will unconsciously adjust what they say after treatment to reflect that they did their best as well as their personal opinion of the treatment.

 

[oceanblue]

I too think the SF36 can provide a decent indication of activity levels, but I think it's prone to self-reporting bias when used in trials, particularly where the changes involved are small, as they were in PACE. I did post an example of how this might work in practice earlier in this thread, but I've no idea how to find that post now.

Here's a study that found a reasonable correlation (0.48) between SF36 and total accelerometry activity levels:
Validation of questionnaires to estimate physical activity and functioning in end-stage renal disease

 

[Dolphin]

I did post an example of how this might work in practice earlier in this thread, but I've no idea how to find that post now.

Don't go back - it's too dangerous - you might never find your way out.

 

[Graham]

Wise words Dolphin. I worked through the thread, and look what it did to me. Before that I was a dead ringer for Jimi Hendrix.

Could the post you are looking for be number 901 on page 91?

 

[oceanblue]

Don't go back - it's too dangerous - you might never find your way out.

!! Maybe that explains why there are now only a few of us left on this thread?

Thanks for the suggestion, Jimi, but that was a different pearl .

 

[Esther12]

I lost track of this thread around page 60. I've still got a bookmark labelled 'catch up from here'.

It's not going to happen.

 

[Snow Leopard]

So we are entitled just to ignore APT as a comparison.

I think it's pretty simple. If you psychologically condition patients into thinking they are improving (even if they are not), they will report improvements on questionnaires. Whereas if you condition them into thinking they are not improving at all, then you won't get the same reported improvements on the questionnaires. Of course things would be a lot clearer if they just measured patient activity levels objectively....

 

[anciendaze]

While there are plenty of other studies in medicine and psychology with similar uses of statistics I'm afraid that doesn't validate this use so much as the idea that a large part of the profession is statistically incompetent. If you represent a one-sided distribution with a mean of 85 and a mode of 95 with a symmetrical one that has mean, mode and median at 85 you invalidate any p values based on that distribution.

Arguments that the individual group distributions are nevertheless normal depend on unstated assumptions that the sampling process which produced those distributions had certain properties like independence. You can't depend on inheriting those properties from a population distribution which lacks them. All the evidence presented is that the sampling was contrived.

If you create something which looks like a normal distribution through sampling, when it lacks any natural cohesion as a group, you must expect it to exhibit regression toward the mean of the original population. Any action which increases group variance will look like a favorable intervention.

This shouldn't even be necessary to discuss when the most successful result of six months of therapy showed a group of people in their 20s, 30s and 40s moving like fragile octogenarians. Subjective measures are subject to all sorts of distortions which people on this thread have brought up. In the absence of better results, these cannot be attributed to much of anything. This is nothing more than a taste test for different flavors of therapy, and tastes are notoriously subject to rapid change.

Try to imagine this result as a political poll. Would you bet significant money on next year's election as a result of a modestly good showing at present?

 

[Marco]

The 'slow puncture' analogy of PACE

Apologies for this little piece of self amusement and I admit its not a perfect analogy but its strange how things pop into your head sometimes.

Imagine that you are happily driving along the road when you suddenly feel that slightly seasick feeling that the car is not handling properly, starting to wander around and pulling to the left. You suspect a puncture and pull over to the side of the road and find that the tyre is not fully deflated but soft. In other words a slow puncture. You carry on driving at a pace you feel comfortable with so as not to be a danger to yourself or other road users.

Imagine also that you have a passenger who is a cognitive behavioural therapist (god forbid - and here I'm stretching the analogy as even the most committed CBT'er would be unlikely to stay on board). Your passenger tells you that he doesn't think that you have a puncture and that sometimes we all get vague feelings that there's something wrong with the car. He advises you just to ignore these vague feelings and that its perfectly safe to drive a little faster. In fact he suspects that the problem is that you just let the pressure drop too low and that the tyre isn't seated tightly against the rim. Driving too slowly just makes things worse and what you really need to do is drive a little more quickly getting heat and pressure in the tyre which will seal it against the rim. You try this but still feel the car pulling to the left. He suggests that this may be the case (perhaps because of the camber of the road) but is nothing to worry about and just to countersteer to the right to compensate.

You do as recommended and soon begin to gain a little more confidence and are eventually able to drive along more comfortably at 30mph rather than the 10mph pace when you initially found the puncture - but its still a little irritating to the other motorists on this 55mph limit road.

So who is right? Both obviously. You do have a puncture but may have erred on the side of caution rather than cause an accident. In all likelihood you could have driven a little faster and in most cases you will make it home safely. On the other hand the tyre might have punctured completely sending you and your passenger off the road or into oncoming traffic.

Longer term which approach would get you home safely? Ideally you would get to a garage immediately and have the puncture repaired or the tyre replaced (we wish) but as a stopgap you would take the B roads, driving well within the limits of safety and stopping as often as necessary to reinflate the tyre until you get home.

Your passenger however recommends that you continue to gradually build up speed and join the motorway just as soon as you drop him off.

Just a little light relief.

 

[Graham]

I love analogies: I think they make us reasses our way of looking at things, and this one is a lovely one!

 

[Dolphin]

Apologies for this little piece of self amusement and I admit its not a perfect analogy but its strange how things pop into your head sometimes.

Imagine that you are happily driving along the road when you suddenly feel that slightly seasick feeling that the car is not handling properly, starting to wander around and pulling to the left. You suspect a puncture and pull over to the side of the road and find that the tyre is not fully deflated but soft. In other words a slow puncture. You carry on driving at a pace you feel comfortable with so as not to be a danger to yourself or other road users.

Imagine also that you have a passenger who is a cognitive behavioural therapist (god forbid - and here I'm stretching the analogy as even the most committed CBT'er would be unlikely to stay on board). Your passenger tells you that he doesn't think that you have a puncture and that sometimes we all get vague feelings that there's something wrong with the car. He advises you just to ignore these vague feelings and that its perfectly safe to drive a little faster. In fact he suspects that the problem is that you just let the pressure drop too low and that the tyre isn't seated tightly against the rim. Driving too slowly just makes things worse and what you really need to do is drive a little more quickly getting heat and pressure in the tyre which will seal it against the rim. You try this but still feel the car pulling to the left. He suggests that this may be the case (perhaps because of the camber of the road) but is nothing to worry about and just to countersteer to the right to compensate.

You do as recommended and soon begin to gain a little more confidence and are eventually able to drive along more comfortably at 30mph rather than the 10mph pace when you initially found the puncture - but its still a little irritating to the other motorists on this 55mph limit road.

So who is right? Both obviously. You do have a puncture but may have erred on the side of caution rather than cause an accident. In all likelihood you could have driven a little faster and in most cases you will make it home safely. On the other hand the tyre might have punctured completely sending you and your passenger off the road or into oncoming traffic.

Longer term which approach would get you home safely? Ideally you would get to a garage immediately and have the puncture repaired or the tyre replaced (we wish) but as a stopgap you would take the B roads, driving well within the limits of safety and stopping as often as necessary to reinflate the tyre until you get home.

Your passenger however recommends that you continue to gradually build up speed and join the motorway just as soon as you drop him off.

Just a little light relief.

I like it. Although maybe a better analogy is that the passenger isn't in the car - maybe somebody giving advice on the phone (like when the passenger left). If one suffers a crash in ME/CFS, the therapist doesn't lose out - just the patient. There's basically no "risk" in it for them.

 

[oceanblue]

Nice, Marco.

I like you analogy of the therapist leaving the car, and Dolphin's of using the phone to never be at risk. When I had a CBT car crash after following the advice, the psychiatrist didn't want to discuss it.

But CBT therapists are not alone in this readiness for others to take risks on their advice. I had an intersting exploratory discussion with a Lightning Process practitioner, who assured me once I had started on the course I would be able to do far, far more than I normally could, and so could easily manage the 5 hours a day plus homework without problems - I should trust him on this. When I suggested he should agree to refund me if I had a relapse (ie he should accept some personal risk for his advice) he refused. Odd, that. Unsurprisingly I did not proceed (which of course is why I'm still ill).