PACE Trial and PACE Trial Protocol

[oceanblue]

Well spotted. :Sign Good Job:
And of course, this was one of the places where CBT was offered during the PACE Trial so one doesn't need to control for factors like that (different population) too much.

Thanks. Good point about the comparability of the samples with PACE.

 

[Dolphin]

Analyses not done/reported on

I'm not sure if these have been pointed out before:

http://www.biomedcentral.com/1471-2377/7/6

This one wasn't done (as switched to continuous measures). However I find the last paragraph interesting:

Primary analyses of efficacy

The primary analysis will be pragmatic, based on intention to treat, and will utilise all available follow-up data from all randomised participants. The primary binary outcomes of response on the fatigue and physical function sub-scales (comparing proportions with categorical adverse deterioration with this scale as well) and both and a combined response with will be analysed by logistic regression adjusted for centre with contrasts for:

(1) APT vs. SSMC alone,

(2) APT vs. CBT,

(3) APT vs. GET,

(4) Trend across SSMC alone, APT, and CBT/GET combined,

(5) CBT vs. SSMC alone,

(6) GET vs. SSMC alone.

Participants not followed to one year will be classed as non-responders unless they show a consistent pattern of outcome across assessments at 10, 24, and 39 weeks or whenever the last assessment is obtained.

There was no talk of a consistent pattern in the final paper. For both "improvement" and "within normal range" they just used 52 week scores using those where they had scores at 52 weeks from what I can see (the denominator is the number of scores they had at 52 weeks). I think if they had done what they talked about, they would have got lower percentages for responders whether they used "improvement" or "within normal range".
BTW, I'm not sure where 39 weeks came from - it's not even mentioned in the Trial Identifier (i.e. before the protocol) - maybe they are using it as an example of when they got the data.

Secondary analyses of efficacy

The secondary continuous outcomes will be analysed by repeated measures analysis of variance using a linear mixed model with AR(1) covariance structure, and including centre, depressive disorder, CDC and London criteria and baseline values as covariates. The same contrasts as those specified for the primary outcomes will be extracted. A summary measure, the area under the curve, will also be reported.

I will admit I don't understand this. But I'm pretty sure there was no area under the curve reported.

Further secondary sensitivity analyses will be used to assess the robustness of conclusions for missing primary outcomes; these will employ repeated binary outcomes, multiple imputation, and imputation analysing all possible outcomes [58].

I don't believe any sensitivity analyses were reported (there was per-protocol, which was just before this in the protocol, but that's not the same)

Loss to follow-up, departures from randomised treatment protocols, and the prevalence of serious adverse events, will be reported at 13, 26, 39, and 52 weeks from randomisation.

Firstly, there's 39 weeks again. There was no talk of anything to do with 39 weeks in the final paper.

Loss to follow-up is discussed in Figure 1 at 12, 24 and 52 weeks.

Also in Figure 1, they have "withdrew" but I think that would only make up a small part of "departures from randomised treatment protocols". This doesn't be talking about a "per protocol" analysis which was covered elsewhere. So don't think that was reported.

"prevalence of serious adverse events" at the different time points definitely doesn't look like it was reported.

Results from all analyses will be summarised as differences between percentages or means together with 95% confidence limits (CL). The significance level for all analyses of primary outcome variables will be P = 0.05 (two-sided); for secondary outcome variables, P = 0.01 (two-sided) unless profiles of response can be specified in advance.

Prior to writing the Analysis Strategy a consensus will be reached on the profiles of response for each secondary outcome within each of the four treatment groups.

As I'm pretty sure I mentioned before, there was no mention of using p=0.01.
Quite a few of the outcomes in Table 6 have p values where 0.01<p<0.05 where the are all bundled in together if they satisfy p<0.05 as "better".

 

[Dolphin]

(Junk Junk)

As I mentioned, the protocol used p<0.01 for secondary outcome variables

Results from all analyses will be summarised as differences between percentages or means together with 95% confidence limits (CL). The significance level for all analyses of primary outcome variables will be P = 0.05 (two-sided); for secondary outcome variables, P = 0.01 (two-sided) unless profiles of response can be specified in advance.

Prior to writing the Analysis Strategy a consensus will be reached on the profiles of response for each secondary outcome within each of the four treatment groups.

I think improvement and "within normal range" would qualify as a secondary outcome measure (they're not a primary outcome measure anyway).

Initially it looks like a few of them wouldn't make it with p<0.01

64 (42%) of 153 participants in the APT group improved
by at least 2 points for fatigue and at least 8 points for
physical function at 52 weeks, compared with 87 (59%) of
148 participants for CBT, 94 (61%) of 154 participants for
GET, and 68 (45%) of 152 participants for SMC. More
participants improved after CBT compared with APT
(p=00033) or SMC (p=00149), and more improved with
GET compared with APT (p=00008) or SMC (p=00043);
APT did not diff er from SMC (p=061; webappendix p 2).

25 (16%) of 153 participants in the APT group were
within normal ranges for both primary outcomes at
52 weeks, compared with 44 (30%) of 148 participants for
CBT, 43 (28%) of 154 participants for GET, and 22 (15%)
of 152 participants for SMC. More participants were
within normal ranges after CBT than APT (p=00057) or
SMC (p=00014), and more were within normal ranges
with GET compared with APT (p=00145) or SMC
(p=00040); APT did not diff er from SMC (p=065).

However, I think P = 0.01 (two-sided) unless profiles of response can be specified in advance means that effectively what the test is is p =0.02. Just as p=0.05 two tailed is p=0.025 one tailed.

It would be (a bit) interesting to see what the p values were for other thresholds rather than these post-hoc ones.

 

[Graham]

You could also consider doing a reliability test (not sure if it is of much value but just brainstorming): ask people to not look at their previous answers and fill it in a week later. This may not work so well with CFS as there is some fluctuation: with many questionnaires, there can be natural fluctutation e.g. ask a healthy person how happy/well they feel today on a scale of 0-100 and ask them 10 minutes later (or some distance in time when they might forget their initial response) and you won't necessarily get the same result. There are probably better exams out there in terms of reliability.
I suppose what I'm partly getting at is that at different moments in time, with some scales, people can give different answers while by some objective measure there is no difference.

Thinking aloud: I wonder would there be any chance that people might overplay their symptoms a little to make sure they could enter into a trial/be ill enough so that they would be seen as worthy of treatment. Then they wouldn't need to really "improve" but there would still be an improvement. Although perhaps this would be equal across all arms of a trial; but it would lead to lower scores initially than might be registered in other circumstances i.e. when trying to work out how ill the cohort was.
(no need to respond to any of this)

Thanks for those suggestions - I'm responding because it is of major importance as far as I am concerned. There are lots of fancy buttons in Excel that will do all sorts of sophisticated statistical tests, but far far more important is the reliability of the data. Now this will come as a complete shock to all of you, who have thought of me as a fine upstanding creature, but I have been known to be a bit crafty. I have got plans to choose and ask a select few to resubmit their scores later without referring to their previous scores. Of course, I know my data has not been gathered in a fair and balanced way, but if it does indicate inherent unreliability greater than the criteria used to measure improvement, it minimises even more any claims of success with GET and CBT.

And of course, with brainfog, none of us, including me, will be able to remember what we scored the last time anyway. The trouble is that I do need decent numbers - I'm up to 15 at the moment.

 

[Dolphin]

Good to hear, Graham.

BTW, you're naked - you don't have your signature on.

 

[Dolphin]

Chalder Fatigue Scale: would using the 'true' SD have made PACE fail?

Sorry for the melodramatic title but it's a geeky point and I wanted to create the illusion it might be interesting.

PACE made heavy use of Standard Deviation (SD) of the baseline group in definging 'success': a 'clinically useful difference' (CUD) was defined as 0.5SD, and as the SD was 3.7 the CUD was set at 2. The group mean difference for CBT and GET were both over 2 (success!) and around 80% of individuals also exceeded the CUD (more success!).

And 65% for the SMC group. I wonder if there is any rule of thumb that one shouldn't choose a value such that over x% (e.g. over 50%) of the control group would satisfy the 'clinically useful difference' (or some other categorical improvement measure - this was basically the measure they came up with to replace the primary outcome measure "overall improvers" that was more strictly defined).

It shows the value in this paper more than others that the "number needed to treat" (NNT) is a useful way of presenting data. Measures such as the "number needed to treat" takes account of how the control group (or those in another arm) do rather than relying on the percentage measure alone i.e. an improvement of 80% for a treatment group is a lot more impressive if only 20% of the control group improved compared to if 65% of the control group improved - the 80% figure alone doesn't give sufficient information to tell the value of the treatment.

 

[Graham]

And the NNT for GET and CBT were 7 and 8 (or vice-versa), I believe.

(it's late at night, I'm ready for bed, so who needs clothes? Intermittent nudity may attract more attention.)

 

[oceanblue]

And 65% for the SMC group. I wonder if there is any rule of thumb that one shouldn't choose a value such that over x% (e.g. over 50%) of the control group would satisfy the 'clinically useful difference' (or some other categorical improvement measure - this was basically the measure they came up with to replace the primary outcome measure "overall improvers" that was more strictly defined).

It shows the value in this paper more than others that the "number needed to treat" (NNT) is a useful way of presenting data. Measures such as the "number needed to treat" takes account of how the control group (or those in another arm) do rather than relying on the percentage measure alone i.e. an improvement of 80% for a treatment group is a lot more impressive if only 20% of the control group improved compared to if 65% of the control group improved - the 80% figure alone doesn't give sufficient information to tell the value of the treatment.

Good points. I think the 65% improvement in the SMC group does show how low a threshold they set (or what a fabulous treatment SMC is). And no surprise that PACE didn't use NNT.

Does anyone know why they never actually call SMC a 'control' group and don't refer to their study as a 'randomised controlled trial'?

 

[Battery Muncher]

Hi Battery Muncher

As you can see, I only joined a few months ago, and the thread was enormous then. It doesn't break down easily into phases, and lots of topics keep getting revisited. I slowly worked through all the posts and kept finding nuggets in all sorts of places. The nature of the posts is that often two or three threads are interwoven.

The big question is, what sort of aspects interest you (changes from the original proposals, the different measures, analysis of results, correspondence with Lancet ...), and what level do you want (e.g. everyday, broad scientific or in-depth analysis). If it is anything specific, we may be able to direct you somewhere.

People here have been very very patient and helpful - a great bunch!

Hi, thank you very much for taking the time to reply.

I'm interested in the main criticisms of the trial. In particular, criticisms that would carry weight in an argument with a Dr.

I have an appointment next week, and my Dr asked me to read it. I just want to make sure I don't get pressured into making decisions I don't want/ force me to make concessions because I haven't researched properly (it's happened before).

Having said that. I understand what you say about the thread having many strands running through it at once. I will definitely try and read it in full eventually.

 

[Esther12]

I will definitely try and read it in full eventually.

*gulp*

The brief thing to make clear to your doctor is that the improvements made were, when you look at the raw data, much lower than they tried to make it sound like. Also, results from trials tend to be more impressive than those seen in the real world, and in the real world therapists aren't operating with the sort of controls seen under PACE, so more can go wrong.

It could be worth giving your local CFS centre a go, but PACE is good evidence that the therapies available there are of limited use to most patients, and have been misleadingly over-sold for the last twenty years.

If your doctor thinks PACE is good evidence that CBT/GET are curative/great, then I'd try to avoid an argument, and instead ask more about the details eg: what SF36 PF or Chalder Fatigue Scores your doctor thinks would indicate good health, and then compare that with population norms and the changes shown under PACE. I'd avoid being too negative about CBT/GET, and instead focus upon the fact that the evidence of benefit is only quite small improvements in questionnaire scores - a benefit minor enough that, if you feel this approach is not likely to suit you, it's entirely reasonable for you to decide against trying.

Equally - I'm hopeful that the poor results from PACE will mean that many CFS centres are going to now have to take a less doctrinal approach to CFS, and the problems many of us have experienced in the past might become rarer. It could well be worth giving them a go, and seeing if they do have any helpful advice for you (until there are some mass firings of those responsible for the quackery of the past, I don't think I could bear to go back... but I'm trying to be hopeful for the future!)

(Actually - I've missed so much out the above is inevitably somewhat misleading. It's so complicated, that discussing it with your doctor could well be tricky. Good luck.)


PS: Thanks to all the comments from others too. I've just gone through trying to take the recent posts in, and there's still interesting stuff been dug out.

 

[Dolphin]

Hi, thank you very much for taking the time to reply.

I'm interested in the main criticisms of the trial. In particular, criticisms that would carry weight in an argument with a Dr.

I have an appointment next week, and my Dr asked me to read it. I just want to make sure I don't get pressured into making decisions I don't want/ force me to make concessions because I haven't researched properly (it's happened before).

Having said that. I understand what you say about the thread having many strands running through it at once. I will definitely try and read it in full eventually.

You might find this paper of interest - free full text at: http://niceguidelines.files.wordpress.com/2009/10/twisk-maes-cbt1.pdf

Neuro Endocrinol Lett. 2009;30(3):284-99.

A review on cognitive behavorial therapy (CBT) and graded exercise
therapy (GET) in myalgic encephalomyelitis (ME) / chronic fatigue
syndrome (CFS): CBT/GET is not only ineffective and not
evidence-based, but also potentially harmful for many patients with
ME/CFS.

Twisk FN, Maes M.


Source

ME-de-patinten Foundation, Limmen, the Netherlands. frank.twisk@hetnet.nl


Abstract

Benign Myalgic Encephalomyelitis (ME) / Chronic Fatigue Syndrome (CFS)
is a debilitating disease which, despite numerous biological
abnormalities has remained highly controversial.

Notwithstanding the medical pathogenesis of ME/CFS, the
(bio)psychosocial model is adopted by many governmental organizations
and medical profes-sio-nals to legitimize the combination of Cognitive
Behavioral Therapy (CBT) and Graded Exercise Therapy (GET) for ME/CFS.

Justified by this model CBT and GET aim at eliminating presumed
psychogenic and socially induced maintaining factors and reversing
deconditioning, respectively.

In this review we invalidate the (bio)psychosocial model for ME/CFS
and demonstrate that the success claim for CBT/GET to treat ME/CFS is
unjust.

CBT/GET is not only hardly more effective than non-interventions or
standard medical care, but many patients report that the therapy had
affected them adversely, the majority of them even reporting
substantial deterioration.

Moreover, this review shows that exertion and thus GET most likely
have a negative impact on many ME/CFS patients.

Exertion induces post-exertional malaise with a decreased physical
performan-ce/aerobic capacity, increased muscoskeletal pain,
neurocognitive impairment, "fatigue", and weakness, and a long lasting
"recovery" time.

This can be explained by findings that exertion may amplify
pre-existing pa-thophysiological abnormalities underpinning ME/CFS,
such as inflammation, immune dysfunction, oxidative and nitrosative
stress, channelopathy, defec-tive stress response mechanisms and a
hypoactive hypothalamic-pituitary-adrenal axis.

We conclude that it is unethical to treat patients with ME/CFS with
ineffective, non-evidence-based and potentially harmful
"rehabilitation therapies", such as CBT/GET.


PMID: 19855350 [PubMed - indexed for MEDLINE]


Publication Types, MeSH Terms

P.S. If your doctor is ok with your battery munching, you probably can persuade him/her about anything.

 

[Graham]

Hi BM

You have had some pretty good advice from the others - I'd like to add a bit of craftiness to the equation. I don't know what your GP is like, nor what your ME support centre is like, so my advice is based on ignorance! First of all, do not rubbish the PACE trial openly - it can set doctors' backs up and make them label you as awkward. Go with the flow, but inject softeners like "yes, I would like it to do me some good, but with the Number Needed to Treat being around 7 or 8, I will need to be realistic about my prospects" (that means, the PACE figures show that for every 1 patient that gets back to their weak idea of normal, there are 6 or 7 who show little improvement or deteriorate). Also throw in the fact that the group that only had Specialist Medical Care showed more improvement than the amount that GET and CBT added on, and one of the things that SMC offered was treatment to help with pain and sleep. Finally, and this is the most important of all, if you do end up with GET or CBT, by all means go along with it - one day you may need your doctor's support in saying that you have tried the official treatments, but always take care and trust in your own judgement if you think it is starting to do you any harm - play the game, but don't take risks.

There is no doubt that CBT and GET, if done wisely and very carefully tailored to the individual, can help a proportion of patients to a small extent. Personally, I am also sure that they both have the potential to set patients back quite badly if they are inflexible (such as insisting on increasing efforts each week) and ignore warning signs from the patient. To be honest, when most of us talk about pacing, we are talking about elements of CBT and GET, but the crucial difference is that in pacing listening to your body is the key to it all. (I would get heavily criticised for this in some areas, but I think that is because CBT and GET is often given inflexibly and "according to the book".) Trust your own judgement, or come back on line and ask for advice. The good thing about the ME centres is meeting others with ME and discussing the common ground and variations between you, and generally the staff are very friendly and supportive.

Bob and I did a blog a while back on one aspect of the PACE trial that may help you get it into context.

Sorry if this sounds as if I am preaching - I'm an ex-maths teacher, and after 40 years it's hard to be a normal person! But if you have any questions or problems, feel free to post here, or send a personal message to any of us on this thread - they have all been really helpful to me, and so patient.

Good luck!

 

[Battery Muncher]

Thank you very much for the informative replies everyone. It's been very helpful. In particular, the advice not to openly rubbish the PACE trial comes through loud and clear.


Esther12: Ha!

Thanks for the practical suggestions on the doctor's appointment, I'll definitely keep them in mind (unless brain fog!).

Interesting comments on the possible fall-out from PACE. Hope for the future?


Dolphin: Thanks for the link! It certainly looks impressive. Did it have any impact in the medical community?

As for the battery munching, you're completely right. Clearly they are more open minded than I thought! Perhaps I should give them a bit more credit.


GM: That wasn't preachy at all! It was, in fact, extremely helpful advice.

Again, thanks for the practical tips regarding the doctor's visit. It's really useful to get this sort of advice from experienced members! I hear your points about CBT, and I will look into ME centres near by.

Finally, thanks for the link to the blog. I'll make a note to read it sometime tomorrow.

 

[biophile]

Summary of criticisms for the PACE trial

Battery Muncher wrote: I'm interested in the main criticisms of the trial. In particular, criticisms that would carry weight in an argument with a Dr. I have an appointment next week, and my Dr asked me to read it. I just want to make sure I don't get pressured into making decisions I don't want/ force me to make concessions because I haven't researched properly (it's happened before). Having said that. I understand what you say about the thread having many strands running through it at once. I will definitely try and read it in full eventually.

A chance to attempt a concise summary of what we have learnt in 6 months! First just to summarize what others have said, for myself mainly, before I attempt to answer. Dolphin suggests starting at the published (http://forums.phoenixrising.me/show...published-and-authors-response-(and-editorial) and unpublished (http://forums.phoenixrising.me/show...-letters-that-were-not-accepted-by-the-Lancet) letters to the Lancet. Esther12 points out that the PACE trial results have been exaggerated, may not occur the same in the real world, and that CBT/GET have been "misleadingly over-sold for the last twenty years". Graham points out that we need to be realistic, about the number of patients needed to treat and show a significant outcome in one patient was about 7 for CBT and 8 for GET, and also refers us to a blog he created with Bob (http://meanalysis.blogspot.com). He and also Esther12 had additional words of wisdom; use your own judgement and express curiosity, "go with the flow" and don't antagonize the GP or "rubbish" PACE (IMO unless it is called for), especially if they are "pro" CBT/GET, therapies which for some people who meet some CFS criteria may help to some extent, but which also have fundamental questions remaining. To compliment what others have already said, you may also wish to consider the following:

Although CBT and GET encourage and presume increases in activity, PACE gave no data for actual increases in activity. The relatively "small" improvements (according to Cohen's rules of thumb, not White et al's loose interpretation of "moderate") were subjective and there was no solid evidence of objective improvement.

The almost total reliance on self-reports is ironic when the authors propose that patients' perceptions and cognitions are distorted/unreliable. The possibility of placebo response is trivialized by the authors, they point out that expectations were poorer in the CBT and GET groups at the beginning of the trial, but ignore the fact that the CBT and GET groups were then conditioned over 52 weeks, as if psychotherapy couldn't possibly induce any response bias or inflated self-efficacy or wishful thinking which isn't accompanied by objective improvements?

As the trial had no adequate placebo control, you may be concerned the results are somewhat illusionary for therapies which are aimed at altering your thought processes, especially considering that a meta-analysis of 3 CBT studies showed that CBT does not lead to increases in physical activity as the proponents of CBT/GET have assumed for over 20 years (http://www.ncbi.nlm.nih.gov/pubmed/20047707). Note that PACE actually had the equipment to run these same tests and did so at baseline but not at followup.

What they defined as "normal" (based on scores for both fatigue and physical functioning at once) and was then erroneously parroted in the news as "recovery", is average for people who are either diseased and/or about 80 years old. Unless you fit that description, this is sub-standard. For physical functioning, this absurdity arose from two dubious methodological processes:

(1) To derive their goalpost for normal ie 60/100 points or more, they claimed to have used a "working age population" but later admitted to using a general population, which included 1/4 who were elderly and 15-22% who were diseased and/or had to cut down on activities recently due to illness. If using datasets from either healthy working age populations, or a similar age group as the PACE participants even from a general population, the definition of normal ie [mean (average) minus 1 standard deviation] should have yielded a lower threshold of 80/100, not 60/100. [EDIT: In a small CFS study by VanNess et al 2010 (http://www.ncbi.nlm.nih.gov/pubmed/20095909), the mean(SD) of healthy sedentary controls was 96.84(4.48), suggesting a lower threshold of 90/100 points!]

(2) They used the [mean -1SD] rule to determine the lower threshold of normal, this rule is supposed to capture about 68% of the population from both sides of the bell curve (see http://en.wikipedia.org/wiki/File:Standard_deviation_diagram.svg) but [may be] useless when the scores follow a non-normal distribution as they do in Bowling et al 1999 as used by PACE, where about 72% of the population are actually scoring 85+/100 in a general population, [possibly] suggesting that even 85/100 points is an abnormally low threshold, which should be more like 90/100 before we even consider using a healthy age-matched population. [EDIT: To be cautious here, the PF/SF-36 frames questions towards limitations rather than abilities or activity levels, and is limited to 100 points. Therefore it is quite possible that some people who score 100 are much more active than others who also score 100 (top box ceiling), so if the PF/SF-36 was measuring activity levels it may follow a more normal distribution anyway, making the "mean -1SD rule" adequate afterall but ONLY when using a reasonable comparison population per above ie a healthy working age population (or better still) healthy age-matched controls.]

No amount of spin doctoring can justify 60/100 as "normal" for people who were on average 38 years old and aiming for recovery and do not have the other diseases present in a general population (because these would exclude one from a CFS diagnosis). For issues with "normal" fatigue, I haven't examined that properly yet, someone else here may be able to explain why that was suspect as well, although I have the impression that it wasn't as OUTRAGEOUSLY FLAWED as "normal" physical functioning.

The closest thing to an (arguably) "objective" test was the distance covered during a brisk 6 minute walk (6MWD). CBT had no advantage over SMC. The GET advantage over SMC not only struggled to become clinically useful, but on average was much lower than what is considered healthy (600-700m) and is still comparable to people with serious diseases (300-500m).

Among the best two references to illustrate this point: compare GET @ 52 weeks of an average 379m [ standard deviation of 100m] to: (1) 393115m for patients (n=1083, 11 studies) from a diverse range of cardiopulmonary disorders: end stage lung disease, dilated cardiomyopathy, congestive heartfailure, pulmonary hypertension, chronic obstructive pulmonary disease, interstitial lung disease [study range, 294139m (ESLD) to 463107m (DC)] from Ross et al 2010 (http://www.ncbi.nlm.nih.gov/pubmed/20504351); (2) 696151m for "completely healthy" elderly participants (n=58, 20M/38F) aged 627yrs from Bautmans et al 2004 (http://www.ncbi.nlm.nih.gov/pubmed/15272934).

Patient groups who frequently report that pacing is the most helpful therapy for their members, are generally saying that the APT in PACE is not the same as the pacing they understand and practice, so you may be interested in finding out more about "real" pacing rather than the potential strawman known as APT.

Outside of PACE, the lead author Peter White has expressed concerns over the Canadian criteria having symptoms which may indicate alternative diagnoses to CFS. I'm still looking into this issue in specific relation to PACE itself, but one example I've noticed so far is that the Canadian definition discusses "spatial instability" as a symptom, but in the NICE 2007 guidelines used by PACE to help exclude patients this is considered a "red flag" indicative of another disease, ie "Spatial disorientation is not generally characteristic of CFS/ME, and is usually indicative of brain damage. Concentration and memory difficulties (brain fog) are, however, typical." (p176) Therefore, if you have this symptom and/or meet Canadian criteria in general you may wonder how the PACE trial findings can be generalized to you.

The authors changed and watered down all their goalposts halfway through the study to "increase sensitivity" of the results, which is bad if APT was a strawman. Too much to summarize right now, but PACE did publish a protocol in 2007 (http://www.biomedcentral.com/1472-6882/7/12) and have made changes to the trial since then, discussion of which is embedded in this large thread somewhere. Two examples related to what I mentioned above, they dropped their stricter definitions of "positive outcome" and "recovery" then replaced them with weaker versions of "clinically useful" (0.5 SD on data which has an artificially lowered SD) and the dubious definition of "normal" (which itself was a "post-hoc analysis" suggesting that it was possibly done AFTER the authors saw the disappointing trial data, a big no no).

The threshold for a serious adverse event or reaction was rather high and you may be concerned that anything less than that will be deemed "safe" even when you are actually suffering a relapse and further impaired for weeks while the therapist insists you maintain current activity levels:

"(a) Death; b) Life-threatening event; c) Hospitalisation (hospitalisation for elective treatment of a pre-existing condition is not included), d) Increased severe and persistent disability, defined as a significant deterioration in the participants ability to carry out their important activities of daily living of at least four weeks continuous duration; e) Any other important medical condition which may require medical or surgical intervention to prevent one of the other categories listed; f) Any episode of deliberate self-harm."

"Serious deterioration in health was defined as any of the following outcomes: a short form-36 physical function score decrease of 20 or more between baseline and any two consecutive assessment interviews; scores of much or very much worse on the participant-rated clinical global impression change in overall health scale at two consecutive assessment interviews; withdrawal from treatment after 8 weeks because of a participant feeling worse; or a serious adverse reaction."

Unfairly, the critics of the PACE trial have been generally branded as fringe radicals hell bent on a hate campaign which includes (alleged) death threats. You may be concerned that you will be falsely implicated and wrongly stereotyped for asking crucial questions about issues which will affect your health and livelihood.

 

[Dolphin]

Dolphin: Thanks for the link! It certainly looks impressive. Did it have any impact in the medical community?

I don't think it did - so far anyway. But then a lot of people have a sorted of vested interest where GET and GET/GAT-based CBT are about all the offer (if you're a non-doctor, you can't offer medication based treatment and a lot of psychiatrists generally aren't into giving medical treatment except drugs that work on the brain).

But is a useful place to get an idea of all the different exercise abnormality studies that have been done, etc.

When clinical practice becomes more scientific, the ideas may take hold more.

 

[Graham]

As for the battery munching, you're completely right. Clearly they are more open minded than I thought! Perhaps I should give them a bit more credit.

I thought you were just on an acid trip.

 

[Dolphin]

A chance to attempt a concise summary what we have learnt in 6 months! First just to summarize what others have said, for myself mainly, before I attempt to answer. Dolphin suggests starting at the published (http://forums.phoenixrising.me/show...published-and-authors-response-(and-editorial) and unpublished (http://forums.phoenixrising.me/show...-letters-that-were-not-accepted-by-the-Lancet) letters to the Lancet. Esther12 points out that the PACE trial results have been exaggerated, may not occur the same in the real world, and that CBT/GET have been "misleadingly over-sold for the last twenty years". Graham points out that we need to be realistic, about the number of patients needed to treat and show a significant outcome in one patient was about 7 for CBT and 8 for GET, and also refers us to a blog he created with Bob (http://meanalysis.blogspot.com). He and also Esther12 had additional words of wisdom; use your own judgement and express curiosity, "go with the flow" and don't antagonize the GP or "rubbish" PACE (IMO unless it is called for), especially if they are "pro" CBT/GET, therapies which for some people who meet some CFS criteria may help to some extent, but which also have fundamental questions remaining. To compliment what others have already said, you may also wish to consider the following:

Although CBT and GET encourage and presume increases in activity, PACE gave no data for actual increases in activity. The relatively "small" improvements (according to Cohen's rules of thumb, not White et al's loose interpretation of "moderate") were subjective and there was no solid evidence of objective improvement.

The almost total reliance on self-reports is ironic when the authors propose that patients' perceptions and cognitions are distorted/unreliable. The possibility of placebo response is trivialized by the authors, they point out that expectations were poorer in the CBT and GET groups at the beginning of the trial, but ignore the fact that the CBT and GET groups were then conditioned over 52 weeks, as if psychotherapy couldn't possibly induce any response bias or inflated self-efficacy or wishful thinking which isn't accompanied by objective improvements?

As the trial had no adequate placebo control, you may be concerned the results are somewhat illusionary for therapies which are aimed at altering your thought processes, especially considering that a meta-analysis of 3 CBT studies showed that CBT does not lead to increases in physical activity as the proponents of CBT/GET have assumed for over 20 years (http://www.ncbi.nlm.nih.gov/pubmed/20047707). Note that PACE actually had the equipment to run these same tests and did so at baseline but not at followup.

What they defined as "normal" (based on scores for both fatigue and physical functioning at once) and was then erroneously parroted in the news as "recovery", is average for people who are either diseased and/or about 80 years old. Unless you fit that description, this is sub-standard. For physical functioning, this absurdity arose from two dubious methodological processes:

(1) To derive their goalpost for normal ie 60/100 points or more, they claimed to have used a "working age population" but later admitted to using a general population, which included 1/4 who were elderly and 15-22% who were diseased and/or had to cut down on activities recently due to illness. If using datasets from either healthy working age populations, or a similar age group as the PACE participants even from a general population, the definition of normal ie [mean (average) minus 1 standard deviation] should have yielded a lower threshold of 80/100, not 60/100.

(2) They used the [mean -1SD] rule to determine the lower threshold of normal, this rule is supposed to capture about 68% of the population from both sides of the bell curve (see http://en.wikipedia.org/wiki/File:Standard_deviation_diagram.svg) but is useless when the scores follow a non-normal distribution as they do in Bowling et al 1999 as used by PACE, where about 72% of the population are actually scoring 85+/100 in a general population, suggesting that even 85/100 points is an abnormally low threshold, which should be more like 90/100 before we even consider using a healthy age-matched population.

No amount of spin doctoring can justify 60/100 as "normal" for people who were on average 38 years old and aiming for recovery and do not have the other diseases present in a general population (because these would exclude one from a CFS diagnosis). For issues with "normal" fatigue, I haven't examined that properly yet, someone else here may be able to explain why that was suspect as well, although I have the impression that it wasn't as OUTRAGEOUSLY FLAWED as "normal" physical functioning.

The closest thing to an (arguably) "objective" test was the distance covered during a brisk 6 minute walk (6MWD). CBT had no advantage over SMC. The GET advantage over SMC not only struggled to become clinically useful, but on average was much lower than what is considered healthy (600-700m) and is still comparable to people with serious diseases (300-500m).

Among the best two references to illustrate this point: compare GET @ 52 weeks of an average 379m [ standard deviation of 100m] to: (1) 393115m for patients (n=1083, 11 studies) from a diverse range of cardiopulmonary disorders: end stage lung disease, dilated cardiomyopathy, congestive heartfailure, pulmonary hypertension, chronic obstructive pulmonary disease, interstitial lung disease [study range, 294139m (ESLD) to 463107m (DC)] from Ross et al 2010 (http://www.ncbi.nlm.nih.gov/pubmed/20504351); (2) 696151m for "completely healthy" elderly participants (n=58, 20M/38F) aged 627yrs from Bautmans et al 2004 (http://www.ncbi.nlm.nih.gov/pubmed/15272934).

Patient groups who frequently report that pacing is the most helpful therapy for their members, are generally saying that the APT in PACE is not the same as the pacing they understand and practice, so you may be interested in finding out more about "real" pacing rather than the potential strawman known as APT.

Outside of PACE, the lead author Peter White has expressed concerns over the Canadian criteria having symptoms which may indicate alternative diagnoses to CFS. I'm still looking into this issue in specific relation to PACE itself, but one example I've noticed so far is that the Canadian definition discusses "spatial instability" as a symptom, but in the NICE 2007 guidelines used by PACE to help exclude patients this is considered a "red flag" indicative of another disease, ie "Spatial disorientation is not generally characteristic of CFS/ME, and is usually indicative of brain damage. Concentration and memory difficulties (brain fog) are, however, typical." (p176) Therefore, if you have this symptom and/or meet Canadian criteria in general you may wonder how the PACE trial findings can be generalized to you.

The authors changed and watered down all their goalposts halfway through the study to "increase sensitivity" of the results, which is bad if APT was a strawman. Too much to summarize right now, but PACE did publish a protocol in 2007 (http://www.biomedcentral.com/1472-6882/7/12) and have made changes to the trial since then, discussion of which is embedded in this large thread somewhere. Two examples related to what I mentioned above, they dropped their stricter definitions of "positive outcome" and "recovery" then replaced them with weaker versions of "clinically useful" (0.5 SD on data which has an artificially lowered SD) and the dubious definition of "normal" (which itself was a "post-hoc analysis" suggesting that it was possibly done AFTER the authors saw the disappointing trial data, a big no no).

The threshold for a serious adverse event or reaction was rather high and you may be concerned that anything less than that will be deemed "safe" even when you are actually suffering a relapse and further impaired for weeks while the therapist insists you maintain current activity levels:

"(a) Death; b) Life-threatening event; c) Hospitalisation (hospitalisation for elective treatment of a pre-existing condition is not included), d) Increased severe and persistent disability, defined as a significant deterioration in the participants ability to carry out their important activities of daily living of at least four weeks continuous duration; e) Any other important medical condition which may require medical or surgical intervention to prevent one of the other categories listed; f) Any episode of deliberate self-harm."

"Serious deterioration in health was defined as any of the following outcomes: a short form-36 physical function score decrease of 20 or more between baseline and any two consecutive assessment interviews; scores of much or very much worse on the participant-rated clinical global impression change in overall health scale at two consecutive assessment interviews; withdrawal from treatment after 8 weeks because of a participant feeling worse; or a serious adverse reaction."

Unfairly, the critics of the PACE trial have been generally branded as fringe radicals hell bent on a hate campaign which includes death threats. You may be concerned that you will be falsely implicated and wrongly stereotyped for asking crucial questions about issues which will affect your health and livelihood.

Great post, biophile. Very useful summary. Thanks for the effort you went to.

 

[ukxmrv]

Wonderful Biophile!

Thank you for such a great summary.

Also a thank you for all the other regular commentators on this thread who have added so much to this terrific body of knowledge

 

[Graham]

Here's something you might like. The Chalder scale link between bimodal and Likert scoring is so woolly you can't really determine a conversion formula, but it may be valid to look at the actual scores given to me in my survey to give me an estimate. So far I have only got 17 scores, so, unless I was a research psychologist studying ME and GET, it would be unreasonable to have any confidence in the values yet, but converting the baseline and 52 week Chalder scores from Likert to binomial according to that limited data

Therapy: baseline: 52 weeks APT:28.5:23.1 CBT:27.7:20.3 GET:28.2:20.6 SMC:28.3:23.8

becomes

APT:10.8:8.35 CBT:10.7:7.3 GET:10.8:7.6 SMC:10.8:8.46 (i.e. a year's GET+SMC improves the average score from 10.8 to 7.6)

So in effect, CBT and GET add 1.16 and 0.86 points improvement on the binomial scale (and the original aim was to halve the score or reduce to 3)

Now I need a lot more data! If this is true, you can see why they changed it.

 

[Sean]

Nice one, Biophile. Very useful.