PACE Trial and PACE Trial Protocol

[Bob]

Dolphin, I hope you don't mind me quoting you from another thread...

The CDC doesn't generally use the Chalder Fatigue Scale. The questionnaire they use in the empiric criteria definition is the MFI-20:

"Reduced activity" isn't a measure of fatigue and so could easily be complained about i.e. if you want to be specific, say you don't want them to use that subscale. It's hard for them to argue with that than simply saying not to use the whole questionnaire.

Similarly, it would be harder for them to argue against not using the role emotional subscale of the SF-36, than if one requested the whole SF-36 not be used:

"We defined substantial reduction in occupational, educational, social, or recreational activities as scores lower than the 25th percentile of published US population [11] on the physical function (≤ 70), or role physical (≤ 50), or social function (≤ 75), or role emotional (≤ 66.7) subscales of the SF-36."

The paper is at: http://www.biomedcentral.com/1741-7015/3/19

Well, this is interesting... I wasn't aware of this...
If I've understood this correctly, the Reeves criteria says that a substantial reduction in activities on the SF-36 PF scale would be a score of 70 or less:

Illness classification by standardized clinically empirical criteria

We used information from the SF-36, MFI and Symptom Inventory to classify subjects empirically according to the 3 main dimensions of CFS: functional impairment (SF-36), fatigue (MFI) and accompanying symptoms (Symptom Inventory). We defined substantial reduction in occupational, educational, social, or recreational activities as scores lower than the 25th percentile of published US population [11] on the physical function (≤ 70)...
http://www.biomedcentral.com/1741-7015/3/19

Helpful evidence for demonstrating that a score of 60 is nowhere near a 'normal' score.

 

[biophile]

Goalposts for improvement shifted down from 9-14 points to 2 (fatigue) and 20-30 points to 8 (physical function).

Has there been an FoI request for the statistical analysis strategy? Too me their complete lack of openness around the results and how they processed them, coupled with their definition of 'normal' suggests that the results they had didn't say what they wanted hence they were scrambling around for something positive to say.

I agree there has been a lack of openness. With a strange delay in publishing additional data. They may even refuse to publish some of it, there was a conference abstract on mediating factors which didn't look like they found anything important, and someone said they aren't going to publish that (despite the protocol mentioning mediating factors being an important question to be answered).

I don't recall exactly if FOIs have specifically targeted the Trial Steering Committee, but I'm sure there have been attempts to find out more about what happened behind the scenes during the protocol changes. I've examined the paper thoroughly and it seems to me that adjunctive CBT/GET was generally only about about 25-50% as effective as expected in the protocol and the change in goalposts was quite large.

The original goalpost for positive outcome was either <=3/11 bimodal score for fatigue or 50% reduction in baseline score, and either >=75/100 score for physical function or 50% increase of baseline score. The 50% reduction criterion equates to: for fatigue, which they changed from bimodal to Likert scoring, an average about 14 out of 33 points, but the 2011 Lancet paper arrived at a mere 2 points for this threshold; for physical function on average it was about 20 out of 100 points, but in the 2011 Lancet paper it was only 8 points.

A minimal "clinically important difference" was defined in the 2007 protocol as somewhere between 2 to 3 times the improvement rate of SMC-alone. This equates on average to about 9-14 out of 33 points for fatigue and 20-30 points out of 100 for physical function. Again, compare that to the 2011 Lancet paper where the goalpost was a mere 2 point improvement in fatigue and 8 point improvement in physical function, these were derived from a "clinically useful difference" or "improvement" defined in the 2011 Lancet paper as half a standard deviation of the group baseline score. It can be argued that the SD was artificially low due to ceiling effects in the scoring and the exclusion of severely and mildly affected patients.

Recovery was defined in the 2007 protocol, but absent from the 2011 Lancet paper, the closest thing to it was the ridiculous definition of "normal" range in fatigue and physical function which overlaps with trial entry criteria, is clearly not "normal" for healthy people, and isn't close to their original definition of "positive outcome" which itself was less strict than recovery. I find it annoying that the accompanying Lancet editorial claimed "normal" was recovery based on healthy persons' scores. It was extremely sloppy and any other explanation involves casting serious doubt on their competence or integrity eg deliberate spin. The Lancet still haven't corrected it, so I guess they don't mind inaccuracies.

Anyway, these large changes in goalposts explain why the response rate was 6 times higher than expected in the SMC-alone quasi control group. It required these greatly reduced thresholds to get the expected response rate from the CBT and GET groups. However, it is possible that the SMC group improved more than expected anyway.

The PACE authors argue that the changes were to improve the "sensitivity" of the results. Sounds good, but might be problematic if too sensitive, especially if APT wasn't pacing. Also, when the objective outcomes do not reach close to the subjective outcomes, it is more difficult to separate small differences in subjective outcomes from reactivity bias in an unblinded trial where the therapy groups favoured by the authors received much more optimistic claims/expectations about improvements or recovery and were trained to view symptoms as less obnoxious. No such improvement in sensitivity was afforded to rates of deterioration or adverse effects, in fact I think one of them was even made stricter or less sensitive.

I think Dolphin pointed out earlier, that even if they did not see the data before protocol changes and "post-hoc" analyses, there could have been an atmosphere of less than expected results, it isn't like those involved were properly blinded, they only mentioned that discussion about how things were going was "discouraged". The whole point of blinding it to eliminate such biases, the fact that blinding was impractical doesn't excuse the fact that it was unblinded. Also, the sibling FINE Trial probably scared them, the PACE Trial protocol changes seemed to coincide with the FINE Trial failure, especially the part about changing the fatigue score from bimodal to Likert which allowed them to squeeze out a significant result in the FINE Trial that wasn't there before with bimodal scoring.

 

[Sean]

Good post, BP.

The huge difference between the PACE author's expectations, and the actual results, is one of the most starkly revealing facts to emerge from the whole study.

 

[Don Quichotte]

You may disagree with me, but I think that the publication of a study on CFS in a leading journal in beneficial for the CFS community regardless of what this study is about.
Because, it significantly increases the awareness to this disease in the medical community.
Politicians know that being in the headlines is more important than their agenda.
Commercials are mostly based on finding the best way to make you remember the name of the product. Many times you do not remember what it is and why you picked it up from the shelf when you went to the supermarket.
The major issue with CFS was that it was not seen as a serious medical problem which requires any intervention.
It was seen as a self-limiting problem which could be the result of a viral illness or the normal stresses of life.
Physicians confidently told their patients (what was written in their text-books) that there is nothing to worry about and within a short period they will be fine.
This may be true for most people with post-viral fatigue or normal life stresses, but this is not true for patients with CFS.
I think the Lancet paper made it clear that this is a serious and true medical condition, of which the exact etiology and pathophysiology are still poorly understood. Even they don't say it is caused by thoughts and emotions, they just say this contributes to the illness (just like they do in cancer, lupus or any other disease).

One can argue if the risk of GET out-weights the benefit or not , just like one can argue regarding risk/benefit of any intervention for any disease.
Some people see CBT as the solution to all human problems and others see it as ridiculous and there is an entire spectrum in between. But, again this is not specific for CFS.

 

[biophile]

You make a good point, Don Quichotte, about exposure. When starting from a position that CFS is not important at all, the PACE Trial is an improvement from that. But that change may come at the cost of people then viewing CFS as a "serious" cognitive-behavioural illness instead.

While it is true that the authors do not claim CFS is simply "caused" by thoughts and emotions, most of the example triggers hypothesized in the cognitive behavioural model of CFS are psychological and social, while all of the significant perpetuating factors (which are presumed to be predominant) are cognitive and behavioural.

CBT for CFS goes beyond coping with a chronic illness and attempts to reverse the illness itself, based on the assumption that thoughts-emotions-behaviours are what perpetuates the symptoms and disability in this functional illness, not disease processes. CBT for cancer and lupus etc is not routine and is aimed more at coping and management compliance, not based on the idea that the disease of cancer or lupus etc itself (or the symptoms and disability ie illness) are predominately perpetuated by cognitive and behavioural factors and even reversible through psychotherapy.

The notion that thoughts-emotions-behaviours contribute to all illness and disease seems to be overused as a justification for applying CBT and for deflecting criticisms of speculated factors. It also generally ignores differences between the strengths of association; the onset and course of some illnesses/diseases do appear to be affected to some degree by such factors, but others less so or not at all. PACE Trial criticism was often dismissed on the basis that patients/advocates were (supposedly) arguing from an outdated position of mind-body dualism and naive about the proverbially powerful mind-body connection where psychological factors can affect all illness/disease.

 

[Don Quichotte]

PACE Trial criticism was often dismissed on the basis that patients/advocates were (supposedly) arguing from an outdated position of mind-body dualism and naive about the proverbially powerful mind-body connection where psychological factors can affect all illness/disease.

This is a ridiculous criticism, because the PACE trial itself is based on this exact dualism.

It is based on the assumption that not only are the mind and body different and easily separable parts, but that you can actually measure accurately the "mind" and intervene in its function just like you can measure and correct a simple physiological parameter such as the blood count or heart rate.

In fact the entire field of psychology/psychiatry is based on this mind-body dualism (or else why would this field be required at all, if the mind and the body are one and the same, physicians would be taking care of the entire patient just like they did in the time of Hipocrates or is done in oriental medicine in which this dualism doesn't exist).

This criticism is similar to a wealthy banker saying that (as opposed to him) his clients are greedy and only interested in money, because they complain about the low interest rate they are receiving. (while he makes enormous profits on their expense).

You can't make a living as a psychologist/psychiatrist and at them same time complain about mind-body dualism.
Or rather you can, if you are good in Rhetorics!

 

[Don Quichotte]

BTW, I personally do not think there is a mind-body dualism.

I think that we are structured to respond to the changing environment. This we basically do by our senses (which are interpreted and integrated by our brain), by our immune system and by our thought process.

Our actions (which are mostly orchestrated by our brain), are the result of the compilation of all this complex data our brain receives continuously in order to monitor changes in our environment and respond accordingly. Most of our actions are programmed to be automatic (which is more economical).

Our thought process can over-ride and modify automatic actions. But, this is a more complex process and requires more time and energy.

For example- when we touch something hot, we automatically move our hand from it. At some point we no longer have to touch it, we know that fire (we see with our eyes) is hot and we will not put our hand into it.
Yet, if we see our child in the fire, our thoughts and emotions will over-ride this instinct and we will put our hand into it to save our child. We also learn that touching a picture of a fire is not dangerous, even tough it looks the same.

This is not "psychology" but cognitive adaptation. Just like your body can adapt to a higher altitude by increasing the level of red cells.

A blind man, can be taught to use other senses to know his place and position. This is not "psychology" either but cognitive adaptation.

we can also control our "autonomic" functions (such as breathing, blood pressure, heart rate) to some extent, by similarly over-riding the automatic input of our brain, using higher (cognitive) functions.

This is not "psychology" but a physiological process of our brain. This gives us the ability to better adapt to numerous situations that could not be foreseen or programmed. This creates the flexibility required for our survival.
Because of its complexity, this is also prone to numerous errors which we can correct (if we realize our mistake).

For the same reason (enormous flexibility) , it is relatively easy to manipulate.
That is why you can convince people to put their and other's life at risk for ridiculous causes that are made to be seen as the essence of life and even more important than life.

My higher brain (cognitive) functions combined with my healthy survival instincts, made me realize I should stay away from physicians who criticize people's outdated positions regarding mind-body dualism and at the same time readily give very mono-minded psychological explanations for everything that requires they use their own higher brain functions to understand. (This may be seen as a form of self-administered CBT, if you like).

 

[Bob]

You may disagree with me, but I think that the publication of a study on CFS in a leading journal in beneficial for the CFS community regardless of what this study is about.
Because, it significantly increases the awareness to this disease in the medical community.
Politicians know that being in the headlines is more important than their agenda.
Commercials are mostly based on finding the best way to make you remember the name of the product. Many times you do not remember what it is and why you picked it up from the shelf when you went to the supermarket.
The major issue with CFS was that it was not seen as a serious medical problem which requires any intervention.
It was seen as a self-limiting problem which could be the result of a viral illness or the normal stresses of life.
Physicians confidently told their patients (what was written in their text-books) that there is nothing to worry about and within a short period they will be fine.
This may be true for most people with post-viral fatigue or normal life stresses, but this is not true for patients with CFS.
I think the Lancet paper made it clear that this is a serious and true medical condition, of which the exact etiology and pathophysiology are still poorly understood. Even they don't say it is caused by thoughts and emotions, they just say this contributes to the illness (just like they do in cancer, lupus or any other disease).

I understand your point, and I hadn't looked at it that way before.
But I don't think that I agree with you, based on how the results have been promoted.
Newspaper headlines were along the lines of "ME patients should exercise to the max in order to recovery" and "30% of ME patients recovered after exercising to their limits."
All nonsense of course. No one recovered (as far as we yet know), and only a mere 13% responded to treatment, and CBT was ineffective at reducing disability.
So the misrepresentation of the results by the authors has harmed us, in my opinion.

One can argue if the risk of GET out-weights the benefit or not , just like one can argue regarding risk/benefit of any intervention for any disease.
Some people see CBT as the solution to all human problems and others see it as ridiculous and there is an entire spectrum in between. But, again this is not specific for CFS.

Only an average only 13% responded to CBT and GET, and CBT was ineffective at reducing disability.
The other three primary outcomes were assessed to have a 'moderate' effect size (pretty small).
Severely affected patients were excluded from the PACE Trial, but they were found not to respond to CBT and GET in the FINE Trial.
So there isn't much of an argument to be had about the benefits. There were hardly any.
When and if we get the data for the deterioration rates, then we'll see how many patients were harmed. My guess is that they were quite high.

 

[Graham]

I wrote to NICE about our PACE analysis, and got back a reasonable but neutral response. Included in it were these lines:

In accordance with our processes this guidance will next be considered for review in August 2013. I can confirm that I have logged your email in our system so that it can be brought to the attention of the clinical guidelines team when they come to review the guidance. When the review takes place, our Information Services team will also conduct a thorough search of new published evidence, so we would also encourage you to make sure that as much information as possible is published in peer-reviewed journals before the review begins.

So, hmmm! Published in peer-reviewed journals. Now where have I heard that before? I'm sure someone somewhere much wiser than me said I ought to be doing that.

Not much chance of getting anything into The Lancet or BMJ of course. So, let's go wild and suppose that I rewrite the survey results, and include the graphics linking bimodal and Likert scoring for the fatigue scale, any advice or suggestions?

On another tack, I did think of producing a short series of notes on statistics aimed at medical analysis. Bob's seen a draft of one of them. Any thoughts about that? The aim is simply to go right back to basics and put things like averages, standard deviation, conditional probablility etc. into proper context. Perhaps I could build up a bit of statistical kudos that way.

It still has to remain a second string though. I haven't given up pushing the PACE analysis.

 

[Don Quichotte]

so we would also encourage you to make sure that as much information as possible is published in peer-reviewed journals before the review begins

I think this is good advice.

I wonder if you can convince one of those journals to publish a debate.
This is a good way to create a discussion, increase awareness and make people think.

 

[user9876]

So, hmmm! Published in peer-reviewed journals. Now where have I heard that before? I'm sure someone somewhere much wiser than me said I ought to be doing that.

.

My feeling is that a really good paper could be written about statistics from medical surveys and the dangers on relying on simple stats. The biggest problem is the lack of any underlying data to back that up. Given the failure to get data through the FoI I had two thoughts.

One is to look at the consequence of various ways data is collected and processed on basic statistics. The idea here would be to set up a model based on a number of distributions (or multidimentional distributions, or copulas) where the model goes through the sets of filling out questionaires, summating results and generating basic stats (mean, std, median and cronburg alpha). Here we understand both the input (distributions) and output of the model and hence can talk about the quality of the metrics provided. I don't know what this would throw up but my feeling is would could talk about conditions where the use of the supmmary stats are justified or wrong. We could then do a brief literature search to give examples of different papers such as PACE, FINE... who quote the stats without appropriate justifications. If we can link the message to a current 'sexy' area such as the need for open data I think this would be a great thing. If it is seen as a ME/CFS paper it will get rejected from medical journals due to the choice of reviewers. The argument then to NICE is that they should examine the raw data from PACE since the results published are not reliable.

The other thought was to collect data from members. I think you may have already started doing this. A longitudanal study (i.e. collect stats over time) would be good. Ideally we could set up some sort of recording system where people can record activities as well as fatigue and SF-36 scores. I'm not sure what we could show but I thing a couple of interesting aspects would be how scores vary over time and how the different components vary with respect to the overall rating. Having a lot of varition may just suggest that the sample set was too small to make conclusions.

I'm quite happy to help with the work and writing.

 

[alex3619]

BTW, I personally do not think there is a mind-body dualism.

I think that we are structured to respond to the changing environment. This we basically do by our senses (which are interpreted and integrated by our brain), by our immune system and by our thought process.

Our actions (which are mostly orchestrated by our brain), are the result of the compilation of all this complex data our brain receives continuously in order to monitor changes in our environment and respond accordingly. Most of our actions are programmed to be automatic (which is more economical).

Our thought process can over-ride and modify automatic actions. But, this is a more complex process and requires more time and energy.

For example- when we touch something hot, we automatically move our hand from it. At some point we no longer have to touch it, we know that fire (we see with our eyes) is hot and we will not put our hand into it.
Yet, if we see our child in the fire, our thoughts and emotions will over-ride this instinct and we will put our hand into it to save our child. We also learn that touching a picture of a fire is not dangerous, even tough it looks the same.

This is not "psychology" but cognitive adaptation. Just like your body can adapt to a higher altitude by increasing the level of red cells.

A blind man, can be taught to use other senses to know his place and position. This is not "psychology" either but cognitive adaptation.

we can also control our "autonomic" functions (such as breathing, blood pressure, heart rate) to some extent, by similarly over-riding the automatic input of our brain, using higher (cognitive) functions.

Hi Don Quichotte, at the risk of going off topic (do we want a thread on mind body dualism?) my personal view is hybrid monist/dualist. I think there is only brain, but the complexity of an adaptive massively parrallel processing system is sometimes easier thought of as mind - this is a descriptive choice, not reality.

There are more than two brains anyway, broadly speaking. Some are unconscious, doing most of the day to day processing, and are responsible for quick decisions and snap judgements. This is not very manipulable, though it does learn from experience - so its what the B part of CBT focusses on. The other is conscious, is grounded in languistic and similar manipulable skills. This is what C part to CBT focusses on.

However the brain takes all data and inputs for its adaptive process. What you hear, and what you tell yourself, are just two more inputs in a sea of inputs. So either C or B of CBT is just a limited input. There is only so much it can do. However by changing self talk habits they modify what stimuli are being received. Its stimulus modification, not cure.

I really dislike the occasional attempt to describe the mind-body problem using a computer analogy. The brain is hardware, our mind is software, for example. This is a nonsense. The hardware is capable of changing itself adaptively, something a computer has trouble doing. "Software" is more a data set - a collection of experiences that the brain has accumulated through adaptation to stimuli, and resulting in actual rewiring of the brain.

A lot of what I think about brain is influenced by the work of Maturana and Varela. I actually met Maturana once.

I have debated whether to write a mind-body blog. I didn't think enough would be intererested to justify it.

Bye, Alex

 

[Sean]

PACE is a short-medium term disaster for patients. No way around that.

But it is a long term catastrophe for psychiatry (and The Lancet). No way around that either.

 

[Don Quichotte]

Hi Don Quichotte, at the risk of going off topic (do we want a thread on mind body dualism?) my personal view is hybrid monist/dualist. I think there is only brain, but the complexity of an adaptive massively parrallel processing system is sometimes easier thought of as mind - this is a descriptive choice, not reality.

There are more than two brains anyway, broadly speaking. Some are unconscious, doing most of the day to day processing, and are responsible for quick decisions and snap judgements. This is not very manipulable, though it does learn from experience - so its what the B part of CBT focusses on. The other is conscious, is grounded in languistic and similar manipulable skills. This is what C part to CBT focusses on.

However the brain takes all data and inputs for its adaptive process. What you hear, and what you tell yourself, are just two more inputs in a sea of inputs. So either C or B of CBT is just a limited input. There is only so much it can do. However by changing self talk habits they modify what stimuli are being received. Its stimulus modification, not cure.

I really dislike the occasional attempt to describe the mind-body problem using a computer analogy. The brain is hardware, our mind is software, for example. This is a nonsense. The hardware is capable of changing itself adaptively, something a computer has trouble doing. "Software" is more a data set - a collection of experiences that the brain has accumulated through adaptation to stimuli, and resulting in actual rewiring of the brain.

A lot of what I think about brain is influenced by the work of Maturana and Varela. I actually met Maturana once.

I have debated whether to write a mind-body blog. I didn't think enough would be intererested to justify it.

Bye, Alex

Hi Alex,

I fully agree with you that comparing our brain (a very complex biological system we know very little about, which functions for years) to a computer (a man created device which usually can't handle what it should do for more than a limited time) is both arrogant (thinking that we can be as smart as nature) and ridiculous.

When I said there is no mind-body dualism I meant it from a pure biological point of view. I think that you can not differentiate what we call physiological responses (which is basically what we can easily measure and comprehend with our senses) from emotions and thoughts (which we can't measure). I think they are all interconnected in one network. Whenever there is failure of a part of the system (namely disease) it effects all other parts. it can lead to warning signals reaching the brain (eg-pain from an abcess) , wrong signals reaching the brain (eg-neuropathy), wrong input coming from the brain (eg-stroke) and the need to "manually" asses automatic processes (eg-Parkinson's disease) . It is inevitably accompanied by a rapid (if there is an immediate threat) or complex and gradual (if there is an ongoing milder threat) process of adaptation of the functioning systems in order to preserve survival.

We diagnose an illness not only by the disease process, but by the way the organism normally responds to it. That is why when there is a complex disease process leading to a different form of adaptation we find it hard to diagnose it, unless we are aware of this complexity.
(For example a patient with neutropenia and bacterial pneumonia may have a completely normal chest X-ray, because what we normally see on an X-ray is not the bacteria but the leukocyte infiltrate in response to it; A patient with schizopherenia may have an abdominal abcess, lytic bone lesion, but not show the normal distress due to the severe accompanying pain, because of abnormal pain perception).

If you look at the history of medicine, you will see that once a certain unmeasurable or invisible process could be measured or assessed by one of the senses, it became a "physiological process". Such as the development of the stethoscope, CT scan, EEG etc. This is also the reason for the constant search for "biomarkers". and the (ridiculous) attempts to "map" thoughts, emotions, personality traits and behaviours to certain areas of the brain by showing increased uptake of glucose in those areas.

The mind as a concept is something different. We can't explain poetry, art, music in biological or scientific terms. We can't map talent. We may look at the brain of a genius using functional MRI, and we may possibly see specific areas of the brain more active, but can this explain why he/she is capable of solving a complex mathematical problem no one else could?

Science is a way of thinking and tool. It can not be applied to everything.
Therefore trying to understand complex cognitive processes using this tool is bound to fail.
Just like you can't measure how much water you have using a ruler.

In a way trying to understand our cognitive function, by science (a tool created by our cognitive function) is like measuring something by that something itself. It will inevitably create a logical loop.

We have to humbly accept that we can not explain or understand everything because of the limits of our cognitive abilities. We can wisely use our abilities only if and when we are ready to accept our disabilities.

 

[alex3619]

Hi Don Quichotte, I almost fully agree with post 1734. I do think that science can indeed explain these things though ... eventually. I doubt it will be this century, and it might not be this millenium, but the future is a big place. For now we are guessing or making simplistic observations. The brain is the most complex thing we have ever studied. Bye, Alex

 

[biophile]

I might contribute to the mind-body discussion another time, but to be frank, I cannot see the 2013 review of the NICE guidelines changing much from recommending optional CBT or GET for mildly to moderately affected patients. At most there are good arguments for additional caveats.

For example, CBT/GET does not generally lead to increased activity according to actigraphy. In the PACE Trial, there is no advantage in 6MWD scores for CBT, and the GET advantage was minimal. The number needed to treat in PACE was about 7 or 8 patients for 1 to experience a small to moderate benefit in self-reported fatigue and physical function, the obvious majority of patients aren't helped so is a blanket recommendation worth it? There were improvements to other symptoms but the psychological benefits appear to be minimal.

Still awaiting data on recovery, employment outcomes, cost-effectiveness, and rate of deterioration according to the same thresholds used for improvements. It is worth mentioning that CBT/GET is only "safe" if conducted properly, which ironically means that patients don't have to increase activity as the rationale presumes occurs.

It will be argued that despite these weaknesses, CBT/GET are still the best there is and better than nothing, helping some patients to reduce perceived fatigue and perceived disability. One possibility is that the PACE Trial will be used to actively discourage pacing and insist CBT/GET are effective and safe. Another possibility is that the FINE Trial results will be misinterpreted to recommend pragmatic rehabilitation for severely affected, or used to discourage supportive listening.

 

[Bob]

I might contribute to the mind-body discussion another time, but to be frank, I cannot see the 2013 review of the NICE guidelines changing much from recommending optional CBT or GET for mildly to moderately affected patients. At most there are good arguments for additional caveats.

For example, CBT/GET does not generally lead to increased activity according to actigraphy. In the PACE Trial, there is no advantage in 6MWD scores for CBT, and the GET advantage was minimal. The number needed to treat in PACE was about 7 or 8 patients for 1 to experience a small to moderate benefit in self-reported fatigue and physical function, the obvious majority of patients aren't helped so is a blanket recommendation worth it? There were improvements to other symptoms but the psychological benefits appear to be minimal.

Still awaiting data on recovery, employment outcomes, cost-effectiveness, and rate of deterioration according to the same thresholds used for improvements. It is worth mentioning that CBT/GET is only "safe" if conducted properly, which ironically means that patients don't have to increase activity as the rationale presumes occurs.

It will be argued that despite these weaknesses, CBT/GET are still the best there is and better than nothing, helping some patients to reduce perceived fatigue and perceived disability. One possibility is that the PACE Trial will be used to actively discourage pacing and insist CBT/GET are effective and safe. Another possibility is that the FINE Trial results will be misinterpreted to recommend pragmatic rehabilitation for severely affected, or used to discourage supportive listening.

Yes, I tend to agree that might be the outcome. I'm hoping that there will be enough evidence to persuade NICE to at least add extra caveats. But I'm optimistic that will be the worst outcome for the next review. I'm hoping for better.

Something to add to your post biophile was that CBT was found to be ineffective in the PACE Trial for physical disability using the primary outcome measure, SF-36 Physical Function.

I'm not sure how the FINE Trial could be misinterpreted? It wasn't promoted as a success at all. Pragmatic rehabilitation failed when measured at follow up. Are you referring to the initial responses to it?

 

[biophile]

Something to add to your post biophile was that CBT was found to be ineffective in the PACE Trial for physical disability using the primary outcome measure, SF-36 Physical Function.

It was? In [Table 3], compared to SMC at 52 weeks, CBT showed an average advantage of 7.1 (95% CI: 2.0 to 12.1) points, which was statistically significant (unadjusted p value 0.0068, Bonferroni adjusted p value 0.0342). It could be argued that 7.1 points was below the threshold for clinically useful difference of 8 points, but these were average scores and 71% patients did improve by 8 points or more (vs 58% for SMC). However, it is still unclear how effective CBT was at improving physical function for these patients, but it doesn't look particularly good for most patients. The number needed to treat for CBT to improve self-reported physical function by 8 or more points in one patient was 8 patients.

I'm not sure how the FINE Trial could be misinterpreted? It wasn't promoted as a success at all. Pragmatic rehabilitation failed when measured at follow up. Are you referring to the initial responses to it?

IIRC, the FINE Trial found a small but statistically significant improvement in self-reported fatigue immediately post-treatment but not at longer followup. There was no improvement in physical function at either time point. When the authors later changed from bimodal to Likert scoring, the small but statistically significant improvement in self-reported fatigue was sustained at longer followup.

You're probably right that it wasn't promoted as much of a success. However, the potential for misinterpretation and spin shouldn't be underestimated. The accompanying editorial avoided a frank assessment, painted a relatively optimistic picture compared to the dismal results, and opined that the results would probably have been better if patients just received more sessions from better trained specialists. Also, a recent systematic review into exercise guidelines for CFS (Van Cauwenbergh et al 2012) claims that FINE "showed significant improvement (P < 0.05) in fatigue and physical functioning". I'm unclear on how they concluded that, and the entire paper is generally devoid of effect sizes.

Some of the FINE Trial nurses regarded patients as the "bastards that didn't want to get better", so let's hope that part doesn't get misconstrued into the NICE guidelines!

 

[Bob]

It was? In [Table 3], compared to SMC at 52 weeks, CBT showed an average advantage of 7.1 (95% CI: 2.0 to 12.1) points, which was statistically significant (unadjusted p value 0.0068, Bonferroni adjusted p value 0.0342). It could be argued that 7.1 points was below the threshold for clinically useful difference of 8 points, but these were average scores and 71% patients did improve by 8 points or more (vs 58% for SMC). However, it is still unclear how effective CBT was at improving physical function for these patients, but it doesn't look particularly good for most patients. The number needed to treat for CBT to improve self-reported physical function by 8 or more points in one patient was 8 patients.

Yes, the 'effect size' for CBT SF-36 PF was 'small'. All the other primary outcomes were 'moderate'. 8 points was the threshold for a 'moderate' effect size. And 8 points was also the threshold for a 'clinically useful difference', meaning that CBT was clinically ineffective at reducing physical disability for both the primary and secondary outcomes.
That's the way the Trial was set up, so it is legitimate to say that CBT was ineffective (clinically ineffective) for that primary outcome measure (SF-36 physical function).

The authors successfully managed to hide this result though, by not highlighting it in the text at all, so hardly anyone knows.

Here's the only place in the text of the publish paper (as opposed to the data tables) where we are told that CBT failed:
“Mean differences between groups on primary
outcomes almost always exceeded predefined clinically
useful differences for CBT and GET when compared
with APT and SMC.”

In my opinion, the authors also misrepresented and obfuscated this result for CBT in the text, and in post-publication promotion, with the clever use of words, and by not qualifying or clarifying their statements.
Here's an example of where they misrepresented the results in the paper:

“Discussion
We suggest that these findings show that either CBT or GET, when
added to SMC, is an effective treatment for chronic
fatigue syndrome, and that the size of this effect is
moderate”

(Although not strictly a lie, they did not qualify that statement, and they should have done, because the effect size of CBT was 'small' for SF-36 PF, and only 'moderate' for Chalder fatigue. I believe that sort of obfuscation is against MRC contractual rules, and I'm looking into that.)

IIRC, the FINE Trial found a small but statistically significant improvement in self-reported fatigue immediately post-treatment but not at longer followup. There was no improvement in physical function at either time point. When the authors later changed from bimodal to Likert scoring, the small but statistically significant improvement in self-reported fatigue was sustained at longer followup.

You're probably right that it wasn't promoted as much of a success. However, the potential for misinterpretation and spin shouldn't be underestimated. The accompanying editorial avoided a frank assessment, painted a relatively optimistic picture compared to the dismal results, and opined that the results would probably have been better if patients just received more sessions from better trained specialists. Also, a recent systematic review into exercise guidelines for CFS (Van Cauwenbergh et al 2012) claims that FINE "showed significant improvement (P < 0.05) in fatigue and physical functioning". I'm unclear on how they concluded that, and the entire paper is generally devoid of effect sizes.

Some of the FINE Trial nurses regarded patients as the "bastards that didn't want to get better", so let's hope that part doesn't get misconstrued into the NICE guidelines!

I thought that all the improvements were clinically insignificant at follow up? That's what I've read repeatedly anyway, but I haven't looked at the results in detail.

 

[biophile]

White et al 2011 wrote:

Mean differences between groups on primary outcomes almost always exceeded predefined clinically useful differences for CBT and GET when compared with APT and SMC. [...] We suggest that these findings show that either CBT or GET, when added to SMC, is an effective treatment for chronic fatigue syndrome, and that the size of this effect is moderate."

Yes, the 'effect size' for CBT SF-36 PF was 'small'. All the other primary outcomes were 'moderate'. 8 points was the threshold for a 'moderate' effect size. And 8 points was also the threshold for a 'clinically useful difference', meaning that CBT was clinically ineffective at reducing physical disability for both the primary and secondary outcomes.

You're correct Bob, they seem to be basing the statement of "moderate" effect on the clinical useful difference, and it can be deduced that "almost always" refers to CBT not reaching this threshold for physical function. And as you said, they do hide this fact in their conclusion by claiming that CBT and GET have a moderate effect size, without clarifying that CBT was not moderate in one of the two primary measures ie physical function.

Their "clinically useful difference" threshold has problems as it was a rather low due to a smaller than usual standard deviation in baseline scores. It led to the situation where an improvement of a mere 2 out of 33 points in fatigue was regarded as "moderate". As CBT and GET both showed an advantage of over 3 points on average, I'm surprised they didn't claim this to be a "large" effect. To demonstrate the problem further, imagine if the SMC group improved by 40 points and the CBT group by an additional 8 points. According to PACE logic, the effect of CBT would still be regarded as "moderate". Another example, imagine if the standard deviation of baseline scores was only 2 points because of a more homogeneous group in terms of fatigue severity and physical function score, then suppose the SMC group improved by 40 points and the CBT group by only an additional 8 points, the effect size for CBT would have been considered to be extremely large despite being relatively small.

The implication of goalpost shifting is also obvious, neither the effect size of CBT nor GET reached their original definition of "clinically important difference" (2 to 3 times the improvement rate of SMC) for improvements in fatigue and physical function. IIRC, the effect sizes or advantage over SMC did not reach what is usually regarded as "moderate" according to Cohens' rules of thumb for interpreting effect sizes (although it was probably somewhat close in some measures and these are just guidelines and depend on the study subject and the measure involve, for example, in other studies a "small" effect may still save thousands of lives).

As I said previously, the equivalent goalposts for improvement shifted down greatly from 9-14 points to 2 (fatigue) and 20-30 points to 8 (physical function), quite a large change. The authors claim that all protocol changes, and presumably post-hoc analyses (ie post- protocol design and post- study initialization but before seeing data), were approved "before outcome data were examined" and "before examining outcome data to avoid outcome reporting bias". However, this makes me wonder, in this context does "outcome data" necessarily include baseline data too? Did they have early access to baseline data before analyzing the outcome results? If so, they could have realized that the standard deviation of baseline scores would allow them to greatly lower the threshold for improvement, for "sensitivity" purposes of course. ;-)

That's the way the Trial was set up, so it is legitimate to say that CBT was ineffective for that primary outcome measure (SF-36 physical function).

Well, obviously not clinically useful on average, but as you mentioned elsewhere it technically had a small effect on average. It also apparently helped 1 person out of 8 (NNT) to achieve a supposedly moderate effect, although even then CBT could have just helped them improve ever so slightly as to scrape over the threshold if they were close to it anyway with just SMC, rather than CBT having a useful or moderate effect. CBT may have helped a slim minority make larger improvements.

I thought that all the improvements were clinically insignificant at follow up? That's what I've read repeatedly anyway, but I haven't looked at the results in detail.

They were. But as I said, changing from bimodal to Likert scoring allowed the small statistically significant improvement in fatigue to be sustained at followup. I don't think it was published, it was mentioned in a BMJ Rapid Response. Also, someone could argue that the initial small effect only faded because they weren't receiving "booster shots" of cognitive behavioural wisdom! I don't know how the systematic review I mentioned came to the conclusion that FINE resulted in improved physical function, I discuss some possibilities in a previous post on a different thread (http://forums.phoenixrising.me/inde...-on-cfs-and-exercise.17782/page-2#post-271208). It appears to be an error, but it is only one of several such "errors" made in professional interpretations of PACE and FINE (think Crawley, Bleijenberg & Knoop, BMJ "Short Cuts", news article coverage, etc).

Considering how much spin has occurred elsewhere, could this be the PACE and FINE appeal to NICE?: "You spin me right round baby right round like a record baby right round round round."