BMJ comments on new PACE trial data analysis

[chipmunk1]

http://www.ncbi.nlm.nih.gov/pubmed/18256342

randomized controlled design was used where 72 patients with MS fatigue were randomly assigned to eight weekly sessions of CBT or relaxation training (RT). RT was designed to control for therapist time and attention. Participants were assessed before and after treatment, and at 3 and 6 months posttreatment. The primary outcome was the Fatigue Scale. Secondary outcomes included measures of stress, mood, and fatigue-related impairment.

INTERPRETATION:
Both CBT and RT appear to be clinically effective treatments for fatigue in MS patients, although the effects for CBT are greater than those for RT. Even 6 months after treatment, both treatment groups reported levels of fatigue equivalent to those of the healthy comparison group.
----

In the real world you would not expect to cure MS fatigue by seeing a therapist a few times. I think this shows all what is wrong with psych research. Studies can show seemingly dramatic results from fairly neutral interventions.

http://www.ncbi.nlm.nih.gov/pubmed/21672300

BACKGROUND:
Chronic fatigue is a common symptom of multiple sclerosis (MS). A randomized controlled trial (RCT) showed that cognitive behavioural therapy (CBT) was more effective in reducing MS fatigue than relaxation training (RT). The aim of the current study was to analyse additional data from this trial to determine whether (1) CBT compared to RT leads to significantly greater changes in cognitions and behaviours hypothesized to perpetuate MS fatigue; (2) changes in these variables mediate the effect of CBT on MS fatigue; and (3) these mediation effects are independent of changes in mood.

METHOD:
Seventy patients (CBT, n=35; RT, n=35) completed the Cognitive and Behavioural Responses to Symptoms Questionnaire (CBSQ), the Brief Illness Perception Questionnaire (B-IPQ) modified to measure negative representations of fatigue, the Hospital Anxiety and Depression Scale (HADS), and the Chalder Fatigue Questionnaire (CFQ), pre- and post-therapy. Multiple mediation analysis was used to determine which variables mediated the change in fatigue.

RESULTS:
Avoidance behaviour and three cognitive variables (symptom focusing, believing symptoms are a sign of damage and a negative representation of fatigue) improved significantly more in the CBT than the RT group. Mediation analysis showed that changing negative representations of fatigue mediated the decrease in severity of fatigue. Change in anxiety covaried with reduction in fatigue but the mediation effect for negative representations of fatigue remained when controlling for improvements in mood.

CONCLUSIONS:
Change in beliefs about fatigue play a crucial role in CBT for MS fatigue. These beliefs and the role of anxiety deserve more attention in the further development of this intervention.

i find it really interesting that they don't want to seem to accept that it is possible to feel fatigued for physiological reasons.

 

[A.B.]

So, about that hypothetical study of CBT for car engine problems. I really think it would show a reduction in car engine problems if we do it King's College style. Maybe it's even cost effective compared to traditional interventions. Oh the possibilities. Where do I apply for a research grant?

 

[chipmunk1]

So, about that hypothetical study of CBT for car engine problems. I really think it would show a reduction in car engine problems if we do it King's College style.

a reduction in false car engine problem beliefs...

 

[Aurator]

So in the PACE trial maybe SMC was seeing a doctor who says "Well you have chosen the standard care - which is actually no care at all

So it appears the term Specialist Medical Care in our case may be an example of that phenomenon referred to by Orwell in Politics and the English Language as "giving an appearance of solidity to pure wind"; a phrase, I see, that was used even more appositely than here by the Countess of Mar when, in the concluding remarks of her House of Lords speech of 2013, she neatly summarized the achievement of the PACE trial.

 

[user9876]

Normally secondary measures are sunk by the Bonferoni correction with marginal power in the first place. If you do statistics on the secondary measures you compromise the validity of your primary analysis too. I think the hypothesis driven approach to statistics may run into too many unascertainables about the certainty of a particular prediction applying to a particular hypothesis and how wide a net a hypothesis can cast.

It seems strange to me that in an experiment a single outcome is preselected and a test is done to see if that particular variable from two different data groups differ. Rather than saying what evidence do we have in favour of the given hypothesis and what against (in this way we can compare a number of different hypothesises rather than saying data from two groups is the same or not the same).

The problem with secondary outcomes is (and I assume Bonferoni) is that some may just show and effect that isn't there. But shouldn't we use all the data we can collect and combine the evidence (both in favour and against). I would go as far as arguing that it is interesting to look at what evidence does and doesn't support a hypothesis and ask why.

Ultimately its a frequentist vs baysian argument and I guess I'm a believer in baysian inference which too me always has seemed to have more solid foundations and be closer to logical reasoning, Whereas I find significance testing arguments a bit counter intuitive when I've tried to understand the equations.

Still I guess for a medical trial to be accepted a frequentist approach needs to be taken.

 

[worldbackwards]

So it appears the term Specialist Medical Care in our case may be an example of that phenomenon referred to by Orwell in Politics and the English Language as "giving an appearance of solidity to pure wind"; a phrase, I see, that was used even more appositely than here by the Countess of Mar when, in the concluding remarks of her House of Lords speech of 2013, she neatly summarized the achievement of the PACE trial.

...and as Lord Winston also amply demonstrated afterwards!

 

[user9876]

a reduction in false car engine problem beliefs...

I could argue that my car recently had a false engine problem belief. The brake lights had gone which caused the car to believe that the brakes were on so the revs wouldn't go up when in gear. Although maybe it should be considered a sensory problem. Needless to say the garage didn't mend the car using CBT but just changed the bulbs.

 

[Bob]

I'm not well read on placebo effects or response bias, but it is possible to design a sort of placebo control into a cognitive-behavioural trial. It's been done in other trials. For example, you can have a placebo pill as a control arm, or sham acupuncture (safely using needles by someone untrained in acupuncture), sham homeopathy, or sham cognitive therapy delivered by an actor, or use medical equipment like a transcranial magnetic stimulation machine with the power turned off. I think all of these have been used as placebo arms. It may not entirely control for response bias though, unless the participants being given the dummy treatment are told that CFS is reversible and that the treatment will lead to a recovery, as they were for CBT/GET, and they are given equal positive encouragement. And the assessment stage should be overseen only by an independent assessor.

 

[Bob]

This is a placebo effect meta-analysis that was pointed out to me recently...

Is the placebo powerless? An analysis of clinical trials comparing placebo with no treatment.
Hróbjartsson A, Gøtzsche PC. (2001)
N Engl J Med. 344:1594-602.
http://www.ncbi.nlm.nih.gov/pubmed/11372012

The paper describes the typical placebo interventions that were used:
"The typical pharmacologic placebo was a lactose tablet. The typical physical placebo was a procedure performed with a machine that was turned off (e.g., sham transcutaneous electrical nerve stimulation). The typical psychological placebo was a nondirectional, neutral discussion between the patient and the treatment provider, referred to as an “attention placebo.” No treatment typically entailed observation only or standard therapy; in the latter case, all patients in the trial received standard therapy, and the placebo was additional."


In this meta-analysis, they compared placebo to no treatment. For continuous outcome measures (as opposed to binary measures) there was a significant placebo effect for subjective outcomes but no significant placebo effect for objective outcomes. For binary measures there was no significant effect for either subjective or objective outcomes. I think the outcomes for pain may have been different from others - I don't understand the differences yet.

 

[Esther12]

This is a placebo effect meta-analysis that was pointed out to me recently...

It's been updated here: http://www.ncbi.nlm.nih.gov/pubmed/20091554

Placebo interventions for all clinical conditions.
Hróbjartsson A1, Gøtzsche PC.
Author information
Abstract
BACKGROUND:
Placebo interventions are often claimed to substantially improve patient-reported and observer-reported outcomes in many clinical conditions, but most reports on effects of placebos are based on studies that have not randomised patients to placebo or no treatment. Two previous versions of this review from 2001 and 2004 found that placebo interventions in general did not have clinically important effects, but that there were possible beneficial effects on patient-reported outcomes, especially pain. Since then several relevant trials have been published.

OBJECTIVES:
Our primary aims were to assess the effect of placebo interventions in general across all clinical conditions, and to investigate the effects of placebo interventions on specific clinical conditions. Our secondary aims were to assess whether the effect of placebo treatments differed for patient-reported and observer-reported outcomes, and to explore other reasons for variations in effect.

SEARCH STRATEGY:
We searched the Cochrane Central Register of Controlled Trials (CENTRAL, The Cochrane Library Issue 4, 2007), MEDLINE (1966 to March 2008), EMBASE (1980 to March 2008), PsycINFO (1887 to March 2008) and Biological Abstracts (1986 to March 2008). We contacted experts on placebo research, and read references in the included trials.

SELECTION CRITERIA:
We included randomised placebo trials with a no-treatment control group investigating any health problem.

DATA COLLECTION AND ANALYSIS:
Two authors independently assessed trial quality and extracted data. We contacted study authors for additional information. Trials with binary data were summarised using relative risk (a value of less than 1 indicates a beneficial effect of placebo), and trials with continuous outcomes were summarised using standardised mean difference (a negative value indicates a beneficial effect of placebo).

MAIN RESULTS:
Outcome data were available in 202 out of 234 included trials, investigating 60 clinical conditions. We regarded the risk of bias as low in only 16 trials (8%), five of which had binary outcomes.In 44 studies with binary outcomes (6041 patients), there was moderate heterogeneity (P < 0.001; I(2) 45%) but no clear difference in effects between small and large trials (symmetrical funnel plot). The overall pooled effect of placebo was a relative risk of 0.93 (95% confidence interval (CI) 0.88 to 0.99). The pooled relative risk for patient-reported outcomes was 0.93 (95% CI 0.86 to 1.00) and for observer-reported outcomes 0.93 (95% CI 0.85 to 1.02). We found no statistically significant effect of placebo interventions in four clinical conditions that had been investigated in three trials or more: pain, nausea, smoking, and depression, but confidence intervals were wide. The effect on pain varied considerably, even among trials with low risk of bias.In 158 trials with continuous outcomes (10,525 patients), there was moderate heterogeneity (P < 0.001; I(2) 42%), and considerable variation in effects between small and large trials (asymmetrical funnel plot). It is therefore a questionable procedure to pool all the trials, and we did so mainly as a basis for exploring causes for heterogeneity. We found an overall effect of placebo treatments, standardised mean difference (SMD) -0.23 (95% CI -0.28 to -0.17). The SMD for patient-reported outcomes was -0.26 (95% CI -0.32 to -0.19), and for observer-reported outcomes, SMD -0.13 (95% CI -0.24 to -0.02). We found an effect on pain, SMD -0.28 (95% CI -0.36 to -0.19)); nausea, SMD -0.25 (-0.46 to -0.04)), asthma (-0.35 (-0.70 to -0.01)), and phobia (SMD -0.63 (95% CI -1.17 to -0.08)). The effect on pain was very variable, also among trials with low risk of bias. Four similarly-designed acupuncture trials conducted by an overlapping group of authors reported large effects (SMD -0.68 (-0.85 to -0.50)) whereas three other pain trials reported low or no effect (SMD -0.13 (-0.28 to 0.03)). The pooled effect on nausea was small, but consistent. The effects on phobia and asthma were very uncertain due to high risk of bias. There was no statistically significant effect of placebo interventions in the seven other clinical conditions investigated in three trials or more: smoking, dementia, depression, obesity, hypertension, insomnia and anxiety, but confidence intervals were wide.Meta-regression analyses showed that larger effects of placebo interventions were associated with physical placebo interventions (e.g. sham acupuncture), patient-involved outcomes (patient-reported outcomes and observer-reported outcomes involving patient cooperation), small trials, and trials with the explicit purpose of studying placebo. Larger effects of placebo were also found in trials that did not inform patients about the possible placebo intervention.

AUTHORS' CONCLUSIONS:
We did not find that placebo interventions have important clinical effects in general. However, in certain settings placebo interventions can influence patient-reported outcomes, especially pain and nausea, though it is difficult to distinguish patient-reported effects of placebo from biased reporting. The effect on pain varied, even among trials with low risk of bias, from negligible to clinically important. Variations in the effect of placebo were partly explained by variations in how trials were conducted and how patients were informed.

 

[Aurator]

...and as Lord Winston also amply demonstrated afterwards!

Yes, and the vague nature of Baroness Northover's touchy-feely reassurances about "many patients" having had a "positive experience", and doctors and other professionals being "committed to providing the best possible care" neatly dovetails with the reality of SMC being all spin and no substance:

"At this point it is worth acknowledging that, while many patients feel let down by the system, many patients have had a positive experience and have worked with health and social care professionals to manage their care effectively. It is also important to acknowledge that doctors and other professionals working in this field are highly committed to providing the best possible care for their patients. We want to see people with CFS/ME being listened to when it comes to decisions about what type of treatment and care may best meet their individual needs and, across the country, many doctors are working with their patients to achieve this."

BTW, I'm intrigued to know how we are to understand the logic of saying that "many patients feel let down by the system" and yet at the same time "many have had a positive experience".

 

[Snow Leopard]

So for me the important point is that a subjective endpoint is the right endpoint if you can design a blinded study. If you cannot do a blinded study you really need something better than even actometry to even think it is worth doing a trial. I am not sure we have that. So it is not that PACE should have used other endpoints. It was never going to be valid anyway. We need different methodologies for answering questions about unblindable therapies. Not just endpoints.

It sounds to me, more of a debate between science based medicine and evidence based medicine (which technically permits the use of placebos, since it doesn't require understanding of how treatments work).

In terms of evidence based medicine, It is simply about weighing the quality and quantity of evidence. You are right, that actigraphy or fitness testing or whatever cannot prove that the effects are not due to placebo (or crystal healing, or sending patients to the moon and back etc.). But when those same measurements contradict the questionnaire results, then questions need to be asked about the quality of those questionnaire results.

 

[Bob]

"At this point it is worth acknowledging that, while many patients feel let down by the system, many patients have had a positive experience and have worked with health and social care professionals to manage their care effectively. It is also important to acknowledge that doctors and other professionals working in this field are highly committed to providing the best possible care for their patients. We want to see people with CFS/ME being listened to when it comes to decisions about what type of treatment and care may best meet their individual needs and, across the country, many doctors are working with their patients to achieve this."

I wonder if that includes the medical professionals who post on the BMJ that ME/CFS is a "meme"?
Oh, and the ones who insist that ME/CFS is a false/unhelpful illness belief?

I'm sure most of us have come across medical professionals who are helpful, caring and compassionate, but I think most of us have come across the opposite as well.

Considering the limitations placed upon the medical profession, I certainly don't agree that doctors are working towards achieving the care that may best meet their patients' individual needs. The best I've received from a GP is an honest and sympathetic apology that they couldn't do anything to help, except to send me on another CBT course.

 

[Sean]

"At this point it is worth acknowledging that, while many patients feel let down by the system, many patients have had a positive experience and have worked with health and social care professionals to manage their care effectively. It is also important to acknowledge that doctors and other professionals working in this field are highly committed to providing the best possible care for their patients. We want to see people with CFS/ME being listened to when it comes to decisions about what type of treatment and care may best meet their individual needs and, across the country, many doctors are working with their patients to achieve this."

I am going to need a bigger bucket.

 

[WillowJ]

What matters for the ME patient is how awful they feel. Walking time might reflect that, but it might not. Going to work might reflect that and it might not. Moreover, all these so-called objective measures like actometry and walking time and employment are, in the terms of the placebo effect, subjective measures. They can all very credibly be affected by beliefs. So actually they are no better and they are less to the point. If there were objective measures like a CRP that would be different.

That's fair point about actometry, walking time, and employment being, to some respect, subjective as well, although the ability to "fake wellness" it does depend on how sick a person is. I managed ok during the first half of the years I was ill, then I began to decline. Now I can fake wellness for 15 minutes or so, but not very long (depends on the day, how long I have been seated, whether my legs are up, and so on). Certainly not long enough to stay in school or stay employed. Sometimes not even long enough to sound sharp through an entire doctor visit any more (which used to throw them off--she looks fine, she sounds fine, but she says she can't do anything or think right!)

While I agree that this is complicated and there are no ideal outcome measures, for me personally, I think that we should not emphasise how people feel over what they can do.

I agree wholeheartedly with that. While HRQOL is important and I think symptom reduction is a valuable goal, I think the most important goal is actually having a life. I guess most ME patients agree, because many will say that they push their limits in some way, and actually choose to do things they know will increase symptoms in favor of being able to do at least some of the most important things, whether this is taking a shower or attending an event.

The great thing about individualised targets is that you can vary them as much as you like as long as you stipulate exactly what they are before you randomise a patient to a treatment. We did this for lupus and it worked well.

That sounds very interesting, and I would never have thought that would be valid, but I can understand that different patients might have different expectations and this might be realistic, as they might have different levels of complications.

I would worry in some cases that patients might be pressured to select something that is non-optimal for them, based on the investigator's prejudices (e.g. CFS/ME patients are overachievers who need to pare down their life expectations... children with CFS/ME are recovered if they can attend school 90% of the time and we don't care whether they do any extracurricular activities or get passing grades...). But probably you had other users in mind.

 

[WillowJ]

I was going to say that SMC should have been called bog-standard-care but, to be fair, I never received a leaflet when I was diagnosed. (Well, really, I diagnosed myself, so it seemed a bit silly to give myself a leaflet.)

I was diagnosed by a doctor, but I didn't get a leaflet. I wrote my own, and gave copies to my doctors!

 

[OverTheHills]

I think that we should not emphasise how people feel over what they can do. I've been able to improve how I feel, but not the amount I can do, and these ongoing limitations are a key part of my health problems. I'd rather be able to do the amount of stuff I was prior to falling ill, without having an improvement in how I feel, than feel as well as I did before falling ill without being able to do any more than I do now.

BRAVO @Esther12. I think you have really hit the nail on the head here.

I suspect that most PWME feel the same way, because being able to do stuff (perhaps while still feeling a bit ill) is the key to independence, security, fulfillment, respect, etc etc

 

[MeSci]

This is a key problem for questionnaires. And not just for ME/CFS patients, for anyone! Even if a person is trying very hard to be truthful and objective it is difficult to accurately list, for example, just what you had to eat yesterday (e.g., those studies trying to link diet to disease). It's even harder to remember how symptoms have changed over time.

I will often forget a key symptom when talking about my illness. The most blatant example for me (that I can remember) is the time when I was in my doctor's office with my husband. I was reporting my symptoms and I said something like, "I haven't had a migraine in weeks." And I really believed that. Then my husband said, "But you just had one on Tuesday." I was embarrassed (which is probably why I still remember this incident) but it was true. I had just had a migraine a few days earlier.

I was not trying to be super positive about my symptoms because I was listing a bunch of other problems. And I don't think I was subconsciously omitting information to make the doctor happy. I had simply forgotten all about it. As soon as my husband mentioned it then I remembered it. So that memory was in my brain somewhere. And yet, when asked to list my symptoms, I had completely forgotten one of the big ones that had just happened recently and wiped me out for half a day.

This kind of thing happens to me all the time. It's so frustrating because I did not used to be so ditzy and flakey (former software engineer, attention to detail, etc.). I actually had a lot less brain fog at the start of my illness. Not sure why.

I know that no one has a perfect memory, and has biases, and so on. But this is ridiculous.

I'm not saying don't use questionnaires for ME/CFS studies, or don't believe ME/CFS patients, or anything like that. I'm just pointing out a problem (which I'm sure is obvious to most of you) with using questionnaires.

Hope this is useful and not a derail.

Reminds me of a routine eye appointment I had about a week ago. The ophthalmologist asked if I had had any headaches. I said no. Then fortunately I recalled that I had had quite severe headaches two days running a few months earlier, accompanied by other symptoms that my GP was sufficiently concerned about to get me rushed to hospital for brain and neck scans, fearing that I had had TIAs! The neuro diagnosed migraine, of which I have no known prior history.

But I had forgotten!

 

[MeSci]

One can improve, at least somewhat, the accuracy of self-reporting by using live, ecological measurements e.g. electronic diaries. These involve getting a prompt asking for the level of one or more particular symptoms at a particular moment in time. Fred Friedberg had tested this in at least one CFS study:

Isn't there a risk for many people (those with an exaggerated startle response) that the prompt will provoke such an adrenaline rush that the patients will feel momentarily highly energised? I used to startle very easily and get a huge rush of nervous energy.

 

[MeSci]

...and as Lord Winston also amply demonstrated afterwards!

Winston is one of those mediocre scientists who is lionised, honoured and quoted ad nauseam by the British establishment as is done with Wessely. Never mind whether he has any expertise in the specific subject area, he is the one who is trotted out to talk vague, patronising, platitudinous hogwash.