BMJ comments on new PACE trial data analysis

[Esther12]

But that is an issue about the difference between doctors' objectives and patients' objectives rather than the statistical issue we were debating. In the end I doubt it will turn out to have been a best endpoint because it will become accepted fairly soon that the trial is uninterpretable - particularly if the authors continue to produce further data that shoots their own hypotheses in the foot.

Maybe I did miss something, but don't those statistical matters depend upon assumptions about researchers having using valuable endpoints? If we're still not confident in our ability to do that for some CFS trials, then wouldn't that favour the sort of analysis User9876 suggested?

 

[Jonathan Edwards]

Maybe I did miss something, but don't those statistical matters depend upon assumptions about researchers having using valuable endpoints? If we're still not confident in our ability to do that for some CFS trials, then wouldn't that favour the sort of analysis User9876 suggested?

I think the two issues are different. User8876 is making the very valid point that often we are interested in the interpretation that seems most likely to be right on the basis of all the facts. But large trials are specifically designed to give us something different - one piece of interpretation that we can be sufficiently sure of to use it as a basis for standard practice. Whether that standard practice is what people really need or want is another matter. I am doubtful that some of the treatments for Alzheimer's produce anything anyone would really want if they thought about it.

I think there is a lot to be said for knocking about ideas for better endpoints - as has been going on on various threads recently. I am getting clearer that what people can do may be a better thing to look at than how they feel but I think it would have to be individualised. I am still of the view that one ought to look for a single preferred measure. That is on the grounds that if your best measure is not adequate then it does not help much to look at even less good measures as well when it comes to being sure. It may well be useful in terms of building deeper understanding and getting a sense of what is most likely to be going on.

 

[Woolie]

Is it even possible implement blinding in psychotherapy research? A placebo treatment must be indistinguishable from the real thing for both patient and therapist. In the context of psychotherapy this sounds like an impossible goal. I think it's also apparent that lowering the standards of research isn't the answer. The logical conclusion is that psychotherapy research of this type is a waste of time and resources.

I think it is possible, if we were able to develop a set of agreed standards. The "placebo" control would need to control for a number of key factors known to affect self-reported improvements in health. Most importantly, it would need to offer: a) the same degree of expectation of improvement as the treatment of interest; b) the same degree of personal investment in the treatment by the patient; and c) the same degree of investment on the part of the therapist, and the same opportunity to develop genuine therapist-patient rapport.

Psychotherapy can be useful for many conditions (e.g., mild depression, phobia, anxiety). I wouldn't want to throw the baby out with the bathwater. But unfortunately, the soft reporting standards in this domain and poor quality control has led to claims that are wildly beyond its scope e.g., increasing the life expectancy of cancer patients!

 

[lansbergen]

I am getting clearer that what people can do may be a better thing to look at than how they feel but I think it would have to be individualised.

The problem is people can do more between flares than during flares. .

A big problem I noticed is that the punish and reward system does not work well.

A donkey does not stumble twice on the same stone but I did it hundreds of times

 

[Woolie]

But thinking more broadly I wonder if one way to mitigate the placebo response is to exclude 'therapy' from the study. A placebo response is generally considered to be a positive response to a therapeutic intervention involving a carer (or team) and a patient.

No, I think this view of the placebo response is too limited. You can get it wherever there is the expectation of improvement from a treatment (therapy or not), and it seems the more the patient invests in the treatment the larger it is. So expensive homeopathic remedies would fit too. I'll try to find you some refs.

especially if assessment is delegated to people hired to score outcomes who have no prior involvement in any therapeutic training .

Yes, scoring by blinded raters is a good way of minimising placebo effects, which are almost entirely restricted to unblinded, self-report measures.

 

[Woolie]

Actually, talk therapy has made a recent comeback in the treatment of schizophrenia, a condition universally acknowledged as real and serious. Apparently the psychologists peddling this nonsense have forgotten that even the father of psychobabble Sigmund Freud was against psychotherapy for psychosis since he saw how completely useless it was for such patients. In the present day it's worse than useless since effective biological therapies exist whereas they did not when Freud was around

Yes, check out this metanalysis which finds no effect of CBT on schizophrenia when objective measures are used, and appropriate control conditions, that control for patient expectation, investment, etc:

Lynch D, Laws KR and McKenna PJ. Cognitive behavioural therapy for major psychiatric disorder: does it really work? A meta-analytical review of well-controlled trials. Psychological Medicine 2010; 40: 9-24
http://uhra.herts.ac.uk/bitstream/handle/2299/5741/903449.pdf?sequence=3

 

[alex3619]

Yes, scoring by blinded raters is a good way of minimising placebo effects, which are almost entirely restricted to unblinded, self-report measures

Which is why I think single blinded studies into these kinds of things are mandatory, and anything less than that cannot be called gold standard.

The reason why I insist on hard objective measures though is that the primary outcome should be an improvement in functional capacity ... most patients demand that. I don't give a damn if I feel a little better, I want to be able to do more. . Subjective measure can be used about quality of life, but for something that that is objective, and can objectively be measured, we should never accept subjective results unless they are used only in support of the primary objective outcome measure to give us better understanding of the nuances of the outcomes.

 

[Woolie]

@MeSci, re "Specialist Medical Care": I wondered whether the specialists might have been psychiatrists. I seem to remember many patients were recruited from psychiatric outpatient clinics. That choice of specialist would certainly fit the researchers' underlying framework.

 

[Woolie]

This is a key problem for questionnaires. And not just for ME/CFS patients, for anyone! Even if a person is trying very hard to be truthful and objective it is difficult to accurately list, for example, just what you had to eat yesterday (e.g., those studies trying to link diet to disease). It's even harder to remember how symptoms have changed over time.

I agree. I think we tend to view placebo responding as a problem with "us silly irrational humans". But it reflects some of the aspects of our design that make us so effective as a species. PDP models of memory consolidation propose that our memory for individual past episodes is limited, and memories that are aligned with previous experience and/or general knowledge therefor have an advantage (they can be more easily integrated within existing networks). So confirmation bias is favoured by efficiency considerations.

The same can be said for human hope. Studies in the 1980s suggested that most people were more optimisitic about their future than their past would predict, and that those that did not show this bias were less effective in life.

These biases are not foibles, merely characteristics. We just need to know about how to control for their effectiveness when trying to answer certain questions.

 

[Dolphin]

Which is why I think single blinded studies into these kinds of things are mandatory, and anything less than that cannot be called gold standard.

The reason why I insist on hard objective measures though is that the primary outcome should be an improvement in functional capacity ... most patients demand that. I don't give a damn if I feel a little better, I want to be able to do more. . Subjective measure can be used about quality of life, but for something that that is objective, and can objectively be measured, we should never accept subjective results unless they are used only in support of the primary objective outcome measure to give us better understanding of the nuances of the outcomes.

And it's not like they think or are saying "physical functioning" is not important in ME/CFS: that's what they claim to be measuring, just using a questionnaire rather than more objective measures.

With something else e.g. depression, there might be more justification for more subjective measures.

 

[Woolie]

And I honestly suspect that very little of this has anything to dow the classical conception of the placebo effect of making the patient feel better. I think it mostly has to do with patients saying what they think the therapist wants to hear.

Bet that's a super biggie amongst you polite and authority-respecting British! I lived in the UK for four years (pre-CFS), and at my first doctor's appointment, I bawled out the doctor for keeping me waiting for an hour and not apologising! You can imagine how that went down!

Agreed, not much evidence that the patient actually feels better in placebo, I don't think the placebo effect is "real" either in that way. But I think not all placebo is "placer" in the Spanish sense. A lot of it is confirmation bias and selective recall of confirmatory episodes.

True about fatigue being possibly particularly vulnerable to placebo, but oddly enough, estimates of placebo effect sizes are not that much bigger for psychological complaints than they are for symptoms of well-attested medical illnesses. I posted a range of articles of estimates for a number of complaints, can't seem to see them now, but will chase up if of value.

But it sounds like we pretty much agree on the rest.

 

[Dolphin]

@MeSci, re "Specialist Medical Care": I wondered whether the specialists might have been psychiatrists. I seem to remember many patients were recruited from psychiatric outpatient clinics. That choice of specialist would certainly fit the researchers' underlying framework.

The appendix for the Lancet paper (White et al., 2011) gives this info:

4 general practitioners║
7 physicians ¶
27 liaison psychiatrists‡

║General practitioners with a special interest in CFS. ¶ All physicians specialised in infectious diseases. ‡ 22 of the 27 liaison psychiatrists were from one centre. 300 (47%) participants were treated by physicians, 184 (29%) by psychiatrists, and 149 (23%) by GPs. All the physicians and GPs had completed training. 4 psychiatrists had completed training; the rest were trainees. Therapists treated a median (range)of 14 (2-33) participants. Doctors provided SMC alone for a median (range) of 3 (1-19) participants.

 

[Dolphin]

I agree. I think we tend to view placebo responding as a problem with "us silly irrational humans". But it reflects some of the aspects of our design that make us so effective as a species. PDP models of memory consolidation propose that our memory for individual past episodes is limited, and memories that are aligned with previous experience and/or general knowledge therefor have an advantage (they can be more easily integrated within existing networks). So confirmation bias is favoured by efficiency considerations.

The same can be said for human hope. Studies in the 1980s suggested that most people were more optimisitic about their future than their past would predict, and that those that did not show this bias were less effective in life.

These biases are not foibles, merely characteristics. We just need to know about how to control for their effectiveness when trying to answer certain questions.

I've read the odd thing to the effect that people with depression often have more realistic views while the supposedly healthier people (i.e. those who don't have depression) often have less realistic (i.e. overly optimistic) views by comparison. Something along those lines.

 

[WillowJ]

Isn't there a risk for many people (those with an exaggerated startle response) that the prompt will provoke such an adrenaline rush that the patients will feel momentarily highly energised? I used to startle very easily and get a huge rush of nervous energy.

Or wobbly and sick.

But maybe the prompt could have enough different options that one could find something sufficiently gentle (and others who wanted something louder could find that).

 

[WillowJ]

I've read the odd thing to the effect that people with depression often have more realistic views while the supposedly healthier people (i.e. those who don't have depression) often have less realistic (i.e. overly optimistic) views by comparison. Something along those lines.

I've read similar things about pessimism helping people to make realistic choices and not take unnecessary risks.

 

[MeSci]

Bet that's a super biggie amongst you polite and authority-respecting British! I lived in the UK for four years (pre-CFS), and at my first doctor's appointment, I bawled out the doctor for keeping me waiting for an hour and not apologising! You can imagine how that went down!

Agreed, not much evidence that the patient actually feels better in placebo, I don't think the placebo effect is "real" either in that way. But I think not all placebo is "placer" in the Spanish sense. A lot of it is confirmation bias and selective recall of confirmatory episodes.

True about fatigue being possibly particularly vulnerable to placebo, but oddly enough, estimates of placebo effect sizes are not that much bigger for psychological complaints than they are for symptoms of well-attested medical illnesses. I posted a range of articles of estimates for a number of complaints, can't seem to see them now, but will chase up if of value.

But it sounds like we pretty much agree on the rest.

Maybe part of the reason we don't tend to argue with or complain to doctors is that it's not easy to find different, better ones and switch to them. So we feel vulnerable, and fear that if we make a doctor angry they will treat us badly. This is even more the case in hospital - one is so vulnerable. And the services are (mostly) free, so we may feel that it's wrong to complain, even though most of us have paid for it through taxes.

But the amount of time patients often have to wait, and the poor service and treatment we often get, makes it outrageous that there have been proposals to fine patients who don't turn up to appointments or are late. If they can do that, we should be able to fine them when they don't meet expectations!

Questionnaires can be worded in such a way as to get the answers one wants. There has been research on this, and you can even get a completely-different/opposite reply if you word a question differently, even if it means the same. I think that people are often more inclined to reply 'yes' than 'no' for example.

 

[MeSci]

The appendix for the Lancet paper (White et al., 2011) gives this info:

4 general practitioners║
7 physicians ¶
27 liaison psychiatrists‡

║General practitioners with a special interest in CFS. ¶ All physicians specialised in infectious diseases. ‡ 22 of the 27 liaison psychiatrists were from one centre. 300 (47%) participants were treated by physicians, 184 (29%) by psychiatrists, and 149 (23%) by GPs. All the physicians and GPs had completed training. 4 psychiatrists had completed training; the rest were trainees. Therapists treated a median (range)of 14 (2-33) participants. Doctors provided SMC alone for a median (range) of 3 (1-19) participants.

What a hotchpotch!

 

[Sidereal]

The appendix for the Lancet paper (White et al., 2011) gives this info:

4 general practitioners║
7 physicians ¶
27 liaison psychiatrists‡
║General practitioners with a special interest in CFS. ¶ All physicians specialised in infectious diseases. ‡ 22 of the 27 liaison psychiatrists were from one centre. 300 (47%) participants were treated by physicians, 184 (29%) by psychiatrists, and 149 (23%) by GPs. All the physicians and GPs had completed training. 4 psychiatrists had completed training; the rest were trainees. Therapists treated a median (range)of 14 (2-33) participants. Doctors provided SMC alone for a median (range) of 3 (1-19) participants.

Wow, what utter nonsense. How can GPs, infectious disease doctors, and consultant psychiatrists or registrars in psychiatry all be said to be delivering the same intervention, this mythical "specialist medical care"? So the SMC group likely consisted of many different interventions ranging from pure contempt, through callous indifference, to maybe sympathetic attitude and supportive care for symptoms like sleep and pain.

 

[Valentijn]

Wow, what utter nonsense. How can GPs, infectious disease doctors, and consultant psychiatrists or registrars in psychiatry all be said to be delivering the same intervention, this mythical "specialist medical care"? So the SMC group likely consisted of many different interventions ranging from pure contempt, through callous indifference, to maybe sympathetic attitude and supportive care for symptoms like sleep and pain.

They're all ME/CFS "specialists", so were working at CBT/GET clinics. The care from all would be pretty similar, in that it's a typical lack of care with the possible exception of pain or sleep meds.

Specialism doesn't matter at all in this context, and is actually often just used to mislead patients. "See, it's real treatment being offered - it's from an immunologist!" But they don't mention that the immunologist, or neurologist, or other type of physician whole-heartedly supports the psychological theories and treatments for ME/CFS.

 

[Sidereal]

I've been able to improve how I feel, but not the amount I can do, and these ongoing limitations are a key part of my health problems. I'd rather be able to do the amount of stuff I was prior to falling ill, without having an improvement in how I feel, than feel as well as I did before falling ill without being able to do any more than I do now. When how I feel and activity levels are so tied up this is a bit difficult to talk about.

This is really such a crucial point when it comes to assessing the effectiveness of various interventions in ME/CFS.

There have been many studies and personal anecdotes on forums like this one over the years suggesting that various strategies like pacing, aggressive rest, medications and nutritional supplements are effective at reducing subjective perception of symptoms and improving quality of life but it is much more rare that any of these interventions end up reliably improving functional capacity. Often when people come on the internet extolling the virtues of various things, upon further questioning it turns out that while it makes them feel better they are still disabled "oh it's a bit easier to make it to the letterbox and back... I think... on some days". Or things that do improve capacity in a minority of patients don't work for others or make others worse.

To an outsider looking in it can certainly seem as though this illness is an ever-changing landscape of messy ill-defined symptoms coming and going seemingly at random but the underlying fundamental problem of extreme disability is always there and for many (most?) of us functional capacity does not differ all that much despite various treatments. Sure, there are good days and bad days, but even on a good day we're not well enough to fake normality or hold down a job or even do all of our ADLs.

In this illness there is really a poor correlation between how someone feels and how functionally impaired they are. In the early years of my illness my symptom load and distress were much greater than now but I was a thousand times more functional. Paul Cheney who has been treating ME/CFS patients for 30 years in the States has spoken about this many times and has identified what he believes are stages of this illness characterised by differing levels of misery vs disability. At first many patients experience very high misery levels but low disability. They feel awful, everything hurts, prominent viral symptoms like sore throats, glands, low-grade fevers etc. But they're still ambulatory, still able to make it to doctor appointments, they may still be trying to work full or part-time etc. Then later on something seems to shift and people begin to feel better in some ways but are now stuck in this locked-in condition of having to drop out of life and being able to do virtually nothing and if they try to do anything, the symptoms immediately come crashing back and the payback is severe. But, with extremely strict pacing and an unfathomably limited rigid lifestyle, it is possible to feel relatively ok once you're many years into this illness.

This staging model of course doesn't apply to people who are consciously or inadvertently overdoing it or those who are on a deteriorating progressive course regardless of how much they rest. Very severe ME is constant unrelenting torture and such people can't make it to doctors' offices or research studies so nothing is known about this form of illness except anecdotes and harrowing reports or movies about people who have died, often after abuse/neglect by GPs and fatigue clinics delivering SMC/CBT/GET/antidepressants.

This is why I think that, paradoxically, while this is a disease defined by subjective symptoms as Chalder pointed out, self-report questionnaires have no role in assessing outcome of any intervention in ME. Stuff like actigraphy, CPET, employment outcomes, receipt of welfare benefits etc. are much less susceptible to placebo effects than ticking boxes on questionnaires. No matter how much you wanna please your therapist or doctor you can't fake normality long enough to hold down a job or change your anaerobic threshold.