NYT letter to editor re: David Tuller article

[SpecialK82]

Boo Hiss - The principal investigators of the PACE trial comment of Davild Tuller's article:

http://www.nytimes.com/2011/03/15/health/15letters-STUDYINGAFAT_LETTERS.html?_r=1

In Defining an Illness Is Fodder for Debate (March 8), your reporter David Tuller correctly noted that the way an illness is defined can often determine what is found in studies of it. He also suggested that this problem of definition had limited the interpretation of our trial of treatments for chronic fatigue syndrome, published recently in The Lancet. That is not the case.

The patients in this trial had a disabling chronic illness in which fatigue was their main symptom and for which no alternative had been found; that is the definition of the syndrome used in Britain. But we also assessed trial participants to see if they met two other definitions of the illness that are favored by some scientists.

We found that both cognitive behavior therapy and graded exercise therapy, when added to specialist medical care, were most effective not only in the whole sample but also in the participants who met these alternative criteria. In addition, these treatments were the most effective whether or not a patient was depressed, a not uncommon accompaniment to this chronic and misunderstood illness. The trial also found that these treatments were safe so long as they were provided by appropriate therapists trained to help patients with chronic fatigue syndrome.

So to Mr. Tullers question Does the evidence from that study prove that these strategies would help patients identified as having chronic fatigue syndrome through very different criteria?, the answer is Yes, it does. Patients and their doctors now have robust evidence that there are two safe treatments that can improve both symptoms and quality of life, however the illness is defined.

P. D. White

T. Chalder

M. Sharpe

London and Edinburgh


The writers are the principal investigators of the PACE trial.


David Tuller replies: The article asked whether findings among a population defined by one set of criteria would apply to populations defined by very different criteria. In this study, all participants were first defined, identified and selected not by different criteria but by the same criteria, the so-called Oxford criteria used in Britain. Subgroups within that already screened population who also meet secondary criteria are not easily compared to patients who have not been screened, since an unknown number who met the secondary criteria might not have met the studys criteria for inclusion. The gold standard for making comparisons across groups of patients identified by three varying case definitions would be a study with three completely separate cohorts, not one large sample with two embedded subgroups.


Science Times welcomes letters from readers. Those submitted for publication must include the writers name, address and telephone number. E-mail should be sent to scitimes@nytimes.com. Send letters to Science Editor, The New York Times, 620 Eighth Avenue, New York, N.Y. 10018.

 

[Sean]

David Tuller replies: The article asked whether findings among a population defined by one set of criteria would apply to populations defined by very different criteria. In this study, all participants were first defined, identified and selected not by different criteria but by the same criteria, the so-called Oxford criteria used in Britain. Subgroups within that already screened population who also meet secondary criteria are not easily compared to patients who have not been screened, since an unknown number who met the secondary criteria might not have met the studys criteria for inclusion. The gold standard for making comparisons across groups of patients identified by three varying case definitions would be a study with three completely separate cohorts, not one large sample with two embedded subgroups.

Nice work, Mr Tuller.

 

[urbantravels]

There seem to be very few flies on David Tuller.

"Robust evidence" my afflicted a$$... Wish there were more reporters out there capable of spotting such obvious bad science.

 

[WillowJ]

yes, very perceptive, Mr. Tuller

 

[CBS]

$8 million

All that money and nothing but "results" that confuse rather than clarify. Par for the course for government funded CFS research (either side of the pond). If I didn't know any better (and I don't), I just might start to think that confusion was the underlying goal.

Nice to see Tuller call them on their antics.

 

[justinreilly]

I sent Tuller a two sentence thank you email. Perhaps everyone doing the same would give him some positive reinforcement. I avoided making it longwinded so I didn't inadvertantly punish him for his good deed. There is an email form here:

http://journalism.berkeley.edu/contact/david_tuller/

 

[slayadragon]

I have SUCH the crush on this reporter.

I sent him a thank you note too, Justin. Thanks for the good idea.

Lisa

 

[SpecialK82]

Good idea on letters to Tuller, he had a nice response.

There is so much wrong with the PACE investigators logic, they excluded all patients that had any neurological disorders. One of the main symptoms of CFS is cognitive dysfunction, many have autonomic nervous system problems, Baraniuk's study as well as the latest proteomic study both point to neurologic abnormalities and some of the additional proteins found are also found in Alzheimers and Parkinson's patients.

If we can extrapolate results into other patient groups that weren't included in the study - maybe we can say that GET and CBT are good treatments for Alzheimer's and Parkinson's as well ----- unbelievable.

 

[Wayne]

I sent Tuller a two sentence thank you email. Perhaps everyone doing the same would give him some positive reinforcement. I avoided making it longwinded so I didn't inadvertantly punish him for his good deed. There is an email form here:

http://journalism.berkeley.edu/contact/david_tuller/

Hey Justin, thanks much for the e-mail link. I sent him a short thank you as well.

Wayne

 

[Sing]

I did too! And I am going to hang onto the link for David Tuller as I am sure he will be writing again.

Sing

 

[Ember]

Does anyone know what "two other definitions of the illness" were used in the Pace trials?

 

[Angela Kennedy]

Good idea on letters to Tuller, he had a nice response.

There is so much wrong with the PACE investigators logic, they excluded all patients that had any neurological disorders. One of the main symptoms of CFS is cognitive dysfunction, many have autonomic nervous system problems, Baraniuk's study as well as the latest proteomic study both point to neurologic abnormalities and some of the additional proteins found are also found in Alzheimers and Parkinson's patients.

If we can extrapolate results into other patient groups that weren't included in the study - maybe we can say that GET and CBT are good treatments for Alzheimer's and Parkinson's as well ----- unbelievable.

Yes. These are good points. It was in the PACE authors' interest NOT to include any person who could have ANY neuro disease (forget about ME for the moment). The example I've been using is MS, which does have similarities of symptoms (and signs) to ME.

 

[Angela Kennedy]

Does anyone know what "two other definitions of the illness" were used in the Pace trials?

i've been discussing this over on the PACE thread. I'll dig out my specific posts on the subject if you like (it's complicated)...

 

[Sing]

He wrote back with his thank you. I made the same point as you, SpecialK82. It is good to know that he is accessible!

 

[Ember]

i've been discussing this over on the PACE thread. I'll dig out my specific posts on the subject if you like (it's complicated)...

I'd appreciate that, Angela. If there were a simplified version, it would be easier to understand the letter to the editor. "Two other definitions of the illness that are favoured by some scientists" sounds good on the surface. Is one of the two the London criteria?

 

[Angela Kennedy]

I'd appreciate that, Angela. If there were a simplified version, it would be easier to understand the letter to the editor. "Two other definitions of the illness that are favoured by some scientists" sounds good on the surface. Is one of the two the London criteria?

Here's my latest post of the problems I've found from the PACE article and PACE protocols. I'm hoping it's clear to understand- but please do ask me any questions you need to.

It is still important that we address the problem of the cohorts- we can't forget this issue, and here's why.

Lacklustre results notwithstanding, this trial has been spun, both in the Lancet and in the press, as substantiating the safety of CBT/GET for ME sufferers (even if they use their term 'CFS/ME'). These results are generally no more impressive than any CBT/GET trials done in the past, but that wasn't necessarily their primary aim. This appears to have been to dismiss the claim that CBT/GET is unsafe for ME sufferers, because this is an extremely serious allegation.

Now they've undertaken some outstanding ontological gerrymandering to do this, but they've managed to pooh-pooh concerns about safety, and THIS will be the serious issue people will be faced with in the future.

Showing how poor the results of this trial are is one thing (in important one). But it will still be business as usual unless we can show that CBT/GET is still potentially unsafe. The reasons we know CBT/GET has not been established as 'safe' for ME sufferers are:

1. The PACE cohorts have potentially eliminated all ME sufferers from the trial. AT BEST, very few will have got in. Maybe none at all were in there. If this happened, it will have been achieved at the doctor examination/history taking stage, which WAS ad hoc (the only standardised form was a sign off by the research nurse after the doctors had 'screened' the patients). It would be highly problematic for the doctors to include any people with neurological deficit in the trial- because those deficits may have represented other neurological illnesses, like MS etc. I believe it is significant that so many people attending the 'specialist clinics' (over 1000) were deemed not to have met Oxford. Previous 'CFS' research cohorts in the UK have been worked so that people with organic dysfunction seen in ME (say Canada, even the historical ME case descriptions etc.) are excluded from these.

2. There is uncertainty over how 'Reeves' were used. On one table they place them as a sub-group of the cohort (which might lead one to believe they were inclusionary criteria performed after Oxford). But the text on page 2 shows that Reeves were used for exclusionary purposes (to "exclude alternative diagnoses") along with NICE (those are the two references given here). There is no literature on the PACE protocol that I can see that sets out standardisation of Fukuda (or Reeves) inclusion or exclusion requirements.

3. As someone has already said, 47% of the cohort had a psychiatric disorder. Now - there is some strange comment on the Pace Trial protocol about the "grey box ineligible for trial" because even on the pdf- there are three shades of grey (and two textures of 'hashed' and 'smooth'!) But it looks like all sorts of people were eligible for inclusion, including agoraphobics, any phobics, OCD, PTSD, and lifelong psychosis, and there appears some confusion between the SCID form and the 'Oxford form about inclusion/exclusion of bipolars, and schizophrenics! Funnily enough- considering the frequent claims about 'personality disorder' in CFS - these are not included as exclusions (so mean all sorts of personality disorders could be included).

4. The PACE version of the London criteria used actually a diagnosis of 'ME' based on: Exercise induced fatigue (who doesn't get tired after exercise?!) but the 'exercise'/'exertion' has to be 'trivial' in self report; impaiment of short term memory and loss of concentration; fluctuation of symptoms (ubuquitous in all health states and difficult to quantify); 6 months plus duration; no primary depressive illness or anxiety/'neurosis'. That is all that is necessary to meet the criteria for ME (though don't get me started on the instability of the terms anxiety and neurosis!)

5. They have not addressed the issue of abnormal physiological response to exertion, either within the biomedical literature or the reports from patients. This is a major omission. They have NOT considered the differential cohort that would have been established by applying say the Canadian Criteria either, even though this was brought to their attention a good few times. This should have been a limitation of study item (I note there was no such section in the article).

6. Obviously seriously affected/ bedbound etc. weren't included. But they are likely to try and claim slightly and moderately affected are still 'safe' with CBT/GET (and pacing is useless).

 

[Angela Kennedy]

It might also be useful, when looking at this letter by White et al, to see my response to Cleare, Wessely et al in respect of the cohort used for the negative British XRMV done by Erlwein et al. Their post and my response can be seen here:

http://www.plosone.org/annotation/listThread.action?inReplyTo=info:doi/10.13\
71%2Fannotation%2F13ea20d1-91e6-49c3-bc4b-8fd1ca18f150&root=info%3Adoi%2F10.1371\
%2Fannotation%2F13ea20d1-91e6-49c3-bc4b-8fd1ca18f150

The authors (Cleare, Wessely et al) wrote:

"Patients in our CFS cohort had undergone medical screening to exclude
detectable organic illness".

They also write:

"Thus patients in our service have also co operated in studies of PET and fMRI
neuroimaging, autonomic dysfunction, neurochemistry, respiratory function,
vitamin status, anti nuclear antibodies, immune function, neuroendocrine
function and genetics "

I've written in response:

While patients being processed for a research cohort may well, indeed are
likely, to have co-operated and had such tests done, this does not necessarily
mean that patients with positive results are part of the research cohort.
Indeed, positive results, which would indicate organic abnormality, would surely
be likely to prevent a patient being selected for a cohort, by the very logic
described in the author's paper here, by their own response (the additional
tests are considered `not clinically necessary'?) and in at least one of their
citations (Quarmby et al)? In the Quarmby et al paper, the cohort is described,
in which the criteria used (in addition to 'Fukuda/CDC') is 'Oxford'. The Oxford
criteria (Sharpe et al 1991) in particular actually do allow for patients who fulfil organic abnormality to be selected out of a research cohort.

I also refer to Anthony David (a co-author of Oxford):

...David, who, referring to these, commented at the time:

"British Investigators have put forward an alternative, less strict, operational definition which is essentially chronic (6 months or more) severe disabling fatigue in the absence of neurological signs with myalgia, psychiatric symptoms and previous viral infections as common associated features."

Here special attention needs to be paid to the term `previous viral infections'and `absence of neurological signs', in order to contextualise the cohortselection process applied using the Oxford Criteria.

Remember flu would, technically, count as a previous viral infection (some viral illness? Who hasn't had those before?) - if we are talking semantics, which we actually are (there's an awful lot of semantics used in literature promoting psychogenic explanations for ME/CFS).

I also wrote this:

...The issue of `disability' also needs to be clarified. The references cited in
the Erlwein paper to support the statement that the patient cohort was of 'high
levels of disability' refer only to 'disability' in psycho-social terms or
feelings of `fatigue`, and not in terms of physical impairment, a key omission.
Mundt et al's paper in particular focuses on specific mental health problems and
the social exclusionary effects of living with these. While in no way
invalidating or trivialising the disability caused by mental health problems, it
must be pointed out that both Mundt et al and Chalder Scales nevertheless fail
to elucidate a high level of physical or physiological (say, for example,
neurological, mitochondrial and/or cardiovascular) impairment, key problems
present in people given a clinical diagnosis of `CFS', usually related to
specific organic abnormalities that can be found, if they are tested for in the
first place.

Note Mundt et al is referenced in the PACE trial protocol also . When White et al talk about their cohort's 'disability'- it's in the above context, as 'mentally disabled' not physically. This is a key issue and their getting all protective and caring for their cohorts' 'disability' deflects attention away from the lack of physiological impairment that would be found say, in patients fulfilling the Canadian criteria!

 

[Ember]

Thanks, Angela. But I'm trying for a simple answer that I wish had been contained in the PACE investigators' letter. Let me try again. What "two other definitions of the illness" were used?

In his article, David Tuller mentions four case definitions for chronic fatigue syndrome: the Oxford criteria (1991), the CDC case definition (1994), the CDC "empirical" case definition (2005) and the Canadian case definition (2003). He points out that the Lancet study authors used the single-symptom Oxford criteria (1991), and he asks, "Does the evidence from that study prove that these strategies would help patients identified as having chronic fatigue syndrome through very different criteria?"

The PACE investigators answer unequivocally, "Yes, it does." In addition to using the Oxford criteria, they "also assessed trial participants to see if they met two other definitions of the illness that are favored by some scientists."

David Tuller's response goes to the mothodology. He answers that "the gold standard for making comparisons across groups of patients identified by three varying definitions would be a study with three completely separate cohorts, not one large sample with two embedded subgoups."

My question is, methodological problems notwithstanding, what were the definitions used for the embedded subgroups? Were they the CDC "empirical case definition (2005) and the London criteria?

 

[urbantravels]

OK, since I've brought up questions about the placebo effect and the nocebo effect, I wanted to take a look at the APT arm protocol to see what was really in there. I had, inexcusably, been relying on some chance comment someone had made about how rigid the PACE version of "pacing" was, and I think I may have even been the one to coin the term "the evil version of pacing."

On the face of the the APT used in PACE doesn't really look *that* bad or that radically unlike what we would understand as "pacing." But even on my first quick read I definitely saw some poison pills in there.

This is the one arm of the trial which claims to use a "pathological" model of ME, i.e. that it is a physical disease. The manual is sprinkled with familiar-sounding quotes from people with ME about how they learn not to do two major errands in one day, for instance, and how they learn to stop *before* they feel really exhausted, stuff like that.

To be noted, though: there is constant emphasis on how this approach should be communicated to patients as "not a cure", and that the best it can do for you is "create the conditions for natural recovery to occur." (So even this model doesn't contemplate ME as a disease that might be incurable, or that an individual might never recover "naturally.") The quotes from patients, though *we* know them all to be accurate representations of how pacing works - I gotta tell you those quotes sound like a lot of doom and gloom to the uninitiated. When I was first ill and would read statements like "don't do the laundry and get the groceries on the same day," I was NOT ready to hear that; in fact it would set off a fury of grief that I had to accept such awful limitations on my life, when I used to do dozens of things each day. The grief process involved in accepting that is a major, major undertaking - and these poor people weren't actually getting any emotional support about it, or any real reason to hope (say, by being told that research is ongoing and someday there might be better treatments available. Because as far as the authors of the PACE trial are concerned, all the necessary research has been done & *they* know the cure already - they're deliberately giving people in this arm a treatment that they themselves think is ineffective.)

On the other hand, the GET and CBT arms are filled with positive messages about self-empowerment, encouraging you not to think of yourself as really (or permanently) limited, "helping" you to "identify" your bad habits that are perpetuating this "vicious cycle" of fear of activity/deconditioning, etc., telling you over and over again that you can overcome this vicious cycle and improve your condition.

And then measure outcomes subjectively, after a good year of inculcating the proper attitude in each group of patients about what they can expect from their therapy.

Now think about the cohort issues: we've got a majority of patients in the trial who would never meet CCC, an unknown number of whom have never even experienced true PEM, a large number of whom probably have primary depression or some other fatiguing condition. Would being trained in "pacing" do these people any good at all? When the therapy is delivered with such a strong underlying message that "this probably won't help you improve at all"?

Even if you somehow accidentally got into this trial with real M.E. (it would have to be a mild case), the expectation that there is some "natural recovery" that might *possibly* occur would certainly lead to disappointment with the APT treatment. As far as I understand pacing, it's not going to be a cure or even make me feel dramatically *better* in any way; what it does is cut down on the worst of the suffering. Not an effect you'd feel if you weren't acutely suffering going in; and those folks were pretty well screened to eliminate anyone who was really suffering physically. And if you actually *were* fatigued because you were depressed and deconditioned, of *course* you wouldn't feel better after your 52 weeks with Eeyore being told to lie down and think of England. And you'd be pretty mad that you didn't even get any "natural recovery."

The folks hanging out with Tigger in the other two arms, where everything is wonderful and the power of positive thinking rules all, are being encouraged to believe they feel better. And, of course, if they really had been deconditioned and depressed, they might feel a bit better, especially in the GET arm - and they'd have that nice "sense of control" that they accomplished it through their own good efforts.

OK, guesses as to which group(s) get the placebo effect and which group gets the nocebo effect?

This is based on a very quick read and I'll have to delve deeper to flesh out these thoughts some more - some things still strike me as odd, such as the pacing group being *forbidden* to use heart rate monitors (?) and rely only on their "perception" of how fatigued they felt ... and the fact that some positive aspects of real pacing seem to have snuck their way into the CBT arm rather than being put in the APT arm.

 

[Angela Kennedy]

Thanks, Angela. But I'm trying for a simple answer that I wish had been contained in the PACE investigators' letter. Let me try again. What "two other definitions of the illness" were used?

In his article, David Tuller mentions four case definitions for chronic fatigue syndrome: the Oxford criteria (1991), the CDC case definition (1994), the CDC "empirical" case definition (2005) and the Canadian case definition (2003). He points out that the Lancet study authors used the single-symptom Oxford criteria (1991), and he asks, "Does the evidence from that study prove that these strategies would help patients identified as having chronic fatigue syndrome through very different criteria?"

The PACE investigators answer unequivocally, "Yes, it does." In addition to using the Oxford criteria, they "also assessed trial participants to see if they met two other definitions of the illness that are favored by some scientists."

David Tuller's response goes to the mothodology. He answers that "the gold standard for making comparisons across groups of patients identified by three varying definitions would be a study with three completely separate cohorts, not one large sample with two embedded subgoups."

My question is, methodological problems notwithstanding, what were the definitions used for the embedded subgroups? Were they the CDC "empirical case definition (2005) and the London criteria?

ok. First - Aaaaagh! I say that because this is complicated- possibly due to discrepancies in PACE methodology itself (at present, it looks like that to me). I will try and keep it as 'simple' as I can- so bear with me!

Firstly, White et al's 'London criteria', contained in the PACE trial protocol - appears different to the 'London' criteria recently put out on the MEA website by Charles Shepherd (we already know the London criteria are controversial). I'm still analysing this- and it appears to be a game of semantics, but neurological signs and symptoms are not necessary (and patients with these will have been already been excluded by Oxford). So at present, one cannot even say 'the London criteria' were used (though what the London criteria actually are is unstable in itself!!!!)

Secondly, as I said above, There is uncertainty over how 'Reeves' were used. On one table they place them as a sub-group of the cohort (which might lead one to believe they were inclusionary criteria performed after Oxford). But the text on page 2 shows that Reeves were used for exclusionary purposes (to "exclude alternative diagnoses") along with NICE (those are the two references given here). There is no literature on the PACE protocol that I can see that sets out standardisation of Fukuda (or Reeves) inclusion or exclusion requirements. But it looks like that 'sub-group' might have been only EXCLUSIONS, not INCLUSIONS. It is possible this is the case. It does need clarifying.

Oh - and even IF they were INCLUSIONARY criteria, they weren't Reeves 2005, but Reeves 2003, which may, or may not, be the same as 'Fukuda'! (i have to go back to that article to check.)

So far- there is overlap between the (actually) three 'embedded sub-groups' on the table provided (if you count the third group as SCID psychiatric disorders). So - the cohorts on sight appear very similar as groups (especially in the lack of neurological dysfunction). I'm trying to get someone to help me carry out a 't-test' to help establish if my hypothesis is correct there.

I'm so glad you're interested in this. I think this is a key issue, but for some reason not many people have been taking on board the problem of the cohorts. Hence I've been rather alone in trying to analyse a lot of complicated and apparently discrepant data, and summarise it in ways people can understand while 'on the hop' so to speak. I still haven't finished looking at this issue.