PACE Trial and PACE Trial Protocol
- Thread starter Dolphin
- Start date
Angela Kennedy
Senior Member
- Messages
- 1,026
- Location
- Essex, UK
Below is my latest letter to Dr Charles Warlow, Ombudsman for the Lancet, which is self-explanatory.
------------------------
Dear Dr Warlow,
I draw your attention to my complaint made to you on June 9th, 2011, regarding the PACE trial and Richard Horton's responses to my own and others concerns that have been expressed.
Please see below for a reminder of my email to you.
I am most concerned that you have not yet seen fit even to acknowledge receipt of my email in the two months since I sent it to you, let alone address the issues contained with this email.
I must therefore ask you, as a matter of urgency, to acknowledge receipt of my original email, and this one, and set out how you propose to address the concerns contained within my complaint.
Please note I will be making all correspondence public, in the interests of fairness and transparency.
Yours sincerely,
Angela Kennedy
----- Original Message -----
From: ANGELA KENNEDY
To: ombudsman@lancet.com
Sent: Thursday, June 09, 2011 10:19 AM
Subject: Complaint about the Lancet publication of the PACE trial
Dear Dr Warlow,
I am writing to you in your capacity as Ombudsman for the Lancet journal. I am writing to make a complaint about three issues, related to the publishing by the Lancet of the article White PD, Goldsmith KA, Johnson AL, et al. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue
syndrome (PACE): a randomised trial. Lancet 2011; 377: 823-836.
The first complaint regards the publication of the article itself, and the unsafe claims made within the Lancet article and accompanying editorial: as well as the unsafe claims made elsewhere as a direct result of the publication.
The second complaint regards Richard Hortons prejudicial and publicly contemptuous treatment of the many concerns raised about this article and its problems, and the people raising them.
The third complaint regards Richard Hortons failure to address my serious concerns.
COMPLAINT 1
The PACE trial was subject to a large amount of concern and objection by advocates for those diagnosed with ME or CFS, from the beginning of the study in 2004 and throughout its course. I was one of those who outlined specific concerns at the beginning of the trial: and various concerns were also outlined in response to the publication of the protocol mid-trial. For evidence of this please see my own and others comments at:
http://www.biomedcentral.com/1471-2377/7/6/comments/comments
I am the mother and long-term advocate of a child (now a woman) diagnosed at 13 with ME/CFS, and who has various objective, medically substantiated organic impairments (especially neurological and cardiovascular) which have led to severe disability. If subjected to
PACE-type CBT and GET, she would be at serious risk of further harm.
There remain a large number of very serious flaws, problems and discrepancies in this whole study, including the published article in the Lancet.
I originally wrote to Dr Horton on 25th April 2011, to reiterate the many substantive and valid concerns raised by Professor Malcolm Hooper in his complaint to Dr Horton about the article and the trial itself, available here:
http://www.meactionuk.org.uk/COMPLAINT-to-Lancet-re-PACE.htm
I am also aware that a large number of valid and substantive criticisms of the trial have been made in letter form to the Lancet and have been rejected for publication, as well as some concerns that were published by the Lancet.
More recently and since I first made my complaint to Dr Horton, I have been made aware that the PACE authors have written to Professor Hooper, and that he has responded to them:
http://www.meactionuk.org.uk/Comments-on-PDW-letter-re-PACE.htm
White et als response to Professor Hooper, and his comments contain some extremely important information which has further ramifications for the concerns I have expressed here.
In addition to the above concerns, I informed Dr Horton that I am specifically gravely concerned about the dangers to patients caused by the unsafe claims that Cognitive Behavioural Therapy of the type advocated by White et al, and Graded Exercise Therapy, are safe treatments for people diagnosed with Myalgic Encephalomyelitis/Chronic Fatigue Syndrome, claims propounded both within the PACE article itself, and the accompanying editorial. This has led to similar unsound claims being made elsewhere. The potential adverse ramifications for patients of these unsound claims are particularly serious, and therefore those claims should not have been made. Bleijenberg and Knoop, in their Lancet editorial accompanying the publication of the PACE article, claim:
"Concerns about the safety of cognitive behaviour therapy and graded
exercise therapy have been raised more than once by patients' advocacy
groups. Few patients receiving cognitive behaviour therapy or graded
exercise therapy in the PACE trial had serious adverse reactions and no more
than those receiving adaptive pacing therapy or standard medical care, which
for cognitive behavioural therapy has already been shown.This finding is
important and should be communicated to patients to dispel unnecessary
concerns about the possible detrimental effects of cognitive behaviour
therapy and graded exercise therapy, which will hopefully be a useful
reminder of the potential positive effects of both interventions."
The PACE article itself states:
"Trial findings show cognitive behaviour therapy (CBT) and graded exercise
therapy (GET) can be effective treatments for chronic fatigue syndrome, but
patients' organisations have reported that these treatments can be harmful
and favour pacing and specialist health care. We aimed to assess
effectiveness and safety of all four treatments."
The same article concludes:
"Findings from the PACE trial suggest that individually delivered CBT and
GET, when added to SMC, are more effective and as safe as APT added to SMC
or SMC alone. Patients attending secondary care with chronic fatigue
syndrome should be offered individual CBT or GET, alongside SMC."
Other parties have repeated these unsafe claims, informed by the PACE trial. One example of a newspaper article is that in the Daily Mail, on 18th February 2011, which claims:
"Got ME? Fatigued patients who go out and exercise have best hope of
recovery, finds study".
I have recently been made aware of various other similar headlines which resulted from publication of the PACE trial article, detailed in Professor Hoopers response to the White et al team, linked as above.
A press release from the Science Media Centre about the trial included various unsound claims of safety from doctors. Alistair Miller, for example, stated:
"It provides convincing evidence that GET and CBT are safe and effective and
should be widely available for our patients with CFS/ME".
Derick Wade, as another example, claimed that the trial:
"...confirms the effectiveness of two treatments, and their safety. The
study suggests that everyone with the condition should be offered the
treatment, and every patient who wishes to be helped should be willing to
try one or both of the treatments".
In addition to the claim of safety of CBT and GET, Wade's comment also indicates that patients may be regarded as recalcitrant (for example, in a context of welfare support or continued medical support) should they, quite rationally, dare refuse to 'try' treatments that actually may be dangerous.
The fundamental problem that needs to be addressed is that the evidence available shows that, contrary to the above claims, the PACE trial did not adequately assess, or even address, safety of CBT and GET, and this study did not disprove patients and doctors' valid and substantive concerns regarding the dangers of CBT and GET. One major discrepancy of the PACE trial and the resulting article was the failure to address the biomedical evidence available detailing serious organic physiological dysfunction in patients who receive a 'CFS' or 'ME' diagnosis. Another is the inadequate treatment of adverse outcomes within the trial. This is discussed in more detail in Professor Hooper's document as detailed above.
I also raised specific concerns about the patient cohort. Evidence indicates that research cohorts for 'CFS' or 'CFS/ME' appear to be obtained (by those promoting psychogenic explanations for these conditions) by excluding patients with signs and symptoms (especially neurological) found in Myalgic Encephalomyelitis case descriptions, or indeed other organic diseases (the 'alternative diagnoses'). The PACE trial used, not just one case criteria to exclude
patients with symptoms and signs of organic disease from the trial, but three: 'Oxford' (Sharpe et al, 1991); Reeves et al (2003), and those from the NICE guidelines (see White et al, 2011: 2).
Of 3158 patients who had been referred to "six specialist chronic fatigue syndrome clinics in the UK National Health Service" (White et al, 2011: 2), 1187 patients (over a third) were actually excluded because they did not actually meet Oxford criteria for 'CFS'. Confusingly, no figures are given for those meeting Reeves et al (2003) and NICE exclusionary criteria, though these are claimed as part of the exclusion process. This is possibly because the Oxford criteria themselves efficiently exclude those with signs and symptoms of neurological myalgic encephalomyelitis, to the point that the Reeves and NICE exclusionary criteria may well have been superfluous.
There are similarities of symptoms and signs of neurological dysfunction found in specific case descriptions of myalgic encephalomyelitis (for example, Ramsay, 1988), or 'ME/CFS' (as defined by Carruthers et al, 2003), with other neurological conditions, for just one example, those found in Multiple Sclerosis (see, for example, Poser 2000). Therefore, to have included patients with neurological symptoms and/or signs might have meant there was a risk of other neurological conditions (such as Multiple Sclerosis) being involved in the trial. Indeed, in his response to me on the Biomed Central site, Peter White discusses the need to keep people with other neurological conditions out of the trial. But, crucially, the key problem here is that, from the evidence available (some of which is detailed by Professor Hooper), Professor White and his colleagues do not appear to believe 'ME' is a neurological condition in the first place, despite the acceptance of this by the World Health Organisation and British agencies, and despite the evidence available to support this, and therefore seem unable to acknowledge that at least some people given an ME or CFS diagnosis have organic neurological and other deficits. It seems therefore likely that ME/CFS patients with signs and symptoms of neurological (and indeed other organic) dysfunction were actively excluded from the PACE trial.
Ironically, if this premise is accurate, White et al cannot have substantiated their claims for the safety and efficacy of CBT/GET for patients they claim such treatments are safe and efficacious, those given an ME or CFS diagnosis who suffer physiological impairments including neurological deficits. It needs to be noted that the PACE article actually claims the results:
"can be generalised to patients who meet alternative diagnostic criteria for chronic fatigue syndrome and myalgic encephalomyelitis but only if fatigue is their main symptom" (citing the London criteria and Reeves et al 2003 as the 'alternative diagnostic criteria').
This is a confusing statement, bearing in mind that: the 'London Criteria' used in PACE were not actually that as referenced by them (as the documentation from the PACE trial protocol shows); the Reeves et al criteria were supposed to have been used by them within the trial itself (so the question arises, why are they 'alternative'?); and their conclusions, and that of Bleijenberg and Knoop and others, presents a blanket claim of safety and efficacy for all people given a diagnosis of ME or CFS, contradicting this statement about "only if fatigue is their main symptom".
Indeed, it is notable that White et al, from the beginning of the trial and throughout, refused to use the criteria of Carruthers et al (2003) to include people with symptoms (and possibly signs) of neurological dysfunction, although they used their own (specifically customised and therefore different) version of a set of criteria claimed to identify ME (the 'London' criteria), already controversial due to lack of peer reviewed publication, uncertainty in authorship, and the existence of different versions. Indeed, as is evident from the PACE Trial protocol, the specifically customized PACE version of the 'London' criteria for ME bore close similarities to the Oxford criteria for CFS, and were fundamentally different to the Carruthers et al criteria (2003).
That so many patients (nearly a third), of whom had been referred to a 'specialist chronic fatigue syndrome unit' by their GP, were actually excluded from the CFS diagnosis favoured by these authors, is extremely important, and leads to the question: what happens to such patients? When
the patient exclusion process of another project (the negative XMRV study by Erlwein et al, 2009) was clarified by co-authors (Wessely et al, 2010), some clinical patients who had attended chronic fatigue/CFS clinics commented in response that they had not been investigated thoroughly in the way the research cohorts appeared to be (ironically in order to exclude organic disease), either at the clinic or by their GP. Another study by Newton et al (2010) found that 40% of patients referred to a 'chronic fatigue syndrome unit' did not have 'CFS', though, crucially, Newton was including, as 'CFS' patients, those with specific physiological conditions such as positional orthostatic tachycardia syndrome (POTS), which are associated with neurological dysfunction (Carruthers et al, 2003). If these patients had been also excluded from a diagnosis of CFS (which, according to the Oxford criteria and indeed the Reeves et al criteria, they should), the amount of patients referred to British 'chronic fatigue syndrome units' (or, often, 'chronic fatigue units'), meeting the Oxford criteria for CFS and having no exclusionary conditions that suggest organic dysfunction, would appear to be very small indeed. But even patients excluded under the rubric of the Oxford criteria from research, will now be exhorted to 'try' CBT and/or GET in a clinical context, because of the unsafe claims of the PACE trial.
Another major discrepancy in the PACE trial that I specifically highlighted is that one of the treatments, 'Adaptive Pacing Therapy', bore no resemblance to the strategy of 'pacing', specifically adopted by ME patients and reported as being helpful by them in charity surveys. 'Pacing' as reported in these surveys is merely an autonomous flexible management strategy utilised by patients with ME in order to cope with the limitations of the illness, like sufferers of other chronic impairments. The PACE trial's 'Adaptive Pacing Therapy' was not autonomous, being therapist led, and imposed a regime upon the patient similar to the GET treatment. This has specific iatrogenic potential in that, informed by the claims of PACE, patients may be told by health care professionals that an autonomous, flexible self-management strategy that is common in patients with chronic impairments, that has been found to be useful in ME/CFS, must not be practised, on the incorrect findings of a trial that did not even study the correct type of 'pacing' in the first place.
In light of the extremely complex and serious problems of confounding inherent in this trial, it is of serious issue that unsafe claims of safety and efficacy of CBT/GET as treatments for ME or CFS were made by the PACE authors and supporters, to the point that iatrogenic harm could be caused to patients because of a resulting lack of understanding, by medics and ancillary staff, misinformed by such unsafe claims, of both the neurological and other physiological impairments in at least some patients given such diagnoses, and the abnormal physiological response to exertion that appears to be a key feature in those patients.
In my complaint I also drew attention to (Lancet member of staff) Zoe Mullan's comment to me in our email correspondence: "We were not aware of any objections to this study". I am very concerned about this as objections to the PACE trial have been publicly mounted, and indeed have been responded to (though in eventuality, not satisfactorily) by authors of the trial, some years prior to publication. As you will see from the email correspondence I had with Ms Mullan, I did draw Dr Hortons attention to these objections.
As I wrote to Dr Horton, at the very least, a much more detailed discussion of limitations to this study should have been undertaken that took into account the concerns that were raised. In the circumstances and to ensure patient safety, the article should be retracted, and the claims that CBT and GET have been found to be safe in ME and/or CFS should be publicly corrected.
COMPLAINT 2
In addition to the above problems, I informed Dr Horton that I believe there should be an unreserved public apology issued to all the ME community and their advocates who have raised legitimate and substantive concerns, in various ways, about the problems in the PACE trial, for the prejudicial misrepresentation of their concerns and motivations, made by him on the ABC radio programme 'The Health Report', in which he made the following offensive and prejudicial comments (I have bolded them for your information. They were, however not bolded initially when I cited them to Dr Horton in April) :
http://www.abc.net.au/rn/healthreport/stories/2011/3192571.htm
"...the criticisms about this study are a mirage, they obscure the fact that
what the investigators did scrupulously was to look at chronic fatigue
syndrome from an utterly impartial perspective."
"Not this kind of orchestrated response trying to undermine the credibility
of the study from patient groups but also the credibility of the
investigators and that's what I think is one of the other alarming aspects
of this. This isn't a purely scientific debate; this is going to the heart
of the integrity of the scientists who conducted this study."
"...indeed in a few examples of allegations have been made to professional
authorities, the General Medical Council here in the UK about the work of
these scientists on the basis of the flimsiest and most unfair allegations.
And indeed the study costs $4 million pounds to undertake but the
allegations and the freedom of information requests and the legal fees that
have been wrapped up over the years because of these vexatious claims has
added another 750,000 pounds of taxpayers' money to the conduct of this
study."
"Indeed, and I think this is where one sees a real fracture in the patient
community. One is seeing a very substantial number of patients very willing
to engage in this study, desperate to get good evidence on which to base
their future treatment but one sees a fairly small, but highly organised,
very vocal and very damaging group of individuals who have I would say
actually hijacked this agenda and distorted the debate so that it actually
harms the overwhelming majority of patients."
"Well what we're doing right now is waiting for the formal response from the
authors to this 43 page attack on their integrity and the study and the
request for a retraction. We plan to publish their response to that attack,
we will invite the critics to submit versions of their criticisms for
publication and we will try as best as we can to conduct a reasonable
scientific debate about this paper. This will be a test I think of this
particular section of the patient community to engage in a proper scientific
discussion."
As I informed Dr Horton, these prejudicial comments should also be retracted. They are inappropriate and inaccurate, and sadly indicate a possible bad faith on Dr Hortons part from the offset, and this is an unusual and extremely worrying response from the editor of a key medical journal to the reasonable concerns that have been raised, in good faith, by a vulnerable patient community and others supporting them. I cannot emphasise enough that the concerns related by myself and others relate specifically to the risks to safety of patients, and our concern to prevent those: this is indeed the motivation for my own actions.
Unfortunately, as detailed below, Dr Horton went on to make further prejudicial comments against people expressing concerns about the PACE trial in a Lancet editorial, after I made my complaint to him.
COMPLAINT 3
In my initial complaint to Dr Horton of 25th April 2011, in addition to my call for a retraction of the article, and a public correction of the inaccurate claims, that CBT and GET have been found to be safe in ME and/or CFS, that were made in the Lancet. I also asked for an apology for the above prejudicial comments made on the ABC radio programme.
I also requested this of him:
I must ask that you keep to your promise that "we will invite the
critics to submit versions of their criticisms for publication and we will
try as best as we can to conduct a reasonable scientific debate about this
paper" made by you on the ABC radio programme 'The Health Report'. I
consider myself as one of those 'critics'. Indeed, I believe a full and
public good faith investigation of my own and others complaints need to be
undertaken by you and other parties, as appropriate.
As is evident, Dr Horton made no response to me personally to my complaint, or requests for information under FOI, until a cursory dismissal of my complaint via Zoe Mullan in an email of June 8th, after I had informed both Horton and Mullan that I intended to take my complaint to Ombudsman and required a response with a one week period. Indeed, after my complaint had been made, Dr Horton made yet another public attack on the motives of people expressing concern about this trial, within another Lancet editorial, that, crucially, accompanied those (few) very letters critical of the trial that were published (after many others were rejected for publication):
http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(11)60696-X/fulltext
"But one cannot help but wonder whether the sheer anger and coordination of the response to this trial has been born not only from the frustration many feel about a disabling condition, but also from an active campaign to discredit the research. White and colleagues have been accused of having formed their opinion about the intended outcome before the trial began. This view is unjustified and unfair. The researchers should be praised for their willingness to test competing ideas and interventions in a randomised trial. The evidence might even suggest that it is the critics of the PACE trial who have formed their opinions first, ignoring the findings of this rigorously conducted work."
These ad hominem misrepresentations (bolded above) of the motives of all those who complained or expressed critiques about this trial and the paper (including those critiques he actually published) set Dr Horton in a blatantly adversarial position against all of us who have made serious, substantive complaints about this trial and the paper in good faith and with good reason. It is clear that Horton had taken an immediate, extremely partisan position against a vulnerable patient community and their supporters, and set upon discrediting their concerns and their motives, and maintains this position. That is unacceptable and fails to meet the standard of professional and impartial practice that one can reasonably expect from the editor of such a highly valued medical and scientific journal.
Further information has come to light that further substantiates the likelihood that the majority of the concerns expressed by people about this trial, some from the beginning, and my own concerns in my complaint in April, are substantive and salient, and will also need to be taken into account in due course.
In light of the above, I ask that the below steps need to be taken:
1. Specifically, to ensure immediate patient safety and removal of risk, the PACE article and its accompanying editorial must be retracted immediately, and an accompanying notice in the Lancet given to the effect that problems in methodology have been found, meaning that the CBT/GET has not yet been demonstrated by the PACE trial to be safe, for people diagnosed with ME/CFS, who have physiological, including neurological, deficits, and might not be in the future.
2. A unreserved public apology needs to be made by Dr Horton, to all those who have expressed concern about the PACE trial, for the many prejudicial and ad hominem comments he has made about their motives for expressing these concerns, both in the ABC radio programme, and in the Lancet editorial discussed in this letter to you.
3. To ensure, in the longer term, patient safety, and scientific standards related to the illness entity of myalgic encephalomyelitis and indeed of the term chronic fatigue syndrome, a full, independent, public investigation needs to be made of the discrepancies of both the PACE trial methodology, results and claims that authors have made about these and the trial throughout its undertaking, and of the many concerns expressed about this trial, from its beginning, and ongoing.
The email correspondence that I sent to both Richard Horton and Zoe Mullan, and Zoe Mullans initial replies to my first emails of concern, will be sent to you in separate emails, and are exhibited here:
http://pacedocuments.blogspot.com/
There is one more problem to be addressed, and that is your own Conflict of Interest in this matter. You are a co-author with at least one of the co-authors of the PACE trial (Michael Sharpe) that I am aware of, for example, from your profile page at the CCBS:
http://www.dcn.ed.ac.uk/pages/profiles/profiles.asp?ProfileId=10
Systematic review of misdiagnosis of conversion symptoms and hysteria. Stone J, Smyth R, Carson A, Lewis S, Prescott R, Warlow C, Sharpe M. BMJ (2005) 331:
You also appear to support psychogenic explanations for illnesses of uncertain aetiology, and therefore your views are likely to be in harmony with the known and published views of key PACE trial authors and assistants about illnesses such as ME or CFS, for example, your above profile states:
I have also had passing interests in motor neuron disease, multiple sclerosis and nowadays in the large number of patients whose neurological symptoms are not explained by any disease (sometimes called somatisation or functional).
With this fundamental Conflict of Interest on your part in mind, I must ask you how you intend to address this problem in order to impartially, fairly and adequately address my complaint.
Please be aware that, in the interests of transparency and accountability, I will be publishing this email to you, and I may publish any responses I receive.
Yours faithfully
Angela Kennedy
------------------------
Dear Dr Warlow,
I draw your attention to my complaint made to you on June 9th, 2011, regarding the PACE trial and Richard Horton's responses to my own and others concerns that have been expressed.
Please see below for a reminder of my email to you.
I am most concerned that you have not yet seen fit even to acknowledge receipt of my email in the two months since I sent it to you, let alone address the issues contained with this email.
I must therefore ask you, as a matter of urgency, to acknowledge receipt of my original email, and this one, and set out how you propose to address the concerns contained within my complaint.
Please note I will be making all correspondence public, in the interests of fairness and transparency.
Yours sincerely,
Angela Kennedy
----- Original Message -----
From: ANGELA KENNEDY
To: ombudsman@lancet.com
Sent: Thursday, June 09, 2011 10:19 AM
Subject: Complaint about the Lancet publication of the PACE trial
Dear Dr Warlow,
I am writing to you in your capacity as Ombudsman for the Lancet journal. I am writing to make a complaint about three issues, related to the publishing by the Lancet of the article White PD, Goldsmith KA, Johnson AL, et al. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue
syndrome (PACE): a randomised trial. Lancet 2011; 377: 823-836.
The first complaint regards the publication of the article itself, and the unsafe claims made within the Lancet article and accompanying editorial: as well as the unsafe claims made elsewhere as a direct result of the publication.
The second complaint regards Richard Hortons prejudicial and publicly contemptuous treatment of the many concerns raised about this article and its problems, and the people raising them.
The third complaint regards Richard Hortons failure to address my serious concerns.
COMPLAINT 1
The PACE trial was subject to a large amount of concern and objection by advocates for those diagnosed with ME or CFS, from the beginning of the study in 2004 and throughout its course. I was one of those who outlined specific concerns at the beginning of the trial: and various concerns were also outlined in response to the publication of the protocol mid-trial. For evidence of this please see my own and others comments at:
http://www.biomedcentral.com/1471-2377/7/6/comments/comments
I am the mother and long-term advocate of a child (now a woman) diagnosed at 13 with ME/CFS, and who has various objective, medically substantiated organic impairments (especially neurological and cardiovascular) which have led to severe disability. If subjected to
PACE-type CBT and GET, she would be at serious risk of further harm.
There remain a large number of very serious flaws, problems and discrepancies in this whole study, including the published article in the Lancet.
I originally wrote to Dr Horton on 25th April 2011, to reiterate the many substantive and valid concerns raised by Professor Malcolm Hooper in his complaint to Dr Horton about the article and the trial itself, available here:
http://www.meactionuk.org.uk/COMPLAINT-to-Lancet-re-PACE.htm
I am also aware that a large number of valid and substantive criticisms of the trial have been made in letter form to the Lancet and have been rejected for publication, as well as some concerns that were published by the Lancet.
More recently and since I first made my complaint to Dr Horton, I have been made aware that the PACE authors have written to Professor Hooper, and that he has responded to them:
http://www.meactionuk.org.uk/Comments-on-PDW-letter-re-PACE.htm
White et als response to Professor Hooper, and his comments contain some extremely important information which has further ramifications for the concerns I have expressed here.
In addition to the above concerns, I informed Dr Horton that I am specifically gravely concerned about the dangers to patients caused by the unsafe claims that Cognitive Behavioural Therapy of the type advocated by White et al, and Graded Exercise Therapy, are safe treatments for people diagnosed with Myalgic Encephalomyelitis/Chronic Fatigue Syndrome, claims propounded both within the PACE article itself, and the accompanying editorial. This has led to similar unsound claims being made elsewhere. The potential adverse ramifications for patients of these unsound claims are particularly serious, and therefore those claims should not have been made. Bleijenberg and Knoop, in their Lancet editorial accompanying the publication of the PACE article, claim:
"Concerns about the safety of cognitive behaviour therapy and graded
exercise therapy have been raised more than once by patients' advocacy
groups. Few patients receiving cognitive behaviour therapy or graded
exercise therapy in the PACE trial had serious adverse reactions and no more
than those receiving adaptive pacing therapy or standard medical care, which
for cognitive behavioural therapy has already been shown.This finding is
important and should be communicated to patients to dispel unnecessary
concerns about the possible detrimental effects of cognitive behaviour
therapy and graded exercise therapy, which will hopefully be a useful
reminder of the potential positive effects of both interventions."
The PACE article itself states:
"Trial findings show cognitive behaviour therapy (CBT) and graded exercise
therapy (GET) can be effective treatments for chronic fatigue syndrome, but
patients' organisations have reported that these treatments can be harmful
and favour pacing and specialist health care. We aimed to assess
effectiveness and safety of all four treatments."
The same article concludes:
"Findings from the PACE trial suggest that individually delivered CBT and
GET, when added to SMC, are more effective and as safe as APT added to SMC
or SMC alone. Patients attending secondary care with chronic fatigue
syndrome should be offered individual CBT or GET, alongside SMC."
Other parties have repeated these unsafe claims, informed by the PACE trial. One example of a newspaper article is that in the Daily Mail, on 18th February 2011, which claims:
"Got ME? Fatigued patients who go out and exercise have best hope of
recovery, finds study".
I have recently been made aware of various other similar headlines which resulted from publication of the PACE trial article, detailed in Professor Hoopers response to the White et al team, linked as above.
A press release from the Science Media Centre about the trial included various unsound claims of safety from doctors. Alistair Miller, for example, stated:
"It provides convincing evidence that GET and CBT are safe and effective and
should be widely available for our patients with CFS/ME".
Derick Wade, as another example, claimed that the trial:
"...confirms the effectiveness of two treatments, and their safety. The
study suggests that everyone with the condition should be offered the
treatment, and every patient who wishes to be helped should be willing to
try one or both of the treatments".
In addition to the claim of safety of CBT and GET, Wade's comment also indicates that patients may be regarded as recalcitrant (for example, in a context of welfare support or continued medical support) should they, quite rationally, dare refuse to 'try' treatments that actually may be dangerous.
The fundamental problem that needs to be addressed is that the evidence available shows that, contrary to the above claims, the PACE trial did not adequately assess, or even address, safety of CBT and GET, and this study did not disprove patients and doctors' valid and substantive concerns regarding the dangers of CBT and GET. One major discrepancy of the PACE trial and the resulting article was the failure to address the biomedical evidence available detailing serious organic physiological dysfunction in patients who receive a 'CFS' or 'ME' diagnosis. Another is the inadequate treatment of adverse outcomes within the trial. This is discussed in more detail in Professor Hooper's document as detailed above.
I also raised specific concerns about the patient cohort. Evidence indicates that research cohorts for 'CFS' or 'CFS/ME' appear to be obtained (by those promoting psychogenic explanations for these conditions) by excluding patients with signs and symptoms (especially neurological) found in Myalgic Encephalomyelitis case descriptions, or indeed other organic diseases (the 'alternative diagnoses'). The PACE trial used, not just one case criteria to exclude
patients with symptoms and signs of organic disease from the trial, but three: 'Oxford' (Sharpe et al, 1991); Reeves et al (2003), and those from the NICE guidelines (see White et al, 2011: 2).
Of 3158 patients who had been referred to "six specialist chronic fatigue syndrome clinics in the UK National Health Service" (White et al, 2011: 2), 1187 patients (over a third) were actually excluded because they did not actually meet Oxford criteria for 'CFS'. Confusingly, no figures are given for those meeting Reeves et al (2003) and NICE exclusionary criteria, though these are claimed as part of the exclusion process. This is possibly because the Oxford criteria themselves efficiently exclude those with signs and symptoms of neurological myalgic encephalomyelitis, to the point that the Reeves and NICE exclusionary criteria may well have been superfluous.
There are similarities of symptoms and signs of neurological dysfunction found in specific case descriptions of myalgic encephalomyelitis (for example, Ramsay, 1988), or 'ME/CFS' (as defined by Carruthers et al, 2003), with other neurological conditions, for just one example, those found in Multiple Sclerosis (see, for example, Poser 2000). Therefore, to have included patients with neurological symptoms and/or signs might have meant there was a risk of other neurological conditions (such as Multiple Sclerosis) being involved in the trial. Indeed, in his response to me on the Biomed Central site, Peter White discusses the need to keep people with other neurological conditions out of the trial. But, crucially, the key problem here is that, from the evidence available (some of which is detailed by Professor Hooper), Professor White and his colleagues do not appear to believe 'ME' is a neurological condition in the first place, despite the acceptance of this by the World Health Organisation and British agencies, and despite the evidence available to support this, and therefore seem unable to acknowledge that at least some people given an ME or CFS diagnosis have organic neurological and other deficits. It seems therefore likely that ME/CFS patients with signs and symptoms of neurological (and indeed other organic) dysfunction were actively excluded from the PACE trial.
Ironically, if this premise is accurate, White et al cannot have substantiated their claims for the safety and efficacy of CBT/GET for patients they claim such treatments are safe and efficacious, those given an ME or CFS diagnosis who suffer physiological impairments including neurological deficits. It needs to be noted that the PACE article actually claims the results:
"can be generalised to patients who meet alternative diagnostic criteria for chronic fatigue syndrome and myalgic encephalomyelitis but only if fatigue is their main symptom" (citing the London criteria and Reeves et al 2003 as the 'alternative diagnostic criteria').
This is a confusing statement, bearing in mind that: the 'London Criteria' used in PACE were not actually that as referenced by them (as the documentation from the PACE trial protocol shows); the Reeves et al criteria were supposed to have been used by them within the trial itself (so the question arises, why are they 'alternative'?); and their conclusions, and that of Bleijenberg and Knoop and others, presents a blanket claim of safety and efficacy for all people given a diagnosis of ME or CFS, contradicting this statement about "only if fatigue is their main symptom".
Indeed, it is notable that White et al, from the beginning of the trial and throughout, refused to use the criteria of Carruthers et al (2003) to include people with symptoms (and possibly signs) of neurological dysfunction, although they used their own (specifically customised and therefore different) version of a set of criteria claimed to identify ME (the 'London' criteria), already controversial due to lack of peer reviewed publication, uncertainty in authorship, and the existence of different versions. Indeed, as is evident from the PACE Trial protocol, the specifically customized PACE version of the 'London' criteria for ME bore close similarities to the Oxford criteria for CFS, and were fundamentally different to the Carruthers et al criteria (2003).
That so many patients (nearly a third), of whom had been referred to a 'specialist chronic fatigue syndrome unit' by their GP, were actually excluded from the CFS diagnosis favoured by these authors, is extremely important, and leads to the question: what happens to such patients? When
the patient exclusion process of another project (the negative XMRV study by Erlwein et al, 2009) was clarified by co-authors (Wessely et al, 2010), some clinical patients who had attended chronic fatigue/CFS clinics commented in response that they had not been investigated thoroughly in the way the research cohorts appeared to be (ironically in order to exclude organic disease), either at the clinic or by their GP. Another study by Newton et al (2010) found that 40% of patients referred to a 'chronic fatigue syndrome unit' did not have 'CFS', though, crucially, Newton was including, as 'CFS' patients, those with specific physiological conditions such as positional orthostatic tachycardia syndrome (POTS), which are associated with neurological dysfunction (Carruthers et al, 2003). If these patients had been also excluded from a diagnosis of CFS (which, according to the Oxford criteria and indeed the Reeves et al criteria, they should), the amount of patients referred to British 'chronic fatigue syndrome units' (or, often, 'chronic fatigue units'), meeting the Oxford criteria for CFS and having no exclusionary conditions that suggest organic dysfunction, would appear to be very small indeed. But even patients excluded under the rubric of the Oxford criteria from research, will now be exhorted to 'try' CBT and/or GET in a clinical context, because of the unsafe claims of the PACE trial.
Another major discrepancy in the PACE trial that I specifically highlighted is that one of the treatments, 'Adaptive Pacing Therapy', bore no resemblance to the strategy of 'pacing', specifically adopted by ME patients and reported as being helpful by them in charity surveys. 'Pacing' as reported in these surveys is merely an autonomous flexible management strategy utilised by patients with ME in order to cope with the limitations of the illness, like sufferers of other chronic impairments. The PACE trial's 'Adaptive Pacing Therapy' was not autonomous, being therapist led, and imposed a regime upon the patient similar to the GET treatment. This has specific iatrogenic potential in that, informed by the claims of PACE, patients may be told by health care professionals that an autonomous, flexible self-management strategy that is common in patients with chronic impairments, that has been found to be useful in ME/CFS, must not be practised, on the incorrect findings of a trial that did not even study the correct type of 'pacing' in the first place.
In light of the extremely complex and serious problems of confounding inherent in this trial, it is of serious issue that unsafe claims of safety and efficacy of CBT/GET as treatments for ME or CFS were made by the PACE authors and supporters, to the point that iatrogenic harm could be caused to patients because of a resulting lack of understanding, by medics and ancillary staff, misinformed by such unsafe claims, of both the neurological and other physiological impairments in at least some patients given such diagnoses, and the abnormal physiological response to exertion that appears to be a key feature in those patients.
In my complaint I also drew attention to (Lancet member of staff) Zoe Mullan's comment to me in our email correspondence: "We were not aware of any objections to this study". I am very concerned about this as objections to the PACE trial have been publicly mounted, and indeed have been responded to (though in eventuality, not satisfactorily) by authors of the trial, some years prior to publication. As you will see from the email correspondence I had with Ms Mullan, I did draw Dr Hortons attention to these objections.
As I wrote to Dr Horton, at the very least, a much more detailed discussion of limitations to this study should have been undertaken that took into account the concerns that were raised. In the circumstances and to ensure patient safety, the article should be retracted, and the claims that CBT and GET have been found to be safe in ME and/or CFS should be publicly corrected.
COMPLAINT 2
In addition to the above problems, I informed Dr Horton that I believe there should be an unreserved public apology issued to all the ME community and their advocates who have raised legitimate and substantive concerns, in various ways, about the problems in the PACE trial, for the prejudicial misrepresentation of their concerns and motivations, made by him on the ABC radio programme 'The Health Report', in which he made the following offensive and prejudicial comments (I have bolded them for your information. They were, however not bolded initially when I cited them to Dr Horton in April) :
http://www.abc.net.au/rn/healthreport/stories/2011/3192571.htm
"...the criticisms about this study are a mirage, they obscure the fact that
what the investigators did scrupulously was to look at chronic fatigue
syndrome from an utterly impartial perspective."
"Not this kind of orchestrated response trying to undermine the credibility
of the study from patient groups but also the credibility of the
investigators and that's what I think is one of the other alarming aspects
of this. This isn't a purely scientific debate; this is going to the heart
of the integrity of the scientists who conducted this study."
"...indeed in a few examples of allegations have been made to professional
authorities, the General Medical Council here in the UK about the work of
these scientists on the basis of the flimsiest and most unfair allegations.
And indeed the study costs $4 million pounds to undertake but the
allegations and the freedom of information requests and the legal fees that
have been wrapped up over the years because of these vexatious claims has
added another 750,000 pounds of taxpayers' money to the conduct of this
study."
"Indeed, and I think this is where one sees a real fracture in the patient
community. One is seeing a very substantial number of patients very willing
to engage in this study, desperate to get good evidence on which to base
their future treatment but one sees a fairly small, but highly organised,
very vocal and very damaging group of individuals who have I would say
actually hijacked this agenda and distorted the debate so that it actually
harms the overwhelming majority of patients."
"Well what we're doing right now is waiting for the formal response from the
authors to this 43 page attack on their integrity and the study and the
request for a retraction. We plan to publish their response to that attack,
we will invite the critics to submit versions of their criticisms for
publication and we will try as best as we can to conduct a reasonable
scientific debate about this paper. This will be a test I think of this
particular section of the patient community to engage in a proper scientific
discussion."
As I informed Dr Horton, these prejudicial comments should also be retracted. They are inappropriate and inaccurate, and sadly indicate a possible bad faith on Dr Hortons part from the offset, and this is an unusual and extremely worrying response from the editor of a key medical journal to the reasonable concerns that have been raised, in good faith, by a vulnerable patient community and others supporting them. I cannot emphasise enough that the concerns related by myself and others relate specifically to the risks to safety of patients, and our concern to prevent those: this is indeed the motivation for my own actions.
Unfortunately, as detailed below, Dr Horton went on to make further prejudicial comments against people expressing concerns about the PACE trial in a Lancet editorial, after I made my complaint to him.
COMPLAINT 3
In my initial complaint to Dr Horton of 25th April 2011, in addition to my call for a retraction of the article, and a public correction of the inaccurate claims, that CBT and GET have been found to be safe in ME and/or CFS, that were made in the Lancet. I also asked for an apology for the above prejudicial comments made on the ABC radio programme.
I also requested this of him:
I must ask that you keep to your promise that "we will invite the
critics to submit versions of their criticisms for publication and we will
try as best as we can to conduct a reasonable scientific debate about this
paper" made by you on the ABC radio programme 'The Health Report'. I
consider myself as one of those 'critics'. Indeed, I believe a full and
public good faith investigation of my own and others complaints need to be
undertaken by you and other parties, as appropriate.
As is evident, Dr Horton made no response to me personally to my complaint, or requests for information under FOI, until a cursory dismissal of my complaint via Zoe Mullan in an email of June 8th, after I had informed both Horton and Mullan that I intended to take my complaint to Ombudsman and required a response with a one week period. Indeed, after my complaint had been made, Dr Horton made yet another public attack on the motives of people expressing concern about this trial, within another Lancet editorial, that, crucially, accompanied those (few) very letters critical of the trial that were published (after many others were rejected for publication):
http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(11)60696-X/fulltext
"But one cannot help but wonder whether the sheer anger and coordination of the response to this trial has been born not only from the frustration many feel about a disabling condition, but also from an active campaign to discredit the research. White and colleagues have been accused of having formed their opinion about the intended outcome before the trial began. This view is unjustified and unfair. The researchers should be praised for their willingness to test competing ideas and interventions in a randomised trial. The evidence might even suggest that it is the critics of the PACE trial who have formed their opinions first, ignoring the findings of this rigorously conducted work."
These ad hominem misrepresentations (bolded above) of the motives of all those who complained or expressed critiques about this trial and the paper (including those critiques he actually published) set Dr Horton in a blatantly adversarial position against all of us who have made serious, substantive complaints about this trial and the paper in good faith and with good reason. It is clear that Horton had taken an immediate, extremely partisan position against a vulnerable patient community and their supporters, and set upon discrediting their concerns and their motives, and maintains this position. That is unacceptable and fails to meet the standard of professional and impartial practice that one can reasonably expect from the editor of such a highly valued medical and scientific journal.
Further information has come to light that further substantiates the likelihood that the majority of the concerns expressed by people about this trial, some from the beginning, and my own concerns in my complaint in April, are substantive and salient, and will also need to be taken into account in due course.
In light of the above, I ask that the below steps need to be taken:
1. Specifically, to ensure immediate patient safety and removal of risk, the PACE article and its accompanying editorial must be retracted immediately, and an accompanying notice in the Lancet given to the effect that problems in methodology have been found, meaning that the CBT/GET has not yet been demonstrated by the PACE trial to be safe, for people diagnosed with ME/CFS, who have physiological, including neurological, deficits, and might not be in the future.
2. A unreserved public apology needs to be made by Dr Horton, to all those who have expressed concern about the PACE trial, for the many prejudicial and ad hominem comments he has made about their motives for expressing these concerns, both in the ABC radio programme, and in the Lancet editorial discussed in this letter to you.
3. To ensure, in the longer term, patient safety, and scientific standards related to the illness entity of myalgic encephalomyelitis and indeed of the term chronic fatigue syndrome, a full, independent, public investigation needs to be made of the discrepancies of both the PACE trial methodology, results and claims that authors have made about these and the trial throughout its undertaking, and of the many concerns expressed about this trial, from its beginning, and ongoing.
The email correspondence that I sent to both Richard Horton and Zoe Mullan, and Zoe Mullans initial replies to my first emails of concern, will be sent to you in separate emails, and are exhibited here:
http://pacedocuments.blogspot.com/
There is one more problem to be addressed, and that is your own Conflict of Interest in this matter. You are a co-author with at least one of the co-authors of the PACE trial (Michael Sharpe) that I am aware of, for example, from your profile page at the CCBS:
http://www.dcn.ed.ac.uk/pages/profiles/profiles.asp?ProfileId=10
Systematic review of misdiagnosis of conversion symptoms and hysteria. Stone J, Smyth R, Carson A, Lewis S, Prescott R, Warlow C, Sharpe M. BMJ (2005) 331:
You also appear to support psychogenic explanations for illnesses of uncertain aetiology, and therefore your views are likely to be in harmony with the known and published views of key PACE trial authors and assistants about illnesses such as ME or CFS, for example, your above profile states:
I have also had passing interests in motor neuron disease, multiple sclerosis and nowadays in the large number of patients whose neurological symptoms are not explained by any disease (sometimes called somatisation or functional).
With this fundamental Conflict of Interest on your part in mind, I must ask you how you intend to address this problem in order to impartially, fairly and adequately address my complaint.
Please be aware that, in the interests of transparency and accountability, I will be publishing this email to you, and I may publish any responses I receive.
Yours faithfully
Angela Kennedy
Nothing that new. Perhaps it has even been said before.
I'm re-reading the full papers at the moment.
As we know, in the Lancet paper, they dropped one part of the primary efficacy measures (see bold below)
http://www.biomedcentral.com/1471-2377/7/6
They didn't tell the readers they dropped this in the Lancet paper.
It is interesting to see the following in the protocol - "both" requires a primary measure that has both - but they never reported it:
Similarly, "both" here requires the measure that includes both:
The paper does include some post hoc calculations that they use at one stage for "both" e.g. improvement of 8+ on SF-36 PF and 2+ on CFQ; 60+ on SF-36 (reminder: you could get in the trial with 65!) and 18- on CFQ (reminder: you could get in the trial with 18). However, one is not supposed to used post-hoc analyses for one's primary objectives.
I'm re-reading the full papers at the moment.
As we know, in the Lancet paper, they dropped one part of the primary efficacy measures (see bold below)
http://www.biomedcentral.com/1471-2377/7/6
They didn't tell the readers they dropped this in the Lancet paper.
It is interesting to see the following in the protocol - "both" requires a primary measure that has both - but they never reported it:
(Note: I'm not exactly sure what (4) is getting at - perhaps a combined comparison which wasn't reported but on single measures, both were significant so obvious what the result would be).
Similarly, "both" here requires the measure that includes both:
The paper does include some post hoc calculations that they use at one stage for "both" e.g. improvement of 8+ on SF-36 PF and 2+ on CFQ; 60+ on SF-36 (reminder: you could get in the trial with 65!) and 18- on CFQ (reminder: you could get in the trial with 18). However, one is not supposed to used post-hoc analyses for one's primary objectives.
Was PACE submitted elsewhere before the Lancet?
This may have been covered before, but we know PACE was published in Feb after an express review process and seems to have been submitted in Jan 2011. Yet it was originally expected to be published in the autumn. Peter White was due to present baseline data at he BACME oct 10 conference (presumably he was planning to do so because publication was imminent) and some Bart's document was expecting publication in Nov '10, implying submission had taken place well before that.
So I wondered if anyone knew if the paper had indeed been submitted to another journal but withdrawn or been rejected?
This may have been covered before, but we know PACE was published in Feb after an express review process and seems to have been submitted in Jan 2011. Yet it was originally expected to be published in the autumn. Peter White was due to present baseline data at he BACME oct 10 conference (presumably he was planning to do so because publication was imminent) and some Bart's document was expecting publication in Nov '10, implying submission had taken place well before that.
So I wondered if anyone knew if the paper had indeed been submitted to another journal but withdrawn or been rejected?
I'd be interested to know too. If anyone has a chance, I'd be interested to know whether other Lancet papers say the date the paper was received - I'm wondering if it's unusual for the journal not to include it; I see it a lot in other journals. But it does seem to have reviewed quickly.
Back on Aug 10, 2010, I posted on a forum the following that somebody had told me:
Best of luck. One way to give it a plug is to put it in your signature. My signature is rather big at the moment but I might put it in after August 27 (but I have an old one with lots of links already so something might have to get the chop)
Graham
Senior Moment
- Messages
- 5,188
- Location
- Sussex, UK
Thanks for that suggestion Dolphin - I'm onto it now.
Graham
Senior Moment
- Messages
- 5,188
- Location
- Sussex, UK
Testing, testing
Battery Muncher
Senior Member
- Messages
- 620
Hi,
This thread is absolutely massive, overwhelmingly so. Does anyone have any suggestions where to start?
This thread is absolutely massive, overwhelmingly so. Does anyone have any suggestions where to start?
Graham
Senior Moment
- Messages
- 5,188
- Location
- Sussex, UK
Hi Battery Muncher
As you can see, I only joined a few months ago, and the thread was enormous then. It doesn't break down easily into phases, and lots of topics keep getting revisited. I slowly worked through all the posts and kept finding nuggets in all sorts of places. The nature of the posts is that often two or three threads are interwoven.
The big question is, what sort of aspects interest you (changes from the original proposals, the different measures, analysis of results, correspondence with Lancet ...), and what level do you want (e.g. everyday, broad scientific or in-depth analysis). If it is anything specific, we may be able to direct you somewhere.
People here have been very very patient and helpful - a great bunch!
As you can see, I only joined a few months ago, and the thread was enormous then. It doesn't break down easily into phases, and lots of topics keep getting revisited. I slowly worked through all the posts and kept finding nuggets in all sorts of places. The nature of the posts is that often two or three threads are interwoven.
The big question is, what sort of aspects interest you (changes from the original proposals, the different measures, analysis of results, correspondence with Lancet ...), and what level do you want (e.g. everyday, broad scientific or in-depth analysis). If it is anything specific, we may be able to direct you somewhere.
People here have been very very patient and helpful - a great bunch!
Good one, Graham.
A lot of the points were summarised in the letters that were submitted.
The published letters can be read here: http://forums.phoenixrising.me/show...published-and-authors-response-(and-editorial)
The unpublished letters (which were often very good but probably didn't make it because of some overlap with the other ones) are at:
http://forums.phoenixrising.me/show...-letters-that-were-not-accepted-by-the-Lancet
If they were a bit technical for you (I have no idea what you know), an easier place to start might be Cort's summary: http://forums.phoenixrising.me/content.php?369-A-Hitch-in-its-Step and the comments under them.
There are other discussions and even wikis and websites but I'll let others mention them - I'm just dipping in.
Chalder Fatigue Scale: would using the 'true' SD have made PACE fail?
Sorry for the melodramatic title but it's a geeky point and I wanted to create the illusion it might be interesting.
PACE made heavy use of Standard Deviation (SD) of the baseline group in definging 'success': a 'clinically useful difference' (CUD) was defined as 0.5SD, and as the SD was 3.7 the CUD was set at 2. The group mean difference for CBT and GET were both over 2 (success!) and around 80% of individuals also exceeded the CUD (more success!).
Lots have people have commented that the SD was artificially constrained both by ceiling effects and a recruitment threshold requiring a bimodal score of 6 or more - this led to a smaller SD making it easier for PACE to succeed. So it's interesting to see the SD for the Chalder Fatigue scale which isn't constrained by ceiling and recruitment thresholds.
Step forward a recent study. Here, the CFQ mean for patients is only 22.4 so probably the ceiling effect is less of an issue (max score is 33). There also wasn't a floor imposed by a CFQ threshold for recruitment. And guess what? Without these floor/ceiling constraints the SD is 7.2. so perhaps this is a 'true' SD for patients. If so, neither GET nor CBT mean differences would have reached the o.5SD threshold for a clinically useful difference:
0.5SD (new study)=3.6 (PACE=1.9)
PACE mean differences: GET=3.2; CBT=3.4
Sorry for the melodramatic title but it's a geeky point and I wanted to create the illusion it might be interesting.
PACE made heavy use of Standard Deviation (SD) of the baseline group in definging 'success': a 'clinically useful difference' (CUD) was defined as 0.5SD, and as the SD was 3.7 the CUD was set at 2. The group mean difference for CBT and GET were both over 2 (success!) and around 80% of individuals also exceeded the CUD (more success!).
Lots have people have commented that the SD was artificially constrained both by ceiling effects and a recruitment threshold requiring a bimodal score of 6 or more - this led to a smaller SD making it easier for PACE to succeed. So it's interesting to see the SD for the Chalder Fatigue scale which isn't constrained by ceiling and recruitment thresholds.
Step forward a recent study. Here, the CFQ mean for patients is only 22.4 so probably the ceiling effect is less of an issue (max score is 33). There also wasn't a floor imposed by a CFQ threshold for recruitment. And guess what? Without these floor/ceiling constraints the SD is 7.2. so perhaps this is a 'true' SD for patients. If so, neither GET nor CBT mean differences would have reached the o.5SD threshold for a clinically useful difference:
0.5SD (new study)=3.6 (PACE=1.9)
PACE mean differences: GET=3.2; CBT=3.4
Graham
Senior Moment
- Messages
- 5,188
- Location
- Sussex, UK
So far with just 13 replies, the average (mean) deterioration in Chalder scores on a bad patch is 3.2 points, and the improvement in a good patch is 2.3 points.
You could also consider doing a reliability test (not sure if it is of much value but just brainstorming): ask people to not look at their previous answers and fill it in a week later. This may not work so well with CFS as there is some fluctuation: with many questionnaires, there can be natural fluctutation e.g. ask a healthy person how happy/well they feel today on a scale of 0-100 and ask them 10 minutes later (or some distance in time when they might forget their initial response) and you won't necessarily get the same result. There are probably better exams out there in terms of reliability.
I suppose what I'm partly getting at is that at different moments in time, with some scales, people can give different answers while by some objective measure there is no difference.
Thinking aloud: I wonder would there be any chance that people might overplay their symptoms a little to make sure they could enter into a trial/be ill enough so that they would be seen as worthy of treatment. Then they wouldn't need to really "improve" but there would still be an improvement. Although perhaps this would be equal across all arms of a trial; but it would lead to lower scores initially than might be registered in other circumstances i.e. when trying to work out how ill the cohort was.
(no need to respond to any of this)
Esther12
Senior Member
- Messages
- 13,774
Yeah - there are so many potential problems like this. I'm amazed that people so often talk as if an improvement in these questionnaires can be confidently assumed to be an improvement in 'fatigue'. re PACE though - presumably many of these confounding factors would be the same for the SMC group.
Well spotted. :Sign Good Job:And of course, this was one of the places where CBT was offered during the PACE Trial so one doesn't need to control for factors like that (different population) too much.
Yes - see bit I added when you were typing.
(not important)
A small point I noticed when re-reading the trial protocol http://www.biomedcentral.com/1471-2377/7/6 is that the entry criteria were given as SF-36 PF of 65 or less. People will remember that initially it was 60 and then 11 months after the trial began, they changed it. So this isn't actually the protocol at baseline/before the trial began (although perhaps it is the only change).
Anyway, the main point I have about that is it might possible give them less of an excuse for changing the outcome measures, as they did in the final paper, or at least they can't claim that the protocol changed again before the trial started.
A small point I noticed when re-reading the trial protocol http://www.biomedcentral.com/1471-2377/7/6 is that the entry criteria were given as SF-36 PF of 65 or less. People will remember that initially it was 60 and then 11 months after the trial began, they changed it. So this isn't actually the protocol at baseline/before the trial began (although perhaps it is the only change).
Anyway, the main point I have about that is it might possible give them less of an excuse for changing the outcome measures, as they did in the final paper, or at least they can't claim that the protocol changed again before the trial started.
(Not important) "clinically important difference" vs "clinically useful difference"
(I think this point may have been made before)
In the trial protocol, clinically important difference is defined in terms of the overall percentage who improve (where improve would be the dropped primary outcome measured).
[Note: One only really needs to read the highlighted lines in the quotes below]
However, in the Lancet paper, they come up with a "clinically useful difference" which has now become 0.5 SD. This definition is actually also sometimes used for "clinically important difference" (see refs in Guyatt et al., 2002).
Of course, even by that measure, they don't reach their target as 2 and 3 times the improvement rate of SSMC is difficult when the improvement rate of SSMC is 45%! It shows in another way that the new definition doesn't fit what they were defining as improvement (they estimated that only 10% of the SMC group would improve)
(I think this point may have been made before)
In the trial protocol, clinically important difference is defined in terms of the overall percentage who improve (where improve would be the dropped primary outcome measured).
[Note: One only really needs to read the highlighted lines in the quotes below]
However, in the Lancet paper, they come up with a "clinically useful difference" which has now become 0.5 SD. This definition is actually also sometimes used for "clinically important difference" (see refs in Guyatt et al., 2002).
Of course, even by that measure, they don't reach their target as 2 and 3 times the improvement rate of SSMC is difficult when the improvement rate of SSMC is 45%! It shows in another way that the new definition doesn't fit what they were defining as improvement (they estimated that only 10% of the SMC group would improve)