PACE Trial - letters that were published and authors' response (and editorial)

[Mark]

I think we need to acknowledge that, whatever their serious failings might be, The Lancet did publish a good number of patient letters, and the ones they chose were a reasonable selection.

I'm not sure about that: the published letters didn't look like a good representative sample of the critique to me. From my scan of the letters, my impression was that the ones that were published were pretty much all along similar lines - mostly statistical and quite technical; relatively opaque to those not familiar with the SF scale issues. There seemed to be an awful lot of key points that didn't make it into print - some analysis of which criticisms were and weren't printed would be interesting.

 

[Sean]

Hear what you are saying, Mark, and I am certainly no fan of the way The Lancet has handled this.

But there will never be agreement on the ideal combination of letters that should have been published. Point is they did publish some, and those letters did make a lot of good points, including stuff about actometers, shifting goal posts, etc. Their choices are at least defensible, if not optimal.

 

[Dolphin]

Discussion on the Lancet page on Facebook

If anyone is so inclined, they could post in the discussion "PACE Trial with particular reference to editorial and correspondence" on the Lancet page on Facebook:
http://www.facebook.com/topic.php?uid=374651963469&topic=16694 - preferably to criticise the PACE Trial and the like (i.e. as opposed to the letters that were sent).

 

[Dolphin]

I think the Lancet letters were probably largely chosen to avoid overlap/repetition of the same points. It would be interesting to collate points that didn't make it into print but I certainly don't have the time.
(or of course, other lists of points could be used and then see how many of them were in the letters).

 

[Bob]

I think the Lancet letters were probably largely chosen to avoid overlap/repetition of the same points. It would be interest to collate points that didn't make it into print but I certainly don't have the time.

I noticed that a certain type of letter was being accepted, and a certain type rejected...
I don't know if I'm correct about this because I haven't given it careful consideration...

Accepted Letters:
Raised discussions about how the authors' (supposedly legitimate) scientific assumptions and interpretations could be interpreted in another way based on different research papers. In other words, where there is an argument to be had about the interpretation of the data, but only when based on different scientific references to the ones that the authors used.

Rejected Letters:
Raised points about the implications of the study or the implications of the results of study.
Raised points about the legitimacy of the results or the conclusions, based purely on the motives or competence of the authors, or the methodology of the Trial.
Raised issues where the competence of the authors was at stake.
Raised issues about the basis of the Trial.

 

[Dolphin]

The authors being somewhat hypocritical

Somebody sent me this in an E-mail and said I could quote it:

One thing I was thinking about the authors reply is that they claimed to do away with
the threshold scores in order to 'improve the interpretability' of the
study, however with the addition of the post-hoc 'normal range' bs
they did precisely what they claimed to have done away with, which was
simply to add a threshold score, except the one they added just so
happened to be 15 points lower than the one they previously defined!

Same description- 1 SD below the mean, basically the same purpose-
determining what a positive outcome was, only difference being a 15
point drop! I bet that did 'improve the interpretability' of the study
for the bastards!

Sure, you can post it wherever and there's no need to credit me. It might be good to add that the Trial ID claimed 1 SD below the mean was 75 (from a mean of 90), the trial protocol said 70 (from a mean of 85 if I remember right) and the published trial was 60 (from a mean of 84).

For anyone who doesn't know, needs their memory jogged, the primary outcome measures in the published protocol said:

http://www.biomedcentral.com/1471-2377/7/6

Primary outcome measures Primary efficacy measures

Since we are interested in changes in both symptoms and disability we have chosen to designate both the symptoms of fatigue and physical function as primary outcomes. This is because it is possible that a specific treatment may relieve symptoms without reducing disability, or vice versa. Both these measures will be self-rated.

The 11 item Chalder Fatigue Questionnaire measures the severity of symptomatic fatigue [27], and has been the most frequently used measure of fatigue in most previous trials of these interventions. We will use the 0,0,1,1 item scores to allow a possible score of between 0 and 11. A positive outcome will be a 50% reduction in fatigue score, or a score of 3 or less, this threshold having been previously shown to indicate normal fatigue [27].

The SF-36 physical function sub-scale [29] measures physical function, and has often been used as a primary outcome measure in trials of CBT and GET. We will count a score of 75 (out of a maximum of 100) or more, or a 50% increase from baseline in SF-36 sub-scale score as a positive outcome. A score of 70 is about one standard deviation below the mean score (about 85, depending on the study) for the UK adult population [51,52].

Those participants who improve in both primary outcome measures will be regarded as overall improvers.

which they claimed they dropped because they wanted continuous measures (but they presumably would have included in the paper the continous figures anyway).

As well as introducing post-hoc definitions for normal functioning and normal fatigue, they also introduced post-hoc definitions for improvement. So their claim that they prefer continuous measures is very debatable; I think they just knew that the figures wouldn't look good if they used the primary outcome measures as they originally published them.

 

[Sean]

The case for a re-analysis according to the original published protocol is overwhelming.

 

[Bob]

Have you all seen this response from White to Malcolm Hooper?:

In their letter, Peter White et al state: The PACE trial paper refers to chronic fatigue syndrome (CFS) which is operationally defined; it does not purport to be studying CFS/ME.

The sentence continues by stating that the PACE Trial studied: CFS defined simply as a principal complaint of fatigue that is disabling, having lasted six months, with no alternative medical explanation (Oxford criteria).

This is exactly what the ME/CFS community has been saying from the outset, namely that the PACE Trial was not studying those with ME.

http://phoenixrising.me/forums/showthread.php?11744-Hooper-s-initial-response-to-White-re-PACE-Trial

http://www.meactionuk.org.uk/Hoopers-initial-response-to-PDW-letter.htm

I think this is a quote to keep handy at all times!

 

[Dolphin]

Have you all seen this response from White to Malcolm Hooper?:



I think this is a quote to keep handy at all times!

I don't think that quote is that exciting. He says it doesn't apply to "CFS/ME" (as for example the Canadian criteria were not used) but in the paper he says it applies to both CFS (International (2003) - very similar to Fukuda) and M.E. (London criteria), albeit, that they have to have fatigue as their main symptom. I might post that on another thread as I don't particular want this thread clogged up on this point.

ETA: I've re-posted the comment here: http://forums.phoenixrising.me/showthread.php?11744-Hooper-s-initial-response-to-White-re-PACE-Trial - please challenge it on that thread if you disagree.

 

[Angela Kennedy]

I don't think that quote is that exciting. He says it doesn't apply to "CFS/ME" (as for example the Canadian criteria were not used) but in the paper he says it applies to both CFS (International (2003) - very similar to Fukuda) and M.E. (London criteria), albeit, that they have to have fatigue as their main symptom. I might post that on another thread as I don't particular want this thread clogged up on this point.

ETA: I've re-posted the comment here: http://forums.phoenixrising.me/showthread.php?11744-Hooper-s-initial-response-to-White-re-PACE-Trial - please challenge it on that thread if you disagree.

Just to say I've commented briefly on that thread as to why that comment from White is extremely important, for those of us especially working on different issues to the outcome measures. Do not underestimate its importance people!

 

[Dolphin]

FWIW, the letters are in the print edition today. The pdfs can be downloaded for free: http://www.thelancet.com/journals/lancet/issue/current

 

[Dolphin]

We determined the normal range by use of the conventional mean plus or minus 1 SD from what we regarded as the most relevant general population data. For physical function, this was a demographically representative sample (in our paper we stated that this was a UK working-age population, whereas more accurately this should have been an English adult population).3

This issue was discussed on one or more other threads before but thought I'd highlight it here after their "correction". The population from which this figure derived included all the old people, all the sick people, etc. in society. Not a good way to calculate what might be a normal level of functioning for people of mean age 39 who have had other conditions excluded and are well enough at the start of treatment to attend clinics for exercise therapy, etc.

 

[Dolphin]

This is one of the references in the authors' reply:

3 Dworkin RH, Turk DC, McDermott MP, et al. Interpreting the clinical importance of group differences in chronic pain clinical trials: IMMPACT recommendations. Pain 2009; 146: 238-244. CrossRef | PubMed

It is interesting that they quote this as they basically didn't include any data on pain in their paper (unless one counts "post exertional malaise" where they just gave presence/absence at baseline and 52 weeks) despite the fact that pain is a very prominent symptom for a lot of people affected. I think they may run away from it as I think GET can make the pain worse for many people with ME/CFS.

For what it's worth, I posted the bits I underlined from this paper and a few comments in a thread at: http://forums.phoenixrising.me/show...e-clinical-importance-of-group-differences-in

 

[oceanblue]

A recent Journal of Internal Medicine editorial by van der Meer and Lloyd critiquing the new International Consensus Criteria makes an extraordinary claim that these letters represent "unscientific and sometimes personal attacks" on PACE and its authors. I imagine those who wrote these letters will want to respond.

This dispute between
the various protagonists recently surfaced with the
PACE trial published in the Lancet [2], which provided
evidence for effectiveness of elements of cognitive-
behavioural therapy (CBT) and graded exercise
therapy (GET) for patients with CFS. This
publication triggered unscientific and sometimes
personal attacks on the researchers in both the
scientific literature [310]...


3 Feehan SM. The PACE trial in chronic fatigue syndrome. Lancet
2011;377:18312.
4 Giakoumakis J. The PACE trial in chronic fatigue syndrome.
Lancet2011;377:1831; author reply 45.
5 Kewley AJ. The PACE trial in chronic fatigue syndrome. Lancet
2011;377:1832; author reply 45.
6 Kindlon T. The PACE trial in chronic fatigue syndrome. Lancet
2011;377:1833; author reply 45.
7 Mitchell JT Jr. The PACE trial in chronic fatigue syndrome.
Lancet2011;377:1831; author reply 45.
8 ShinoharaM.ThePACEtrial inchronic fatigue syndrome.Lancet
2011;377:18334; author reply 45.
9 Stouten B, GoudsmitEM, Riley N. The PACE trial in chronic fatigue
syndrome.Lancet2011;377:18323; author reply 45.
10 Vlaeyen JW, Karsdorp P, Gatzounis R, Ranson S, Schrooten M.
The PACE trial in chronic fatigue syndrome. Lancet 2011; 377:
1834; author reply -5.

Firestormm has posted a thread on the ICC critique itself.

 

[Snow Leopard]

Yes, criticising a study is de-facto unscientific. Nothing good has ever come from publishing letters to the editor.

 

[Sean]

Yes, criticising a study is de-facto unscientific. Nothing good has ever come from publishing letters to the editor.

Even just considering such an action clearly indicates major psychopathology, and probably moral and criminal deviance as well.

 

[Dolphin]

I've just noticed that decimal points have disappeared from the published letters. I think this may have happened with the forum software change. I can't edit the letters (written too long ago).

 

[Dolphin]

An error was subsequently corrected for the PACE Trial authors' reply:
http://download.thelancet.com/pdfs/journals/lancet/PIIS0140673611611239.pdf

Department of Error

White PD, Goldsmith KA, Johnson AL, et al. The
PACE trial in chronic fatigue syndrome—Authors’
reply. Lancet 2011; 377: 1834–35—In this
Correspondence (May 28), references 2 and 3
in the reference list should have been
transposed. This correction has been made to
the online version as of July 15, 2011.

i.e. Dworkin et al should be reference 2.

 

[biophile]

An error was subsequently corrected for the PACE Trial authors' reply:
http://download.thelancet.com/pdfs/journals/lancet/PIIS0140673611611239.pdfi.e. Dworkin et al should be reference 2.

So the Lancet have issued a correction for a reference in the PACE authors' reply, but not for content errors in the original paper and the accompanying editorial? WTF?

I've just noticed that decimal points have disappeared from the published letters. I think this may have happened with the forum software change. I can't edit the letters (written too long ago).

I had the same problem with some of my earlier posts, certain symbols weren't carried over.

 

[Dolphin]

So the Lancet have issued a correction for a reference in the PACE authors' reply, but not for content errors in the original paper and the accompanying editorial? WTF?

Yes, although just to be thorough, we did get this clarification published on February 18, 2012 (exactly a year after the online publication).

These sorts of little clarifications/corrections could even make the authors look very scrupulous (i.e. if you are not aware of the things they're not correcting).

The Lancet, Volume 379, Issue 9816, Page 616, 18 February 2012​

<Previous Article|Next Article>​

doi:10.1016/S0140-6736(12)60267-0 Cite or Link Using DOI
PACE trial clarification

T Chalder a , M Sharpe a, PD White a
In the PACE trial,1 we stated that we used the Chalder fatigue questionnaire.2 We would like to clarify that we used an updated version of the scale.3 The item “Do you have problems thinking clearly?” was replaced with “Do you find it more difficult to find the correct word?” This updated version has slightly better reliability than the original scale.2, 3 In practice, either item can be used without altering the interpretation of the scale.2
TC has done consultancy work for insurance companies and has received royalties from Sheldon Press and Constable and Robinson. MS has done voluntary and paid consultancy work for government and for legal and insurance companies, and has received royalties from Oxford University Press. PDW has done voluntary and paid consultancy work for the UK Departments of Health and Work and Pensions and Swiss Re (a reinsurance company).

References

1 White PD, Goldsmith KA, Johnson AL, et al. Comparison of adaptive pacing, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet 2011; 377: 823-836.Summary | Full Text | PDF(309KB) | CrossRef | PubMed
2 Chalder T, Berelowitz G, Hirsch S, et al. Development of a fatigue scale. J Psychosom Res 1993; 37: 147-153. CrossRef |PubMed
3 Cella M, Chalder T. Measuring fatigue in clinical and community settings. J Psychosom Res 2010; 69: 17-22. CrossRef | PubMed​