PACE Trial and PACE Trial Protocol

[Dolphin]

Yes, I know of the method of tying numbers/names to something meaningful in order to remember them. My dad taught me that when I was quite young: he got me to learn the names of the roads and towns that we had to pass through to go on holiday by building up a story around them, so that I could navigate for him, The trouble is that I am too quick at that. What happens is that if I say get confused over a person's name (David or Eric), before I have had a chance to find out which is correct, my mind has already come up with a way to tie both of them to him. Then I can't remember which is the right tie. And for those of you who think I am joking, I'm not. Of course, the best way is to tie the name to the person right at the start when introduced, but by the time the introduction is completed, I have already forgotten the name.

For what it's worth:
I've found post-ME but if doubt suddenly enters my mind about a memory, I'm not so good at remembering it into the future. So I could have known something well for 10+ years; then one day for some reason I have difficulty recalling it or I doubt it. Then after that, I tend not to be sure about it again. (At least I think it is an ME phenomenon - I became ill at 16 so hard to remember exactly how things were before that, over two decades ago).

 

[MeSci]

For what it's worth:
I've found post-ME but if doubt suddenly enters my mind about a memory, I'm not so good at remembering it into the future. So I could have known something well for 10+ years; then one day for some reason I have difficulty recalling it or I doubt it. Then after that, I tend not to be sure about it again. (At least I think it is an ME phenomenon - I became ill at 16 so hard to remember exactly how things were before that, over two decades ago).

I get that too to some extent, maybe when I am tired and/or have PEM. Passwords and usernames, for example. I often find that if I try too hard to remember them, it gets even harder, in the same way as remembering how to perform a common physical task has become more instinctive than intellectual so is a type of procedural memory. I have to relax and gently coax it back to the front of my brain. Or just try the most likely combinations I can think of!

After a spell in hospital with hyponatraemia I couldn't even remember how to make a cup of tea at first!

 

[Tom Kindlon]

My new 1000-word BMJ rapid response (i.e. e-letter) on the £5m PACE Trial, "PACE Trial: Simply giving a reason why an outcome measure was changed is not necessarily sufficient".

It lists all sorts of reasons why I'm unhappy with the changes made to the PACE Trial outcome measures.

I'm afraid I knew it would be too long to get published, so didn't spend as much time on the wording as other letters where I'm hoping they might be published in the print edition.

http://www.bmj.com/content/347/bmj.f5963?tab=responses

 

[Tom Kindlon]

My new 1000-word BMJ rapid response (i.e. e-letter) on the £5m PACE Trial, "PACE Trial: Simply giving a reason why an outcome measure was changed is not necessarily sufficient".

It lists all sorts of reasons why I'm unhappy with the changes made to the PACE Trial outcome measures.

I'm afraid I knew it would be too long to get published, so didn't spend as much time on the wording as other letters where I'm hoping they might be published in the print edition.

http://www.bmj.com/content/347/bmj.f5963?tab=responses

Sean Lynch, a psychiatrist, has now responded, approx. 24 hours after my comment went up. Authors get automatic alerts for responses - I wonder whether there might be a connection?

His defence of the changes to the PACE Trial protocol largely seem to depend people trusting him as some sort of expert, perhaps an unbiased expert. There is little substance in them. It's like an appeal to authority.

Sean Lynch was one of the co-authors of the Royal Colleges of Report on CFS (1996) (Word file of this is available here:
http://bit.ly/HFQeOM ). As well as recommending CBT and GET, and little else, it gave very few suggestions for biological research.

He calls for more trials like the PACE Trial:

I feel that there is a need for a new debate now. PACE has in my view made an important contribution to the research evidence and has been a well-designed and conducted trial, but there is a need for further research to build on its contribution in my opinion.

[..]

Pragmatic trials might have advantages of larger sample size and seeing how well the best- evidenced treatments of that time are applied and fare in real world situations and their cost-effectiveness (6).

so that'd be CBT and GET trials and the like.

 

[Dolphin]

Here's Sean Lynch's own letter that he references:

Letter

Patient reported outcome measures

Measures need to capture patients’ views and experiences more effectively

BMJ 2013; 346 doi: http://dx.doi.org/10.1136/bmj.f1553 (Published 19 March 2013)
Cite this as: BMJ 2013;346:f1553

Sean Lynch, consultant psychiatrist and honorary associate professor1
Author Affiliations
seanlynch@nhs.net

I agree that using patient reported outcome measures (PROMs) in a “tick box way” to feed a management led IT beast is a dreadful scenario.1 However, how have we got into the situation where PROMS are needed in the first place?

I can comment only from the viewpoint of my specialty and cannot speak for others. In psychiatry, outcome measures that are interviewer led can be helpful, but can lead to serious bias (misinterpretation, patients not feeling able to be honest because of a perceived power difference between doctor and patient). In addition, self rated questionnaires are only as good as their design and the questions they ask, and patients often tell me that they don’t ask the right questions or have options that would provide more useful information.

These are real life clinical concerns in assessing how somebody is getting on with treatment and if it is making a difference, not “ivory tower” stuff or necessarily a research agenda.

I think that it is counter productive to develop PROMS and other measures for their own sake. Instead we should use fewer and better measures that come closer to capturing patients’ views and experiences.

Notes
Cite this as: BMJ 2013;346:f1553

Footnotes
Competing interests: None declared.

References
01.↵ Black N. Patient reported outcome measures could help transform healthcare. BMJ 2013;346:f167. (28 January.)

 

[Dolphin]

James C. Coyne PhD in a recent blog said people who write commentaries on papers are often the reviewers (he suggested they volunteer; it would make sense also as they would have an indepth knowledge of the paper which is what one would want from a commentator). This suggests one of PACE Trial reviewers were Knoop or Bleijenberg. Peter White suggested a reviewer suggested the normal range data; they used it in their full recovery paper (Knoop et al., 2007). It was also mentioned in their commentary on the PACE Trial.

 

[biophile]

James C. Coyne PhD in a recent blog said people who write commentaries on papers are often the reviewers (he suggested they volunteer; it would make sense also as they would have an indepth knowledge of the paper which is what one would want from a commentator). This suggests one of PACE Trial reviewers were Knoop or Bleijenberg. Peter White suggested a reviewer suggested the normal range data; they used it in their full recovery paper (Knoop et al., 2007). It was also mentioned in their commentary on the PACE Trial.

The normal range analysis was plainly stated as post hoc, given in response to a reviewer’s request.
http://www.meactionuk.org.uk/whitereply.htm

I am still not 100% clear whether this means the peer reviewer requested the normal range be added or just that it should be described as post-hoc. I am leaning towards Dolphin's interpretation because normal range is not mentioned in the Statistical Analysis Plan. Neither is recovery mentioned, so maybe that wasn't changed yet, although no mention at all is odd. Perhaps PACE liked how the normal range gave an inflated sense of success and decided to adopt it for their redefining of recovery?

If either Knoop or Bleijenberg were peer reviewers of the Lancet paper and suggested the normal range, that just makes their associated blunders even worse and possibly more incestuous, since White was a co-author of Knoop et al 2007. Knoop and Bleijenberg claimed that the Lancet paper had a "strict" criteria for "recovery" based on a "healthy" population. All aspects of their claim were utterly and irrefutably false. Hooper said the Lancet promised a proper correction, but this never happened.

It was later revealed that White even approved the editorial in question, making his later impartially-sounding non-apologetic "correction" (i.e. we did not report on recovery) about in the authors' reply a tad disingenuous.

Furthermore, both Knoop and Bleijenberg have previously published papers in which 60/100 points in physical function was clearly regarded as major disability. Perhaps a simple mistake, albeit bizarre one? Nope. After being called out for it, they fully stood by their comment. After initially defending them in the "strongest terms possible", the PCC later conceded that this statement in the editorial by Knoop and Bleijenberg was misleading.

White was a co-author of Knoop et al 2007, yet he claims in the recovery paper that PACE used the more conservative mean minus 1SD to derive their threshold rather than the less conservative mean minus 2SD used in Knoop et al. Their comparison was also wrong. Knoop et al used the same mean minus 1SD that PACE used and came up with 80 points. Describing their method as "conservative" was already questionable, but White seems to have trouble interpreting papers he has co-authored himself. To err is human, but such a series of blunders is baffling until you realize that every single one of them just happens to make their pet therapies look better than they otherwise would have. Coincidence?

 

[ukxmrv]

Sorry, this is off topic a little but shows how things changed from the original plan of the PAC study. Was looking through my old emails and found this one from Chris Clark - ex-AFME CEO in 2004.

From: Chris Clark
Subject: action for m.e. statement - PACE criteria


ACTION FOR M.E. STATEMENT

I note that there has been some comment on the e-groups
regarding the criteria for the PACE study. Some time ago the
researchers agreed at our request to also record all those in
the study against BOTH the Fukuda criteria and the
operationalised criteria for myalgic encephalomyelitis.

It is intended that this should provide information on whether
any of the treatments being compared are more helpful or more
harmful than the others for different definitions of CFS / M.E.

Chris Clark
Chief Executive

 

[Valentijn]

ACTION FOR M.E. STATEMENT

I note that there has been some comment on the e-groups
regarding the criteria for the PACE study. Some time ago the
researchers agreed at our request to also record all those in
the study against BOTH the Fukuda criteria and the
operationalised criteria for myalgic encephalomyelitis.

It is intended that this should provide information on whether
any of the treatments being compared are more helpful or more
harmful than the others for different definitions of CFS / M.E.

Chris Clark
Chief Executive

It's always nice to see AfME standing up to represent the interests of everyone except ME/CFS patients.

 

[Bob]

@ukxmrv, they did assess the therapies for patients diagnosed using the London ME criteria and the CDC's Reeves criteria, in the main (Lancet) PACE paper. However, these criteria were used only after patients had been selected to the study using Oxford. This isn't good practise. I suppose it could be compared to selecting patients with progressive MS to a study, and then re-assessing these patients for relapsing-remitting MS. It just isn't appropriate.

 

[Valentijn]

@ukxmrv, they did assess the therapies for patients diagnosed using the London ME criteria and the CDC's Reeves criteria, in the main (Lancet) PACE paper. However, these criteria were used only after patients had been selected to the study using Oxford, which isn't good practise. I suppose it could be compared to selecting patients with progressive MS to a study, and then re-diagnosing these patients to see if they have relapsing-remitting MS. It just isn't appropriate.

And it doesn't matter who they're studying, when the outcome measurements are all subjective and the recovery thresholds are meaningless.

 

[Bob]

And it doesn't matter who they're studying, when the outcome measurements are all subjective and the recovery thresholds are meaningless.

And the self-reported results were no better than would be expected for placebo or homeopathy, etc. etc. etc. etc.

 

[Dolphin]

Lots of BMJ Rapid Responses that are critical of the PACE Trial in one way or another have been posted on this discussion (last one went up in last 24 hours):
http://www.bmj.com/content/347/bmj.f5963?tab=responses

 

[Dolphin]

James C. Coyne PhD in a recent blog said people who write commentaries on papers are often the reviewers (he suggested they volunteer; it would make sense also as they would have an indepth knowledge of the paper which is what one would want from a commentator). This suggests one of PACE Trial reviewers were Knoop or Bleijenberg. Peter White suggested a reviewer suggested the normal range data; they used it in their full recovery paper (Knoop et al., 2007). It was also mentioned in their commentary on the PACE Trial.

I was thinking about this a little more and remembered that the PACE Trial peer review was fast-tracked (I think only started in January 2011? - it was published on Feb 18, 2011). This would seem to make it even more likely Knoop and/or Bleijenberg were a reviewer.

Also, it would seem that it was even more likely they might already have been thinking about their editorial, if they were the reviewer who asked the PACE Trial investigators for a normal range analysis.

It also seems a bit odd for a reviewer to ask this of investigators given how much the investigators were already trying to squash into the 4,000-word limit for the Lancet incl. some secondary outcome measures that weren't reported on, etc. Also seems a bit odd given this was a fast-track review.

ETA: Given all the things I mention in the last paragraph, I think the PACE Trial investigators could probably have resisted it if they'd wanted.

 

[Dolphin]

FWIW:
I'm just reading:

Ann Intern Med. 1995 Jul 15;123(2):81-8.
Chronic fatigue and the chronic fatigue syndrome: prevalence in a Pacific Northwest health care system.
Buchwald D, Umali P, Umali J, Kith P, Pearlman T, Komaroff AL.

It uses the Holmes CDC criteria for CFS (1988) which are generally seen as more strict than the Fukuda criteria.

They didn't find many (just 3) with CFS.
Anyway, the average SF-36 PF score of these 3 was 75 i.e. they would on average have been "back to normal".

In total, they found 74 with chronic fatigue alone i.e. 71 of whom didn't satisfy the CFS criteria for one reason or another but who did say yes to:
"Have you felt unusual fatigue or loss of energy, either constantly or repeatedly, for at least the past 6 months"
and
"Does this state of fatigue interfere with your work or responsibilities at home such that you have had to reduce your level of activity by at least one half?"
This group with chronic fatigue had an average SF-36 physical functioning score of 81.3.
(The average for the matched controls was 93.6).
The point is that people can have scores in the 70s or 80s and still answer yes to both those questions.
[These questions are a bit like the Oxford criteria questions, although the Oxford criteria also does have exclusions].

 

[Tom Kindlon]

Queen Mary FOI Request 2014/F6

I didn't get any new data directly but they have promised me it will be in a subsequent paper

From: Tom Kindlon
Sent: 07 January 2014 23:38
To: PD White
Cc: 'Tom Kindlon'
Subject: Request for data on baseline staff expectations in PACE Trial

Dear Prof. White,

In your reply to comments of mine and others on the PACE Trial protocol, you (and others who were part of the PACE Trial management group) said on behalf of the group[1]:
-------------

Beliefs and expectations of treatment and who is running the trial

The trial has been designed and is being managed by many different healthcare and research professionals, including doctors, therapists, health economists, statisticians and a representative of a patient charity. The Trial Management Group includes five physicians and four psychiatrists. To measure any bias consequent upon individual expectations, all staff involved in the PACE trial recorded their expectations as to which intervention would be most efficacious before their participation, and we will publish these data after the end of the trial.

-------------
The statistical plan for the PACE Trial that was recently published also mentioned that [2]:
-------------

Baseline staff expectations regarding the outcome of the trial were recorded.

-------------
As it is now nearly three years since the main paper on the trial was published in the Lancet[3], I would be like to request such data.

Thanking you in advance,
Tom Kindlon
Irish ME/CFS Association

References:
[1] White PD, Sharpe MC, Chalder T, DeCesare JC, R Walwyn R, for the PACE trial management group: Response to comments on "Protocol for the PACE trial" http://www.biomedcentral.com/1471-2377/7/6/comments#306608

[2]. Walwyn R, Potts L, McCrone P, Johnson AL, Decesare JC, Baber H, Goldsmith K, Sharpe M, Chalder T, White PD. A randomised trial of adaptive pacing therapy, cognitive behaviour therapy, graded exercise, and specialist medical care for chronic fatigue syndrome (PACE): statistical analysis plan. Trials. 2013 Nov 13;14:386. doi: 10.1186/1745-6215-14-386.
[3]. White PD, Goldsmith KA, Johnson AL, Potts L, Walwyn R, DeCesare JC, Baber HL, Burgess M, Clark LV, Cox DL, Bavinton J, Angus BJ, Murphy G, Murphy M, O'Dowd H, Wilks D, McCrone P, Chalder T, Sharpe M; PACE trial management group. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet. 2011 Mar 5;377(9768):823-36. doi: 10.1016/S0140-6736(11)60096-2. Epub 2011 Feb 18.

From: QM FOI Enquiries

Dear Mr. Kindlon

Thank you for your email of 7th January, which has been forwarded to be dealt with as a request under the Freedom of Information Act 2000.

We do hold the information that you have requested but I am afraid that I cannot supply it to you. This is because we believe it to be exempt under s.22 as it is information intended for future publication. As you have quoted, there was an indication that this information would be published after the end of the trial. This is indeed the case; there is a planned paper relating to moderators of outcomes in the trial, which is currently in review. Although there is no precise timescale, primarily because this is not within the control of the authors or Queen Mary, it should be in the coming months that this paper is released in to the public domain.


This exemption is subject to the public interest test. On balance we believe that, at this time, the public interest in maintaining the exemption outweighs the public interest in disclosing the information. On the one hand there is always a public interest in the transparency of a public authority. In addition, adding to the understanding of the PACE trial also favours the release of these data.


To date there have been a number of publications and papers arising from the PACE trial. These have all been subjected to a rigorous peer-review process that has ensured that the data and all other commentaries undergo similar scrutiny as to their validity and quality. It is in the public interest that all the publications have the same level of analytical vigour that peer reviewing brings, applied to them – peer-review being a fundamental of academic good practice. Since it has always been the intention to publish this information and it is in the process of being prepared for a future release, the public interest favours the release in that context, rather than at the current time. The First-tier Tribunal (Information Rights) has recognised this approach as reasonable.


If you are dissatisfied with this response, you may ask Queen Mary to conduct a review of this decision. To do this, please contact the College in writing (including by fax, letter or email), describe the original request, explain your grounds for dissatisfaction, and include an address for correspondence. You have 40 working days from receipt of this communication to submit a review request. When the review process has been completed, if you are still dissatisfied, you may ask the Information Commissioner to intervene. Please see www.ico.org.uk for details.

Yours sincerely


Paul Smallcombe

Records & Information Compliance Manager

 

[biophile]

To date there have been a number of publications and papers arising from the PACE trial. These have all been subjected to a rigorous peer-review process that has ensured that the data and all other commentaries undergo similar scrutiny as to their validity and quality. It is in the public interest that all the publications have the same level of analytical vigour that peer reviewing brings, applied to them – peer-review being a fundamental of academic good practice.

LOL @ all the talk about analytical vigour and a rigorous peer-review process and similar scrutinies for commentaries.

How did the misleading claims and frankly schoolboy errors pass through then?

http://www.oxforddictionaries.com/definition/english/vigour

• physical strength and good health

• effort, energy, and enthusiasm

http://www.oxforddictionaries.com/definition/english/rigour

• the quality of being extremely thorough and careful

• severity or strictness

• (rigours) harsh and demanding conditions

The "rigorous peer-review process" (or as chief editor Richard Horton put it, "endless rounds of peer review") of the Lancet spawned the so-called normal range in fatigue and physical function, you know, the one which overlaps with the trial eligibility criteria for disabling fatigue. It also failed to spot the mislabeling of the dataset for the normative population.

White personally waved through an editorial comment in the Lancet written by sympathetic peers, you know, the one which the Press Complaints Commission later ruled to be misleading about the issue of "recovery". White et al acted innocent when called on the 2011 recovery fiasco by David Tuller as to why reporters commonly confused normal for recovery.

The "rigorous peer-review process" of Psychological Medicine allowed the normal range absurdity to continue, and failed to spot that the PACE reasoning for abandoning the original definition of recovery was completely bogus and based on an error which even high-school students should have been able to spot easily. It also allowed more false claims to be made in the form of a misleading and erroneous comparison to a previous CBT trial on recovery in which White was even a co-author of, so there should be less excuse for just another blunder.

It is difficult to have confidence and respect for peer-reviewers who collectively display incompetence.

Not to mention all the other problems elucidated in the 2000+ posts on this current thread.

 

[MeSci]

If you are dissatisfied with this response, you may ask Queen Mary to conduct a review of this decision.

Anyone know a good medium?

 

[alex3619]

http://www.nice.org.uk/media/3F6/9E/ClinicalGuidelineStaticListAndSHCommentsNov13.pdf

Comment 90 on p80 of this collection of submissions to NICE: the PACE trial did not, if this is correct, use the Fukuda criteria, but the Reeves criteria here:

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC317472/

Identification of ambiguities in the 1994 chronic fatigue syndrome research case definition and recommendations for resolution

 

[biophile]

Technically, PACE did not use either the Fukuda criteria or what ended up being known as the Reeves 2005 criteria, as the latter is slightly different than the Reeves 2003 paper, although I do not recall what those differences were. And whatever CDC criteria they used, PACE later admitted to not using it properly at some stage i.e. not requiring symptoms for 6 months.