AHRQ's responses to comments on draft AHRQ review on CFS

[Dolphin]

http://effectivehealthcare.ahrq.gov/ehc/products/586/2064/chronic-fatigue-disposition-150416.pdf

Start:

Evidence Report/Technology Assessment Disposition of Comments Report

Research Review Title: Diagnosis and Treatment of Myalgic Encephalomyelitis/Chronic Fatigue Syndrome

Draft review available for public comment from September 23, 2014, to October 20, 2014.

Research review citation: Smith MEB, Nelson HD, Haney E, Pappas M, Daeges M, Wasson N, McDonagh M. Diagnosis and Treatment of Myalgic Encephalomyelitis/Chronic Fatigue Syndrome. Evidence Report/Technology Assessment No. 219. (Prepared by the Pacific Northwest Evidence-based Practice Center under Contract No. 290-2012-00014-I.) AHRQ Publication No. 15-E001-EF. Rockville, MD: Agency for Healthcare Research and Quality; December 2014. www.effectivehealthcare.ahrq.gov/reports/final.cfm.

Comments to Research Review

The Effective Health Care (EHC) Program encourages the public to participate in the development of its research projects. Each research review is posted to the EHC Program Web site in draft form for public comment for a 4-week period. Comments can be submitted via the EHC Program Web site, mail or email. At the conclusion of the public comment period, authors use the commentators’ submissions and comments to revise the draft comparative effectiveness research review.

Comments on draft reviews and the authors’ responses to the comments are posted for public viewing on the EHC Program Web site approximately 3 months after the final research review is published. Comments are not edited for spelling, grammar, or other content errors. Each comment is listed with the name and affiliation of the commentator, if this information is provided. Commentators are not required to provide their names or affiliations in order to submit suggestions or comments.

The tables below include the responses by the authors of the review to each comment that was submitted for this draft review. The responses to comments in this disposition report are those of the authors, who are responsible for its contents, and do not necessarily represent the views of the Agency for Healthcare Research and Quality.

Jennie Spotila just highlighted this on her blog: http://www.occupycfs.com/2015/05/29/running-in-the-background/

I don't recall seeing it before but I don't read all the threads on Phoenix Rising

It's a long document:
http://effectivehealthcare.ahrq.gov/ehc/products/586/2064/chronic-fatigue-disposition-150416.pdf

Some times it can be interesting to read about particular experts' views. For example, I found it interesting to read all sorts of weird and wonderful views Peter White had in the comments on the draft NICE guidelines (read here: http://forums.phoenixrising.me/inde...ice-guidelines-insight-into-their-views.1239/) and see that he has comments in the AHRQ document. If anyone sees anything of potential interest, perhaps you could post them as I imagine most of us won't read much of it.

 

[Valentijn]

Their responses regarding references to Wiborg were pretty disappointing. Basically they claimed it was only a review of other trials, and did not contain new data. But it did include new data (deliberately withheld from the papers which were being "reviewed"), regarding actometers showing no improvement even though patients gave better answers on questionnaires.

Between that and their approach to the Oxford criteria ("not a good definition, but lets base all of our recommendations for treatment on it"), they either aren't very bright or went into the project intending to spin results.

At least the IOM did well. Otherwise reading this crap would be leaving me highly pissed off instead of moderately annoyed

 

[Esther12]

At least the IOM did well. Otherwise reading this crap would be leaving me highly pissed off instead of moderately annoyed

I started reading it thinking this was for the unreleased P2P report, and was thinking 'oh-oh... they sound a bit rubbish'. (TBH, I spent about 45 minutes getting increasingly concerned, then realised I'd been an idiot).

A bit interesting to see why the Evidence Review was worse than it should have been.

I only skimmed through and was mainly looking for interesting replies and the reading comments on the basis of that. I got a bit of a sense that they had so many responses that they dealt with a lot of them in a rather cursory manner. I noticed a lot of critical stuff about PACE and their recovery claims... go patient peer reviewers!

edit: After reading more of their responses to comments I feel like we were lucky that the evidence review was not worse.

 

[Valentijn]

I got a bit of a sense that they had so many responses that they dealt with a lot of them in a rather cursory manner.

They also seemed to have approached the research in a rather cursory manner. Their understanding of it didn't seem to go beyond what someone would get from quickly skimming abstracts.

 

[jimells]

They also seemed to have approached the research in a rather cursory manner. Their understanding of it didn't seem to go beyond what someone would get from quickly skimming abstracts.

That's probably the most to be expected from any modern office today. I heard a report recently that the average person checks their smart phone ONE HUNDRED FIFTY times a day. Wow. That doesn't leave much time to do any actual work.

 

[WillowJ]

They also seemed to have approached the research in a rather cursory manner. Their understanding of it didn't seem to go beyond what someone would get from quickly skimming abstracts.

Or possibly the titles, in some cases.

Someone gave them the cost effectiveness PACE paper, helpfully pointing out the null employment results, and they replied that cost effectiveness was beyond the scope of the review!

Guess White et al. played their cards right to bury unwanted data.

 

[alex3619]

I have only just started reading but it looks as if PDW may get the anti PACE commentary weakened.

 

[Nielk]

I really got a kick out of reading their reasoning for why they decided to include Oxford criteria based studies even though they agree that Oxgord criteria are inappropriate.

They state that they had to include them because as it is, there is a lack of studies included for this report.!!!

So, they had to include studies that were actually not based on real ME/CFS patients in order to have an adequate amount of research.

Are they not embarrassed to admit that QUALITY was not an aim of this report?

 

[user9876]

I have only just started reading but it looks as if PDW may get the anit PACE commentary weakened.

One of his comments is about safety

ES - 28 Several previous studies have found worsening effects with exercise and a survey sponsored by the ME Association found that patients believed that GET made more people worse compared with other treatments.71,72

The problem with generalising from surveys of patient organisations are two -
fold:

1) We do not know what the survey members’ diagnoses were, and we are aware of one study showing high rates of non - CFS diagnoses in such a patient organisation. Brimmer and colleagues (2013) found that 59% of 49 US patient support group members had an exclusionary condition, and only 35% met criteria for CFS.

2) We do not know if they really did receive graded exercise therapy; one qualitative study of such a survey found significant variation in content and delivery of treatment received (Gladwell et al, 2014). Since the randomised controlled trials do not generally suggest that harm follows GET, we suggest that caution is necessary before generalising from such surveys

The logical implication of his first point is that GET does harm some patients who may be misdiagnosed as having CFS. I seem to remember that mis-diagnosis included depression and sleep disorders but not sure. Haven't there been claims that GET helps with depression although I think there was a study that suggested it made little difference. But given his claim and the implications he should be justifying his claim by showing evidence that those who could have been misdiagnosed could been harmed by GET and that it is more likely than harm to people with ME. His statement just tries to muddy the water without any real critical thinking.

His second point should also cause concern since the implication is that GET is hard to role out as a service even within specialist centers (given that patients attending specialist centers report deterioration). Of course his claim that trials do not suggest harm should not be taken seriously until he has published all the data defined in the PACE protocol.

 

[Bob]

ES - 28 Several previous studies have found worsening effects with exercise and a survey sponsored by the ME Association found that patients believed that GET made more people worse compared with other treatments.71,72

The problem with generalising from surveys of patient organisations are two -
fold:

1) We do not know what the survey members’ diagnoses were, and we are aware of one study showing high rates of non - CFS diagnoses in such a patient organisation. Brimmer and colleagues (2013) found that 59% of 49 US patient support group members had an exclusionary condition, and only 35% met criteria for CFS.

The logical implication of his first point is that GET does harm some patients who may be misdiagnosed as having CFS.

It's a weak argument because we do know what the patients were diagnosed with; They were diagnosed with ME/CFS, and that's why they are responding to the survey. Whether or not they actually have ME/CFS is an issue for the health services that provided the survey respondents' treatment.

Reading the comments from MEA's latest public survey, it seems that many patients who deteriorated after GET were simply given an appallingly inappropriate service by therapists who didn't understand their patients or the illness. The comments suggest it has nothing to do with misdiagnosis.

 

[worldbackwards]

His statement just tries to muddy the water without any real critical thinking.

I'm shocked, shocked...

 

[user9876]

It's a weak argument because we do know what the patients were diagnosed with; They were diagnosed with ME/CFS, and that's why they are responding to the survey. Whether or not they actually have ME/CFS is an issue for the health services that provided the survey respondents' treatment.

Reading the comments from MEA's latest public survey, it seems that many patients who deteriorated after GET were simply given an appallingly inappropriate service by therapists who didn't understand their patients or the illness. The comments suggest it has nothing to do with misdiagnosis.

My point was that even if we were to accept that there were significant people misdiagnosed then he needs to demonstrate that those who may have been included with a misdiagnosis were more likely, given the possible diseases that they had, to be harmed by GET than those with a correct ME diagnosis. I think he would find any evidence to support that yet that is the consequence of him dismissing the various patient surveys on a mis-diagnosis argument.

At the very least his statement is accepting that there is a group of people who get harmed by GET and that there is a good chance that they will be diagnosed with ME and then harmed by GET. Hence if you took this interpretation of his argument it would suggest that GET is a bad treatment as there is a failure to apply good diagnostic process.

Unfortunately the AHQS response was to accept his point without further analysis. I don't know if comments can be made on comments.

My belief is that GET is likely to be harmful to many with ME and with little benefit it isn't worth the risk. I noticed (on a quick scan) that Whites main emphasis with the AHQS report was to try to claim GET was safe and to increase his citation count. He did a search for his name on the comments and he didn't seem to want to justify his recovery claims. I wonder if this indicates that he is most concerned to justify his safety claims (because to be seen to have got those wrong would be pretty damning) or if that is all that is left that he feels is defensible.

 

[jimells]

The tobacco companies insisted for decades that their products were fun and safe. They also falsified data and hid unfavorable results. They are still paying for their crimes today, at least a little.

I doubt if PD White has pockets as deep as the tobacco companies. When the time comes the disability insurance mob will dump him and Sir Simon overboard. Perhaps he is starting to fear for his own skin...

 

[Sidereal]

I couldn't agree more with this comment:

I offer here a few comments on the recently released preliminary draft of the AHRQ report on Diagnosis and Treatment of Myalgic Encephalomyelitis / Chronic Fatigue Syndrome (ME/CFS). First, I want to point to the intellectual absurdity of first admitting that ME and CFS may well describe different populations, and that definitions that do not make PEM mandatory may exacerbate this problem. And yet the authors go ahead and include all definitions on the same level. They then list their Key Questions that intentionally omit all reference to attempts to understand the underlying processes of this disease/these diseases. They are interested only in Diagnosis and Treatment. But how can one arrive at an accurate Diagnosis without some understanding of the disease(s) being diagnosed? They set out to answer a question already made unanswerable before they begin. The whole project is premature and doomed, as many of us protested to NIH some time ago.

 

[Anne]

My point was that even if we were to accept that there were significant people misdiagnosed then he needs to demonstrate that those who may have been included with a misdiagnosis were more likely, given the possible diseases that they had, to be harmed by GET than those with a correct ME diagnosis. I think he would find any evidence to support that yet that is the consequence of him dismissing the various patient surveys on a mis-diagnosis argument.

At the very least his statement is accepting that there is a group of people who get harmed by GET and that there is a good chance that they will be diagnosed with ME and then harmed by GET. Hence if you took this interpretation of his argument it would suggest that GET is a bad treatment as there is a failure to apply good diagnostic process.

Excellent point!

So White is admitting that there are patient groups who get harmed by GET and as user9876 says, these groups (according to White) are commonly misdiagnosed as having ME/CFS.

I wonder which illnesses he is thinking of - since PEM and deterioration following activity/exercise is often thought of as rather exclusive to ME/CFS and with most other illnesses patients are considered to benefit from exercise?

 

[Tom Kindlon]

Regarding how GET is done, here's an extract from a letter I had published:

Disabil Rehabil.2015;37(5):466-7. doi: 10.3109/09638288.2014.952456. Epub 2014 Aug 19.
Elements of rehabilitative strategies associated with negative outcomes in CFS/ME: the need for further investigations.
KindlonT1.

The authors posit that the level of supervision may explain the results that are less than satisfactory in patient organisation surveys. They have noted that in one survey, 40% of those trying GET had been unsupervised and that in a 2008 Action for ME report, respondents whose GET was supervised by their General Practitioners rather than by ‘‘NHS specialists’’ more often reported worse outcomes [7]. However, this does not prove that unsupervised therapy is necessarily worse. Indeed, the third group in that survey, described as ‘‘other’’, reported marginally lower rates of deterioration (28.93%, 46/181) than those who had undertaken GET with a NHS specialist (31.27%, 111/355).

 

[Tom Kindlon]

Also, from:

Kindlon T. Reporting of harms associated with graded exercise therapy and cognitive behavioural therapy in myalgic encephalomyelitis/chronic fatigue syndrome. Bulletthe IACFS/ME 2011;19: 59–111. http://iacfsme.org/ME-CFS-Primer-Ed...of-Harms-Associated-with-Graded-Exercise.aspx

3.3. Limitations of survey data

Some researchers have been dismissive of the survey results,contrasting them with what they see as the safety that has been proven in RCTs and suggesting the discrepancy might be due to improper implementation of GET outside of RCTs. Even if safety had been shown in RCTs, which is debatable given there appears to be scope for improvement in the reporting of harms (see sections 4 and 5),it has been observed in other medical domains that outcomes from routine practice may be more relevant than the “artificial” environment of a clinical trial (86, 87). Moreover, a subgroup analysis of a GET survey performed by Action for M.E./AYME and published in 2008 (82) found that there was no statistically significant difference in the rate of people saying they were made worse from engagingin GET under a “NHS specialist” (31.27%, 111/355) compared to the rest of those reporting such an outcome from GET in another scenario (33.02%, 70/212).

 

[Snowdrop]

So White is admitting that there are patient groups who get harmed by GET and as user9876 says, these groups (according to White) are commonly misdiagnosed as having ME/CFS.

I wonder which illnesses he is thinking of - since PEM and deterioration following activity/exercise is often thought of as rather exclusive to ME/CFS and with most other illnesses patients are considered to benefit from exercise?

Keep digging that hole PDW. You might just find you have something you want to bury in it.

 

[Tom Kindlon]

Peter White sometimes tries to claim that GET doesn't cause problems if done under an ME/CFS specialist.
The data from 2015 ME Association ME/CFS ‬survey report http://bit.ly/1N38qkp says otherwise:

 

[Dolphin]

PD White, T Chalder, R Moss-Morris, M Sharpe, AJ Wearden
Executive Summary

ES-12 “…and there were more withdrawals in the GET group in several trials.” This is not the case. Therehave been6 RCTs of GET for CFS published (Fulcher, Powell, Wearden, Moss-Morris, Wallman, White), although thereare published trials of other exercise interventions. The proportions withdrawing from GET versus the control arm were similar in all but onetrial (Wearden et al, 1998). The proportions of participants withdrawingfrom GET in the largest(PACE) trial were the smallest (6%) compared to all other treatment arms(7, 9, and 11%), although differences were not significant (White et al, 2011; table 2). Wearden’s (1998)trial intervention was designed asa fitness training intervention rather than graded exercise therapy. The intervention had higher starting levels of exercise intensity than the other trials, and exercise progression was based on change in heart rate, which probably explains the higher drop-out rates (Wearden et al, 1998).

Response:
Thank you for your comments. We have reviewed the withdrawal rates of the trials where this data is available and reported them accordingly.

Anyone know why White are them are treating Powell et al. (2001) as a GET trial but the FINE Trial (Wearden et al. (2010)) (a different trial from Wearden et al., 1998) when the FINE Trial was just a big version of Powell et al. 2001?
My suspicion is because Wearden et al. (2010) did not give a good result.