1. Patients launch $1.27 million crowdfunding campaign for ME/CFS gut microbiome study.
    Check out the website, Facebook and Twitter. Join in donate and spread the word!
A disease with two faces? Re-naming ME/CFS
Persuasion Smith covers the bases on the misleading and disreputable name for our disease we've all been saddled with ...
Discuss the article on the Forums.

'Recovery' from chronic fatigue syndrome after treatments given in the PACE trial

Discussion in 'Latest ME/CFS Research' started by Sam Carter, Jan 31, 2013.

  1. Firestormm

    Firestormm Guest

    Messages:
    5,824
    Likes:
    5,982
    Cornwall England
    Sam Carter and Dolphin like this.
  2. Firestormm

    Firestormm Guest

    Messages:
    5,824
    Likes:
    5,982
    Cornwall England
    This is it in a nutshell for me:

    From Charles's published letter.

    I mean if they hadn't used that word and had reported everything else: I wouldn't have been faced with so much grief. People latch on to words like 'recovery' and interpret PACE as recovery. Doctors do it, health professionals do it, and family members do it.

    One of the concerns raised by some members of this ME Management course I am on was the feeling that we might be expected to embrace the course; that if we didn't it would count against us in terms of benefits etc. It was a fair point - but my issue was with the expectation others might have that we should be experiencing some sort of sustained recovery as a result of CBT GET and SMC.

    That if we didn't then inevitably it was our fault. Incidentally, this isn't how our course is being run; but the pressure that has come from PACE and the media, transmitted through the health service, and taken on by patients - whether real, implied or imagined - did lead us to raise the matter.

    The more informed health professional will say something like 'It's the best we can offer at the moment and it might not help everybody'. But that never made it into the media. And even I feel an ominous shadow approaching in the shape of the DWP what with the way things are going there.

    And why the hell didn't 22% get to in the headlines? All I remember is 40% = 'moderate'. Now White is using the 22% figure. Could be my foggy-head but I thought it was Bob who had unearthed the 22% figure from the inferno.

    'Moderate' Pah! There's another bloody term that gets my back up. Compared to what? What might we expect from an 'Effective' treatment if 22% = Moderate and only then if you use a rather loose and stupid definition for 'recovery'? :zippit:
     
    Sam Carter, Sean and Bob like this.
  3. Firestormm

    Firestormm Guest

    Messages:
    5,824
    Likes:
    5,982
    Cornwall England
    From White's Reply - first paragraph:

    We were talking about this somewhere else recently on PR. Because of what ME is and how it is defined; I wonder if it will ever be possible to ideally determine 'recovery' from a single treatment: if what he says here is true? It seems very strange to me that he should say such a thing. Why use the word 'recover' at all then if it is never relevant?

    There has to be something better than simply asking a patient if they found such-and-such a programme 'helpful'. Something that can be applied to fluctuating conditions until such time as a treatment is found that will address the underlying biological process/es. Of course to do that we will need to learn to better define people with ME from people with other things: and that means better understanding what ME is... and perhaps more importantly - isn't.

    I can see that when talking on the one hand about ME being heterogeneous, of patients also having possibly other factors that contribute to their continuing disability - it might be hard to say X caused recovery. But it doesn't stop them saying so, does it? And lest we forget the SMC in this Trial was pathetic. It might have been representative but it was not something that patients should expect from the GPs - let alone specialists.

    I am all for learning how to better live with a chronic condition: but please don't pretend that these management tools are anything more than they are - or any more useful to someone with ME than they are to someone with Rheumatiod Arthritis. Let's work on the wording a little. Some better consideration of the impact of terminology would go a long way.
     
    Sean likes this.
  4. Esther12

    Esther12 Senior Member

    Messages:
    5,387
    Likes:
    5,902
    Possible source of confusion (and this is from memory, so I might have some details wrong): Bob has rightly pointed out that, according to their (weak) criteria for clinically significant difference, only 18% [about that anyway] of patients received benefit from CBT/GET+SMC vs SMC alone. White et al did not choose to focus on this figure.

    They've gone on to define 'recovery' in such a way that they can claim that CBT/GET led to more people recovering, than it led to people gaining a clinically significant improvement, as fewer people from the SMC alone group fell into the 'recovered' criteria than fell into the 'clinically significant improvement'.

    It is a bit complicated and confusing, and I hope my possibly inaccurate post hasn't made things worse! I thikn I got the gist right, even if I'm not sure of all the details/terminology used is exactly right.
     
    Sean, Valentijn and Firestormm like this.
  5. Esther12

    Esther12 Senior Member

    Messages:
    5,387
    Likes:
    5,902
    Also, patients need to have an accurate understanding of the evidence for efficacy for a treatment before they can provide informed consent for it. If I'd been made aware of the evidence, I never would have spent any of my time or effort on the 'rehabilitation' available on the NHS. There are other more fun and useful things I'd rather do instead. I think that a lot of patients on agree to these treatments, and the medicalisation of their cognitions and behaviour, because they have been misled.
     
    Sean, biophile, Dolphin and 1 other person like this.
  6. Bob

    Bob

    Messages:
    8,913
    Likes:
    12,611
    South of England
    For self-reported 'improvement' it was a maximum of 15% of participants who reported a 'clinically useful' improvement in the CBT+SMC and GET+SMC groups when compared to SMC alone.

    For the 'recovery' paper, the net results for CBT and GET (when the results of the SMC control group are subtracted) were almost exactly the same, for 'recovery', as they were for the reported clinically useful 'improvements': 15% were declared 'recovered' for the CBT+SMC & GET+SMC groups when compared to SMC alone.

    Main results in the recovery paper:
    GET+SMC = 22%
    CBT+SMC = 22%
    SMC = 7%
     
    Dolphin, Simon and Firestormm like this.
  7. user9876

    user9876 Senior Member

    Messages:
    796
    Likes:
    1,961
    Whites statement doesn't make any sense. All they have done is asked a patient how they feel after some treatment. They have done this using a number of questionaires so asking a patient 'do you feel you have recovered?' is no difference. The point is that the randomisation in the trial along with a control group is intended to deal with the variability assuming a sufficient sample size. Dr Shepard didn't phrase his comment in the best way and so White has interpreted as asking do you credit treatment to your recovery.

    The problem with an RCT of this type is it is basically black box testing (as in you try to judge results looking only from the outside with no attempt to look at mechanism) and so they can't judge by any more methods that asking patients and also testing their abilities. The admission by White here is important in that he basically says that they didn't bother to conduct the only objective test (6 minute walking test) in an adaquate manner.
     
    WillowJ, Sean and Valentijn like this.
  8. Esther12

    Esther12 Senior Member

    Messages:
    5,387
    Likes:
    5,902
    I wonder if some will see that as White being a bit clever-clever, and slipping out of an argument. White has chosen to define 'recovery' in a way which is far from what most people or patients would understand by the use of the word.
     
    Sean, Valentijn and Firestormm like this.
  9. urbantravels

    urbantravels disjecta membra

    Messages:
    1,333
    Likes:
    507
    Los Angeles, CA
    "Oh no, I feel worse after this treatment! I must be regressing to the mean!"

    And as for that "composite measure of recovery" - was that developed post facto or was it in the trial protocol at the start? Because if you decided to custom-blend a "measure of recovery" after your data was already in, that's as good as an admission that you're cooking the books to get the result you want.
     
    Sam Carter, Sean, ukxmrv and 3 others like this.
  10. Dolphin

    Dolphin Senior Member

    Messages:
    6,872
    Likes:
    6,166
    Yes, it's an important issue: the hyping of results.

    Also, there can be different types of hype. One could have the hype of the percentage improved. And then hype about the percentage that recover. "Recovery" suggests that all that stopped these people from being well was a poor lifestyle. However, if very few actually recovered, it suggests a poor lifestyle isn't sufficient to explain ongoing problems.
     
    WillowJ, Sam Carter, Sean and 2 others like this.
  11. Dolphin

    Dolphin Senior Member

    Messages:
    6,872
    Likes:
    6,166
    What was in the protocol was:
    They only left (iv) unchanged. They loosened the requirements for the other three.
     
    Bob, Firestormm and Valentijn like this.
  12. SOC

    SOC Senior Member

    Messages:
    5,367
    Likes:
    6,465
    USA
    Talk about bad science. :rolleyes: I'm appalled that this was published given the way they cooked the books. In my field that would be called falsification. Why does psychology get to play fast and loose with the basics of scientific research?
     
    Sam Carter, Sean and Valentijn like this.
  13. biophile

    biophile Places I'd rather be.

    Messages:
    1,410
    Likes:
    4,944
    Some commentary on the White et al reply to the letters (Simon below summarizes each letter):

    Claim 1 of 8: [Using several domains of improvement and combining them negates the problem where no single threshold is sufficient on its own.]

    True to some extent, except all the domains used were still inadequate even when combined, for reasons already discussed on this thread so I do not need to repeat them at depth. The fatigue requirement was based on inadequate normative data and the physical function requirement was disgracefully atrocious. The CGI requirement did not indicate recovery per se but perceived improvement relative to baseline, and the Oxford criteria requirement is not all that it appears either, it was relatively easy to get knocked out of a diagnosis due to a minor change in one symptom despite ongoing problems in other domains, or even if a non-fatigue symptom become worse than fatigue, while the judgement of CFS caseness was left up to people who knew about treatment allocation and possibly sympathetic to CBT/GET.

    The original definition of recovery, which was abandoned, was better, but the authors claim the post-hoc one was better and even describe it as "conservative".

    Claim 2 of 8: [Asking patients whether they were recovered as a result of treatment is meaningless because their attribution may be incorrect.]

    The point of the question should be more about whether the patient's health is restored to where it would be without CFS and whether they genuinely feel normal and healthy again (as opposed to say, merely scoring >60/100 points on a scale), but this also has its own problems, because CBT and GET encourages the re-attribution of symptoms to normal everyday life rather than illness, which is why objective outcomes are important too.

    White et al have claimed before that "recovered" PACE Trial participants can expect an average of 3 symptoms each. There are multiple anecdotes of patients who have done CBT or LP without much improvement but have worked themselves into a denial about their ongoing symptoms and impairments, claiming to be recovered when clearly not so.

    Claim 3 of 8: [It may be true that self-ratings may be biased because participants were not masked to treatment allocation, but this is inconsistent with CBT being more effective than APT despite lower expectations before treatment.]

    That defence has been wheeled out before, but it completely fails to address that CBT and associated reading material actively attempts to change the participants' perceptions and expectations about their illness. Imagine a trial where the drug group also received repetitive encouragement/optimism and were told the drug is "powerful and safe" as they were in the PACE Trial? I do not think the authors of such a trial would be getting away with claiming that subjective biases are irrelevant to outcomes simply because of participants' perceptions and expectations before treatment even began.
     
    Sean, WillowJ, Svenja and 3 others like this.
  14. biophile

    biophile Places I'd rather be.

    Messages:
    1,410
    Likes:
    4,944
    Claim 4 of 8: [Multiple self-reported measures are more important than a single objective measure i.e. 6MWD. The way we conducted the 6MWD was not exactly the same as others do it, so it may not be directly comparable, and we only had 72% follow-up anyway.]

    Their main defense seems to be "we did not conduct the test properly", lacking additional encouragement and 30-50m of walking space. These can indeed make a difference, but I highly doubt it would be enough to explain such low scores in the PACE Trial which have a 250-300m discrepancy with healthy scores. Get healthy controls to walk as far as possible in 6 minutes up and down a 10 meter corridor, it should be much more than 350 meters or so. Not all 6MWT studies are conducted the same anyway, it might be possible to find some which were similar to the method that PACE used.

    [Edit: in a later post I discovered that the sources which White et al cite suggest that encouragement only accounts for about 30m on average, not the 250-300m required to explain the dismal 6MWD scores].

    White et al claim that, "Rather than encouragement, we told participants, 'You should walk continuously if possible, but can slow down or stop if you need to.'" This statement is not contradictory to encouragement and they did also say "Please walk as far as you can." The slow down statement has appeared in studies which also had encouragement.

    For example, one of the letters cites Lipkin et al 1986 for 6MWD scores in chronic heart failure and healthy controls:
    http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1339640/pdf/bmjcred00224-0015.pdf

    * McGavin et al 1976 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1639415/pdf/brmedj00510-0042.pdf

    Maybe I will later try to find studies which applied the 6MWT exactly as PACE did, but again my money is on healthy controls doing much better than PACE participants did. Irrespective of problems with direct comparisons, the 6MWD is still useful in a relative sense, and here we know that CBT made no significant difference, and the GET improvement was not clinically significant on average according to their definition of clinical significance for other measures (0.5SD).

    White et al are simply not serious about objective measures in the first place. Look at how they handled the actigraphy issue, and avoid discussing the fact that sickness-related benefits were unchanged despite reported recoveries. Subjective measures make their therapies look somewhat successful, but the objective measures do not, so (apparently) they have been avoiding them. In what other field of medical research is objective evidence downplayed and dismissed so casually?
     
    WillowJ, Svenja, Sam Carter and 3 others like this.
  15. biophile

    biophile Places I'd rather be.

    Messages:
    1,410
    Likes:
    4,944
    Claim 5 of 8: [We agree there is a difference between sustained recovery and temporary remission, so we were careful to point out that these results are relevant to one time point and the current episode of the illness.]

    Yet they seemed quite happy conflating recovery and remission in general. If they were so concerned, they should have just used the tentative term "remission" and not repeatedly called it "recovery" with minimal caveats. As Samuel Carter's letter points out, the source that White et al used frowns upon how White et al have used it.

    Claim 6 of 8: "We believe our approach is reasonable."

    I am glad they are not the final arbiters of what is reasonable in ME/CFS research. These are the same researchers who for years have been discouraging medical investigations, and have been claiming that CBT and GET lead to increased activity levels and function, despite objective evidence to the contrary. Their approach to research appears to be more about spinning results in their favour and ignoring the stark implications of objective data while relying exclusively on how much they can change the (allegedly initially faulty) perceptions of their patients.

    It would be refreshing and appreciated to see a frank honest admission that their cognitive behavioural model was an over-reach and now they want to tease out the true effect on patients' lives.

    They and their supporters/apologists previously rejected the idea that employment outcomes at 12 months from baseline are relevant to improvement and recovery, arguing that people who have been out of work for years cannot find work quickly, especially in a time of economic downturn. Despite the reported improvements and "recoveries", in general the participants were not getting off sickness-related benefits either, not just unemployment benefits. In other words the reported improvements may have never been road tested in the real world without being cushioned by financial support. It is one thing to feel better while not working, but let us see how that "recovery" holds up when working 40 hour weeks again.

    White et al did not respond to Shepherd's point about the rates of sickness-related benefits increasing over time and being the same between groups at followup despite all the supposed recoveries in the CBT and GET groups. One cannot be "recovered" from an illness and also genuinely receive sickness-related benefits for the same ongoing illness!

    Furthermore, employment is not a black and white activity, some people would have been part-time employed with the option to work more when possible. PACE calculated the number of total lost work days due to illness, and found no significant difference between groups. I also looked into the economic downturn argument and found that about half of the trial participants were processed and discharged/followed-up before the economic downturn even began.
     
    WillowJ, Svenja, Sam Carter and 4 others like this.
  16. biophile

    biophile Places I'd rather be.

    Messages:
    1,410
    Likes:
    4,944
    Claim 7 of 8: "The findings from the PACE trial are clear; however we measured recovery, CBT and graded exercise therapy (GET) were more likely to lead to recovery, when added to specialist medical care (SMC), compared to either adding APT or SMC alone. Recovery after SMC alone, using our composite criteria, was only 7% - the same as that without treatment (Cairns & Hotopf, 2005) - whereas three times as many (22%) recovered after receiving CBT or GET."

    Number Needed to Treat is about the same no matter how they defined improvement and recovery, suggesting that there is little difference between a small improvement and a full recovery. Those who "recovered" were probably already close to the threshold of "recovery" and only needed to make relatively minor improvements to step over the border into pseudo-recovery land. CBT and GET groups were more likely to report these improvements, so demonstrate an advantage on these self-reported measures, but using the word "recovery" was going too far, especially without objective measures.

    White et al have repeatedly cited the Cairns & Hotopf 2005 review to back up their claim that the recovery rate of SMC was about the same as what would be expected from a natural recovery without special treatment i.e. 7% (note that the review did not necessarily exclude studies where basic "treatment" similar to SMC was being offered).

    http://occmed.oxfordjournals.org/content/55/1/20.full.pdf

    As others have already noted about this review, the rates and definitions of improvement and recovery were highly variable between studies, making it difficult to compare with confidence. Furthermore, the figure White et al cite had combined all sources i.e. primary care, secondary care, and community settings. The irony here is that the PACE Trial was conducted in secondary care and this is what Cairns & Hotopf stated about the analysis of secondary care studies:

    Either White et al are unaware of this (did not read the paper properly) or are engaging in deceptive spin. Of course, a figure of 23.5% is higher than their best therapies CBT and GET at 22%, so it would not look good to cite such a figure when promoting the success of their adjunctive therapies! White et al must have been smiling with glee when finding the 7% figure instead, which just happens to coincide with their control group. The authors expected a NNT of about 2, with the associated response size also being significantly greater than what they ended up using. PACE must have clearly been a disappointment behind the scenes when the actual NNT for a much smaller improvement was higher at 7 or 8 or so.

    Claim 8 of 8: "The PACE trial has shown that both CBT and GET are moderately effective, safe, cost-effective, and are more likely to lead to recovery (White et al. 2011, 2013; McCrone et al. 2012). These treatments should now be routinely offered to all those who may benefit from them (Crawley et al. 2013)."

    All these are debatable to some extent. CBT and GET show a limited statistical advantage in some domains but not others, while the nature of these improvement in practice are uncertain when considering how CBT and GET aim to change patients' perceptions. The total disregard for more objective data is concerning, especially when what is available from other studies suggests no benefit. And who may benefit from these therapies? The generalizability of the findings to most "ME/CFS" patients is debatable, and as one letter pointed out, we are not given enough information about the "recovered" as a group, such as how much did they improve and what their final scores were on average. We have yet to see the paper on mediators and predictors. There is evidence which suggests a efficacy-effectiveness gap between trials and routine clinical practice. Once all these limitations and biases are taken into account, there may be little if anything left.
     
  17. biophile

    biophile Places I'd rather be.

    Messages:
    1,410
    Likes:
    4,944
    They claim that the post-hoc changes to the protocol were made before analysing the data. We are still waiting on publication of the "statistical analysis plan", which is supposed to outline the justification for the changes. Unsurprisingly, some people may not believe that White et al finalized all the changes before seeing the data. I think someone mentioned earlier that they may have not needed to see the data before getting the impression that the trial was not going as well as initially expected, since assessors were not masked to treatment allocation during the trial.

    Hehe Valentijn @ "psychobabble maths". Reminds me of their claim that half of the UK working age population score 85 points or more in physical function (their entire justification for shifting the threshold in physical function), another false claim. And people wonder why we question the validity of their claims.
     
    Sam Carter and Valentijn like this.
  18. Simon

    Simon

    Messages:
    1,531
    Likes:
    4,911
    Monmouth, UK
    Summary of the Letters criticising the PACE Recovery criteria (Psychological Medicine, July '13)
    edit: added comparison of original protocol criteria with new, weaker criteria:

    Original criteria
    1. Chalder Fatigue score of <= 3 (0-11 scale, 0 is best)
    2. SF-36 Physical Function score of >=85 (0-100 scale, 100 best)
    3. Clinical Global impression score of 1 (=”Very much better” than before trial, 1-7 scale, 1 is best)
    4. No longer meet Oxford, Fukuda or “ME” Criteria

    Revised criteria
    1. Worse fatigue allowed: threshold raised to <=18 (0-33 scale, equivalent to 4-7* on 0-11 scale, was <=3/11)
    2. Worse physical function allowed: SF36 PF score reduced to >=60 (was 85)
    3. Less improvement needed: Clinical Global Impression score of 1 or 2 (“Much better” now acceptable, as well as “very much better”)
    4. No change on case definition criteria

    *Technically a Chalder Fatigue score of 18 (0-33 scoring) could represent a score up as high as 9 on 0-11 scoring, although my guess is that very few with a score of 18 have above 7 on 0-11.
    Letter 1 – Need for Objective Measures (Susanna Agardy)
    The Recovery criteria do not include the sole objective outcome measure, the 6 minute walking test (6MWT). Important as there was no difference on 6MWT between SMC and CBT groups, and a gain of only 35m over one year for the Graded exercise group vs SMC.

    Letter 2 – Limited PACE follow-up period means this is remission not recovery (Samuel Carter)
    Due to the fluctuating nature of CFS, and the limited follow-up data (52 weeks from recruitment), the authors ‘recovery’ definition in fact only defines ‘remission’. Quotes the author of the last Cochrane Review of Graded exercise for CFS who says the PACE trial needs longer follow-up, to determine whether patients who respond to exercise stay well or relapse.

    Letter 3 – Physical Function ‘Recovery’ criteria also defines severe impairment (Robert Courtney)
    The PACE recovery threshold of 60 or more on the SF36 Physical Function scale overlaps with the trial recruitment criteria, and has been used to indicate severely impaired physical function in similar patient groups.

    Letter 4 – Each of the Recovery criteria is weak (Duncan Cox)
    1. Fatigue and function ‘recovery’ scores are possible at baseline where they define severe and disabling fatigue.
    2. Minimal gains on baseline fatigue or function scores could mean patients no longer meet the Oxford Criteria (a recovery criterion), highlighted by the fact that 41% of the SMC control group no longer met Oxford Criteria at 12 months.
    3. Patients rating themselves ‘much better’, but not ‘very much better’, may be improved - not recovered.

    Letter 5 – Response bias could explain self-rated gains in CBT, objective 6MWT data needed (Carly Maryhew)
    Possibility of significant response bias as the trial was unblinded. This could explain the higher self-rated scores for the CBT score vs SMC & APT despite no difference between groups on the objective 6MWT. Other CFS studies have found no correlation between self-reported fatigue or function and objective measures of activity. 6MWT for results for those who met recovery criteria could have been compared with healthy norms.

    Letter 6 – Hard to judge Recovery without real world measures eg employment (Charles Shepherd)
    Participants were not asked if they consider themselves recovered as a result of treatment. `Recovery' implies a sustained return to symptom-free health with the ability to repeatedly and reliably participate in all aspects of normal life- employment, education, social activities. etc. Three objective markers of recovery would have improved the paper:
    • Receipt of state Benefits indicates people are still ill (no change in PACE data)
    • Return to employment/education (or ability to do so). (No change in PACE employment)
    • Ability to mobilise. (No or minimal change in 6MWT PACE data.)
     
    WillowJ, Svenja, Sam Carter and 4 others like this.
  19. Firestormm

    Firestormm Guest

    Messages:
    5,824
    Likes:
    5,982
    Cornwall England
    I didn't mention the knock-on effect with Insurance companies/claims assessors either as a concern. Picked this up from IiME Facebook just now. I don't know when it was posted by 'Swiss Re' (I think it might have been in 2011/12) but you might want to take a butcher's look:

    Also of interest (a little off piste perhaps) was this:

    I wouldn't want to encourage comments in relation to that second bit here of course but I hadn't read that before and found it intriguing if perhaps out of date?
     
    MeSci likes this.
  20. Bob

    Bob

    Messages:
    8,913
    Likes:
    12,611
    South of England
    I'm not even certain about that... Didn't participants only have to be excluded from the Oxford Criteria for a single week (or something similar) to be consider 'recovered'? So even that criteria was drastically watered down. (I'm not sure I've got this entirely accurate, without checking.)
     

See more popular forum discussions.

Share This Page