1. Patients launch $1.27 million crowdfunding campaign for ME/CFS gut microbiome study.
    Check out the website, Facebook and Twitter. Join in donate and spread the word!
AVIVA Semi-Finals: National ME/FM Action Network is competing for $100,000
The National ME/FM Action Network in Canada is competing for $100,000 for biomedical research of ME and FM in the Aviva Community Fund contest. With thanks to all who helped, they made it through the first round of voting into the Semi-Finals.
Discuss the article on the Forums.

'Recovery' from chronic fatigue syndrome after treatments given in the PACE trial

Discussion in 'Latest ME/CFS Research' started by Sam Carter, Jan 31, 2013.

  1. alex3619

    alex3619 Senior Member

    Messages:
    7,725
    Likes:
    12,655
    Logan, Queensland, Australia
    Does anyone know what percentage of the population scores 60 or worse at SF-36? Then what percentage of working age adults score 60 or less?

    In addition, what about PEM/PENE? We get these scores when we pace. When we overdo things I bet our scores crash. That might not be true of misdiagnosed patients in the study, but it would be true for any subset of ME patients that got through.

    The modified London ME criteria has been discussed before. One of the authors of the original criteria did not like the changes.
     
    Valentijn likes this.
  2. biophile

    biophile Places I'd rather be.

    Messages:
    1,411
    Likes:
    4,950
    alex3619. According to my statistical calculator, on papers referenced for normative data in the original 2007 protocol of PACE, and without commenting on the non-normal distribution of scores:

    If the mean(SD) of the general population from Bowling et al is 84(24), 60 is the 15.9th percentile.

    If the mean(SD) of the working age without reported illness from Jenkinson et al is 92.5(13.4), 60 is the 0.8th percentile.

    Wait, WTF? That last figure sounds too absurd. So the threshold for a "recovery" in physical function in PACE is scoring less than about 99% of the working age population without self-reported illness? So much for providing a "comprehensive and conservative definition of recovery".

    [edit 1: 70 points, assuming that is included in no longer meeting Oxford criteria, is the 4.7th percentile in Jenkinson et al]

    [edit 2: when using a mean(SD) of 96.8(4.5) from healthy sedentary controls from VanNess et al (2010), both 60 and 70 gives error messages for being too low, being 6 and 8 standard deviations from the mean]
     
    Bob, DaiWelsh, Simon and 6 others like this.
  3. Mark

    Mark Acting CEO

    Messages:
    4,532
    Likes:
    2,016
    Sofa, UK
    Sounds like a pretty good start towards the kind of referenced, indisputable soundbite we're needing here....
     
    peggy-sue, Valentijn and Purple like this.
  4. Dolphin

    Dolphin Senior Member

    Messages:
    6,872
    Likes:
    6,167
    Initially when I read the paper first, I thought the cut-off for recovery was >=60 on SF-36 PF, although I wasn't sure.
    However, on closer reading, I think patients can't fit the entry criteria for Oxford which would seem to suggest they need to score more than 65 (generally 70, unless somebody didn't fill in all the questions and they averaged the scores which ended up with a score between 65 and 70).
    I think the criterion, >65 (~70) on SF-36 PF, is also required for recovery from ME or CFS (CDC).

    Similarly, I think it requires a score of <6 to satisfy recovery from Oxford criteria.
    And similarly I think it requires a score of <6 to satisfy recovery from ME or CFS (CDC).

    The one thing that makes me doubt this is how small the drop is for the Oxford criteria. I would have thought going from >=60 on SF-36 PF and <=18 on Chalder Fatigue "Likert"
    to
    >65 (~70) on SF-36 PF and <6 on Chalder Fatigue "bimodal"
    might have "knocked off" more.
    I believe I have worked out what happens in this post: http://forums.phoenixrising.me/inde...en-in-the-pace-trial.21628/page-9#post-332130
     
    Simon and biophile like this.
  5. alex3619

    alex3619 Senior Member

    Messages:
    7,725
    Likes:
    12,655
    Logan, Queensland, Australia
    If one percent is accurate for working age people at SF-36 or below (we may need to look into this further), and the disability rate was 7.3% for working age (1991 census data http://unstats.un.org/unsd/demographic/sconcerns/disability/disform.asp?studyid=156) though this might be very different from current rates, then the PACE criteria rewrites disability such that 86% of the disabled population is fully recovered? Forget DWP and ATOS, the UK government should have hired White et. al.. Of course I have not taken into account the percentage of mental disability, but with a little research this might be an interesting argument.

    There is more accurate data here: http://www.papworth.org.uk/downloads/factsandfigures_disabilityintheuk_july2011_110721132605.pdf

    Disability is about 20% of working age adults as of 2011. This makes the PACE claim even more outragious.

    Official data from DWP here: http://odi.dwp.gov.uk/docs/res/factsheets/disability-prevalence.pdf

    Their figure of 5.2 million roughly translates to 8%, though I wonder if this reflects unworking disabled rather than just those with a disability.

    From this http://archpsyc.jamanetwork.com/article.aspx?articleid=210038 it appears that physical disability is more common than mental, though not by much. So the percentage of the UK population who were physically disabled, and now can be said to be recovered, is about 3% of the working age population, give or take and presuming the data is accurate.

    I am very tired today, it would be nice if others with better math ability could look at this.
     
    Valentijn and biophile like this.
  6. biophile

    biophile Places I'd rather be.

    Messages:
    1,411
    Likes:
    4,950
    Initial assessment, may be revised later

    This new paper on "recovery" (or "remission" as they are calling it too) is an unexpected and somewhat surprising repeat of the 2011 "normal range" stunt, with the addition of optional criteria bolted on (e.g. not meeting Oxford criteria, and/or feeling "much better"). I was expecting the physical function threshold to be higher than 60 points. Table 1a and 1b describes "normal range" as recovery, even though the authors had to clarify in 2011 after much criticism that it was not so! So no wonder the Lancet stubbornly refused to correct the accompanying editorial which erroneously claimed so.

    White et al on one hand want to claim to be deriving their score thresholds from healthy norms, then on the other hand want to also claim that their use of mean minus SD is conservative. They cannot have it both ways and appear to be misrepresenting the results. Subtracting 1 SD from a healthy average score gives at least 80 or even 90, that is why the threshold in the original definition of recovery was 85 points or more. Similar problem with their fatigue norms from mean plus 1SD (18 points in Likert scoring can be anywhere between 4-9 points in bimodal score, while their original criterion here was 3 points or less for recovery, now it is 4-9 on a scale where 11 is the maximum).

    When discussing how their PACE recovery rates compare with other similar studies, White et al (erroneously) claim that a previous paper by Knoop et al (2007), which White even co-authored, used a less conservative rule (mean +/- 2SD) to calculate their thresholds, implying that the PACE definition for recovery was stricter than theirs. However, that earlier paper actually used the same "conservative" mean+/-1SD as PACE did, but using healthier norms data it arrived at 80 points or more as the threshold, which is ironically stricter than PACE. White cannot keep track of his own previous work before making misleading claims in medical journals? As two studies were discussed, I considered the possibility that it was a citation error, but the 2nd study (Deale et al, 2001) has an even higher threshold of 83 points.

    When comparing CBT and GET to SMC, the NNT in PACE is usually about the same no matter how improvement or recovery is defined, which is odd because the NNT should generally be increasing as the definitions become stricter. As the "clinically useful difference" is only 2 points in fatigue or 8 points in physical function, this is a red flag that all their definitions for improvement and thresholds for recovery are also minimal. If it takes about 7 patients for one to report at least a small improvement, how can one patient out of that 7 also report a full recovery? Basically it suggests that participants close to or even at the rather low thresholds for "normal" fatigue and physical function at trial entry (despite meeting criteria for "disabling fatigue") are making minor improvements crossing the thin border into "recovery" land a little more often than the SMC group. However, the set of patients required for one response eg fatigue, is not necessarily the same set required for another response eg physical function.

    As others have already said, what about employment, welfare, and walking? These measurements may not be perfect on their own, but they are not less important than elderly-like scores in physical function. It would be strange if the "recovered" participants were unemployed, on welfare, and unable to walk properly over 6 minutes. Welcome to the post-modern definition of recovery after CBT/GET. Are we still supposed to believe in confidence that all these post-hoc changes to the protocol were definitely all made before seeing the disappointing trial data, and that the refusal to release the recovery data according to the original definition is against the "public interest"?

    The peer-review process is sloppy when participants can enter a trial with "disabling fatigue", make no improvement or even decline slightly, then be classified as "normal" and "recovered". Imagine the gnashing of teeth that would occur if this was a famous publicly funded non-blinded drug study without placebo control conducted by developers of the drug and allowing the favoured drug groups to receive more encouragement and optimism?
     
    Bob, SOC, wdb and 3 others like this.
  7. alex3619

    alex3619 Senior Member

    Messages:
    7,725
    Likes:
    12,655
    Logan, Queensland, Australia
    biophile

    [satirical in part]

    Why would anyone object to a drug trial in which a number of patient were recruited at the start who could be redefined as recovered without any improvement or even some worsening from the drug, and then this was used to promote drug efficacy?

    This is of course compounded by not releasing data showing how many fit this category, or even acknowledging that it must exist, by definition.

    Lets imagine this is a fast food restaurant survey. They stack the deck by deliberately including in their survey an undisclosed percentage of people who just love their food, and then tout the results as saying 22% of the of the people who ate there love their food. Why would we object to that?

    What would the FDA say about this if it were a drug? Would it be classed as deliberate scientific fraud? Would the FDA permit highly disabled people to be reclassified as recovered on the basis of a change in definitions?
     
    Valentijn likes this.
  8. Dolphin

    Dolphin Senior Member

    Messages:
    6,872
    Likes:
    6,167
    I'm not sure whether you're factoring in SMC responses. NNT in the study is relative to SMC rather than relative to baseline scores/imagining a group that didn't change.
     
  9. Dolphin

    Dolphin Senior Member

    Messages:
    6,872
    Likes:
    6,167
    Unfortunately changing from the bimodal to the Likert scale is complicated. One can't say 3/11 = 9/33. 11/33 is the neutral score.
     
  10. Dolphin

    Dolphin Senior Member

    Messages:
    6,872
    Likes:
    6,167
    Just to be exact: it is the 6-minute walking test (how far one can walk in six minutes).

    Sorry if I look like I'm picking on you but I like to educate people in these threads so we can all act. I thought you made some other interesting observations
     
    Valentijn likes this.
  11. Dolphin

    Dolphin Senior Member

    Messages:
    6,872
    Likes:
    6,167
    I think talking about the PCS and MCS (physical composite score and mental composite score) will likely confuse people. They use all the eight SF-36 questionnaires - most weren't used in the PACE Trial.

    Also, if you use the non-normative physical functioning scale, your score should be multiples of 5. You say your non-normative PF was 51 - that looks more like a normed score.

    By the way, the entry criteria of 65 isn't a composite score. It is simply based on the 10 questions that make up the SF-36 physical functioning scale.
     
    alex3619 likes this.
  12. biophile

    biophile Places I'd rather be.

    Messages:
    1,411
    Likes:
    4,950
    Of course. I just made an edit so it is clearer.
     
    Dolphin likes this.
  13. biophile

    biophile Places I'd rather be.

    Messages:
    1,411
    Likes:
    4,950
    I think Dolphin may be correct. To no longer meet Oxford criteria, the paper says:

    Although it is not entirely clear enough, this would mean that in order for patients to no longer meet Oxford criteria, patients needed to score 70 or more in physical function, and 5 or less in fatigue (bimodal scoring), both of course which are still less than the original definition of 85 or more and 3 or less respectively.

    However, I share his suspicions that adding this criteria to "normal range" barely changes the figures at all.
     
  14. Dolphin

    Dolphin Senior Member

    Messages:
    6,872
    Likes:
    6,167
    This is a link to the 12MB file of the full PACE Trial Protocol, obtained using a FOI request as I understand it. It includes the exact questionnaires used e.g. for CFS and ME criteria, physical functioning, fatigue, etc. - having such info makes it much easier to understand such studies.

    People should be able to download it for free.

    http://www.mediafire.com/view/?92x9s920bsftyx0

    Alternative download link: http://www.yousendit.com/download/UW16K0drdGo4Q1JsQXNUQw (this lasts for a maximum of 100 downloads and 7 days)
     
    Roy S likes this.
  15. biophile

    biophile Places I'd rather be.

    Messages:
    1,411
    Likes:
    4,950
    In other words, they did not use CDC criteria.
     
    SOC, Valentijn, WillowJ and 1 other person like this.
  16. biophile

    biophile Places I'd rather be.

    Messages:
    1,411
    Likes:
    4,950
    It is difficult to know for sure because the data is not normally distributed. However, assuming a normal distribution was actually being very generous, as the distribution of scores in the histogram strongly suggests otherwise. If anything, the percentiles are even lower than the ones calculated!

    The authors do not care about distribution of scores, so why should I? ;-)

    Here are two more since the original post:

    70 points, assuming that is included in no longer meeting Oxford criteria, is the 4.7th percentile, for working age population without reported illness.

    When using a mean(SD) of 96.8(4.5) from healthy sedentary controls matched to CFS patients in VanNess et al (2010), both 60 and 70 gives error messages for being too low, being 6 and 8 standard deviations from the mean.

    PS: I cannot wait for the predictors/mediators paper, where correlations of r=0.2 will be promoted as convincing evidence for major cognitive and behavioural factors in CFS.
     
  17. WillowJ

    WillowJ Senior Member

    Messages:
    3,149
    Likes:
    2,851
    WA, USA
    That's nice that you have a great neighbor to chat about this with.

    Another way to look at your score is that you scored less than 65, so you are 'severely fatigued' and qualify to enter the study to try treatments.
     
    Svenja and biophile like this.
  18. Kati

    Kati Patient in training

    Messages:
    2,126
    Likes:
    1,792
    What kills me is they had to publish and spread a press release all over the world just hours before the FDA decision on Ampligen is due.

    Every time happenings in our community happens- it has to be counteracted and forced into newspapers, doctors who don't know any better, by the UK psychiatrists who are defending CBT and GET like they had a big secret that could not be revealed, ever.
     
  19. user9876

    user9876 Senior Member

    Messages:
    796
    Likes:
    1,962
    I was wondering that however the oxford criteria are a conjunction of 4 things so if the research assessor decides that fatigue is no longer the main symptom or that fatigue is not present 50% of the time then a patient would not meet the Oxford criteria even with much lower scores and hence could be deamed recovered.
     
    Simon likes this.
  20. Valentijn

    Valentijn Activity Level: 3

    Messages:
    6,716
    Likes:
    10,234
    Amersfoort, Netherlands
    The trial was randomized, but not controlled or blinded. Blinding wouldn't work because patients can tell which treatment they're getting. And without blinding patients, I don't see why they'd bother blinding the researchers or assistants involved in treatment. I'm not sure if SMC was provided by unaffiliated local clinics or the trial researchers, but the unblinded researchers are still the ones determining who is meeting the various CFS criteria.
     

See more popular forum discussions.

Share This Page