'Recovery' from chronic fatigue syndrome after treatments given in the PACE trial

[alex3619]

Does anyone know what percentage of the population scores 60 or worse at SF-36? Then what percentage of working age adults score 60 or less?

In addition, what about PEM/PENE? We get these scores when we pace. When we overdo things I bet our scores crash. That might not be true of misdiagnosed patients in the study, but it would be true for any subset of ME patients that got through.

The modified London ME criteria has been discussed before. One of the authors of the original criteria did not like the changes.

 

[biophile]

alex3619. According to my statistical calculator, on papers referenced for normative data in the original 2007 protocol of PACE, and without commenting on the non-normal distribution of scores:

If the mean(SD) of the general population from Bowling et al is 84(24), 60 is the 15.9th percentile.

If the mean(SD) of the working age without reported illness from Jenkinson et al is 92.5(13.4), 60 is the 0.8th percentile.

Wait, WTF? That last figure sounds too absurd. So the threshold for a "recovery" in physical function in PACE is scoring less than about 99% of the working age population without self-reported illness? So much for providing a "comprehensive and conservative definition of recovery".

[edit 1: 70 points, assuming that is included in no longer meeting Oxford criteria, is the 4.7th percentile in Jenkinson et al]

[edit 2: when using a mean(SD) of 96.8(4.5) from healthy sedentary controls from VanNess et al (2010), both 60 and 70 gives error messages for being too low, being 6 and 8 standard deviations from the mean]

 

[Mark]

alex3619So the threshold for a "recovery" in physical function in PACE is scoring less than about 99% of the healthy working age population?

Sounds like a pretty good start towards the kind of referenced, indisputable soundbite we're needing here....

 

[Dolphin]

As I see it, trial recovery criteria
1) Chadler fatigue score below 18
and
2) SF-pf score above 60
and
not meets oxford criteria as judged by research assessor at 52 weeks. That is
not ( fatigue main symptom and
fatigue present > 50% of time and for more than 6 months and
SF-pf >= 65 and fatigue score>6 out of 11 (binary))
and a clinical global impression score of 1 or 2

The last bit of the oxford one in interesting. They basically say to meet the oxford criteria you need to meet the revised entry conditions + a couple of other things. That's an interesting one considering their other thresholds but the research assesor can just say that fatigue is not the main symptom.

I've not finished the paper but I haven't seen it say that the research assessors were blinded in their methods section. I would take the lack of any statement that they were to mean they weren't.

Initially when I read the paper first, I thought the cut-off for recovery was >=60 on SF-36 PF, although I wasn't sure.
However, on closer reading, I think patients can't fit the entry criteria for Oxford which would seem to suggest they need to score more than 65 (generally 70, unless somebody didn't fill in all the questions and they averaged the scores which ended up with a score between 65 and 70).
I think the criterion, >65 (~70) on SF-36 PF, is also required for recovery from ME or CFS (CDC).

Similarly, I think it requires a score of <6 to satisfy recovery from Oxford criteria.
And similarly I think it requires a score of <6 to satisfy recovery from ME or CFS (CDC).

The one thing that makes me doubt this is how small the drop is for the Oxford criteria. I would have thought going from >=60 on SF-36 PF and <=18 on Chalder Fatigue "Likert"
to
>65 (~70) on SF-36 PF and <6 on Chalder Fatigue "bimodal"
might have "knocked off" more.
I believe I have worked out what happens in this post: http://forums.phoenixrising.me/inde...en-in-the-pace-trial.21628/page-9#post-332130

 

[alex3619]

If one percent is accurate for working age people at SF-36 or below (we may need to look into this further), and the disability rate was 7.3% for working age (1991 census data http://unstats.un.org/unsd/demographic/sconcerns/disability/disform.asp?studyid=156) though this might be very different from current rates, then the PACE criteria rewrites disability such that 86% of the disabled population is fully recovered? Forget DWP and ATOS, the UK government should have hired White et. al.. Of course I have not taken into account the percentage of mental disability, but with a little research this might be an interesting argument.

There is more accurate data here: http://www.papworth.org.uk/downloads/factsandfigures_disabilityintheuk_july2011_110721132605.pdf

Disability is about 20% of working age adults as of 2011. This makes the PACE claim even more outragious.

Official data from DWP here: http://odi.dwp.gov.uk/docs/res/factsheets/disability-prevalence.pdf

Their figure of 5.2 million roughly translates to 8%, though I wonder if this reflects unworking disabled rather than just those with a disability.

From this http://archpsyc.jamanetwork.com/article.aspx?articleid=210038 it appears that physical disability is more common than mental, though not by much. So the percentage of the UK population who were physically disabled, and now can be said to be recovered, is about 3% of the working age population, give or take and presuming the data is accurate.

I am very tired today, it would be nice if others with better math ability could look at this.

 

[biophile]

Initial assessment, may be revised later

This new paper on "recovery" (or "remission" as they are calling it too) is an unexpected and somewhat surprising repeat of the 2011 "normal range" stunt, with the addition of optional criteria bolted on (e.g. not meeting Oxford criteria, and/or feeling "much better"). I was expecting the physical function threshold to be higher than 60 points. Table 1a and 1b describes "normal range" as recovery, even though the authors had to clarify in 2011 after much criticism that it was not so! So no wonder the Lancet stubbornly refused to correct the accompanying editorial which erroneously claimed so.

White et al on one hand want to claim to be deriving their score thresholds from healthy norms, then on the other hand want to also claim that their use of mean minus SD is conservative. They cannot have it both ways and appear to be misrepresenting the results. Subtracting 1 SD from a healthy average score gives at least 80 or even 90, that is why the threshold in the original definition of recovery was 85 points or more. Similar problem with their fatigue norms from mean plus 1SD (18 points in Likert scoring can be anywhere between 4-9 points in bimodal score, while their original criterion here was 3 points or less for recovery, now it is 4-9 on a scale where 11 is the maximum).

When discussing how their PACE recovery rates compare with other similar studies, White et al (erroneously) claim that a previous paper by Knoop et al (2007), which White even co-authored, used a less conservative rule (mean +/- 2SD) to calculate their thresholds, implying that the PACE definition for recovery was stricter than theirs. However, that earlier paper actually used the same "conservative" mean+/-1SD as PACE did, but using healthier norms data it arrived at 80 points or more as the threshold, which is ironically stricter than PACE. White cannot keep track of his own previous work before making misleading claims in medical journals? As two studies were discussed, I considered the possibility that it was a citation error, but the 2nd study (Deale et al, 2001) has an even higher threshold of 83 points.

When comparing CBT and GET to SMC, the NNT in PACE is usually about the same no matter how improvement or recovery is defined, which is odd because the NNT should generally be increasing as the definitions become stricter. As the "clinically useful difference" is only 2 points in fatigue or 8 points in physical function, this is a red flag that all their definitions for improvement and thresholds for recovery are also minimal. If it takes about 7 patients for one to report at least a small improvement, how can one patient out of that 7 also report a full recovery? Basically it suggests that participants close to or even at the rather low thresholds for "normal" fatigue and physical function at trial entry (despite meeting criteria for "disabling fatigue") are making minor improvements crossing the thin border into "recovery" land a little more often than the SMC group. However, the set of patients required for one response eg fatigue, is not necessarily the same set required for another response eg physical function.

As others have already said, what about employment, welfare, and walking? These measurements may not be perfect on their own, but they are not less important than elderly-like scores in physical function. It would be strange if the "recovered" participants were unemployed, on welfare, and unable to walk properly over 6 minutes. Welcome to the post-modern definition of recovery after CBT/GET. Are we still supposed to believe in confidence that all these post-hoc changes to the protocol were definitely all made before seeing the disappointing trial data, and that the refusal to release the recovery data according to the original definition is against the "public interest"?

The peer-review process is sloppy when participants can enter a trial with "disabling fatigue", make no improvement or even decline slightly, then be classified as "normal" and "recovered". Imagine the gnashing of teeth that would occur if this was a famous publicly funded non-blinded drug study without placebo control conducted by developers of the drug and allowing the favoured drug groups to receive more encouragement and optimism?

 

[alex3619]

biophile

[satirical in part]

Why would anyone object to a drug trial in which a number of patient were recruited at the start who could be redefined as recovered without any improvement or even some worsening from the drug, and then this was used to promote drug efficacy?

This is of course compounded by not releasing data showing how many fit this category, or even acknowledging that it must exist, by definition.

Lets imagine this is a fast food restaurant survey. They stack the deck by deliberately including in their survey an undisclosed percentage of people who just love their food, and then tout the results as saying 22% of the of the people who ate there love their food. Why would we object to that?

What would the FDA say about this if it were a drug? Would it be classed as deliberate scientific fraud? Would the FDA permit highly disabled people to be reclassified as recovered on the basis of a change in definitions?

 

[Dolphin]

The NNT in PACE is usually about the same for CBT and GET no matter how improvement or recovery is defined, which is odd because the NNT should generally be increasing as the definitions become stricter. As the "clinically useful difference" is only 2 points in fatigue or 8 points in physical function, this is a red flag that all their definitions for improvement and thresholds for recovery are also minimal. If it takes about 7 patients for one to report at least a small improvement, how can one patient out of that 7 also report a full recovery? Basically it suggests that participants close to or even at the rather low thresholds for "normal" fatigue and physical function at trial entry (despite meeting criteria for "disabling fatigue") are making minor improvements crossing the thin border into "recovery" land a little more often than the SMC group.

I'm not sure whether you're factoring in SMC responses. NNT in the study is relative to SMC rather than relative to baseline scores/imagining a group that didn't change.

 

[Dolphin]

So they started with 3 out of 11 or lower as "normal", then shifted to 18 out of 33. If looking at the math (3/11 = 9/33), they effectively doubled the normal range.

Unfortunately changing from the bimodal to the Likert scale is complicated. One can't say 3/11 = 9/33. 11/33 is the neutral score.

 

[Dolphin]

That's a good point. This paper doesn't seem to mention the 600 meter walking test at all. Why make that data disappear after the main PACE paper made such a big deal about it?

Just to be exact: it is the 6-minute walking test (how far one can walk in six minutes).

Sorry if I look like I'm picking on you but I like to educate people in these threads so we can all act. I thought you made some other interesting observations

 

[Dolphin]

To see what all the fuss is about, I ran an SF-36 online for me. I tried to post the results. It wont go as a graph, so here is my final data summation:

PCS (Physical)18.7

MCS (Mental) 47.2

http://www.sf-36.org/cgi-bin/demos/report.cgi

Both these results are NORMATIVE and I think the norm is 50.

On non-normative my physical PF is 30! I am only a MODERATE patient too, and I have not been particularly bad lately, aside from a brief crash the other day. (Under the ICC I would class as severe though, as I am mostly homebound though not bedbound.)

So what does this say about an entry criteria of 65? (Though presumably this is a composite score.) I wonder how low a severe patient would score? I am going to run a hypothetical severe patient, based on my limited knowledge, just to get an idea.

OK, as a general idea I get normative PCS of 10.9 and MCS of 43.9. Of course I have no idea how accurate this is. Non-normative physical functioning was 15.2.

Given the vast difference between moderate and severe patients, and severe and very severe, I have to ask how useful is SF-36 for ME? Ok, lets compare with how I was in the late 80s when I was first diagnosed, though my memory of that time is fubar so again I can't say this is really accurate.

My normative PCS was 26.6. Yet I was a LOT better physically than I am now. My MCS was 48.1. My non-normative PF was 51!

This was when I had a LOT more energy and was able to attend university full time (then come home and collapse). Something is really wrong here, and I have no way to be sure if the test I did is a typical SF-36.

I think talking about the PCS and MCS (physical composite score and mental composite score) will likely confuse people. They use all the eight SF-36 questionnaires - most weren't used in the PACE Trial.

Also, if you use the non-normative physical functioning scale, your score should be multiples of 5. You say your non-normative PF was 51 - that looks more like a normed score.

By the way, the entry criteria of 65 isn't a composite score. It is simply based on the 10 questions that make up the SF-36 physical functioning scale.

 

[biophile]

I'm not sure whether you're factoring in SMC responses. NNT in the study is relative to SMC rather than relative to baseline scores/imagining a group that didn't change.

Of course. I just made an edit so it is clearer.

 

[biophile]

I think Dolphin may be correct. To no longer meet Oxford criteria, the paper says:

To satisfy the third criterion for severity of fatigue and disability, participants had to meet trial entry thresholds for fatigue (a binary score of ≥6 out of 11 on the CFQ) and abnormal levels of physical function (a score of ≤65 out of 100 on the SF-36 physical function subscale) (White et al. 2007).

Although it is not entirely clear enough, this would mean that in order for patients to no longer meet Oxford criteria, patients needed to score 70 or more in physical function, and 5 or less in fatigue (bimodal scoring), both of course which are still less than the original definition of 85 or more and 3 or less respectively.

However, I share his suspicions that adding this criteria to "normal range" barely changes the figures at all.

 

[Dolphin]

This is a link to the 12MB file of the full PACE Trial Protocol, obtained using a FOI request as I understand it. It includes the exact questionnaires used e.g. for CFS and ME criteria, physical functioning, fatigue, etc. - having such info makes it much easier to understand such studies.

People should be able to download it for free.

http://www.mediafire.com/view/?92x9s920bsftyx0

Alternative download link: http://www.yousendit.com/download/UW16K0drdGo4Q1JsQXNUQw (this lasts for a maximum of 100 downloads and 7 days)

 

[biophile]

For the purposes of this study, the four or more symptoms needed to be present within the previous week of the assessment date, rather than the previous 6 months (Reeves et al. 2003).

In other words, they did not use CDC criteria.

 

[biophile]

Sounds like a pretty good start towards the kind of referenced, indisputable soundbite we're needing here....

It is difficult to know for sure because the data is not normally distributed. However, assuming a normal distribution was actually being very generous, as the distribution of scores in the histogram strongly suggests otherwise. If anything, the percentiles are even lower than the ones calculated!

The authors do not care about distribution of scores, so why should I? ;-)

Here are two more since the original post:

70 points, assuming that is included in no longer meeting Oxford criteria, is the 4.7th percentile, for working age population without reported illness.

When using a mean(SD) of 96.8(4.5) from healthy sedentary controls matched to CFS patients in VanNess et al (2010), both 60 and 70 gives error messages for being too low, being 6 and 8 standard deviations from the mean.

PS: I cannot wait for the predictors/mediators paper, where correlations of r=0.2 will be promoted as convincing evidence for major cognitive and behavioural factors in CFS.

 

[WillowJ]

Ps just did the SF-36 Physical Function test with my frail but healthy 88 year old neighbour (she's charming and fun.) We scored the same (60points). I guess that proves I'm already fully recovered: I meet the PACE recovery threshold, and I function at the same level as a healthy person. QED

That's nice that you have a great neighbor to chat about this with.

Another way to look at your score is that you scored less than 65, so you are 'severely fatigued' and qualify to enter the study to try treatments.

 

[Kati]

What kills me is they had to publish and spread a press release all over the world just hours before the FDA decision on Ampligen is due.

Every time happenings in our community happens- it has to be counteracted and forced into newspapers, doctors who don't know any better, by the UK psychiatrists who are defending CBT and GET like they had a big secret that could not be revealed, ever.

 

[user9876]

Initially when I read the paper first, I thought the cut-off for recovery was >=60 on SF-36 PF, although I wasn't sure.
However, on closer reading, I think patients can't fit the entry criteria for Oxford which would seem to suggest they need to score more than 65 (generally 70, unless somebody didn't fill in all the questions and they averaged the scores which ended up with a score between 65 and 70).
I think the criterion, >65 (~70) on SF-36 PF, is also required for recovery from ME or CFS (CDC).

Similarly, I think it requires a score of <6 to satisfy recovery from Oxford criteria.
And similarly I think it requires a score of <6 to satisfy recovery from ME or CFS (CDC).

The one thing that makes me doubt this is how small the drop is for the Oxford criteria. I would have thought going from >=60 on SF-36 PF and <=18 on Chalder Fatigue "Likert"
to
>65 (~70) on SF-36 PF and <6 on Chalder Fatigue "bimodal"
might have "knocked off" more.

I was wondering that however the oxford criteria are a conjunction of 4 things so if the research assessor decides that fatigue is no longer the main symptom or that fatigue is not present 50% of the time then a patient would not meet the Oxford criteria even with much lower scores and hence could be deamed recovered.

 

[Valentijn]

So if the patient didn't self-rate, the doctor's rating of that patient was used. And the doctors rated all patients, yet that rating doesn't seem to be included unless the patient didn't self-rate. This seems like a great opportunity for the researchers to take a look at the self-rating versus the doctor rating and cherry pick the best set of results to use.

If they were blinded to which treatment arm the patient was in, this would be okay. If they weren't blinded, then that's pretty ridiculous.

The trial was randomized, but not controlled or blinded. Blinding wouldn't work because patients can tell which treatment they're getting. And without blinding patients, I don't see why they'd bother blinding the researchers or assistants involved in treatment. I'm not sure if SMC was provided by unaffiliated local clinics or the trial researchers, but the unblinded researchers are still the ones determining who is meeting the various CFS criteria.