The P2P Draft Systematic Review Is Up

[Dolphin]

None of the CFS definitions exclude patients who have co-morbid fibromyalgia to the best of my knowledge.

You are correct; they don't. My point, and Professor Hooper's, was simply that people with fibromyalgia could be admitted to the study without meeting the necesssary fatigue criteria associated with the Oxford definition.

I don't what is the evidence for this.

 

[zzz]

As I previously mentioned, it starts on page 57 of Professor Hooper's document. Here are some specific quotes from that section:

To the consternation of medical scientists and contrary to accepted scientific practice, the Wessely School decided to include fibromyalgia patients in the MRC PACE Trial, which means that the PACE Trial includes at least three distinct disorders ‐‐ ME/CFS (ICD‐10 G93.3); fibromyalgia (ICD‐10 M79.0) and psychiatric fatigue (ICD‐10 F48.0).

At the International Science Festival held on 9th April 2004 in Edinburgh, Michael Sharpe spoke in a debate entitled “Science and ME” and was specifically asked if patients with fibromyalgia (FM) were to be included in the PACE Trial of “CFS/ME”. Sharpe replied in the affirmative, implying that patients with FM needed to be included in order to reach the recruitment target. He said (verbatim): “We want broadness and heterogeneity in the trial”
....
That FM patients were to be included in the PACE Trial was further confirmed on 12th May 2004 by Parliamentary Under Secretary of State at the Department of Health, Dr Stephen Ladyman, at an All Party Parliamentary Group on Fibromyalgia, who announced that doctors were being offered financial inducements to persuade patients with FM to attend a “CFS” Clinic to aid recruitment to the PACE Trial (EIF: Spring/Summer 2004, page 19).
...
FM has a distinct biological profile that is different from ME/CFS, so it is unclear how the intentional inclusion of different disorders in an MRC trial evaded detection by the allegedly rigorous monitoring process. The MRC was asked how the deliberate inclusion of different disorders could not result in skewed and meaningless conclusions when, from the outset, patients being entered into the PACE trial were not clearly defined, a question that elicited no response.

Apparently neither the MRC nor the West Midlands MREC is concerned with diagnostic accuracy.

 

[Dolphin]

As I previously mentioned, it starts on page 57 of Professor Hooper's document. Here are some specific quotes from that section:

To the consternation of medical scientists and contrary to accepted scientific practice, the Wessely School decided to include fibromyalgia patients in the MRC PACE Trial, which means that the PACE Trial includes at least three distinct disorders ‐‐ ME/CFS (ICD‐10 G93.3); fibromyalgia (ICD‐10 M79.0) and psychiatric fatigue (ICD‐10 F48.0).

At the International Science Festival held on 9th April 2004 in Edinburgh, Michael Sharpe spoke in a debate entitled “Science and ME” and was specifically asked if patients with fibromyalgia (FM) were to be included in the PACE Trial of “CFS/ME”. Sharpe replied in the affirmative, implying that patients with FM needed to be included in order to reach the recruitment target. He said (verbatim): “We want broadness and heterogeneity in the trial”
....
That FM patients were to be included in the PACE Trial was further confirmed on 12th May 2004 by Parliamentary Under Secretary of State at the Department of Health, Dr Stephen Ladyman, at an All Party Parliamentary Group on Fibromyalgia, who announced that doctors were being offered financial inducements to persuade patients with FM to attend a “CFS” Clinic to aid recruitment to the PACE Trial (EIF: Spring/Summer 2004, page 19).
...
FM has a distinct biological profile that is different from ME/CFS, so it is unclear how the intentional inclusion of different disorders in an MRC trial evaded detection by the allegedly rigorous monitoring process. The MRC was asked how the deliberate inclusion of different disorders could not result in skewed and meaningless conclusions when, from the outset, patients being entered into the PACE trial were not clearly defined, a question that elicited no response.

Apparently neither the MRC nor the West Midlands MREC is concerned with diagnostic accuracy.

That doesn't convince me of this:

My point, and Professor Hooper's, was simply that people with fibromyalgia could be admitted to the study without meeting the necesssary fatigue criteria associated with the Oxford definition.

Fibromyalgia can be co-morbid with CFS. So I interpreted that (and still interpret that) to mean patients with fibromyalgia could be invited, but that doesn't mean they would get in without satisfying the Oxford criteria.

 

[user9876]

In other words, people with fibromyalgia were included whether or not they met the fatigue criteria.


No, we don't; that was never stated. I see no reason to believe that they all did.


You are correct; they don't. My point, and Professor Hooper's, was simply that people with fibromyalgia could be admitted to the study without meeting the necesssary fatigue criteria associated with the Oxford definition.


That's true. In that case, I don't know of a single document that encompasses everything. However, I believe that Professor Hooper's document is a severe and detailed indictment of the trial itself, and I know of no other document that is as detailed in that respect.

Given we don't understand the processes behind ME its easy to argue that a broad group of people should be included and the Oxford definition does define a very broad group. I don't see the real problem with that as long as it is clear who was included. I don't think they had an exception for people with fibromyalgia to allow them to be included without fatigue or even without fatigue being the main symptom. You may quote they come under different diagnostic labels but without an understanding of mechanism these are speculative and many doctors will know this and have questions (both reasonable and unreasonable) about the categorization. I tend to think there may be two distinct diseases of fibromyalgia and ME but there is also quite a lot of overlap and it is easy for a patient with ME to get a diagnosis of fibromyalgia particular if pain is a major component. Its also unlikely that ME exists as such but rather there are likely to be a number of different diseases leading to similar symptoms (for example see the thread http://forums.phoenixrising.me/index.php?threads/do-mes-cause-cfs.31930/ ).

There do seem to be issues with the Oxford criteria as they used them in that they seemed to do an initial filter using those criteria and then a subsequent one where they removed more people. Also in their recovery paper there were quite a few subjects who didn't meet their very low SF36 and CFQ recovery thresholds but still didn't meet the Oxford criteria. The Oxford criteria were changed at this point to include thresholds for the sf36 and CFQ scales to judge fatigue (I think).


The PACE trial basically took a set of people who met the oxford criteria. Divided them into 4 groups. 2 groups were told we can cure you if you ignore your symptoms and 2 groups were told if you do this there may be some natural recovery. The results showed a slight change in the way the 2nd two groups reported their symptoms and what they felt they could do and a slightly bigger change in the 1st group. In the more objective measures there was no real differences. So basically they showed that by telling sick people they are not ill they can influence how they fill out a questionnaire a small amount.

Given the results were so poor I'm not sure that the debate on who was included is valuable since we don't really know who should be included as we don't know what the mechanisms behind ME are.

The way the spun the results in the Lancet paper was bad and I think their statistics are highly flawed.

The way they spun results in the recovery paper suggests to me that they know their trial failed to produce the results but are suffering from cognitive dissonance and will do anything to avoid admitting its failure. They lowered the thresholds they used to define recovery to such a low level that it should set red flags flying for any reader (although apparently not the peer reviewers or editors). Even lowering the recovery thresholds to at or below the trial entry conditions their results were poor.

Given the lengths they are going to to avoid releasing any data that might contradict their spin including many of the measures in the original protocol I think its easy to conclude the trial didn't really lead to significant improvements. I think we need to declare the writeup of PACE as untrustworthy due to their failures to follow protocol; through bad methodology in setting different expectations and their ridiculous spinning of results.

The arguments about patient selection I think are weaker until we have better understanding of mechanisms. However, the way they have spun data suggests they had a complete failure to help anyone whatever the diagnostic criteria.

We could attack them on some very poor statistics but the errors are pervasive in some of the medical literature so we end up with subtle arguments that challenge normal practice. So we need to do some of that but I'm not sure how much.

 

[Dolphin]

Also in their recovery paper there were quite a few subjects who didn't meet their very low SF36 and CFQ recovery thresholds but still didn't meet the Oxford criteria. The Oxford criteria were changed at this point to include thresholds for the sf36 and CFQ scales to judge fatigue (I think).

No, I don't believe the Oxford criteria were changed at this point (i.e. when discussing recovery).

 

[Dolphin]

The PACE trial basically took a set of people who met the oxford criteria. Divided them into 4 groups. 2 groups were told we can cure you if you ignore your symptoms and 2 groups were told if you do this there may be some natural recovery. The results showed a slight change in the way the 2nd two groups reported their symptoms and what they felt they could do and a slightly bigger change in the 1st group. In the more objective measures there was no real differences. So basically they showed that by telling sick people they are not ill they can influence how they fill out a questionnaire a small amount.

Given the results were so poor I'm not sure that the debate on who was included is valuable since we don't really know who should be included as we don't know what the mechanisms behind ME are.

The way the spun the results in the Lancet paper was bad and I think their statistics are highly flawed.

The way they spun results in the recovery paper suggests to me that they know their trial failed to produce the results but are suffering from cognitive dissonance and will do anything to avoid admitting its failure. They lowered the thresholds they used to define recovery to such a low level that it should set red flags flying for any reader (although apparently not the peer reviewers or editors). Even lowering the recovery thresholds to at or below the trial entry conditions their results were poor.

Given the lengths they are going to to avoid releasing any data that might contradict their spin including many of the measures in the original protocol I think its easy to conclude the trial didn't really lead to significant improvements. I think we need to declare the writeup of PACE as untrustworthy due to their failures to follow protocol; through bad methodology in setting different expectations and their ridiculous spinning of results.

The arguments about patient selection I think are weaker until we have better understanding of mechanisms. However, the way they have spun data suggests they had a complete failure to help anyone whatever the diagnostic criteria.

Yes, agree. Good points.

 

[user9876]

No, I don't believe the Oxford criteria were changed at this point (i.e. when discussing recovery).

From the recovery paper

This was the definition of CFS used to define eligibility
for participation in the trial. Research assessors judged
whether participants still met Oxford criteria for CFS
at 52 weeks; specifically they determined if : (1) fatigue
was the main symptom, (2) it was of definite onset
and not lifelong, (3) fatigue was severe, disabling and
affected physical and mental function, and (4) fatigue
had persisted for 6 months or more and was present
50% of the time (Sharpe et al. 1991). To satisfy the third
criterion for severity of fatigue and disability, participants
had to meet trial entry thresholds for fatigue
(a binary score of o6 out of 11 on the CFQ) and abnormal
levels of physical function (a score of f65 out
of 100 on the SF-36 physical function subscale) (White
et al. 2007).

I was assuming the way they say the third condition was satisfied represented a modification but they may have used this all the time. But that's the bit I was thinking about.

 

[Dolphin]

From the recovery paper

This was the definition of CFS used to define eligibility
for participation in the trial. Research assessors judged
whether participants still met Oxford criteria for CFS
at 52 weeks; specifically they determined if : (1) fatigue
was the main symptom, (2) it was of definite onset
and not lifelong, (3) fatigue was severe, disabling and
affected physical and mental function, and (4) fatigue
had persisted for 6 months or more and was present
50% of the time (Sharpe et al. 1991). To satisfy the third
criterion for severity of fatigue and disability, participants
had to meet trial entry thresholds for fatigue
(a binary score of o6 out of 11 on the CFQ) and abnormal
levels of physical function (a score of f65 out
of 100 on the SF-36 physical function subscale) (White
et al. 2007).

I was assuming the way they say the third condition was satisfied represented a modification but they may have used this all the time. But that's the bit I was thinking about.

No, that's the same as in the protocol:

http://www.biomedcentral.com/1471-2377/7/6

Inclusion criteria
1. Both participant and clinician agree that randomisation is acceptable.

2. The participant has given written informed consent.

3. The participant meets operationalised Oxford research diagnostic criteria for CFS 2].

4. The participant's Chalder Fatigue Questionnaire score is 6 or more 27].

5. The participant's SF-36 physical function sub-scale score 28] is 65 or less.

6. The participant is aged at least 18 years old.

 

[Hate ME/CFS]

@Hate ME/CFS I didn't mean so much for her to be the messenger of explaining it medically vs. fluff. What I was trying to say was that it would take a MAJOR politician or celebrity to actually be struck with ME/CFS as a patient for anything to be done about the illness. They don't care when millions of us regular people have it but if someone of that level of celebrity and wealth had it, only then might it be taken seriously. Sorry if I didn't explain it well!

Gingergrrl,
You explained it correct, I just didn't comprehend it correctly. Lol Yes, we do need a well known figure, like Michael J Fox is with Parkinson's Disease.

 

[biophile]

@user9876 wonders if the Oxford criteria were changed for the recovery paper to include entry thresholds for the SF-36 and CFQ, but @Dolphin points out that the fatigue (CFQ ≥6 bimodal) and physical function (SF-36 ≤65) thresholds used in the operationalization of the Oxford criteria were already in the protocol.

I think you are both correct, because while the above thresholds are mentioned in the protocol, the fatigue (CFQ ≤3 bimodal) and physical function (SF-36 ≥85) thresholds used in the original definition of recovery rendered them irrelevant. Any participant meeting the original recovery thresholds would automatically score much better than the entry criteria. However, when they changed these recovery thresholds to CFQ ≤18 Likert for fatigue and SF-36 ≥60 for physical function, a participant still could have met the entry criteria thresholds, so it was probably then that they decided to use CFQ ≤5 bimodal and SF-36 ≥70 to make sure. The problem though is that participants were disqualified from meeting Oxford criteria at followup if they just failed either of the fatigue and/or physical function thresholds but otherwise still met Oxford criteria.

@zzz. It is possible that referral standards were somewhat relaxed if recruitment targets were waning, but that doesn't mean those FM candidates were then screened and made participants without meeting Oxford criteria. I'm not sure if the Hooper quote explicitly claims that occurred anyway, he just raises some possible concerns.

80% of candidates thought to have CFS were excluded from the trial! The most common reason was for not meeting Oxford criteria. The argument that the Oxford criteria excludes proven organic brain disease therefore it automatically excludes ME, is not quite correct but neither is the sentiment entirely incorrect: IIRC the medical assessment in the PACE Trial excluded or casted doubt on some clinical characteristics which are allowed in original definitions of ME and the CCC.

 

[medfeb]

Given the results were so poor I'm not sure that the debate on who was included is valuable since we don't really know who should be included as we don't know what the mechanisms behind ME are.

The arguments about patient selection I think are weaker until we have better understanding of mechanisms.

@user9876

Can you help me understand the above statements?

In a generic sense, aside from its usage in PACE, Oxford is essentially chronic fatigue, does not require hallmark criteria like PEM and allows primary psychiatric illness. Oxford's non-specificity means that it can encompass any number of fatiguing conditions, including ones not related to ME. Definitions like CCC and ME-ICC require PEM/PENE and other hallmark criteria and exclude primary psychiatric illness. GIven that, it's hard to imagine that Oxford could possibly select the same patient cohorts as CCC or ME-ICC. Oxford just encompasses too many other conditions that meet its criteria. Even Fukuda and CCC show that difference if we look at the work of Jason, some of Maes' work and also Nacul's prevalence work.

Then if I look at PACE specifically, 35% of patients had any depressive disorder and 47% of patients had any psychiatric illness, at least according to the P2P review. I recognize that some percent of the 35% could have secondary psychiatric illness versus primary but the point is that Oxford's loosened exclusionary criteria would allow primary psychiatric illness.

So if a subpopulation of PACE or any other Oxford study has primary psychiatric illness - or deconditioning or any other causes of chronic fatigue - then we can expect that their response to CBT and GET would be different than a population selected by CCC in which PEM is at play.

If the above is true, then patient selection issues in a general sense are an issue in addition to the study conduct issues that you outlined.

What am I missing?

 

[alex3619]

Also, for objectively measured physical disability (using the six minute walking distance test), CBT failed to improve average physical disability, and GET failed to improve average physical disability by a clinically useful amount.

Or to put it another way, patients started as severely disabled using physical exercise measures, ended that way, and the improvement was within the expected bias range.

 

[Sean]

...and the improvement was within the expected bias range.

Their results were not robust enough to clear the hurdle of possible bias and confounding.

 

[Kati]

Their results were not robust enough to clear the hurdle of possible bias and confounding.

The problem is that NIh and its team of untrained people sitting on the P2P committee is thinking otherwise.

 

[daisybell]

How can they justify excluding any treatment studies with durations of less than 12 weeks? Radiotherapy for cancer doesn't last 12 weeks, and nobody would say that it isn't relevant for a chronic condition. That's nuts. I can't believe the Rituximab studies have been excluded. The protocol followed was that used for RA. How can it be argued that it isn't long enough, or the appropriate course of treatment?

What planet are they on?

I'm not normally one for conspiracy theories but this fits.

 

[user9876]

No, that's the same as in the protocol:

http://www.biomedcentral.com/1471-2377/7/6

The way I was seeing it is that these criteria are essentially the same but slightly different in detail (I'm being pedantic).

But in the selection criteria has an inequality defined as

RandomisationAgreed &Consent & Oxford & (SF36<=65) & (CFQ>=6)&(Age>=18)

Where as for recovery they have a recovered inequality of:
(SF36 >=60) & (CFQ <= ?) & (CGI = 1 or CGI=2 ) & not(Oxford) .

But at this point they roll the (SF36<=65) & (CFQ>=6) part of the selection inequality into the Oxford criteria which was
(c) The fatigue is severe, disabling, and affects physical and mental functioning.

and Now becomes (c) (SF36<=65) & (CFQ>=6)

Hence they could have differences of interpretation between the two versions where I'm assuming severe and disabling is subjective initially but defined for the recovery criteria based on questionnaire answers. Maybe the question is how to they deal with (c) in their initial oxford criteria and whether this is subjective. I think a big concern is that a lot of potential patients were removed because they didn't meet the Oxford criteria so either they had some very unspecific referral process or some specific filtering.

 

[user9876]

@user9876

Can you help me understand the above statements?

In a generic sense, aside from its usage in PACE, Oxford is essentially chronic fatigue, does not require hallmark criteria like PEM and allows primary psychiatric illness. Oxford's non-specificity means that it can encompass any number of fatiguing conditions, including ones not related to ME. Definitions like CCC and ME-ICC require PEM/PENE and other hallmark criteria and exclude primary psychiatric illness. GIven that, it's hard to imagine that Oxford could possibly select the same patient cohorts as CCC or ME-ICC. Oxford just encompasses too many other conditions that meet its criteria. Even Fukuda and CCC show that difference if we look at the work of Jason, some of Maes' work and also Nacul's prevalence work.

Then if I look at PACE specifically, 35% of patients had any depressive disorder and 47% of patients had any psychiatric illness, at least according to the P2P review. I recognize that some percent of the 35% could have secondary psychiatric illness versus primary but the point is that Oxford's loosened exclusionary criteria would allow primary psychiatric illness.

So if a subpopulation of PACE or any other Oxford study has primary psychiatric illness - or deconditioning or any other causes of chronic fatigue - then we can expect that their response to CBT and GET would be different than a population selected by CCC in which PEM is at play.

If the above is true, then patient selection issues in a general sense are an issue in addition to the study conduct issues that you outlined.

What am I missing?

I'm not sure if I'm answering your points but I've tried and also to try to explain my thinking (but I may have a different view tomorrow!).

I would describe the PACE trial as aimed at people with unexplained fatigue. Which may or may not match ME definitions. There are exclusion criteria within Oxford but we don't know what checks they did. So if I were to design a trial I wouldn't use this definition but they did that so we need to interpret results in line with this.

I would also argue that using PEM as a defining feature is quite arbitrary. I guess its less than that since it is based on clinical experience but I worry that it might exclude people who also need help. We don't understand what is happening with ME so we don't know if this is a reasonable defining feature. Also I worry about that PEM is maybe less when ME is less severe and hence less noticeable. I've observed PEM with my child but she has denied it in the past (she has short term memory issues) but I wonder if it is something that some may not observe but have.

So PACE gives us a set of patients who meet the Oxford criteria and the question do the CBT/GET/APT interventions improve things for that group. (Question 1)

There is another question of how the Oxford group overlaps or doesn't overlap with other groups and hence how we can interpret the first question for people who meet other diagnostic criteria. Question 2

So for question 1 having looked at the results and taking account of what they refused to publish I would conclude that interventions change the way people answer questionnaires but not by much and there doesn't seem to be any objective improvements (benefit/work changes we know and showed no change; the step test we don't know; and the 6mwt showed a slight improvement for GET but not much). Given how little data we have from them its hard to make real conclusions. Hence my conclusion would be improvements are uncertain and based on my belief if they were good they would have published the objective data I conclude real improvements are unlikely.

Thus I conclude (differently to the PACE people but I think justified by what is and isn't published) that given a group of people with unexplained fatigue who meet the Oxford criteria CBT/GET/APT make little real difference.

Then we get to the question of what about other criteria and different subgroups.

Yes there were people suffering from depression (who may be excluded in other defns), I can't remember quite how they measured that but there was a paper that looked at the HADS scale (which I think they used) for people with fibromyalgia and they concluded that it does not measure depression or anxiety but only measures a vaguer concept of psychological distress. So I'm not sure what I conclude about that.

I seem to remember that they did a subgroup analysis around CDC criteria, London(ish) criteria and depression and that they showed much the same as their overall results.

The CBT they did was not aimed at depression but aimed at getting people to reinterpret symptoms so if there were people suffering from primary depression then I'm not sure they would be helped. I've not looked at CBT for depression but everytime I see CBT mentioned for other illnesses it seems to be around positive spins in papers on poor results (see the psychosis debate that Keith Laws and others have had).

So I guess I see the results being so poor and covered up as the major issue rather than patient selection. Patient selection is hard and I think needs to be trial specific and research criteria need to be different from clinical criteria when trying to understand mechanisms.

What I would hate is for us to say we only care about this group of people who are ill but that group whose illness is equally unexplained don't fit our pattern so lets ignore them (or it must be psychological for them). We need to be pushing for research that leads to understanding. What I hate about things like the PACE trial is that they are not science in that they don't hypothesis and test a mechanism but instead they are black box testing within a hard to control and measure environment with a very badly applied cookbook of statistical methods.

My conclusion would be PACE spend £5million to tell us nothing given they refuse to publish the original protocol and objective data.

 

[Bob]

@user9876

Can you help me understand the above statements?

In a generic sense, aside from its usage in PACE, Oxford is essentially chronic fatigue, does not require hallmark criteria like PEM and allows primary psychiatric illness. Oxford's non-specificity means that it can encompass any number of fatiguing conditions, including ones not related to ME. Definitions like CCC and ME-ICC require PEM/PENE and other hallmark criteria and exclude primary psychiatric illness. GIven that, it's hard to imagine that Oxford could possibly select the same patient cohorts as CCC or ME-ICC. Oxford just encompasses too many other conditions that meet its criteria. Even Fukuda and CCC show that difference if we look at the work of Jason, some of Maes' work and also Nacul's prevalence work.

Then if I look at PACE specifically, 35% of patients had any depressive disorder and 47% of patients had any psychiatric illness, at least according to the P2P review. I recognize that some percent of the 35% could have secondary psychiatric illness versus primary but the point is that Oxford's loosened exclusionary criteria would allow primary psychiatric illness.

So if a subpopulation of PACE or any other Oxford study has primary psychiatric illness - or deconditioning or any other causes of chronic fatigue - then we can expect that their response to CBT and GET would be different than a population selected by CCC in which PEM is at play.

If the above is true, then patient selection issues in a general sense are an issue in addition to the study conduct issues that you outlined.

What am I missing?

I think you've got it all correct, above. (I also think @user9876 makes many valid points - In our arguments I think we have to find a balance between nuanced scientific arguments and strong advocacy. Not an easy task.)

Also, remember that the published response rates (i.e. the percentage of participants who responded to treatment and whose improvement met a minimal clinically useful threshold) (ignoring all the weaknesses of the trial) was between 11-15% after treatment with CBT/GET. So it's hardly a treatment to recommend for all ME patients, esp when so many harms have been reported for CBT/GET, and when the trial recruited patients with unexplained chronic fatigue. Also, severely affected patients were excluded from the trial.

It's correct to say that the PACE trial recruited patients using a broad criteria for unexplained chronic fatigue rather than a widely recognised criteria for CFS or ME. (e.g. the Oxford criteria do not select the same cohort as the CDC's CFS criteria.) The fact that participants were later subgrouped using Fukuda Reeves is problematic because participants were previously filtered using the Oxford recruitment criteria, and because Reeves is no longer widely used because of its weaknesses.

 

[worldbackwards]

To summarize, Wessely and others have stated that there is only one "functional somatic syndrome".

Whenever I see this phrase, it reminds me of 'There is only one God and Muhammad is his prophet'. Lets try it out:

There is only one Functional Somatic Syndrome and Simon is it's prophet. Verily, praises be!

 

[catly]

Perhaps this summary on the PACE trial could be helpful to some. For me, I liked this video the best: