The P2P Draft report is out

[alex3619]

@jspotila, I had to present a paper at a conference of international experts in like 1994 or 5. What happens when fog hits and you don't recall anything? During my Honors thesis presentation I got tunnel vision. Lecturing to several hundred students once in 1995 I got a mental crash, and struggled to think. Fortunately I had preprepared a nifty video of something so I ran that.

If you can get through a public engagement with ME its a great achievement.

 

[Nielk]

Jerrold Spinhirne has written an excellent comment criticizing the P2P report. It's available as a Google document which makes it easy to share. Jerry has submitted this as a public comment to the January CFSAC meeting.

https://drive.google.com/file/d/0B4uD-VyWmIw2b1pXMzZOZWVUUE0/view?pli=1

I would recommend for all to read this amazing critique of the p2p by Jerrold Spinhirne. You can read the full comment HERE on PR as well.

 

[joshualevy]

I don't understand why anyone thinks that rejecting Oxford will result in rejecting PACE. There are two obvious reasons why it won't:

First, Oxford also did it's analysis based on patients who met the Fukuda criteria, and got the same results (results were NOT statistically different). So even if Oxford got tossed, the Fukuda results would be there, just as strong. Fukuda and Oxford would both need to be retired to get any traction there.

Second, the normal scientific process is NOT to toss all studies based on a definition, when that definition is replaced / updated / or falls out of fashion. This happens all the time. (If you think of it, diseases are always defined early in the process, before they are really understood, and so definitions often change as more is learned.) However, these changes don't result in the studies getting tossed. It's just that the people in the field look at them differently. Obviously the studies say "I used definition A" and everyone knows that "definition B" is the better one, so people view the results differently, but they are not ignored completely (as you want in this case).

 

[Sasha]

First I'm like: and then I'm like: and then I'm all: and at the end I feel like:

Pretty sure we've all been there.

 

[Sasha]

I don't understand why anyone thinks that rejecting Oxford will result in rejecting PACE. There are two obvious reasons why it won't:

First, Oxford also did it's analysis based on patients who met the Fukuda criteria, and got the same results (results were NOT statistically different). So even if Oxford got tossed, the Fukuda results would be there, just as strong. Fukuda and Oxford would both need to be retired to get any traction there.

My understanding is that the authors said they used Fukuda but actually didn't - patients didn't fulfil the length-of-time criteria for Fukuda symptoms. Correct me if I'm wrong, someone.

@Dolphin, is what I'm saying correct? I saw it on this gigantic thread somewhere.

Also, if you throw out a load of people and effect size remains the same, statisical significance will be less impressive - depends on the number left behind.

Second, the normal scientific process is NOT to toss all studies based on a definition, when that definition is replaced / updated / or falls out of fashion. This happens all the time. (If you think of it, diseases are always defined early in the process, before they are really understood, and so definitions often change as more is learned.) However, these changes don't result in the studies getting tossed. It's just that the people in the field look at them differently. Obviously the studies say "I used definition A" and everyone knows that "definition B" is the better one, so people view the results differently, but they are not ignored completely (as you want in this case).

I take your point, but I think it's very unusual for a disease to start with a tight defintion (Ramsay) and have that exploded to be 'all unexplained fatigue' (Oxford). I don't think that that's remotely normal in medical science.

 

[Wildcat]

.
http://www.investinme.org/Article42...oper to undated letter by White to Horton.htm

Initial response by Professor Malcolm Hooper to an undated letter sent by Professor Peter White to Dr Richard Horton, Editor-in-Chief of The Lancet

18th May 2011


On 17th May 2011 Zoe Mullan, Senior Editor at The Lancet, sent an email to Professor Hooper in response to the complaint he submitted about the PACE Trial article published online by The Lancet on 18th February 2011 and subsequently in the journal on 5th March 2011. In her email, Zoe Mullan wrote: “We asked the authors of the PACE trial to respond to your concerns, which they have duly done. Your complaint and their response were discussed at the highest management level and this group of executive editors was fully satisfied that there were no grounds whatsoever on which to take further action. We attach the response provided to us here. From an editorial perspective, the case is now closed”.

The undated response to Professor Hooper’s complaint by Professors White, Sharpe and Chalder that was sent to Dr Richard Horton (Editor-in-Chief of The Lancet) on behalf of all the co-authors will, in the interests of openness and transparency, be placed in the public domain and will be fully addressed in due course, as will Professor Hooper’s concerns over what he believes is the failure of The Lancet’s editorial process in this instance, but there is one point in Professor White’s letter that is of particular importance, so it is addressed in this initial response.


‘In their letter, Peter White et al state: “The PACE trial paper refers to chronic fatigue syndrome (CFS) which is operationally defined; it does not purport to be studying CFS/ME”.

The sentence continues by stating that the PACE Trial studied: “CFS defined simply as a principal complaint of fatigue that is disabling, having lasted six months, with no alternative medical explanation (Oxford criteria)”…… ‘
.

 

[Bob]

I think the apparent effectiveness of CBT is an artifact of poor research methods, much like the placebo effect. What we really need are some objective measures of improvement and long term follow ups.

Thanks A.B. I agree. I'm including those arguments as well. The Oxford issue is perhaps one more thing to throw at them.

 

[Esther12]

Thanks A.B. I agree. I'm including those arguments as well. The Oxford issue is perhaps one more thing to throw at them.

I've not been following this thread closely, but my impression is that the Oxford part of the argument against CBT/GET is a minor thing.

 

[Sasha]

I've not been following this thread closely, but my impression is that the Oxford part of the argument against CBT/GET is a minor thing.

It hinges on the fact that the P2P report is recommending ditching the Oxford criteria because they're damaging to research. To us, it therefore makes sense that they ditch all Oxford-critiera trials.

 

[Bob]

I don't understand why anyone thinks that rejecting Oxford will result in rejecting PACE. There are two obvious reasons why it won't:

First, Oxford also did it's analysis based on patients who met the Fukuda criteria, and got the same results (results were NOT statistically different). So even if Oxford got tossed, the Fukuda results would be there, just as strong. Fukuda and Oxford would both need to be retired to get any traction there.

Second, the normal scientific process is NOT to toss all studies based on a definition, when that definition is replaced / updated / or falls out of fashion. This happens all the time. (If you think of it, diseases are always defined early in the process, before they are really understood, and so definitions often change as more is learned.) However, these changes don't result in the studies getting tossed. It's just that the people in the field look at them differently. Obviously the studies say "I used definition A" and everyone knows that "definition B" is the better one, so people view the results differently, but they are not ignored completely (as you want in this case).

The executive summary isn't a scientific document - it's a political document based on a selection the scientific evidence and other evidence/submissions (i.e. it's an interpretation of the evidence base and of the clinical needs of patients) - so they can choose how to interpret the evidence. If they choose to, they can make a statement about how the evidence base is weak or inappropriate for CBT/GET.

Re the PACE trial, the recruitment criteria were Oxford. Subsequent analysis using Fukuda does not necessarily represent a Fukuda cohort.
The P2P evidence document states that the PACE trial recruited using Oxford. As far as I can see, it does include data for the secondary Fukuda analysis.

 

[Bob]

I've not been following this thread closely, but my impression is that the Oxford part of the argument against CBT/GET is a minor thing.

It's just one argument among many that can be used against their evidence base for CBT/GET.
I've perhaps mentioned it disproportionately in this thread, because i've recently been exploring this particular issue specifically in relation to their evidence document. (I hadn't analysed that particular aspect of their evidence carefully before now.)

We've explored most of the other issues, re CBT/GET, on this forum before, and so I haven't highlighted them all in this thread.

 

[Esther12]

It hinges on the fact that the P2P report is recommending ditching the Oxford criteria because they're damaging to research. To us, it therefore makes sense that they ditch all Oxford-critiera trials.

Yeah - I get that, and it could be worth mentioning, but I don't think it's that great an argument against CBT/GET.

 

[Bob]

@Anne has pointed out on the other thread that they seem to have published different versions of the draft executive report, with different line numbers.
I've now got two different versions: A version that I downloaded early on, and a different version that's currently available for download.
The text might be identical in both (I'm not certain), but the line numbers are different in some sections.
It seems that Anne might have yet another version of the report, so there might be more than two versions - but I'm not certain about that.

So, that's going to throw a heap of confusion into the whole process.

While we sort out what's going on, i suggest that we need to quote the entire section of text we are responding to, rather than just the line numbers - because the line numbers seem to be meaningless at the moment.

Anne says that Jennie is onto it.

 

[Scarecrow]

My understanding is that the authors said they used Fukuda but actually didn't - patients didn't fulfil the length-of-time criteria for Fukuda symptoms. Correct me if I'm wrong, someone.

I recall trying to find out what 'international' criteria was used in PACE and finding it very obscure. I think it was Empirical and I could not make head nor tail of that paper [edit: Reeves et al].

 

[Valentijn]

First, Oxford also did it's analysis based on patients who met the Fukuda criteria, and got the same results (results were NOT statistically different). So even if Oxford got tossed, the Fukuda results would be there, just as strong.

Not really. Oxford is not just broader than other definitions, but can actively exclude ME patients. Oxford requires that fatigue be the primary symptom, which the other definitions do not, and would necessarily exclude patients who don't have much "fatigue" but rather have a primary symptom of PEM.

Thus by having Oxford as the primary criteria for inclusion, the Oxford+Fukuda patients would generally have to have symptoms closer to Oxford (physical fatigue, cognitive fatigue, headaches, unrefreshing sleep) than to ICC/CCC (PEM, swollen lymph nodes, sore throat). As a result, many Fukuda patients could have been excluded by the use of Oxford, resulting in PACE having a "Fukuda" subgroup which was not a truly representative sample of Fukuda patients.

 

[Ember]

I would recommend for all to read this amazing critique of the p2p by Jerrold Spinhirne. You can read the full comment HERE on PR as well.

Many thanks to Jerrold Spinhirne for his public comment. He writes:

Why would HHS ever implement any of the grand proposals of the P2P draft report when HHS stubbornly refuses to take even the low-cost, simple step of removing the harmful, inaccurate CFS Toolkit from the CDC website and disseminating the urgently needed IC Primer to healthcare professionals?

Comments about CBT/GET treatment findings relevant to the wider community don't seem relevant to ME. According to the Evidence Report, CBT/GET treatment findings can't be generalized to ME patients:

In summary , there is moderate strength of evidence that counseling techniques was associated with global improvement (41 to 70% vs. 25 to 31%), and low strength of evidence that counseling techniques improved overall functioning (SF-36 physical function weighted mean difference 7.73; 95% CI, 3.58 to 11.87), fatigue (27 to 76% vs. 7 to 65%), and employment outcomes in ME/CFS patients. The effects may not be generalizable to a more disabled population as no study used a case definition for ME and only one study analyzed patients fulfilling the London ME (Dowsett, 1994) case definition and may have been underpowered to detect a difference.121

GET was superior to control groups in measures of fatigue, clinical impression of change, and function. These findings however cannot be generalized to all patients with ME/CFS as no study enrolled patients whereby PEM was a diagnostic requirement (ME case definitions) and only one study performed a subgroup analysis to determine if similar results were found in patients meeting the diagnosis of ME.

The Evidence Review acknowledges that CFS, ME and ME/CFS may “identify separate groups entirely.” Why then expect a joint agreement on a single case definition to arise from the wider community?

 

[Hope123]

When one of the panelists gave a presentation on the review of the research, I'm pretty sure she said that 7 is used as the cut-off point for being clinically useful. Hence getting the points to anything under 7 for CBT or GET would be quite useful in showing how useless they are for ME/CFS patients (versus CF patients).

Do not to get too hung up on the actual difference in points (5 vs. 7 vs. etc.) because in truth, there has been no study done that I know of in ME/CFS subjects where they looked at SF-36 point difference and correlated it to a subjective change in quality of life, function, etc. The technical term for this is "minimal clinical signficant difference." Sometimes researchers use MCIDs established for other illnesses when a MCID does not exist for a specific illness but that's not the optimal method as other studies have shown the MCID for a specific scale like SF-36 may differ for say multiple sclerosis vs. seasonal allergies. People with MS may rate a 5 point change in difference as improvement they can feel/see whereas people with allergies may rate a 10 point change. I'm making this example up but you get the idea.


(Note that this is different from studies that have shown improvements in both SF-36 score and say, clinical global improvement, in ME/CFS groups as that does not answer the question. Researchers need to look at individual subject SF-36 scores vs. another measure(s) at more than one timepoint.)

 

[Wildcat]

.

David Tuller wrote about CFS criteria in the New York Times in 2011 Here:
http://www.nytimes.com/2011/03/08/health/research/08fatigue.html



PD White and the other PACE Trial Researchers respond in the NYT Letters page. And David responds to the response (also in the NYT Letters page).

http://www.nytimes.com/2011/03/15/health/15letters-STUDYINGAFAT_LETTERS.html?_r=1


Letters
Studying a Fatigue Illness (1 Letter)
Published: March 14, 2011

To the Editor:

In “Defining an Illness Is Fodder for Debate” (March 8), your reporter David Tuller correctly noted that the way an illness is defined can often determine what is found in studies of it. He also suggested that this problem of definition had limited the interpretation of our trial of treatments for chronic fatigue syndrome, published recently in The Lancet. That is not the case.

The patients in this trial had a disabling chronic illness in which fatigue was their main symptom and for which no alternative had been found; that is the definition of the syndrome used in Britain. But we also assessed trial participants to see if they met two other definitions of the illness that are favored by some scientists.

We found that both cognitive behavior therapy and graded exercise therapy, when added to specialist medical care, were most effective not only in the whole sample but also in the participants who met these alternative criteria. In addition, these treatments were the most effective whether or not a patient was depressed, a not uncommon accompaniment to this chronic and misunderstood illness. The trial also found that these treatments were safe so long as they were provided by appropriate therapists trained to help patients with chronic fatigue syndrome.

So to Mr. Tuller’s question “Does the evidence from that study prove that these strategies would help patients identified as having chronic fatigue syndrome through very different criteria?,” the answer is “Yes, it does.” Patients and their doctors now have robust evidence that there are two safe treatments that can improve both symptoms and quality of life, however the illness is defined.

P. D. White

T. Chalder

M. Sharpe

London and Edinburgh

The writers are the principal investigators of the PACE trial.



David Tuller replies: 'The article asked whether findings among a population defined by one set of criteria would apply to populations defined by “very different criteria.” In this study, all participants were first defined, identified and selected not by different criteria but by the same criteria, the so-called Oxford criteria used in Britain. Subgroups within that already screened population who also meet secondary criteria are not easily compared to patients who have not been screened, since an unknown number who met the secondary criteria might not have met the study’s criteria for inclusion. The gold standard for making comparisons across groups of patients identified by three varying case definitions would be a study with three completely separate cohorts, not one large sample with two embedded subgroups.'

.

 

[Hope123]

From my unreliable memory, this was only in relation to the recovery paper, and not to the main paper. And I think they might only have done this for the Oxford criteria in the recovery paper (because Fukuda didn't feature in the recovery paper.) Hopefully someone will correct me if I'm wrong.

Devil is in the details and White et al. likely got criticized by peer reviewers so they had to put in the info but hid it in the Discussion.

1) If you look in the "Discussion" section of the "Recovery" paper, it says they only assessed minor Fukuda symptoms for 1 week.

"The prevalence of the case-level International (CDC) definition of CFS may have been inaccurate because we only examined for accompanying symptoms in the previous week, not the previous 6 months."
(http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3776285/)

2) This applies to BOTH the original PACE and "Recovery" paper. You can track this by comparing the total number of patients in both studies.

In the "Recovery" section, the "Results" read:

"We studied 640 participants (excluding one participant who withdrew consent after the study). The mean (s.d.) age was 38 (12) years, 77% were female, and 93% were Caucasian. All participants met the Oxford criteria for CFS; 67% of participants also met the International (CDC) criteria for CFS and 51% met the London criteria for ME (White et al.2011)."

They would not be quoting the original PACE paper (2011) if this was a subset of the original PACE cohort. In addition, there would be some mention somewhere of this being a subset of the original group.

Peter White's body of work is interesting because he has some papers that are about the biology of ME/CFS -- cytokines, inciting factors, etc. -- that are worthwhile and intelligent. The problem is he NEVER publicizes those papers much and instead focuses on these types of papers. (I'm not the only one that has observed this) It's also a a rather stupid error to not count Fukuda minor symptoms for 6 months when that is explicitly clear in the case definition. ("These symptoms must have persisted or recurred during 6 or more consecutive months of illness and must not have predated the fatigue.') It's an error a reasonably conscientious high school student would not make much less a multi-million-dollar study that had vetting from multiple professionals.

So I have to conclude that either he is being nefarious or, more likely, the UK government wants to control the story and not face reality. I don't believe much in gov't conspiracies per se but I can see why people do.

 

[Hope123]

.

David Tuller wrote about CFS criteria in the New York Times in 2011 Here:
http://www.nytimes.com/2011/03/08/health/research/08fatigue.html



PD White and the other PACE Trial Researchers respond in the NYT Letters page. And David responds to the response (also in the NYT Letters page).

http://www.nytimes.com/2011/03/15/health/15letters-STUDYINGAFAT_LETTERS.html?_r=1


Letters
Studying a Fatigue Illness (1 Letter)
Published: March 14, 2011

To the Editor:

In “Defining an Illness Is Fodder for Debate” (March 8), your reporter David Tuller correctly noted that the way an illness is defined can often determine what is found in studies of it. He also suggested that this problem of definition had limited the interpretation of our trial of treatments for chronic fatigue syndrome, published recently in The Lancet. That is not the case.

The patients in this trial had a disabling chronic illness in which fatigue was their main symptom and for which no alternative had been found; that is the definition of the syndrome used in Britain. But we also assessed trial participants to see if they met two other definitions of the illness that are favored by some scientists.

We found that both cognitive behavior therapy and graded exercise therapy, when added to specialist medical care, were most effective not only in the whole sample but also in the participants who met these alternative criteria. In addition, these treatments were the most effective whether or not a patient was depressed, a not uncommon accompaniment to this chronic and misunderstood illness. The trial also found that these treatments were safe so long as they were provided by appropriate therapists trained to help patients with chronic fatigue syndrome.

So to Mr. Tuller’s question “Does the evidence from that study prove that these strategies would help patients identified as having chronic fatigue syndrome through very different criteria?,” the answer is “Yes, it does.” Patients and their doctors now have robust evidence that there are two safe treatments that can improve both symptoms and quality of life, however the illness is defined.

P. D. White

T. Chalder

M. Sharpe

London and Edinburgh

The writers are the principal investigators of the PACE trial.



David Tuller replies: 'The article asked whether findings among a population defined by one set of criteria would apply to populations defined by “very different criteria.” In this study, all participants were first defined, identified and selected not by different criteria but by the same criteria, the so-called Oxford criteria used in Britain. Subgroups within that already screened population who also meet secondary criteria are not easily compared to patients who have not been screened, since an unknown number who met the secondary criteria might not have met the study’s criteria for inclusion. The gold standard for making comparisons across groups of patients identified by three varying case definitions would be a study with three completely separate cohorts, not one large sample with two embedded subgroups.'

.

Fun Fact: David Tuller has a PhD in public health but he doesn't use the title when writing.