The P2P Draft report is out

[alex3619]

I don't think the research focus on men part is right.

I took this as a grammatical issue, and a comment on the lack of a research focus on men. Male/female is an obvious subgrouping for study to see if its important.

 

[snowathlete]

What, not a stale cheese sandwich with a pickle?

Its stale because its old. We already read pretty well all of this, from advocates and CFSAC. Result? No result, NIH and government inaction.

Best case scenario: the government wants change, and can't get political leverage. So these reports are to give it justification for action, new and impressive action.

Worst case scenario: the reports (I am including IOM) are intended to keep us busy, and via extreme reactions make us look like crazy people. They can then ignore us.

The reality: a report making recommendations is not tangible action. Its not even a promise of action. The NIH or other parts of government, and even other governments (its NOT all up to the US, when is most of the world going to step up?) need to take action. Governments are very very good at getting reports then doing nothing. Just look at the history of CFSAC.

I agree, it may be bread (pick your choice of fillings) that doesn't mean the government's going to swallow it. That is ultimately what matters.

 

[user9876]

To be honest 'diagnostic criteria' of this sort belong to the nineteenth and first part of the twentieth century. In almost all other branches of medicine they have been abandoned. The only reason they are still around for ME is that without any sort of physiological explanation it is hard to provide reassurance to patients that at least the medical profession has some idea of what the prognosis is without tagging that to a name. In simple terms we cannot decide that PEM is the best way to recognise something when we have no idea what we think it is that we are trying to recognise.

I would have thought that the significant proportion of any diagnostic criteria for ME would be to describe where confusions with other known diseases are so that people don't get mis-diagnosed with ME when they have something else that is diagnosable. There have been a few papers suggesting that a lot ~50% of people diagnosed with ME in primary care have a different illness.

I assume the same issues apply to looking for biomarkers where we don't know how many different versions of ME there are so they could be hard to find? Hence the need to explore potential different measurements but keeping subgroups in mind.

 

[Sasha]

I don't see why the diagnosis of ME should be limited by that specifically if we have no idea what an ME is or how many sorts of ME there are. If we start by assuming that there is one 'special disease' called ME then I think we are sunk scientifically. I see no reason to think that. To be scientific we have to know what we mean in the first place. I don't think there is any analogy with cancer because we know precisely what we mean by cancer - it is a cellular proliferation that invades other tissues. If we want proper epidemiological studies we have to start out assuming we have no idea how many sorts of ME or ME-like problems there are or how much they overlap.

To be honest 'diagnostic criteria' of this sort belong to the nineteenth and first part of the twentieth century. In almost all other branches of medicine they have been abandoned. The only reason they are still around for ME is that without any sort of physiological explanation it is hard to provide reassurance to patients that at least the medical profession has some idea of what the prognosis is without tagging that to a name. In simple terms we cannot decide that PEM is the best way to recognise something when we have no idea what we think it is that we are trying to recognise.

I'm afraid you've started us off now...

My understanding is that ME was first defined off the back of epidemic outbreaks, when you could be sure that whatever the disease was, it was the same thing that you were looking at in your cases. PEM was identified as a cardinal feature (someone correct me if my understanding of the history is wrong here).

Then, because no specialism picked up ME, psychiatrists stuck their oars in and broadened the definition to 'fatigue wtih knobs on' Oxford CFS, which 'ate' ME.

If we had just stuck to PEM-ME, wouldn't we now have biomarkers and a mechanism? Wouldn't we have emerged from the 19th century?

Instead, we've got this binbag diagnosis with who knows how many diseases in it. It makes no sense to me to look at the binbag.

 

[alex3619]

The draft starts out in stating that there are no biomarkers, yet there possibly are.

The issue is that with current science what we have is a lot of candidate diagnostic biomarkers. By no biomarkers they mean no diagnostic biomarkers that have been sufficiently validated. What we have is a huge number of possible candidate biomarkers that have just sat there due to lack of funding and interest. Some have been replicated a bit, validated a bit, but we need large, well designed and funded studies to go further.

We also have a lot of pathophysiology biomarkers, that clearly show things are going wrong in at least large subgroups. That also needs more research.

 

[Ember]

What we have is a huge number of possible candidate biomarkers that have just sat there due to lack of funding and interest. Some have been replicated a bit, validated a bit, but we need large, well designed and funded studies to go further.

I didn't notice any mention of replicating the two-day exercise test.

 

[Valentijn]

We can point out to them, in our submissions: that CBT/GET do not actually lead to real-world improvements for physical function, or improvements in objectively measured physical function; that the evidence in favour of CBT/GET is based on non-blinded studies (i.e. low quality research); and that CBT/GET research is based on the Oxford criteria that is "flawed" and inclusive of "people with other conditions". And hopefully we can get their recommendations re CBT removed altogether.

That's what I did in my comment. There are other aspects of the report which I'm not happy with, but the CBT/GET blurbs seem to be the most important flaw currently. They are also the flaw with the strongest argument against them, since the supporting studies use the very criteria which the report advocates throwing out.

It's not just a matter of patient experience, or psych-versus-bio dogma, but of very basic logic: if those criteria suck so much that they shouldn't be used in the future, then the past studies using those same criteria also shouldn't be a foundation for conclusions regarding treatment.

If it's an issue with the panel being forced by HHS to include those studies, then HHS needs a very firm smack on the nose.

 

[snowathlete]

Snowathlete, I want to preface my response with a declaration that anything I say is not personal, and is not intended to start an argument. Not that you've given me any reason to believe you may assume so, I just want to be sure to avoid getting into a potentially stressful situation.

That said, I don't believe your argument makes sense. There are many symptoms included in the CCC and ICC that aren't required to receive a diagnosis of ME. For example, digestive issues is listed as a non-required criteria as well as pain. I'm certainly not suggesting that PEM be the ONLY symptom required for diagnosis. However, if exercise makes a patient's symptoms worse, he/she has PEM. If exercise doesn't make a patient's symptoms worse, he/she doesn't have PEM and therefore should have no problem exercising. This is why the PACE trial claimed that exercise is a helpful treatment; it included patients without PEM.

No worries GalaxiiGrl, I want to avoid stressful situations too and I don't have any problem with you disagreeing with me, on this or anything else. Please feel free.

You're right about the ICC not having pain as a requiremnet, but what I mean is if they came up with a new definition and it said you had to have pain (as an example that is relevant to me) that might throw me out into the cold. For others it might be a different symptom. For some PEM. As Prof Edwards points out above, even in more established diseases like RA there are some with the disease who's symptoms do not conform to the usual norm. So is it right to exclude people at this stage, who actually might genuinely have the same disease, perhaps even caused by the same thing, but presenting differently for who knows what reason?

 

[Valentijn]

The problem though is that research might happen for those with PEM but not for others. There might be a danger that the psychobabblers then say, "well, this group dont have ME and are the CFS group we've always said existed. They need CBT and GET." And thus they end up out in the cold.

Psychobabblers think all patients have the same thing, and aren't likely to change their tune until they can no longer get funding or jobs. They're the ones who have deliberately conflated ME with CFS, then denied that ME can have neurological symptoms, and turned the whole illness into idiopathic fatigue. They're going to make a nasty mess of things regardless of how subtypes and atypical ME and non-ME CFS are treated, so there's no point in worrying about what they'll do, because we already know, and it will be irrelevant to anyone involved in actual science

I'd rather see as many patients as possible decisively taken away from them. It might leave some in the "who knows" camp of diagnosis, but it will be a major blow to the credibility of psychosomatic zealots. And that does benefit everyone, even if they're not clearly scooped up into the biological ME category.

The other thing to keep in mind is that P2P is all about research and only really has implications for research definitions (unless someone screws around, as HHS is certainly wont to do). A narrow research definition should not exclude atypical patients from receiving sensible clinical treatment. Rigid NHS dogma might exclude atypical patients from receiving sensible treatment, but that is an entirely different (bureaucratic) problem which is not helped by using a looser research definition.

 

[GalaxiiGrl]

No worries GalaxiiGrl, I want to avoid stressful situations too and I don't have any problem with you disagreeing with me, on this or anything else. Please feel free.

You're right about the ICC not having pain as a requiremnet, but what I mean is if they came up with a new definition and it said you had to have pain (as an example that is relevant to me) that might throw me out into the cold. For others it might be a different symptom. For some PEM. As Prof Edwards points out above, even in more established diseases like RA there are some with the disease who's symptoms do not conform to the usual norm. So is it right to exclude people at this stage, who actually might genuinely have the same disease, perhaps even caused by the same thing, but presenting differently for who knows what reason?

I think that PEM, (or PENE) has been found to be exclusive to ME, while the other symptoms (pain, digestive problems) are not. Therefore, if a patient has PEM, he/she almost certainly has ME. If a patient does not have PEM, he/she most likely has a different illness. If the patients without PEM have had all other potential illnesses ruled out, then there needs to be a new illness defined. I have no idea what it would be called, but I would have no doubt that the people who are diagnosed with it have a physical illness and are suffering greatly. But without the hallmark symptom of PEM, we're just lumping together patients with fatiguing illnesses.

 

[PennyIA]

While CBT/GET may not be able to help patients with ME-like symptoms but that DO NOT GET PEM? I'm thinking that part of the problem with folks with PEM is that CBT/GET is readily and easily seen as harmful as you can witness the event of pushing yourself to do that one more thing; AND easily see it create a crash event. If they don't crash after exercise? Then that's one of the factors that make it so harmful for PWME. That said... it doesn't mean that it's a good treatment for them. BUT it would probably have been better identified that if the patients that met the Oxford criteria originally that did report improvements (if there truly was any) that they were likely patients that didn't experience PEM. So, again - could this be a subtype? I don't know the answer to that, but I can guarantee that lumping them in together without clearly identified statistics in studies that show how patients with PEM vs patients without PEM respond to treatment? Will not get us the kind of studies we really need.

 

[snowathlete]

I think that PEM, (or PENE) has been found to be exclusive to ME, while the other symptoms (pain, digestive problems) are not. Therefore, if a patient has PEM, he/she almost certainly has ME. If a patient does not have PEM, he/she most likely has a different illness. If the patients without PEM have had all other potential illnesses ruled out, then there needs to be a new illness defined. I have no idea what it would be called, but I would have no doubt that the people who are diagnosed with it have a physical illness and are suffering greatly. But without the hallmark symptom of PEM, we're just lumping together patients with fatiguing illnesses.

I have PEM and cutting out those without it might help focus the research. That suits me. Which would be good. But we don't know that the absence of PEM means you are ME negative. Only that it's presence is specific to the disease, so it indicates a positive.

There are other ways to exclude other fatigue illnesses, though likely not as completely. If we did that, how many are left that don't have PEM? What's the split? Do these people look different in studies? (PEM aside)

I'd just hate to be one of those poor sods who ends up being dumped and all that means. Especially if fifty years later it turned out they do have the same disease after all, just a PEM-symptom-negative version for some reason.

 

[Sidereal]

Ok so I had to wade through the 477 page evidence report on which this report is based because I saw during one of the presentations I was watching live that the meta-analysis forest plots looked suspect but I couldn't take a screenshot in time.

In this meta-analysis, which is one of the main outcomes in the context of the overall conclusions of the report, they've compared CBT vs control on SF-36 physical function subscale. Three studies - O'Dowd, Wearden (FINE trial) and White (PACE trial) - are included with a CBT arm and 2 control groups (see footnotes in the graph). What they've done is enter the same CBT group from those studies twice, comparing it to each of the two control groups. You can't do that. They are the same participants featuring twice in one analysis. It's like taking two bites at a cherry.

For a complex data structure like this you need more advanced statistical techniques to pool data because you have one treatment arm (in this case CBT) being compared to two control groups and you can't just duplicate the CBT group because this massively inflates the contribution of that particular study to the overall effect size. A sensible option here would be to combine the two control groups into one and compare that merged control group to CBT. Or you could just compare the CBT arm with one control group that makes the most theoretical sense, preferably chosen a priori before you've seen the data and have had the chance to engage in cherry picking of the control group that makes your treatment look better.

The same problem is in the meta-analysis of GET vs control, the other crucial outcome. The PACE GET arm is entered twice, compared to each of the control groups in that study.

Secondly, what's suspicious is that they don't report heterogeneity statistics (they say in the method section they computed Q and I-squared but I can't find the numbers anywhere in the report). Why not? This wouldn't be accepted in a peer-reviewed publication. Is it because there is significant heterogeneity and so they would have to do sensitivity analyses to figure out what the sources of heterogeneity are? It looks to me as though the studies using Oxford criteria show an effect while the others don't.

Oh and by the way, look at that ludicrous outlier, Deale 1997. LOL. Simon Wessely is the senior author on that paper. Obviously a bogus study never to be replicated.

I don't know what the deal is with these governmental agencies but incompetence in their literature reviews is rife and I don't know if they have statisticians involved or is it just non-expert bureaucrats and their research assistants plugging random numbers into Revman with no clue of what they are doing. I wouldn't ascribe this to conspiracy because I've seen worse than this in a report on another illness that's not politically controversial. Back when I was still working I remember looking at an FDA report of a treatment for that illness and the meta-analysis was faulty, laughable really.

 

[Ecoclimber]

I have PEM and cutting out those without it might help focus the research. That suits me. Which would be good. But we don't know that the absence of PEM means you are ME negative. Only that it's presence is specific to the disease, so it indicates a positive.

There are other ways to exclude other fatigue illnesses, though likely not as completely. how many are left that don't have PEM? What's the split? Do these people look different in studies? (PEM aside)

I'd just hate to be one of those poor sods who ends up being dumped and all that means. Especially if fifty years later it turned out they do have the same disease after all, just a PEM-symptom-negative version for some reason.

The problem with cutting off those with the absence of PEM is the lack of understanding on the progression of ME/CFS. Many do not develop PEM until they are in later stages of this illness.

Research has to be conducted to determine the symptomatic progression of this illness over time otherwise you are creating an abritary subset of patients that could be excluded in the earlier stages that do in fact have ME/CFS. Let's not cast everything in hard and fast rules until we have conclusive evidence from further research. Could PEM be cyclical? Could treatment protocol affect the reduction of PEM to the point that it would skew research data by excluding of ME/CFS patients? Somethings to consider.

Can't post fast enough before two or more posts are created after the one I am quoting.

 

[Sing]

Awesome analysis, @Sidereal! Is it possible for you to estimate what a more accurate "Weighted mean difference" then might be? I recall that there was supposedly nearly a 10% improvement shown, according to the Panel, which was at the low end of significance, they said--but they took it as significant since it did barely qualify.

 

[Sean]

I took this as a grammatical issue, and a comment on the lack of a research focus on men. Male/female is an obvious subgrouping for study to see if its important.

That is what it looks like to me, just poor phrasing, and they meant that there has not been enough focus on men.

 

[Ecoclimber]

@Sidereal

Excellent job on catching this. The Pace trial is flawed and should be excluded. This needs to address in the comments to P2P.

Scientific research papers must include all the data so that other scientists can reproduce or replicate the results. Whenever researchers refuse to release their data as in the Pace Trials even under the FOIA, then the data and the results must be suspect. Replication research is required as a standard in all medical science for the results to be accepted and verified by the scientific community as it was the case of XMRV. Therefore, the same standard and level of threshold must be applied to all ME/CFS research.

Without this level of verification and replication, the Pace Trial results must be considered as invalid without further proof and the requirement for the release of all the data including data on the co-mingling of cohorts which include individuals who suffered major depressive disorders and not ME/CFS. Such suspicious behavior on the part of the authors of the Pace Trial for failure to disclose, gives rise that they are hiding or manipulating data.

 

[Sidereal]

Awesome analysis, @Sidereal! Is it possible for you to estimate what a more accurate "Weighted mean difference" then might be? I recall that there was supposedly nearly a 10% improvement shown, according to the Panel, which was at the low end of significance, they said--but they took it as significant since it did barely qualify.

It is possible and I've considered it but it would be a project. I would need to get all the papers, read them, extract the data (because I can't trust what they've done, who knows what other mistakes they've made), figure out the best approach for analysis. I am currently in a crash feeling quite ill so I really can't take on something like this right now.

 

[Sing]

@Ecoclimber, I was just about to bring up the point you make about the progression of ME/CFS. Like quite a few, I had a "gradual onset", or "staged onset", with more and more symptoms as the years have gone by. Now I have got virtually all of them.

Researchers, bureaucrats, doctors and others prefer simple illnesses, the simpler the better. That is one of the main reasons knowledge has not generally progressed for us, but goes round and round perpetually--or in reverse.

In an era of computers and more collaboration, we could move past this futile course as long as we can either get public or private funding, because we won't be getting help from pharma until they've got a quick, straight path to the $.

 

[user9876]

Ok so I had to wade through the 477 page evidence report on which this report is based because I saw during one of the presentations I was watching live that the meta-analysis forest plots looked suspect but I couldn't take a screenshot in time.

In this meta-analysis, which is one of the main outcomes in the context of the overall conclusions of the report, they've compared CBT vs control on SF-36 physical function subscale. Three studies - O'Dowd, Wearden (FINE trial) and White (PACE trial) - are included with a CBT arm and 2 control groups (see footnotes in the graph). What they've done is enter the same CBT group from those studies twice, comparing it to each of the two control groups. You can't do that. They are the same participants featuring twice in one analysis. It's like taking two bites at a cherry.

For a complex data structure like this you need more advanced statistical techniques to pool data because you have one treatment arm (in this case CBT) being compared to two control groups and you can't just duplicate the CBT group because this massively inflates the contribution of that particular study to the overall effect size. A sensible option here would be to combine the two control groups into one and compare that merged control group to CBT. Or you could just compare the CBT arm with one control group that makes the most theoretical sense, preferably chosen a priori before you've seen the data and have had the chance to engage in cherry picking of the control group that makes your treatment look better.

The same problem is in the meta-analysis of GET vs control, the other crucial outcome. The PACE GET arm is entered twice, compared to each of the control groups in that study.

Secondly, what's suspicious is that they don't report heterogeneity statistics (they say in the method section they computed Q and I-squared but I can't find the numbers anywhere in the report). Why not? This wouldn't be accepted in a peer-reviewed publication. Is it because there is significant heterogeneity and so they would have to do sensitivity analyses to figure out what the sources of heterogeneity are? It looks to me as though the studies using Oxford criteria show an effect while the others don't.

Oh and by the way, look at that ludicrous outlier, Deale 1997. LOL. Simon Wessely is the senior author on that paper. Obviously a bogus study never to be replicated.

View attachment 9297

I don't know what the deal is with these governmental agencies but incompetence in their literature reviews is rife and I don't know if they have statisticians involved or is it just non-expert bureaucrats and their research assistants plugging random numbers into Revman with no clue of what they are doing. I wouldn't ascribe this to conspiracy because I've seen worse than this in a report on another illness that's not politically controversial. Back when I was still working I remember looking at an FDA report of a treatment for that illness and the meta-analysis was faulty, laughable really.

I don't think it is even valid to quote the mean change since there is no evidence that he sf36 physical function scale is linear.