• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

KPAX Synergy (methyphenidate/Ritalin) trial results are out

Messages
15,786
Cort's spinnning the results quite a bit, and/or doesn't understand what it means when the results were found to be statistically insignificant.

A la "PACE", he's referring to a 22% reduction in symptoms, when the placebo had a 13% reduction and the treatment can only be attributed to the remaining 9% reduction:
Cort Johnson said:
But what does a 22% decline in a CIS score mean for a patient? The average beginning CIS score was about 110. A 22% decline in that score would put a patient at about 88. Seventy-six is considered the cutoff point for remaining in the workforce. Go above 76 and you’re probably going to drop out of the workforce. Go below 76 and you’re probably going to stay in the workforce.

The average response didn’t put the “average” patient back into the workforce but it put them within about 10 points of possibly entering it. That indicates that a significant increase in functionality occurred.

And he's badly misusing the the term "significant" in the context of the study. The problem is that the study actually found that the reduction in symptoms was not statistically significant.

A little hope is a good thing, but he's really stretching reality to try to promote the treatment. And I don't think most ME/SEID patients ever thought it would be a useful treatment in the first place, since stimulants either do nothing or wipe us out afterward, and everyone's tried various supplements, typically with very modest gains at most.

Anyhow, I'm pretty disappointed to see someone in the ME community engaging in the sort of spin which has also been widely employed by a variety of psychobabblers.
 
Last edited:

Sasha

Fine, thank you
Messages
17,863
Location
UK
I agree he doesn't seem willing to accept that it's the dud that the results suggest. If a difference between treatments is statistically nonsignificant, it means that the difference could easily be attributable to chance: it doesn't mean that you just need to do a bigger trial to make the difference statistically significant.

It means that you can't treat the outcomes (22% vs 13%) as though they're accurate estimates of the real effect sizes associated with each intervention. So all that stuff about being within 10 points of going back to work means nothing.
 

Snow Leopard

Hibernating
Messages
5,902
Location
South Australia
It is interesting that they have released some of the data before publishing a paper. I will be very disappointed if they don't go on to publish a paper, or if they do publish a paper and go on to claim efficacy based on post-hoc analysis.

Significus.jpg

From:
http://statistically-funny.blogspot.com.au/2014/03/if-at-first-you-dont-succeed.html
 
Messages
83
Doesn't surprise me, I know stimulants have barely any effect on me. My DNA results showed im a fast metabolizer of caffeine and this likely applies to stimulants as well so they don't help much.
 

Kati

Patient in training
Messages
5,497
With the placebo response being so high, it shows that patients are willing to believe anything they are taking is helping them. However one body cannot be fooled in the long term especially with this illness. We know this illness can have a fluctuating course for some, so feeling better from one intervention needs to be tested and measured over time, 6months, 1 year, 18 months, 2 years. Chances are, patients are not significantly better after 2 years.

We are in desperate need of measurable biomarkers as we move forward. We need to know who has the same disease and which subset, who is sicker, and why, amongst other things.
 
Messages
25
Location
Canada
From a statistics point of view, the sample size is relatively small. A bigger study with a similar distribution of outcomes could easily lead to statistically significant results. The 9-point difference is quite large.
 

FancyMyBlood

Senior Member
Messages
189
Like others I expected this study to fail. Stimulants have already been studied with little success and some herbs and antioxidants aren't going to do anything.

Still, the high placebo rate seems kinda odd. Wasn't it Wessely who claimed that the placebo rate in ME/CFS is very low? I really wonder what kind of patients they studied.
 

Sasha

Fine, thank you
Messages
17,863
Location
UK
From a statistics point of view, the sample size is relatively small. A bigger study with a similar distribution of outcomes could easily lead to statistically significant results. The 9-point difference is quite large.

But the fact that it's not statistically significant is an indication that you can't trust that the 9-point difference is real. The lack of statistical significance indicates that in a larger population it could really be a 0-point difference.
 
Messages
15,786
But the fact that it's not statistically significant is an indication that you can't trust that the 9-point difference is real. The lack of statistical significance indicates that in a larger population it could really be a 0-point difference.
Exactly. The very best we can say is that maybe future research can determine if there's a subgroup who responds, or similar. To think that an insignificant result somehow suggests it's still worth trying is completely incorrect at this point.

And I'd still be far happier to see some more physical and objective outcome measurements, even in a blinded trial. I'm a helluva lot more concerned with my ability to function than how I "feel" about things. It also sounds like the primary outcome measurement was Checklist Individual Strength, which was developed by a CFS psychobabble group from the Netherlands. Pretty damned useless. Maybe they'd get more useful results with a more relevant questionnaire.

I'm getting the impression that the researchers have little understanding of ME/SEID, and think that everything will be better if we're just forced to perk up a bit :rolleyes:
 

Sasha

Fine, thank you
Messages
17,863
Location
UK
I expected this intervention to make things worse relative to placebo so I am surprised by these results. Perhaps the follow-up was not long enough to detect the detrimental effects of exhaustion of already marginal dopamine stores in ME patients.

The fact that the difference is non-significantly positive statistically includes the possibility that in a large population, the intervention would be shown to be worse than placebo (I should have said that in my statement above).
 

Snow Leopard

Hibernating
Messages
5,902
Location
South Australia
Still, the high placebo rate seems kinda odd. Wasn't it Wessely who claimed that the placebo rate in ME/CFS is very low? I really wonder what kind of patients they studied.

They were using it to claim that because the placebo rate in ME/CFS is low, that the changes after CBT/GET must therefore be real (despite no placebo control).

I don't buy the argument at all. ME/CFS patients are just as suggestive as patients with other diseases when it comes to answering questionnaires.
 
Messages
15,786
I don't buy the argument at all. ME/CFS patients are just as suggestive as patients with other diseases when it comes to answering questionnaires.
I think it also depends on the questionnaire. The more subjective it is, and less focused on specific physical activities, the less reliable it's going to be. It's a lot easier to fudge on "I feel like I can do things" than "I can walk up a flight of stairs without struggling". Though maybe that doesn't directly relate to the placebo response.