• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of, and finding treatments for, complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia, long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Dr James Coyne tackles the PACE follow-up paper

alex3619

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
I posted a comment on his blog:

As a patient I was very pleased to read your analysis. Its obviously of interest to a great many patients. I wonder though how many in science, the medical profession, and related government and industry organizations will be interested. They should be, very much so, but there is a long history of problems with this kind of research being ignored.

I do hope you write a follow-up article.

It is still a mystery to me why a disease that costs the world on the order of a 100 billion dollars per year (projected from estimates in the USA) does not receive more scrutiny, and the science taken more seriously. Anyone with financial interest in outcomes, not just patients, should in theory want to see the most robust science, and should not tolerate science that fails to meet reasonable criteria.

I have long regarded the PACE trial as far below evidence based Gold Standard, due to not being double blinded, the use of a manufactured and irrelevant comparison arm with adaptive pacing, and the heavy reliance of subjective measures in a psychological intervention trial. There is provision within Evidence Based Medicine to downgrade the ranking of trial results due to problems. These alone prevent it being Gold Standard (though this is a nebulous and debatable term), before you even get to effect sizes, statistical and methodological errors, and so on.

I wonder what you think about the PACE trial's papers from an evidence based perspective?
Click to expand...
 
Jonathan Edwards

Jonathan Edwards

"Gibberish"
Messages
5,256
I think Coyne does a superb job on this. The business of the authors saying that the one bit of evidence that does not suit them is post-hoc is quite intriguing. I actually think there is may be interesting mathematical fallacy here that @user9876 might be able to identify. I suspect that an analysis that confirms the null hypothesis rather than your chosen deviation from that null hypothesis does not actually require a Bonferoni correction or equivalent. And if it is the only and obvious way to show your chosen hypothesis is wrong then there is no correction to be made. I am not quite sure what it is but I suspect that basic statistical theory could show that the authors' point is not just inappropriate because it is arbitrary - it is actually wrong.
 
Esther12

Esther12

Senior Member
Messages
13,774
I didn't expect this to be so good, considering the speed with which it was done. Amazing effort from the many patients working to highlight problems with PACE... but it does seem to take you all a bit longer.

Very handy that Coyne had already been thinking about ways of measuring fatigue.
 
U

user9876

Senior Member
Messages
4,556
Jonathan Edwards said:
I think Coyne does a superb job on this. The business of the authors saying that the one bit of evidence that does not suit them is post-hoc is quite intriguing. I actually think there is may be interesting mathematical fallacy here that @user9876 might be able to identify. I suspect that an analysis that confirms the null hypothesis rather than your chosen deviation from that null hypothesis does not actually require a Bonferoni correction or equivalent. And if it is the only and obvious way to show your chosen hypothesis is wrong then there is no correction to be made. I am not quite sure what it is but I suspect that basic statistical theory could show that the authors' point is not just inappropriate because it is arbitrary - it is actually wrong.
Click to expand...

There is a good paper by David Freeman 'shoe leather and statistics' that talks about the use of increasingly complex methods being used in stats to try to deal with bad experiments and impossible to control situations. Basically he argues that stats cannot be a substitute for the hardware necessary to understand the mechanisms. Seems like all these corrections for different correlations fall into the too complex camp to me. We don't know (from a mechanisms perspective) which variables correlate and which are independent. We don't know about the dependence between their outcome variables, or the non-itineraries within the scales.

Basically if the stats are that complex but not based on any real mechanism and hypothesis then I think its impossible to really do any sensible analytics.
 
ahimsa

ahimsa

ahimsa_pdx on twitter
Messages
1,921
@Cheshire - that's a great image, but it's even better when seen as part of a series...

1When-Cherished-Beliefs-Clash-with-Evidence-218x300.png

2When-Cherished-Beliefs-Clash-with-Evidence-300x255.png

3aWhen-Cherished-Beliefs-Clash-with-Evidence-page-0-300x273.jpg
 
You must log in or register to reply here.