• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of, and finding treatments for, complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia, long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Accessible analysis of PACE?

Graham

Graham

Senior Moment
Messages
5,188
Location
Sussex, UK
Earlier this year, a group of us produced an analysis of the PACE trial, which many of you may have looked at. We used graphics and analogies to make it easier to follow.

http://evaluatingpace.phoenixrising.me/homepageanim.html

There are now two short Youtube videos which explain some of our findings, with two more in the pipeline. The idea is that these are even easier to understand, and hopefully will widen our audience, and can be shown to anyone who wonders why we are sceptical of the assertion that CBT and GET "cure" 30% of people with ME.

Don't be too picky, please. It is a complex minefield, so everything has been whittled down to the bare essentials. Hope you approve.


 
Esther12

Esther12

Senior Member
Messages
13,774
You're doing this wrong Graham - all of that analysis of the evidence, and how the results were spun....

You'd get far more attention if you just claimed that patients were refusing to accept that PACE showed 30% of patients were cured by CBT/GET because they were crazy militants who hate psychiatry. If you try to work out what's true, and show why patients are reasonably angry about the way false claims of efficacy are made to them, that's not going to be popular with any important people. You're not going to be able to make any money out of this.
 
Graham

Graham

Senior Moment
Messages
5,188
Location
Sussex, UK
Thanks Esther. I'm intending on the fourth video to return to my rocking chair with George and admit that many people will think that I am a reckless and violent terrorist. Anyway, I'm not after money, just world domination. Modest enough aims I would have thought.
 
Don Quichotte

Don Quichotte

Don Quichotte
Messages
97
one of the problems with studies that look at subjective/psychological variables is that there is no clearly defined normal. Therefore there is a lot of place for statisticobable, which can easily confuse someone who tries to quickly read the article during a busy day.

I tried to "translate" their study to a study in my field of practice.


Other eligibility criteria consisted of a Hgb. level of less than 9.0 . (normal >12.0) . 11 months after the trial began, this requirement was changed from 9.0 to 11.0 to increase recruitment

We summarised continuous variables with mean (SD) or median (IQR) and categorical variables with frequencies and proportions. Differentiation of treatment compared independent ratings of therapy sessions with actual treatment. We calculated the inter-rater reliability (κ and 95% CI) between the two assessors. We used Kruskal-Wallis tests for comparisons of therapy received, therapeutic alliance, and manual adherence. We compared categorical variables with Fisher's exact test. and blah, blah blah....
In another post-hoc analysis, we compared the proportions of participants who had scores of both primary outcomes within the normal range at 52 weeks. This range was defined as a Hgb. of more than 10.0
Click to expand...

I can promise you that any reviewer would throw this paper to the bin, without reading any further. Because a Hgb. of 10.0 would not be accepted as normal by any standard.

Very few physicians (who would be the reviewers of this paper) know what a normal chalder or SF-36 score is. By the time they reach the results section they have forgotten what was defined as normal in the methods section. They have gone through a very long and detailed description of the statistical methods being used, giving them the impression that so much details and terminology prove a very meticulous approach.
I am not sure that even the authors of the paper were aware that their lower limit of normal is below the lower limit of ill. I believe they too got lost in their forest of words and supposedly accurate terminology.
 
Don Quichotte

Don Quichotte

Don Quichotte
Messages
97
Also, the SD (24) is higher than the expected change (10).
This makes it a very poor parameter to assess.
 
Graham

Graham

Senior Moment
Messages
5,188
Location
Sussex, UK
Thanks all. Sonia and Jane! World domination is getting closer.;)

Don, I'm not sure about what the authors thought. I was thinking of adding a comment about this in the fourth video (the first three are to be factual, the last to be a sort of summing up). I'd be interested in people's thoughts on this.

If all the changes and faults in design had been accidental, then I would have thought that some of them would have worked against the effect. But all of them work the same way, either boosting weak effects, or creating positive effects out of nothing. So I think that one or two of the authors, convinced that their therapies were effective, and convinced that the majority of patients would show improvement (after all, they have been saying this for years), decided that their methods or targets needed tweaking to show the truth as they saw it. They failed to see that in reality the trial was proving them wrong. Sadly, we are in an area where medical status counts much more than scientific scepticism, and the other authors, the editors, and the peer-reviewers, did not bother to do a proper analytic job because it was clear that this was a high-status, highly respected study that had used sophisticated statistical techniques (or in my terms, pressed clever buttons on their spreadsheets). Then NICE simply assumed that the peer-reviewers had done their job properly. I would say that it was a case of the emperor's new clothes, but in that case people thought that what they saw was wrong. I don't think anyone much in the medical world has actually looked at it, and has simply let high status take over. Only those of us not dazzled by high medical status have dug around for the reality.
 
B

Battery Muncher

Senior Member
Messages
620
This is a brilliant piece of work, Graham (et al.)! I'm not finished yet - I can only manage it in very very short bursts... but have found it immensely helpful. Surprised it didn't ignite more discussion when first posted!
 
Graham

Graham

Senior Moment
Messages
5,188
Location
Sussex, UK
Thanks Battery Muncher. We are now working on video 6. You aren't a singer are you? I want to start it with a song, with a different person singing each line, and I am a couple of singers short. I could let some sing two lines, but it would be nice not to have to. The song is a Beverley Hillbillies type, and purely for fun.
 
Daisymay

Daisymay

Senior Member
Messages
754
Hi Graham et al, just wanted to congratulate you on these excellent videos which show so clearly the problems with PACE, thanks, I'm sure it took a lot of effort and energy.
 
Graham

Graham

Senior Moment
Messages
5,188
Location
Sussex, UK
Thanks Daisy, but the real bonus is they keep me off the streets and stop me from annoying the family (much). They are a group effort though.
 
Sea

Sea

Senior Member
Messages
1,286
Location
NSW Australia
@Graham if you're still looking I don't mind singing for you. It's the not knowing how to do computer stuff that's held me back from offering before. I'd need help with that.
 
You must log in or register to reply here.