• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Carlisle’s statistics bombshell names and shames rigged clinical trials

Ysabelle-S

Highly Vexatious
Messages
524
Carlisle.JPG


Someone posted that link on Carlisle on Twitter earlier and I thought it was interesting. I particularly liked the above.
 

A.B.

Senior Member
Messages
3,780
I suspect this particular technique would not show up any problems with ME studies. It only looks for falsification in patient's basic measured data at the beginning of the study. It doesn't look at the problems we see in PACE of outcome switching etc.

You never know what they might find. At the very least, in the case of PACE, it would be interesting to see just how much the selected patients deviated from the norm.
 

RogerBlack

Senior Member
Messages
902
You never know what they might find. At the very least, in the case of PACE, it would be interesting to see just how much the selected patients deviated from the norm.

AIUI, it relies on understanding of the statistics.
So if you don't understand the correlates of severity of disease, and the statistics of that versus the normal population, it may be less than useful.
 
Messages
2,087
AIUI, it relies on understanding of the statistics.
So if you don't understand the correlates of severity of disease, and the statistics of that versus the normal population, it may be less than useful.

I am not sure if I have understood it correctly so hoping others might explain it.
But to me he analysed the "randomised" baseline data in the subgroups to see if it was truly random or not. If the baseline data of the randomised subgroups was 'too' normal or if it was not normal enough then this was raised as potential fraud because the chances of the data being randomised was too small.
 
Messages
3,263
I suspect this particular technique would not show up any problems with ME studies. It only looks for falsification in patient's basic measured data at the beginning of the study. It doesn't look at the problems we see in PACE of outcome switching etc.
Yes. There is no evidence the PACE authors falsified anything. All their data checks out. Its just they presented it in a way that was designed to maximise the apparent success of the trial. But that's not the same as falsification.
 
Last edited:
Messages
2,087
Yes. There is no evidence the PACE uathors falsified anything. All their data checks out. Its just they presented it in a way that was deigned to maximise the apparent success of the trial. But that's not the same as falsification.
It's more than likely you are correct but...
When you say all their data checks out - has PACE been analysed using this technique?

Remember, for the trials listed in this report, there was no evidence of fraud before this analysis took place, so prior absence of evidence of fraud, is not a predictor of whether this technique will find anything or not.

I think this technique will find if someone has played with the figures, speculating that it might or might not reveal something in an individual trial ( whatever that trial might be) is kind of pointless. Nobody knows until the analysis is done.

ETA : Do we have enough data to do this on PACE. ?
 

RogerBlack

Senior Member
Messages
902
To sum up the important bits of the blog above it would probably be fair to say 'The technique is interesting, but risks falsely flagging papers as problematic when there are significant correlations between the reported values at baseline'. For example, if height looks a bit unlikely, and gender looks a bit unlikely, you can't simply multiply the likelyhoods together, as an unlikely gender mix is likely to imply an unlikely height mix, as gender and height are strongly correlated.