• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

‘Academic freedom’ overrides individual concerns in PACE Trial tribunal

Bob

Senior Member
Messages
16,455
Location
England (south coast)
The previous Cochrane reviews did have occasionally frank conclusions and interesting commentary.
Perhaps I didn't read the entire commentary, and my memory is useless anyway, but I seem to remember thinking that any criticisms of methodology weren't prominent or bold enough.
 

biophile

Places I'd rather be.
Messages
8,977
Perhaps I didn't read the entire commentary, and my memory is useless anyway, but I seem to remember thinking that any criticisms of methodology weren't prominent or bold enough.

Yes, well, I would not call it excellent e.g. the 2008 version, but it was sometimes refreshing from the usual spin and raised interesting points, such as the lack of effect for physical function and other measures, lack of evidence for sustained effect in general, and that the effect size of CBT disappeared when compared to a waiting list control group, suggesting a placebo response at work. The 2000 version also raised the possibly that the Lloyd study did not find an effect for CBT because it was combined with placebo medication. You rarely find this sort of discussion in pro-CBT papers.

It will be interesting to see if the next Cochrane review considers the sobering implications of data from actometers, 6MWD, welfare etc. The discrepancies between subjective and objective measures should be explored.

I have occasionally donated money to biomedical projects. I would be willing to donate money to a research group (e.g. Jason et al) who is able to professionally access and re-analyze the PACE Trial data according to the goalposts set out in the original protocol, and have the results and commentary published in an open-access journal. CBT/GET would probably still be found to have an advantage for broadly-defined CFS, but at least we would have more realistic figures to work with.
 

user9876

Senior Member
Messages
4,556
Simon, wasn't at least part of the appeal asking for minutes of meetings and not therefore data connected to this research? I mean such information would not readily become part of any 'full release of all data' or what the AllTrial campaign is asking for - is it? There must be some protection for academics and scientists: or do you think everything they say - and who says it regardless of who they are in a meeting - should be made publicly available? Thanks and nice post :)

I think that the request was to get the trial minutes to understand the reasons for protocol changes. I read from one european agency document that reasons should be well explained within trial reports which I don't think they were in PACE.

I would also argue that this is important data for interpreting the published results. Data includes unstructured information as well as the raw numbers.
 

user9876

Senior Member
Messages
4,556
Maybe wasn't clear: I was only commenting on that specific part of the ruling (quoted above) as it might be relevant to future FOI requests, and does imply you can trust researchers to release the data that is relevant. FOI rulings already provide quite a bit of protection from requests for blanket disclosure of discussions, and I don't have a problem with that.

I think there is a difference between research discussions and what I would see as a formal committee structure for reviewing evidence, suggestions and decisions. I would see this particularly for publicly funded projects as information that should be public.
 

user9876

Senior Member
Messages
4,556
What exactly is the Norway Cochrane re-analysis of the PACE data supposed to achieve?

[Edit: I am not sure if this is just the next update or a special edition just for PACE.]

Unless a major error is uncovered or the re-analysis is going to use different thresholds (such as the ones set out in the original protocol), then merely redoing the same analysis is just going to confirm the previous one, adding nothing to the debate except another round of back-patting from the ivory towers.

Is it supposed to demonstrate that PACE did not fake data or that they are not incompetent?

A faker would have already cleverly faked the data and then just sent that over. I do not believe the data was faked, and I doubt anyone seriously believes it was. What critics are really objecting to is the post-hoc protocol changes, the inadequate justification and lack of details, and possibly the timing and potential COI. On the issue of incompetence, the threshold for normal physical function certainly reflects poorly enough on the researchers and professionals involved.

Objections have also been raised which have nothing to do with the data itself, such as the nature of the improvements, the interpretation of the results, and the generalizability of the cohort. The Norway Cochrane re-analysis of PACE data will probably function as a good publicity stunt while failing to address the key criticisms.

Reanalysing the data using the same algorithms would only check their QA processes. I'm not convinced that they have those so errors could be found. I would also expect a few transcription errors on the original patient forms unless they have a good data entry QA process. But I don't think any errors here would be significant.
 

user9876

Senior Member
Messages
4,556
I'm just reading through at the moment. QMUL seem to make a big point about trying to protect junior researchers opinions in a committee. I had assumed that junior researchers wouldn't be on the trial steering committee and the ethics committee but it would include some indepentant people and the PIs. However, I don't know who was on the PACE committees.

Another worry is that where there are discussions around reasons to change protocols these should be reported otherwise the readers of a paper aren't being given the full view and reasons for changes.
 

user9876

Senior Member
Messages
4,556
I am concerned about the claim they make that it is normal to change the trial protocol as I'm surprised that it is normal to change it as much as they did. In the transcript it claims that White gave a 'clear explanation of such changes' its a pitty he didn't do that in the PACE papers and where he did give explanations at least one was just wrong. His witness statement as reported in the transcript says that such changes are normal but it would be interesting to know what percentage of trials change so many primary outcomes.

I have read some proposals suggesting the analysis code should be prewritten prior to the trial suggesting that this isn't a good thing.
 

Dolphin

Senior Member
Messages
17,567
I have occasionally donated money to biomedical projects. I would be willing to donate money to a research group (e.g. Jason et al) who is able to professionally access and re-analyze the PACE Trial data according to the goalposts set out in the original protocol, and have the results and commentary published in an open-access journal. CBT/GET would probably still be found to have an advantage for broadly-defined CFS, but at least we would have more realistic figures to work with.

I'd be willing to donate also.

Peter White says he won't give the data to patients; however, I think he would have difficulty not giving the data to another research group/institution given "All Trials" and similar initiatives in that direction, that this was a publicly funded trial (with substantial funding in fact), etc.
 

Snow Leopard

Hibernating
Messages
5,902
Location
South Australia
I don't doubt they changed their plan before they had seen their results, but the motivation is still the same - to exaggerate their results as much as possible, because they knew that their results wouldn't have looked as impressive if presented in the way of the original protocol.
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
I don't doubt they changed their plan before they had seen their results, but the motivation is still the same - to exaggerate their results as much as possible, because they knew that their results wouldn't have looked as impressive if presented in the way of the original protocol.

I see it as cherry picking what data to present, what data to gather and avoid gathering, and how that data is analyzed and presented. There is bias here, but I doubt they think of it as exaggerating. Its equally disturbing that there is no real discussion of the major flaws and issues: they just use a throwaway line to dismiss anything they do not like or that contradicts them.

Vagueness, and redefinition of words, and blurring of definitions: stock in trade in this research.

I honestly do not know if getting the minutes of any meetings would help us much. I think the problems are deeper than would show up in any such meeting, though I would love to have a recording of the meetings where researchers got together to debate how to counter criticism, especially well founded and substantiated criticism.

Cochrane reviews have the purpose of amalgamating data to increase sample size, thereby reducing error from sampling issues or chance variation. Inherent methodological bias is not addressed. It would be disturbing however if they did not recognize the improper statistical analyses used in PACE, especially with respect to standard deviations from the mean.
 

heapsreal

iherb 10% discount code OPA989,
Messages
10,089
Location
australia (brisbane)
I wish they would answer lady marrs questions. Even when they say people have improved on cbt/get they are still disabled. How do they explain that. They have even said that it cant hurt now even if this is the case, isnt it a waste of time. Money and effort. Definately cherry picking8ng, spin etc to protect themselves. Even the harassment stuff is a smoke screen to avoid answering lady marr's questions.
 

user9876

Senior Member
Messages
4,556
I'd be willing to donate also.

Peter White says he won't give the data to patients; however, I think he would have difficulty not giving the data to another research group/institution given "All Trials" and similar initiatives in that direction, that this was a publicly funded trial (with substantial funding in fact), etc.


Rulings like this one should really worry the All Trials campaign. I'm not sure I quite got what was being said in the conclusion about academic freedom but I take it as the academic should be allowed to choose what results to publish through a processess of peer review. I persume that this ruling goes to form case law and can be used for other studies.

The judge seemed to put a huge amount of trust in what White was saying and the correct functioning of the academic process whilst being extremely dismissive of the person bringing the case. This opinion seems to contradict an earlier one on deteriation rates. My underlying reading is that the judge doesn't understand science and research. Interestingly an economist Dixit (in Lawless Economics) makes a comment about companies not liking to use courts as the results can be somewhat random due to the lack of knowledge of judges.
 

user9876

Senior Member
Messages
4,556
I don't doubt they changed their plan before they had seen their results, but the motivation is still the same - to exaggerate their results as much as possible, because they knew that their results wouldn't have looked as impressive if presented in the way of the original protocol.

Since they have yet to publish their statistical analysis plan and it has taken them so long to publish results I am more suspicious. I was particularly amused by Whites comments around complex statistical analysis when they are running a few scripts in a stats program. In alot of papers they are just quoting mean and SD which is not complex.
 
Messages
13,774
The judge seemed to put a huge amount of trust in what White was saying and the correct functioning of the academic process whilst being extremely dismissive of the person bringing the case. This opinion seems to contradict an earlier one on deteriation rates. My underlying reading is that the judge doesn't understand science and research.

The judge previously worked for the BMJ, and on a DWP ethics committee. It seems that he has a particular sort of understanding.
 

Dolphin

Senior Member
Messages
17,567
I think it could be easy to think after reading this judgement that in hindsight it might have been better not to ask for the minutes. But hindsight is perfect 20/20 vision; I think any discussion on the merits or otherwise of doing it shouldn't be based too much on the knowledge of how they have reacted.
 
Messages
13,774
This is a judgement that White can use as propaganda for anyone foolish enough to trust it. The judgement itself is so lacking in supporting evidence for the claims made about the motivations of the FOI, that presumably some people would see it as evidence of CFS patients being mistreated.

The FOI request for these minutes seems perfectly reasonable to me, particularly considering earlier minutes from the steering committee had been released, and have been helpful.
 
Messages
13,774
It's worth noting that the Judge did not find any inaccurate claims from the appellant.

Also, this was the closest they came to any evidence of extremism in his submissions:

18. In his appeal to the tribunal Mr Mitchell argued that disclosure would increase confidence in the appropriateness of the decisions made... He was markedly suspicious in his approach, and cited the leader of an American CFS group stating: -

"while the authors state that the funding agencies had no role in the study design will conduct, it is difficult to ignore the UK government's strong stake in a good outcome. The study was funded by the country's Medical Research Council Department of Health and Department for Work and Pensions. It was conducted for the benefit of making or revising health policy of the treatment of CFS by the National Health Service. It came at the cost of some £5 million (British) pounds or $8 million (US). In essence, it was too big to fail to reinforce existing UK policy that favours the provision of psychological approaches over medical ones."


So he quoted someone else expressing concerns, as evidence that many patients and patient groups do not have confidence in the decisions that were made about how to present data from PACE, and that there is a public good argument for the release of more information. He can clearly be dismissed as a militant with no interest in the truth!

Really, it seems that White and QMUL were just trying to respond to allegations which had not been made, rather than dealing with the real and legitimate concerns about the way in which data was presented. They decided to assume the appellant was implying impossible collusion:

None of the appellant's implied collusion of government, funders or researchers took place, or was possible.

This seems to be a common theme, with the PACE researchers keen to answer questions no-one has asked, and defend themselves from criticisms no-one has made, while doing all they can to avoid dealing with the real and legitimate concerns of patients.