PACE trial: fraud or incompetence?

[alex3619]

Oh I was thinking we had already established that, but perhaps you mean that we can establish it further?

Are you saying that it's too early to put a case together to put forward to the GMC? Because we are still trying to establish the facts and the data is still being scrutinized?

We have established it to some degree, though not certainty (for me I would need to read the 2007 paper for certainty) but if we want to establish it in a formal sense in the UK then we have to convince enough of the relevant parties to make a difference. It will not be easy.

Currently we cannot provide enough evidence for any kind of criminal fraud.

It does appear we can demonstrate scientific misconduct. I say appear because we need the facts of the 2007 paper to be firmly and publicly established. What is that paper?

As for the rest we know this is very bad science. In all likelihood we might have to settle for that, and in any case once false belief CBT and GET are thoroughly discredited in the view of most doctors we might even find that our advocacy efforts are better placed pushing for research and making effective treatments widely available.

 

[SilverbladeTE]

To restate this in a somewhat different manner - it becomes a crime when it is legally a crime, not when we would wish so.

I note that Andrew Wakefield, despite having his key paper retracted for fraud, despite doing procedures on children for which he had no authority has only suffered loss of his medical licence and no criminal prosecution, despite, reasonably arguably causing many thousands of deaths.

Unfortunately prosecution for harms to patients they did not personally cause are almost impossible due to the multiple layers between them and the patient.
Prosecution for patients directly harmed by PACE is slightly less unlikely.

Prosecution for direct harms would require unearthing of some statement (or him coming out and saying) that he intentionally harmed people with CFS/ME because he hates them.
Prosecution for indirect harms would require unearthing a plan to intentionally harm patients through altering regulations and policies in ways he knows would be harmful.

Being negligent as to possible harm might barely possibly be enough for direct harms, but indirect ones are just too far removed from the trial.

I use 'he' here, when the additional problem with criminal action would be that there are many authors of the PACE trial, and for the most serious charges, there would need to be one or several people knowing that it was a fraud and intending to hurt patients with no reason, and acting to convince the others who would not have gone ahead otherwise.

Knowing that some patients might be made worse, in the belief that you're doing the whole group a favour is medicine as usual.

However, the fact that many of those involved, not just in PACE, but the ones we often refer to as "The Wessely School" or such, have strong links with health insurance companies, government bodies in charge of welfare etc, gives a predisposition for unethical influence.
That and certain behaviours, statements etc over the years move it into "contempt for patients and their suffering".


and the blinkered zealotry of many a murderous, tyrannical, horrendous bastard throughout the centuries, even in medicine, does not excuse in anyway their crimes.
Limits of resources and capabilities, or being given the wrong information, are real excuses for doctors to be unable to help people and that alas, is life.
However, zealotry, or worse, sadism or financial/social benefits from abusing victims, are not.

 

[Laelia]

Currently we cannot provide enough evidence for any kind of criminal fraud.

It does appear we can demonstrate scientific misconduct. I say appear because we need the facts of the 2007 paper to be firmly and publicly established

Thank you for clarifying these thing.

What is that paper?

Yes indeed, what is that paper? Please can somebody help Alex in locating this crucial document? (!)

in any case once false belief CBT and GET are thoroughly discredited in the view of most doctors

I fear that we might still have a lot of work to do before we reach this point

we might even find that our advocacy efforts are better placed pushing for research and making effective treatments widely available.

Yes, it will always be case of considering where our advocacy efforts are best placed.

Thank you very much Alex for taking the time to answers all my questions. I'm sure other people on here have found your answers helpful, not just me!

 

[Laelia]

The right way to fix this is not to try to prosecute people after the fact for the narrow gap between reasonable error and unreasonable error, but:
Ensure that peer review looks at all of the paper, not just the bits the reviewers happen to pick up on.
Look at the statistics to see they are robust - without implicit assumptions that SF36 is linear, or misstatements about the population levels.
Examine trial protocols and adherence to them beforehand for risk of bias.
Ensure pre-specified analysis are done, and put up front in the abstract, before any post-hoc stats-hacking.
Check any measures claimed are reasonable for the disease state, and only accept surrogates unwillingly. (for example, claiming your patients are fully recovered, when they are still bedbound because you've convinced them that they're better, rather than looking at employment and other measures).

I agree that all these measures need to be taken during the peer review process and that the whole process needs to be made more robust. Are there an organisations campaigning for better science practices does anyone know? If so, we should really be supporting them.

But even so, who would enforce these measures? And what happens when they inevitably fail? The only way I can see these problems being solved is if there are consequences to not "playing by the rules".

 

[RogerBlack]

I agree that all these measures need to be taken during the peer review process and that the whole process needs to be made more robust. Are there an organisations campaigning for better science practices does anyone know? If so, we should really be supporting them.

But even so, who would enforce these measures? And what happens when they inevitably fail? The only way I can see these problems being solved is if there are consequences to not "playing by the rules".

Most of the problems with PACE were due to institutions not following their policies.
The published policies of the institutions involved were in general adequate to cover this.

In some ways it's quite irrelevant if PACE authors were careless, honest and mislead or actively fraudulent. I do not believe there is any possibility 'scientific misconduct' would have had any relevance here.
The right solution is not to punish people for misconduct (in border line cases), but to refuse to publish their work, so that their funders and institutions know and they get sidelined or leave the field.

Punishment for misconduct requires a level of proof which is entirely inappropriate for this.

The flaws in the paper were evident on a casual read.
The protocol changes were obviously unjustified and should not have been approved by the institution.

https://en.wikipedia.org/wiki/Scholarly_peer_review#Open_peer_review
http://rsos.royalsocietypublishing.org/open-peer-review

Traditional peer review is a relatively private affair, with the referee reports being seen by only a handful of individuals and the name of the referee being seen by even fewer. The move towards transparency in scientific publishing has lead to a number of journals adopting a more open model.

On Royal Society Open Science we operate the following model. At all points in the peer review process we will encourage referees to sign their reports, thereby disclosing their name to the author. Where authors agree, we will make the editorial process transparent by publishing referee reports, the substantive part of the decision letter after review and the associated author responses, alongside published articles. This will allow readers to better assess the published paper and provide post-publication comments (which will also be published).

Do I believe that the authors of PACE should be punished in various ways - absolutely.
Retraction of PACE and related studies, lack of further funding, and being professionally disciplined.

Some sort of nebulous 'scientific misconduct' sanction would not have stopped this in the least, as they'd never have believed it would be an issue.

'Our science is the best science. It's gonna be yuge'.

 

[Laelia]

Most of the problems with PACE were due to institutions not following their policies.

So do you think it's the institutions that need to be punished in that case?

The published policies of the institutions involved were in general adequate to cover this.

This is good to hear!

The right solution is not to punish people for misconduct (in border line cases), but to refuse to publish their work, so that their funders and institutions know and they get sidelined or leave the field.

But this not happening. Journals are not refusing to publish their work.

Do I believe that the authors of PACE should be punished in various ways - absolutely.
Retraction of PACE and related studies, lack of further funding, and being professionally disciplined.

This would be a good start

Some sort of nebulous 'scientific misconduct' sanction would not have stopped this in the least, as they'd never have believed it would be an issue.

I don't know about that. I'm not sure there is such a thing as 'nebulous' scientific misconduct. Surely it either is scientific misconduct or it isn't. It seems that we have yet to establish whether there was any misconduct in this case.

There's no reason why they should not understand the issues, the issues are quite simple. If they knew that there would be serious consequences to their actions then it seems likely to me that none of this would have ever happened.

[Edit: 2 typos]

 

[RogerBlack]

I don't know about that. I'm not sure there is such a thing as 'nebulous' scientific misconduct. Surely it is either scientific misconduct or it isn't. It seems that we have yet to establish whether there was any in this case.

It's really not.
There is a wide spectrum:
Completely fabricating data because you hate a specific community and are maliciously and knowingly doing so;
Fabricating data to further your career;
Knowingly using real data in such a manner as to mislead people for gain;
Being sloppy and violating best practices while believing strongly in your idea while taking the fact you can get it published as proof of that idea;
Unintentional errors due to carelessness;
Unintentional errors that could in principle have been caught, but to do so would involve things nobody in the field does.


'Serious consequences' only simply apply to the first two or three.
Trying to make them go further than that is really problematic.

 

[trishrhymes]

Knowingly using real data in such a manner as to mislead people for gain;
Being sloppy and violating best practices while believing strongly in your idea while taking the fact you can get it published as proof of that idea;

Exactly what they did.
Plus a whole lot more.

 

[Laelia]

It's really not.
There is a wide spectrum:
Completely fabricating data because you hate a specific community and are maliciously and knowingly doing so;
Fabricating data to further your career;
Knowingly using real data in such a manner as to mislead people for gain;
Being sloppy and violating best practices while believing strongly in your idea while taking the fact you can get it published as proof of that idea;
Unintentional errors due to carelessness;
Unintentional errors that could in principle have been caught, but to do so would involve things nobody in the field does.

I'm going by the Medical Research Council (MRC) definition of scientific misconduct also posted earlier on in this thread.

"The Medical Research Council (MRC) definition of misconduct and fraud (or a variation of the MRC code) is widely used. This code states the following definition:

The fabrication, falsification, plagiarism or deception in proposing, carrying out or reporting results of research or deliberate, dangerous or negligent deviations from accepted practices in carrying out research. It includes failure to follow established protocols if this failure results in unreasonable risk or harm to humans, other vertebrates or the environment and facilitating of misconduct in research by collusion in, or concealment of, such actions by others. It also includes intentional, unauthorised use, disclosure or removal of, or damage to, research-related property of another, including apparatus, materials, writings or devices used in or produced by the conduct of research. It does not include honest error or honest differences in the design, execution, interpretation or judgement in evaluating research methods or results or misconduct unrelated to the research process. Similarly it does not include poor research unless this encompasses the “intention to deceive” (MRC, 1997).[4]"

According to this definition, some of the things you list are considered scientific misconduct, others are not.

[Edit: typo]

 

[Laelia]

Here is David Tuller explaining why he thinks the PACE trial is a fraud (from Trial By Error 22 September 2016):

"‘The PACE trial is a fraud.’ Ever since Virology Blog posted my 14,000-essord investigation of the PACE trial last October, I’ve wanted to write that sentence. (I should point out that Dr. Racaniello has already called the PACE trial a “sham,” and I’ve already referred to it as “doggie-poo.” I’m not sure that “fraud” is any worse. Whatever word you use, the trial stinks.)

Let me be clear: I don’t mean “fraud” in the legal sense—I’m not a lawyer–but in the sense that it’s a deceptive and morally bankrupt piece of research. The investigators made dramatic changes from the methodology they outlined in their protocol, which allowed them to report purported “results” that were much, much better than those they would have been able to claim under their originally planned methods. Then they reported only the better-looking “results,” with no sensitivity analyses to analyze the impact of the changes—the standard statistical approach in such circumstances.

This is simply not allowed in science. It means the reported benefits for cognitive behavior therapy and graded exercise therapy were largely illusory–an artifact of the huge shifts in outcome assessments the authors introduced mid-trial."

http://www.virology.ws/2016/09/22/trial-by-error-continued-the-real-data/

 

[Laelia]

I'm not sure there is such a thing as 'nebulous' scientific misconduct. Surely it either is scientific misconduct or it isn't.

I'm going to take back what I said here @RogerBlack. The definition of what constitutes scientific misconduct does appear to be open to interpretation in some areas. My apologies!

However, I think it's premature to be saying that any misconduct that occured in the PACE trial lies within the nebulous category. Judging by what David Tuller said about the PACE trial ("it's a deceptive and morally bankrupt piece of research"), he does not appear to think that this is the case.

It seems we have yet to reach a consensus on this. And don't forget that there may be missing pieces of evidence that have yet to come out that might paint a completely different picture.

 

[RogerBlack]

However, I think it's premature to be saying that any misconduct that occured in the PACE trial lies within the nebulous category. Judging by what David Tuller said about the PACE trial ("it's a deceptive and morally bankrupt piece of research"), he does not appear to think that this is the case.

It can (as a whole) be a deceptive and morally bankrupt piece of research and not have done anything illegal at the same time.

 

[Laelia]

It can (as a whole) be a deceptive and morally bankrupt piece of research and not have done anything illegal at the same time.

Yes, that is correct. But I don't believe it can be a deceptive and morally bankrupt piece of research and not be scientific misconduct.

 

[Sean]

From Mark Vink's response to White et al's response:

The biopsychosocial model however is at odds with the abnormal physiological findings in ME yet an important part of the model is ignoring evidence even if they have produced it themselves. For example, they disproved deconditioning in 2001 and fear avoidance behaviour in 2005 thereby disproving their own biopsychosocial model, professor White co-authored this study just like a 2004 study which found that exercise caused immunological abnormalities in ME/CFS but not in healthy controls and these were still present in the blood 3 days later.

Deliberately ignoring your own previous findings, and those from others (e.g. the Wiborg review on the critical lack of correlation between self-reported and actometer measured physical activity levels, etc), just because they contradict your own model, is dangerously close to fraud by any practical, commonly understood sense of the word.

 

[RogerBlack]

is dangerously close to fraud by any practical, commonly understood sense of the word.

Not, alas the 'we can lock you up' commonly understood sense.

 

[Laelia]

So was David Tuller correct to describe the PACE trial as "a deceptive and morally bankrupt piece of research"? Did he have enough evidence to make this statement? If so, what were the crucial pieces of evidence that lead him to this conclusion?

It would be enormously helpful if we can establish the answers to these key questions.

[Edit: typo]

 

[RogerBlack]

So was David Tuller correct to describe the PACE trial as "a deceptive and morally bankrupt piece of research"? Did he have enough evidence to make this statement? If so, what were the crucial piece of evidences that lead him to this conclusion?

It would be enormously helpful if we can establish the answers to these key questions.

I think it's safe to say there was nothing private about it now, or that he needs to answer. (though of course, he helped enormously to reveal and publicise some of these issues)

It's simply everything we've come to understand about the trial:
Initial moderately dodgy design;
problems with the trial that seem almost designed to drive placebo effects (leaflets saying 'CBT has helped lots');
Problems with the other arms than CBT being able to discuss things with the patient that the CBT had to by protocol ignore;
Outcome switching and making the bad trial design worse, apparently after seeing some results (though they did this with permission);
Bad reporting (using linear interpretations for non-linear scales), misrepresenting what 'recovered' meant when that meaning is not what anyone reading the paper would understand, ...;

For me, the real moral bankruptcy kicks in after the trial. Before is just 'bad' science done more or less as usual. Refusing to look critically at problems claimed with the trial, using 'eminence based' not 'evidence based' refutations of criticism, demonising the patient community publically, propaganda around the trial (SMC), misleading, factually incorrect and dangerous interviews on the basis that 'because we got it published it must be right'. Pushing for more CBT//CFS trials, using the bald assertion that CBT is positive, never mind the criticism, failed objective improvement or any other things.
And that's without mentioning the fact that a large number of BPS people have gained influence and are on editorial boards, or have been funded by agencies with an apparent interest in making CFS 'treatable' by CBT.

 

[Esther12]

So was David Tuller correct to describe the PACE trial as "a deceptive and morally bankrupt piece of research"? Did he have enough evidence to make this statement? If so, what were the crucial piece of evidences that lead him to this conclusion?

It would be enormously helpful if we can establish the answers to these key questions.

I think that claim is justifiable, but that in most contexts it would be unwise for a patient to jump to making those claims, partly because there is no one crucial piece of evidence that can be easily pointed to. It's lots of different things and problems.

 

[alex3619]

Let me say it another way, if we can find the 2007 PDW paper, establish its contents are what David Tuller says they are in a public arena, then we have proof that the use of a biased and mathematically flawed normal calculation from SF36-PF data was intentional. Its the only evidence that goes directly to intent. If we can do that then all the other evidence becomes secondary with respect to intent and scientific misconduct. Indeed, how that evidence is viewed may well be different once we know, beyond doubt, that the inappropriate calculation of normal was intentional. Until that is established the bad science defense might well work.

 

[RogerBlack]

then we have proof that the use of a biased and mathematically flawed normal calculation from SF36-PF data was intentional.

Err - how does that work.
It's clearly wrong - but how do you come to the conclusion that it proves intent.