Nick Brown (good science star): My (current) position on the PACE trial

[alex3619]

They did do something bad in their analyses.

Their view may be its not wrong if its peer accepted, so they do not feel bad about it. If other psychogenic researchers use similar methods, why are we picking on them? The incestuous comment is rather important, though I think its more a form of an informal review circle.

So few people involved, the reviewers are people with a biased interest, more often than not.

So much money involved, so much reputation, so much power. If it all comes crashing down there will be secondary and tertiary consequences.

Psychogenic medicine needs to be thrown out of medicine. Its at best alternative therapy. Historically it started off asking the important questions, but got railroaded primarily by Freud. Its never got back on the scientific track, never embraced modern scientific methods. For example, in terms of treatment efficacy, including cure, not quality of life, it is objective outcome measures that really matter.

Its also important to not just stockpile facts in your favour, and ignore contrary evidence. Not that they do even this very well, its more stockpiling claims, because the data is not fully shown.

 

[Marco]

With the ongoing privatization of the NHS this may no longer hold. More and more will need private health insurance as the public system goes into decline.

Depite the rhetoric I doubt this is the case but even if it was then even more reason not to be wasting scarce resources on a therapy that their own analysis shows has no lasting benefit.

 

[Art Vandelay]

Nick, we are lucky to have you weigh in on this debate at this time especially if you have health issues of your own.

Having worked in the bureaucracy, I can understand Nick's points as to how bureaucratic organisations work. One thing I learnt very early on in my career is that the organisation will never admit in public that it was wrong. Even if a poor, immoral or unethical decision was made many years before by people who had since left the organisation, you were expected to toe the line to avoid a public admittance of guilt or do anything which would cause embarrassment to the organisation. So the explanation as to why they continue to resist sharing their data even though it is unethical and anti-science makes sense.

As for the argument that the PACE team are not evil, I usually subscribe to Hanlon's razor: "Never attribute to malice that which is adequately explained by stupidity."

However, in this case I am not inclined to give them the benefit of the doubt on both points.

After I came down with ME/CFS, I was told by a large number of doctors that the only appropriate and effective treatment was graded exercise. I was naive enough to believe this. Additionally, given that many of these doctors were appointed by my employer, I had to at least make a good attempt with GET or I would lose my job. After six years, GET caused my health to decline permanently to the point where I could no longer work in my chosen profession.

A few years ago, I decided to read the research which claimed that GET was an effective and safe treatment. In short, I was stunned to find that these studies of immensely poor quality. Moreover, I was stunned that no-one apart from patients had seen fit to challenge this research. It also was mind-boggling that the researchers involved were not in the slightest bit interested in sharing their data.

In my former life I was an economist and I would like to think that there is no way that research of such a dismal standard would ever be published in a mainstream economics journal. If such a study had slipped through peer review, it would have been challenged and retracted within months. Also, as I understand it, an economist who refused to share data would be looked upon very dimly by the profession. Perhaps this is because economics is not a science but rather is a constant battle of competing ideas.

As for the research published by the PACE team, my main observation is that all the methodological problems, flaws and errors within the studies are intended to bias the results in one direction -- that is, to favour their favoured treatment approaches. Moreover, the PACE team are guilty of making claims which aren't even substantiated by their own results. If they are not incompetent, then they are guilty of academic fraud.

Their claims of harassment and threats do not seem credible. Requests for data does not constitute 'harassment' and as far as we are aware, not a single person with ME/CFS has been charged for making threats to these researchers. (Incidentally, unfounded claims of harassment and death threats has been used a tactic used by other dodgy researchers who weren't keen on sharing their data.)

Bureaucracies will resist scrutiny and will fight any threat to their continued funding and existence no matter if it is in the public interest. However, the individuals from the Wessley school have made themselves wealthy and have gained power and prestige (knighthoods for example) through their efforts. They have built up formidable taxpayer-funded empires and bureaucracies. Thus they have the most to lose if their school of thought is discredited.

They may not be evil people but their actions have resulted in evil. People with ME/CFS are ridiculed, ignored, discredited and are forced to undergo treatments which are not only a waste of time and money, they are also unsafe and dangerous.

 

[user9876]

But if I didn't want to share my data, I might ask for management help to mount an argument to defend my decision.

But that is where good corporate governance should kick in and management should say you committed to releasing the data, not to do so will harm our brand. Then they should help manage the situation. In some cases that might mean helping anonymize data but in others that may just mean helping with any PR around re-reported results.

I say corporate governance because universities and big businesses these days and many trade on brand to get students (particularly overseas students). It should for example concern Oxford university that their press office spun a null trial result as if it supported two particular treatments.

 

[Daisymay]

Thanks, @Jonathan Edwards, for a clear, well-written, and sympathetic comment. We need more like that from other eminent scientists.

I couldn't agree more, thank you for speaking out professor Edwards, I hope others will follow.

 

[Sean]

As for the research published by the PACE team, my main observation is that all the methodological problems and errors within the studies are intended to bias the results in one direction -- that is, to favour their favoured treatment approaches.

If that is not a huge flashing red warning sign, I don't know what is.

 

[elliepeabody]

Nick, we are lucky to have you weigh in on this debate at this time especially if you have health issues of your own.

Having worked in the bureaucracy, I can understand Nick's points as to how bureaucratic organisations work. One thing I learnt very early on in my career is that the organisation will never admit in public that it was wrong. Even if a poor, immoral or unethical decision was made many years before by people who had since left the organisation, you were expected to toe the line to avoid a public admittance of guilt or do anything which would cause embarrassment to the organisation. So the explanation as to why they continue to resist sharing their data even though it is unethical and anti-science makes sense.......

Bureaucracies will resist scrutiny and will fight any threat to their continued funding and existence no matter if it is in the public interest.

They may not be evil people but their actions have resulted in evil. People with ME/CFS are ridiculed, ignored, discredited and are forced to undergo treatments which are not only a waste of time and money, they are also unsafe and dangerous.

Sadly this is what our Patient / Carer Group have found during our attempts to improve NHS care for PWME in our area. Some, but by no means all, are more interested in their corporate image than caring for patients. In some parts of the NHS patient care seems to be almost an accidental side effect of justifying the existence (and continued funding) of the 'corporation'. There are some very 'good eggs' within the NHS and we do have the framework of the NHS Constitution, and the 2012 Health and Social Care Act the former of which states

‘’Patients must come first in everything the NHS does. All parts of the NHS system should act and collaborate in the interests of patients, always putting patient interest before institutional interest, even when that involves admitting mistakes. As well as working with each other, health service organisations and providers should also involve staff, patients, carers and local communities to ensure they are providing services tailored to local needs.’’

But changing the culture within the NHS is a massive task and resistance significant. But until the culture changes at a structural level scandals like Mid Staffs, Winterbourne View and most recently Southern Health will continue to occur with depressing regularity. The way that the significant and justified concerns of patients and carers have been ignored regarding PACE and regarding ME care in general is symptomatic of the structure and framework within which NHS care is commissioned and delivered.

 

[sTeamTraen]

Lots of interesting reading here, and I did appreciate Jonathan Edwards' comment on my blog. Clearly there is a big story to be written here. I hope I can get Private Eye interested.

I just had a brief part in a very interesting exchange on Twitter (

https://twitter.com/i/web/status/676737147639058432

is one of the tweets in the thread; Neuroskeptic is a fairly influential guy in science blogging circles, by the way.). That thread sums up my position very well: neutral scientists can help with a critique of the methodology of the trial. But if the methodology is sound, the whole deal with science is that the outcomes have to be provisionally accepted, until someone comes along with better evidence. If you want to recruit neutral scientists to your cause, emphasise the methodology of the study. Try to avoid direct criticism of the outcome, even if you can come up with a coherent explanation of why the outcome might suit the researchers. The fact that is suits them doesn't tell us anything about whether it's true or not. It can raise suspicions, certainly, but it can't then become part of the evidence.

For analogous reasons, I would suggest that invoking direct financial gain as an explanation for the behaviour of this or that researcher may not be terribly helpful. Especially in the UK, and whatever one might think of current government policy towards the NHS, it's rare for individuals in medicine or academia to become personally wealthy from the involvement of, say, insurance companies. Indeed, such accusations are usually fairly easily disproved (or at least, very hard to prove), and then tend to rebound on the accusers.

The main financial incentives in this kind of situation are those that apply to anyone with a regular job, namely, /a/ the maintenance of one's monthly salary, and /b/ the (typically relatively small, at least until you get to board level) incremental gains in wealth that accompany the (usually far larger) motives of status and power. And even for the latter, people don't typically seek them out, rubbing their hands together à la C. Montgomery Burns with glee at their evil plans for world domination; instead, they find themselves playing little games and making minor deals, and it's all rather agreeable, and confirms their status as Senior People. Who would question that? And yet, it's that sort of thing that leads to the cover-ups and maintenance of morally untenable positions --- something akin to what Hannah Arendt called "The Banality of Evil" --- rather than some Spectre-like cabal.

 

[adreno]

Thanks for your involvement @sTeamTraen. I think it's clear that the PACE methodology certainly isn't sound. Here are a few examples, taken from Tuller's critique:

*The study included a bizarre paradox: participants’ baseline scores for the two primary outcomes of physical function and fatigue could qualify them simultaneously as disabled enough to get into the trial but already “recovered” on those indicators–even before any treatment. In fact, 13 percent of the study sample was already “recovered” on one of these two measures at the start of the study.

*In the middle of the study, the PACE team published a newsletter for participants that included glowing testimonials from earlier trial subjects about how much the “therapy” and “treatment” helped them. The newsletter also included an article informing participants that the two interventions pioneered by the investigators and being tested for efficacy in the trial, graded exercise therapy and cognitive behavior therapy, had been recommended as treatments by a U.K. government committee “based on the best available evidence.” The newsletter article did not mention that a key PACE investigator was also serving on the U.K. government committee that endorsed the PACE therapies.

*The PACE team changed all the methods outlined in its protocol for assessing the primary outcomes of physical function and fatigue, but did not take necessary steps to demonstrate that the revised methods and findings were robust, such as including sensitivity analyses. The researchers also relaxed all four of the criteria outlined in the protocol for defining “recovery.” They have rejected requests from patients for the findings as originally promised in the protocol as “vexatious.”

*The PACE claims of successful treatment and “recovery” were based solely on subjective outcomes. All the objective measures from the trial—a walking test, a step test, and data on employment and the receipt of financial information—failed to provide any evidence to support such claims. Afterwards, the PACE authors dismissed their own main objective measures as non-objective, irrelevant, or unreliable.

*In seeking informed consent, the PACE authors violated their own protocol, which included an explicit commitment to tell prospective participants about any possible conflicts of interest. The main investigators have had longstanding financial and consulting ties with disability insurance companies, having advised them for years that cognitive behavior therapy and graded exercise therapy could get claimants off benefits and back to work. Yet prospective participants were not told about any insurance industry links and the information was not included on consent forms. The authors did include the information in the “conflicts of interest” sections of the published papers.

 

[alex3619]

Depite the rhetoric I doubt this is the case but even if it was then even more reason not to be wasting scarce resources on a therapy that their own analysis shows has no lasting benefit.

Actually this is what I used to think, once upon a time. It is not the case. There are no great ME therapies that are approved. By claiming ME and CFS are psychiatric, then they can deny the need for tests. This benefits both insurance companies and health bureaucracy.

With health insurance companies, plus disability insurance, there is often a cap of two years for psychiatric conditions. So they limit payments to two years.

Combining the two years with limited testing and a cheap therapy (even though it does not work) is even cheaper. By the time you get to the two year mark you will have run out of options. Game over. Money saved.

The idea that these organizations always operate in patient's best interests is just not tenable. The whole insurance industry push to psychologize ME and CFS was initiated by them because they were losing a lot of money. (This information is publicly available in their own publications.) Government funded PACE (including DWP) and there is more a cultural and social reaction to this kind of thinking, rather than a rational or economic one.

The economic cost argument is only valid if we can discredit CBT/GET at this time. Even then it would not change insurance industry policy, just government policy, and not by much.

The other thing that will be a game changer is a relatively cheap and fast therapy for ME or CFS that is effective. We are not there yet. Rituximab might also turn out to be too expensive for the insurance industry to change their policy.

 

[worldbackwards]

I hope I can get Private Eye interested.

Unlikely. Dr. Phil Hammond is not a friend of ours.

 

[jimells]

For analogous reasons, I would suggest that invoking direct financial gain as an explanation for the behaviour of this or that researcher may not be terribly helpful.

Then why bother with conflicts of interest statements? There seems to be a general perception that if someone lists their conflicts, then somehow they don't matter, as if listing them "proves" that the researcher is honest. Honesty and integrity are central issues of this research - can one really separate the politics from the methodology when evidence is "sexed up" to support pre-determined conclusions?

Even the funding of the research is suspect: it has been reported that PACE is the first and only study funded by the Dept for Work and Pensions. Is anyone unsure of the outcome that they would like to purchase?

What is really needed here is a proper investigation as recommended six years ago by the UK Parliament Group on Scientific Research into ME (the Gibson Inquiry):

"CFS/ME is defined as a psychosocial illness by the Department for Work and Pensions (DWP) and medical insurance companies. Therefore claimants are not entitled to the higher level of benefit payments. We recognise that if CFS/ME remains as one illness and/or both remain defined as psychosocial then it would be in the financial interest of both the DWP and the medical insurance companies."

The Group called for investigation of what they called, "numerous cases where advisors to the DWP have also had consultancy roles in medical insurance companies. Particularly the Company UNUMProvident. Given the vested interest private medical insurance companies have in ensuring CFS/ME remain classified as a psychosocial illness there is blatant conflict of interest here."

https://en.wikipedia.org/wiki/Controversies_related_to_chronic_fatigue_syndrome#Political

I certainly understand that researchers will want to concentrate on research methodology and issues of scientific fraud, being their area of expertise. That's why we also need the involvement of troublemakers to expose the political fraud and financial corruption. I'm pretty sure both avenues of attack were important in cutting the tobacco industry down to size, for example.

 

[jimells]

Rituximab might also turn out to be too expensive for the insurance industry to change their policy.

This would create an interesting puzzle to pick apart. In the US, the "regular" health insurers have very different priorities than the disability insurers. The disability insurers may see that effective expensive treatment means they pay fewer claims. How will insurers that actually pay treatment claims view short-term treatment expense vs. the expense of long-term ill health? I don't think we know. And then there is the influence of the re-insurers. I understand corporations like Swiss Re end up holding the bag for disability claims, but I don't know how they are related to the medical insurance companies.

An examination of how various insurers and re-insurers have responded to the hugely expensive Hepatitis C treatments could give us a clue about what to expect.

 

[worldbackwards]

The main financial incentives in this kind of situation are those that apply to anyone with a regular job, namely, /a/ the maintenance of one's monthly salary, and /b/ the (typically relatively small, at least until you get to board level) incremental gains in wealth that accompany the (usually far larger) motives of status and power. And even for the latter, people don't typically seek them out, rubbing their hands together à la C. Montgomery Burns with glee at their evil plans for world domination; instead, they find themselves playing little games and making minor deals, and it's all rather agreeable, and confirms their status as Senior People. Who would question that? And yet, it's that sort of thing that leads to the cover-ups and maintenance of morally untenable positions --- something akin to what Hannah Arendt called "The Banality of Evil" --- rather than some Spectre-like cabal.

I've long since thought that, were the whole story to come out (which I very much doubt) most of it would be far more mundane than we can imagine but there'd be at least one bit where everyone spat their cornflakes out. None of us know what the truth is here, but what is certainly true is that anything sounding like a conspiracy theory (whether true or false) makes outsiders switch off - the more extravagant it sounds, the less likely it is to be believed and to make others to want to help us.

I think that this is a battle that needs to be won on the science. There is unlikely to be a big finale to this story, with James Coyne whipping off Peter White's mask as he exclaims "And I'd have got away with it too if it wasn't for those meddling kids at Phoenix Rising". Just a moment where their position is shown to be untenable on the science of PACE and they retreat into the background, hopefully with the scorn of their peers ringing in their ears, (though more likely with their colleague whispering "damned unruly patients" and reassuring them about their tenure).

It's a shame really. Proper evil is always more reassuring, it's give you a sense of order about things. Part of me wishes that I really thought there was a monthly meeting chaired by Simon Wessely, where they all sit around a big table and plotted how to make our lives a misery, after which they all put on leather hoods and Esther Crawley leads off on "the ritual".

 

[snowathlete]

Then why bother with conflicts of interest statements? There seems to be a general perception that if someone lists their conflicts, then somehow they don't matter, as if listing them "proves" that the researcher is honest. Honesty and integrity are central issues of this research - can one really separate the politics from the methodology when evidence is "sexed up" to support pre-determined conclusions?

Even the funding of the research is suspect: it has been reported that PACE is the first and only study funded by the Dept for Work and Pensions. Is anyone unsure of the outcome that they would like to purchase?

What is really needed here is a proper investigation as recommended six years ago by the UK Parliament Group on Scientific Research into ME (the Gibson Inquiry):



I certainly understand that researchers will want to concentrate on research methodology and issues of scientific fraud, being their area of expertise. That's why we also need the involvement of troublemakers to expose the political fraud and financial corruption. I'm pretty sure both avenues of attack were important in cutting the tobacco industry down to size, for example.

I think this speaks of a wider problem in science, that most of the science community (and politics and elsewhere) appear totally accepting of conflicts of interest, and as a proportional part of that, corruption. There isn't an interest in cleaning it up because everyone is doing it (the conflicts of interest. Not all are corrupt). Even if you get some that acknowledge it, it takes a great many before anything actually changes. In the case above, we didnt get that investigation. With that being the environment, the pragmatic approach might be as Nick suggests.

But Ido think that is a fault with how things are done as I never think you should divorce human nature from anything, science or whatever. People are conflicted and often their decisions favour themselves over truth (science). A conflict of interest does not automatically mean that the research is dodgy, but it is reason for greater scrutiny of the work. In the case of PACE, until recently, no one, other than the patients affected did any scrutiniy of the work at all. Worse, it was held up as a beacon of good science originally. For science to progress more efficiently that has to change, but may not be realistic given the human nature I am talking of. Sadly, until you are at the wrong end of the stick, vague corruption of this kind doesn't seem all that evil, but when it impacts your life directly in a serious way you see it for what it is. I think it is a serious social failing.

 

[Isabelle]

Animal researchers get death threats as a matter of course. I know some personally. You don't see them whining about it in the media. Because that would escalate the situation. People in real danger from activists generally like to keep a low profile.

A colleague of mine once received a death threat. Animal rights activists mistakenly assumed her research involved animal experimentation. There was a police investigation and some commotion around the water cooler. To my knowledge she never prefaced future presentations by discussing the incident, never used it to silence debate, never milked it in the media, never had a panic button installed, never warned other scientists against biomedical research in her area, never blamed animal lovers or even all members of animal liberation. She just kept a low profile and got on with her job.

I agree with worldbackwards that the battle needs to be won on the science. PWME want independent reanalysis of the PACE trial data. QMUL, KCL etc. want the scientific community to look the other way.

 

[sTeamTraen]

I've long since thought that, were the whole story to come out (which I very much doubt) most of it would be far more mundane than we can imagine but there'd be at least one bit where everyone spat their cornflakes out. None of us know what the truth is here, but what is certainly true is that anything sounding like a conspiracy theory (whether true or false) makes outsiders switch off - the more extravagant it sounds, the less likely it is to be believed and to make others to want to help us.

This is a good summary of how I feel, including the cornflakes.

Unlikely. Dr. Phil Hammond is not a friend of ours.

Hmmm. I presumed he would have a position on this (most doctors do), but I didn't know what it was. And if it ends up going to the Medicine Balls desk, it will certainly be him deciding. On the other hand, if it goes to the In The Back team, who like stories about injustice, who knows.

That said, Hammond was the guy who got Private Eye to stop supporting Andrew Wakefield's terrible science, which was not their finest hour (I know a scientist who stopped subscribing over that and never returned).

 

[worldbackwards]

Hmmm. I presumed he would have a position on this (most doctors do), but I didn't know what it was. And if it ends up going to the Medicine Balls desk, it will certainly be him deciding. On the other hand, if it goes to the In The Back team, who like stories about injustice, who knows.

That said, Hammond was the guy who got Private Eye to stop supporting Andrew Wakefield's terrible science, which was not their finest hour (I know a scientist who stopped subscribing over that and never returned).

Hammond has more than a position on this, he has a direct interest. He works with kids with CFS under Esther Crawley, who is as big a name in CBT/GET as the trial authors. I don't know how Private Eye works but I suspect that if he got wind of this story getting anywhere near it he'd chuck a fit.

 

[picante]

Emerson sums up the problem: