• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Trial by Error continued is PACE a case of research misconduct

First four paragraphs as a teaser...
I have tip-toed around the question of research misconduct since I started my PACE investigation. In my long Virology Blog series in October 2015, I decided to document the trial’s extensive list of flaws—or as many as I could fit into 15,000 words, which wasn’t all of them—without arguing that this constituted research misconduct. My goal was simply to make the strongest possible case that this was very bad science and that the evidence did not support the claims that cognitive behavior therapy and graded exercise therapy were effective treatments for the illness.

Since then, I have referred to PACE as “utter nonsense,” “complete bullshit,” “a piece of crap,” and “this f**king trial.” My colleague and the host of Virology Blog, Professor Racaniello, has called it a “sham.” Indeed, subsequent events have only strengthened the argument against PACE, despite the unconvincing attempts of the investigators and Sir Simon Wessely to counter what they most likely view as my disrespectful and “vexatious” behavior.

Virology Blog’s open letters to The Lancet and Psychological Medicine have demonstrated that well-regarded experts from the U.S, U.K. and many other countries find the methodological lapses in PACE to be such egregious violations of standard scientific practice that the reported results cannot be taken seriously. In the last few months, more than a dozen peer-reviewed commentaries in the Journal of Health Psychology, a respected U.K.-based academic publication, have further highlighted the international dismay at the study’s self-evident and indisputable lapses in judgement, logic and common sense.

And here’s a key piece of evidence that the trial has lost all credibility among those outside the CBT/GET ideological brigades: The U.S. Centers for Disease Control still recommends the therapies but now insists that they are only “generic” management strategies for the disease. In fact, the agency explicitly denies that the recommendations are related to PACE. As far as I can tell, since last year the agency no longer cites the PACE trial as evidence anywhere on its current pages devoted to the illness. (If there is a reference tucked away in there somewhere, I’m sure a sharp-eyed sleuth will soon let me know.)
http://www.virology.ws/2017/06/24/trial-by-error-continued-is-pace-a-case-of-research-misconduct/
 

Invisible Woman

Senior Member
Messages
1,267
I've added a comment, because the reasoning of Professor McCrone baffles me.

If anyone can help me understand, I would appreciate it.

If my brain fog is allowing me to grasp this I think there are two separate issues:

1) which is the one David is referring to is the fact that, yet again, they said they were costing one thing and then substituted another. The substitution, as usual, backs up the argument the researchers want to make and is potentially very misleading.

2) How should they have costed this informal care in the first place? A different but still valid issue.
 
Messages
724
Location
Yorkshire, England
If my brain fog is allowing me to grasp this I think there are two separate issues:

1) which is the one David is referring to is the fact that, yet again, they said they were costing one thing and then substituted another. The substitution, as usual, backs up the argument the researchers want to make and is potentially very misleading.

2) How should they have costed this informal care in the first place? A different but still valid issue.

Yes, thanks, :hug:

I was more interested in issue 2, as I think David is right about the first issue. These people induce brain fog whenever I try to understand their 'reasoning'.
 
Messages
44
I am afraid that that article has multiple problems. I posted a critical comment on the Virology blog but it seems to have disappeared. It mainly concerned errors in his discussion of the Plos One economics Paper.

This is the comment I posted:

"If you cannot back up a claim of research misconduct, which you acknowledge above that you can't , then why mention it at all? All that does is cause distress in a vulnerable group of patients, leads to ill- informed aggressive M.E. activism, and undermines valid criticism of the PACE trial.
There are various things not correct in this article, a lot of the things where the authors have already shown where you are incorrect, but also the whole thing on the economics paper is a mixture of error and ill-founded speculation. The Plos One article valued informal care at the opportunity cost based on national mean earnings and not at the average rate of a health care worker as you erroneously state above. You don't need to conduct a sensitivity analysis to know that the lower the value you assign to family care, the less benefit CBT and GET show in terms of cost effectiveness, that is just common sense, so Professor Mc Crone is obviously perfectly correct in his response. The paper's claim that the sensitivity analysis did not show a large impact on cost effectiveness for alternative assumptions is I think referring to the alternative assumptions discussed in the actual paper to value informal care i.e. minimum wage and unit cost of a homecare worker and not to the statistical analysis plan (so the zero value was not part of the analysis) so once again your point is ill-founded. It was never arguing that there was no difference in costs, it was comparing treatments and establishing which treatment had the greatest probability of being cost effective, so it is likely that changing the value of informal care as they did in the actual paper's sensitivity analysis did not change these results and therefore their reporting is correct, there is no basis to think otherwise.
The authors acknowledge in their outline of limitations that there are uncertainties regarding which is the most appropriate method to value informal care. Are you seeking to argue above that the time families spend caring for people with M.E. should be valued at zero? On what basis do you suggest this?
You are also failing to report the most important comment by Professor McCrone in his responses to comments on the article. His main point was "What should be stressed above all else is that there is uncertainty around all of these cost and outcome estimates and therefore the acceptability curves are the more informative indicators of the relative cost-effectiveness of these interventions." I feel it is highly irresponsible to be discussing it in the manner you are when it is obvious you do not understand it ."
 
Well, I can't see your comment in the comment section but the blog itself has been revised
Virology Blog said:
June 25, 2017: The last section of this post, about the PLoS One study, has been revised and corrected.
I don't have the brain power today to understand the arguments and whether David has addressed them or not, but this is what he says about the changes
Virology Blog said:
*Explanation for the changes: In the original version, I should have made clear that my concerns involved an analysis of what the investigators called cost-effectiveness from the societal perspective, which included not only the direct health-care costs but other considerations as well, including the informal costs. I also mistakenly wrote that the paper only presented the results under the assumption that informal care was valued at the cost of a home-care worker. In fact, for unexplained reasons, the paper’s main analysis was based on none of the three assumptions mentioned in the statistical analysis plan but on a fourth assumption based on the national mean wage.

In addition, I mistakenly assumed, based on the statistical analysis plan, that the sensitivity analyses conducted for assessing the impact of different approaches included both the minimum wage and zero-cost assumptions. In fact, the sensitivity analyses cited in the paper focused on the assumptions that informal care was valued at the cost of a home-care worker and at the minimum wage. The zero-cost assumption also promised in the protocol was not included at all. I apologize to Professor McCrone and his colleagues for the errors and am happy to correct them.

However, this does not change the fact that Professor McCrone’s subsequent comments contradicted the paper’s claim that, per the sensitivity analyses, changes in how informal care was valued “did not make a substantial difference to the results” and that the findings were “robust” for the alternative assumptions. This apparently false claim in the paper itself still needs to be explained or corrected. The paper also does not explain why the investigators included the zero-cost assumption in the detailed statistical analysis plan and then decided to drop it entirely in the paper itself.
 
Messages
13,774
The paper's claim that the sensitivity analysis did not show a large impact on cost effectiveness for alternative assumptions is I think referring to the alternative assumptions discussed in the actual paper to value informal care i.e. minimum wage and unit cost of a homecare worker and not to the statistical analysis plan (so the zero value was not part of the analysis) so once again your point is ill-founded.

I've not had time to read Tuller's piece yet, but just looked through the comments here, and thought it seemed odd to try to defend the PACE claim that "sensitivity analyses revealed that the results were robust for alternative assumptions", when they ommited the assumptions laid out in the statistical analysis plan which would have undermined their finding. I think that this claim of theirs is misleading and that it's quite right to criticise them for this.

Also, it's worth bearing in mind that there was not direct measure of time spent caring for participants, and that those in the CBT group (maybe GET group too?) had been encouraged to provide less care as a part of the intervention, so there's a potential for reporting bias here.

Your comment is visible to me on Virology.
 
Last edited:
Messages
2,087
Are you seeking to argue above that the time families spend caring for people with M.E. should be valued at zero? On what basis do you suggest this?

There is an expression of concer on this paper, probably worth reading the background...

http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0040808

Do read the reader comments too - a lot of valuable information in there, especially this

Just to point out that the statistical plan mentioned another analysis, zero cost for informal care[3]:
"The main analyses will use an informal care unit cost based on the replacement method (where the cost of a homecare worker is used as a proxy for informal care). We will alternatively use a zero cost and a cost based on the national minimum wage for informal care."
 
Messages
44
@jodie100
I am just curious about your defence of all things PACE and MEGA; have you had CBT/GET therapy?
Did you improve?

I am critical of quite a lot of PACE- the fear avoidance theory, the implication that there is nothing physically wrong, the media spin, the failure to fully acknowledge the potential for harm if incorrectly advised, inadequate respect for patients' autonomy etc.. (eta But I am also critical of opposition to it that does not make a well-argued case and potentially creates more stigma for people with M.E.)

I have never had any form of CBT. I haven't had formal GET just a general recommendation from a doctor to increase my activity gradually as a way of improving but this was long ago, pre PACE and he failed to advise me that I needed to get out of my boom-bust pattern by first reducing my activity and then gradually rebuilding, so it very quickly became unsustainable and it also led to me thinking in later stages of my illness that as much exercise/activity as I could push through was a good thing and this led to some very serious long term crashes, so mostly I think that the PACE style GET approach is not applied in real life and that GET is therefore dangerous. However I do see a role for certain elements from PACE that were common to CBT, GET and APT- avoiding boom-bust behaviour, setting a baseline you can manage each day without getting worse, and good sleep management in helping people in the earlier stages of their illness. These elements might have prevented me getting as sick as I did if I had been aware of them when I first developed M.E.
 
Last edited:

Invisible Woman

Senior Member
Messages
1,267
However I do see a role for certain elements from PACE that were common to CBT, GET and APT- avoiding boom-bust behaviour, setting a baseline you can manage each day without getting worse, and good sleep management in helping people in the earlier stages of their illness.

This type of advice pre dates PACE by a long way. It was not dreamed up by any of the PACE team. Many patients found it very useful & this is what I was taught as pacing.

PACE style advice is to keep increasing activity regardless of symptoms.
 

A.B.

Senior Member
Messages
3,780
avoiding boom-bust behaviour, setting a baseline you can manage each day without getting worse

You seem to be referring to pacing. If one interprets the recommendation to avoid boom-bust behaviour as recommendation to avoid doing too much and crashing from it. I don't like the term boom and bust because observational studies failed to find evidence for high peaks in activity. In other words, patients are crashing from minor exertion which is not compatible with the concept of boom. I think they're trying to work backwards from a conclusion (that it's psychosomatic and behavioural) and have to find some way to explain crashes without pathology, and the only way to do that is to postulate high levels of activity.

Over time GET has morphed somewhat: it used to be about steadily increasing activity, with emphasis on ignoring symptoms, which is very dangerous. Now it's more flexible and doesn't push as hard, but the underlying belief that the patient can and should increase activity levels and can be rehabilitated in this manner is still there.
 
Last edited:

Wolfiness

Activity Level 0
Messages
482
Location
UK
I don't like the term boom and bust because observational studies failed to find evidence for high peaks in activity. In other words, patients are crashing from minor exertion which is not compatible with the concept of boom. I think they're trying to work backwards from a conclusion (that it's psychosomatic and behavioural) and have to find some way to explain crashes without pathology, and the only way to do that is to postulate high levels of activity.

Yeah, it was never clear to me what exactly my OT imagined (ETA: physiologically) caused the "bust". It's not a dangerous organic dysfunction but you still have to kind of take it seriously when it hits you?

I suspect if you ask them they would fudge it. We allow too much deference to the B&B concept.
 
Last edited:
Messages
2,391
Location
UK
"If you cannot back up a claim of research misconduct, which you acknowledge above that you can't , then why mention it at all? All that does is cause distress in a vulnerable group of patients, leads to ill- informed aggressive M.E. activism, and undermines valid criticism of the PACE trial.
I find this a disturbing statement. Sounds like the usual PACE-fervour emotion laden statement that does not really reflect what patients really feel. Thank god we have people like David Tuller to stand up for what is right. Or does me saying that make me an aggressive activist. You are of course right to pick out any factual flaws if need be.

Anyway, hopefully over the coming year the truth, the whole truth, and nothing but the truth will come into the light ... Much of that thanks to DT's work.