• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

A cost effectiveness of the PACE trial

Dolphin

Senior Member
Messages
17,567
Another thing that I don't understand yet, is how they can talk about CBT and GET as separate therapies in terms of cost benefit, because they were only tested as additions to SMC. So, presumably, SMC would have to be administered as well as CBT and GET to see the cost benefits, and so the costs of SMC would have to be paid for as well as CBT and GET. I think they might have taken this into account, but it is confusing.
The comparisons are with SMC alone.
 

Dolphin

Senior Member
Messages
17,567
This seems to indicate that all of the interventions failed, as the societal costs at least doubled after treatments, and health care costs at least doubled.
The top figures are for 6 months, the bottom figures, 12 months.
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
On Lost Production costs
"Lost production costs were significantly higher for APT compared to CBT (difference £1279, 95% CI £141 to £2772)."

Thanks Dolphin.
I tend to ignore APT results, for my own purposes, so I sometimes forget to include the details for other purposes.
I've added the details to my post now.
 

Simon

Senior Member
Messages
3,789
Location
Monmouth, UK
Why no recovery data?
I'd always assumed that recovery data would be essential in calculating cost effectivenes. The published analysis is based on a single year of data, presumably as there is on guarantee that gains would continue in future years. But if patients had recovered (say, by the protocol definition of recovery) then the authors could surely make a case for counting at least some of those gains for at least some future years. With zero further costs, that would improve the cost-effectiveness considerably.

However, as no recovery data was included (18 months after the Trial paper was published and more than a year after a FOI request for recovery was denied on the grounds of 'future publication') I'm inclined to assume that the recovery data didn't show a significant benefit from CBT or GET. Have I missed something?
 

biophile

Places I'd rather be.
Messages
8,977
Good point Simon. If the recovery data looked good, it probably would have been published by now. However, it would not look good for them if this was never published, having already stated that this will be published (http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(11)60651-X/fulltext).

Also, when looking at the primary and secondary objectives (not outcomes) in the 2007 protocol, recovery is not specifically mentioned, the first 4 primary objectives are about which therapies are "more effective", while the 5th and last one is "What are the relative cost-effectiveness and cost-utility of these treatments?" So, I would expect cost-effectiveness to take priority even if they decide to prolong the publication of the recovery data because it was poor.

Re recovery and cost-effectiveness, recovery was a secondary outcome and there is this from the 2007 protocol (http://www.biomedcentral.com/1471-2377/7/6):
The main economic evaluation will be a cost-effectiveness analysis conducted from a societal perspective, examining comprehensive costs (treatment and service costs plus lost productivity) and the two primary efficacy measures (fatigue and physical function). Cost-effectiveness acceptability curves will be plotted as necessary. A supportive cost-consequences analysis will be conducted, examining comprehensive costs alongside all (primary and secondary) efficacy measures. To inform special interests, evaluations will also be conducted from the perspectives of the NHS, and also by using utility scores in the cost-effectiveness analysis (computed from either the EQ-5D [39] or the WSAS [33], there being arguments for and against each as the basis for health-related quality of life measurement).

Also there is this which is what we should expect will be published eventually:
The PACE trial will be the largest randomised trial of available treatments for CFS/ME. It will provide important information about efficacy, adverse events, cost-effectiveness, process and predictors. This will inform patients, their carers, healthcare providers and commissioners which treatments are most useful for which patients, and provide information regarding the essential process of both recovery and improvement from CFS/ME.

The authors make this statement in the 2011 Lancet paper (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3065633):
We plan to report relative cost-effectiveness of the treatments, their moderators and mediators, whether subgroups respond differently, and long-term follow-up in future publications.

Whatever happened to this PACE analysis on baseline variables/mediators? (http://www.trialsjournal.com/content/12/S1/A144). That was about 9 months ago without anything important being found, but does that mean we will never see a paper on it even though it has been promised repeatedly?

As the trial ended sometime late 2009 or early 2010, and their 2.5 year from baseline followup data should have been collected recently last year, perhaps they are going to next publish a paper on recovery and long-term followup? I hope it isn't going to take several more years to publish everything that was promised!
 

Firestormm

Senior Member
Messages
5,055
Location
Cornwall England
I don't know Simon.

I think collectively we are missing something - an alternative. Until there is one/several/more than those being touted and ones that are accepted (in whatever way acceptance occurs) then these are the only therapies they need/feel are suitable - and therefore there is no real pressure on them to do more than they have.

More broadly, as it is acknowledged that these therapies will be used on people whose principle complaint is fatigue there will be positive results out in the real world.

This will also be in part because of the way such positives are measured - if they are measured/recorded at all but, in the case of our condition, and according to NICE currently and the PACE Trial itself - they won't be offered to patients in the 'severe' category.

But I think personally, that my first point is the main one. There are others of course, but essentially why bother dotting the i's and crossing the t's when there is no imperative to do so?

And I am not demeaning these interventions totally. If you think of all those who have a need for help with their health, then I am sure that they (and perhaps even I) will find that if offered, the intervention of a therapist will help in some (perhaps small perhaps large) regard - but perhaps enough for them (or me) to record this improvement on a form that will justify the investment.

Not everyone who engages with these therapies will be offered the protocols used in PACE specifically. And even those that are could well find that they help as I said. Not everyone is me or shares my opinions or willingness (or resistance) to engage (not to engage), and although we are considering the published results here I think we need sometimes to consider their application.

I suppose one of the main questions is: will this paper result in greater investment i.e funding in this area by individual PCT/Clinical Commissioning Groups? First thing I thought of when these latest headlines hit was something that was said recently to the Chairman of our local support group.

We are being placed in a model that appears to be trying to treat more than just our condition with similar interventions.

The comment made was 'this is where the NHS are putting the funding now' or words to that effect.

Made me think...
 

user9876

Senior Member
Messages
4,556
I've started looking for EQ-5D results from placebo and alternative treatments. To me, a big potential concern is that the much of the reported improvement could be down to politeness, positive thinking, etc. As there was no placebo control group in PACE, it might be interesting if we could find our own (I've not found anything great yet).





/
One on CBT for SAD

The figure isn't goint in so I will describe it:

There are three lines but if I just describe one then it makes the point about improved utility values:

CBT
1 : 0.42
2.5 : 0.78
4 : 0.9
10 : 0.85 ish
13 : 0.85ish




http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1018&context=usuhs
One technique for QALS they use the area under the curve. What I thought was noticable is the change in the utilities (individual measurements) which jump from various levels of moderate to in remission. Compare this to the very small changes reported in the PACE trial.
 

biophile

Places I'd rather be.
Messages
8,977
Good point Firestormm, cost-effectiveness is used to help decide where to put healthcare funding, and there is no widely-accepted alternative treatment. I suggested earlier that the "relevant authorities" aren't going to be impressed by the poor performance of CBT and GET in terms of overall service costs, employment losses, welfare benefits and other financial payments. I still doubt they will be busting themselves to rapidly fund a major rolling out of CBT/GET, but they may allocate more resources to it. However, the cost-effectiveness of CBT and GET is based on hypothetical scenarios calculated from self-reported improvements in quality of life, fatigue, and physical symptoms. CBT and GET are not going to save the NHS or DWP much money or justify another push for "welfare to work", so there should be less incentive to goad ME/CFS patients into CBT or GET in order to save money, although they can still make it difficult in other ways.
 

Simon

Senior Member
Messages
3,789
Location
Monmouth, UK
Good point Simon. If the recovery data looked good, it probably would have been published by now.

..Also, when looking at the primary and secondary objectives (not outcomes) in the 2007 protocol, recovery is not specifically mentioned, the first 4 primary objectives are about which therapies are "more effective", while the 5th and last one is "What are the relative cost-effectiveness and cost-utility of these treatments?" So, I would expect cost-effectiveness to take priority even if they decide to prolong the publication of the recovery data because it was poor.

Re recovery and cost-effectiveness, recovery was a secondary outcome and there is this from the 2007 protocol (http://www.biomedcentral.com/1471-2377/7/6):
Thanks for digging that out, very helpful.

I think it's worth expanding on why recovery data could have such a big impact on cost-effectiveness. The treatment costs are the same as for non-recovered patients, but there would be substantial gains in employment, reduced informal care and presumably reduced healthcare costs too. However, let's focus on EQD-5 QALY scores (i.e. effectivenss rather than costs) as they are the reason CBT & GET come out ahead in the current paper:
  • Baseline QALY across all groups is around 0.5 (ex 1.0) for all treatments, improving to about 0.6 accrued QALY for CBT & GET.
  • A recovered person would score either 1.0 (no problems at all) or say 0.85 (no problems with mobility, self-care, pain or usual activities, some problems with anxiety/depression as these are common in the population). Let's conservatively settle on 0.85 (see this paper Table 1 for some similar figures)
  • This is +0.25 higher than the mean QALY score of 0.6. Given that the net QALY gain for CBT & GET relative to SMC was only 0.05-0.03, if a substantial number of patients recovered (relative to SMC), that would make a big difference to the overall effectiveness rating.
  • And of course, with recovered patients it's much easier to make the case for counting QALY benefits in future years too, even making some provision for relapses. This guesstimate also ignores the impact in future years of cost savings in healthcare costs, employment and informal care.
edit: To put this into a picture:

McCrone2012.jpg





As the trial ended sometime late 2009 or early 2010, and their 2.5 year from baseline followup data should have been collected recently, perhaps they are going to next publish a paper on recovery and long-term followup? I hope it isn't going to take several more years to publish everything that was promised!
As they had 12 month data when they published 18 months ago, you might even think that by now they could have the 2.5 year data in publishable format.
 

user9876

Senior Member
Messages
4,556
Good point Firestormm, cost-effectiveness is used to help decide where to put healthcare funding, and there is no widely-accepted alternative treatment. I suggested earlier that the "relevant authorities" aren't going to be impressed by the poor performance of CBT and GET in terms of overall service costs, employment losses, welfare benefits and other financial payments. I still doubt they will be busting themselves to rapidly fund a major rolling out of CBT/GET, but they may allocate more resources to it. However, the cost-effectiveness of CBT and GET is based on hypothetical scenarios calculated from self-reported improvements in quality of life, fatigue, and physical symptoms. CBT and GET are not going to save the NHS or DWP much money or justify another push for "welfare to work", so there should be less incentive to goad ME/CFS patients into CBT or GET in order to save money, although they can still make it difficult in other ways.

There are a couple of good things in this paper. Firstly it releases the benefit figures and states that"Lost
employment was common in all treatment groups". These are important outcomes.

Secondly it has some sort of cost calculation around societal costs.

And thirdly it demonstrates that there was only a very minor gain with each treatment arm under the EQ-5d survey.

This leads to an argument that ME is very costly to society and current treatment options won't relieve that cost. Hence new research is necessary. The MRC call for research is good here in suggesting direction.

The problem is the way they have spun their results and papers pick up on the spin. But important information is in the paper. Journalists don't like to look foolish yet when they report this spin they do. Perhaps we need a strategy of making the spin clear so that in the future they lose trust in the press release and look further.
 

Dolphin

Senior Member
Messages
17,567
The problem is the way they have spun their results and papers pick up on the spin. But important information is in the paper. Journalists don't like to look foolish yet when they report this spin they do. Perhaps we need a strategy of making the spin clear so that in the future they lose trust in the press release and look further.
Except in this case it was an MRC press release: http://www.mrc.ac.uk/Newspublications/News/MRC008797 so probably difficult to cause journalists to lose trust in them. But still interesting idea.
 

Dolphin

Senior Member
Messages
17,567
I didn't find this piece that exciting: http://www.nhs.uk/news/2012/08augus...onic-fatigue-syndrome-not-cost-effective.aspx

However, was interested in the second paragraph here:

Based on the statistical models used by the researchers, CBT and graded exercise therapy were found to be most cost-effective, while specialist medical care and pacing were the least cost-effective.

The researchers did not consider patient preference, which could have an impact.

I am interested in this concept of patient preference. I don't know too much about it. It appears to me it could be a concept that could be useful for us.
I made a similar comment elsewhere and somebody replied:

It might be reference to ongoing debate, e.g.:

http://www.ncbi.nlm.nih.gov/pubmed/20511562

Med Decis Making. 2010 Jul-Aug;30(4):E57-63. Epub 2010 May 28.
Theoretical foundation of patient v. population preferences in calculating QALYs.
Gandjour A.

Pennington Biomedical Research Center, Louisiana State University, Baton Rouge, Louisiana 70808, USA. afschin.gandjour@pbrc.edu
Abstract

The cost-effectiveness of health care interventions is often evaluated using quality-adjusted life years (QALYs) as a measure of outcome. There is a debate on whether QALYs should use patient preferences as opposed to community preferences. This article shows that patient preferences have a theoretical foundation in preference-utilitarian theory and welfare economics. In contrast, this study found no compelling theoretical basis for community preferences. There is a need for further development of a normative framework to inform the choice of preference source.

There are heated exchanges between John Harris, a prominent long time QALY critic, and NICE, e.g.

http://jme.bmj.com/content/31/12/685.full

http://en.wikipedia.org/wiki/John_Harris_(bioethicist)

First came across Harris in Greenhalghs book (old but useful) "How to read a paper":

http://www.fk.unair.ac.id/pdfiles/ebook/How to Read a Paper.pdf
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
I haven't done any work on QALYs yet, so the following disregards QALYs.

Thinking further about my earlier post, it seems that CBT/GET didn't help reduce welfare benefits costs (according to the text of the paper), or private pension costs, or health care costs. (In fact I think that CBT/GET increased medical costs and private pension costs.) I think that the only thing that CBT/GET helped to reduce 'substantially' was societal costs, but if the very convenient 'informal care' costs are not included, then even societal costs look no different between groups.

Does anyone agree with this?
 

Dolphin

Senior Member
Messages
17,567
I haven't done any work on QALYs yet, so the following disregards QALYs.

Thinking further about my earlier post, it seems that CBT/GET didn't help reduce welfare benefits costs (according to the text of the paper), or private pension costs, or health care costs. In fact I think that CBT/GET increased medical costs and private pension costs. I think that the only thing that CBT/GET helped to reduce substantially was societal costs, but if the very convenient 'informal care' costs are not included, then even societal costs look no different between groups.

Does anyone agree with this?
Yes, I think that's right.
 

Dolphin

Senior Member
Messages
17,567
Although GET didn't do better overall with benefits, there was a (numerical) smaller increase in people starting taking Illness/disability benefits. The two GET centres were at Barts. As was posted earlier, Barts don't write letters for benefits. It would be interesting if this disadvantaged those in Barts over other centres/legs of the PACE Trial.

ETA: Not 100% sure of this upon reflection. Thought I knew somebody in Scotland getting GET.
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
OK, scrap this post, I've just been reminded that, for Societal Costs, the paper says that the differences between CBT, GET and SMC "were not statistically significant."

Exploring the paper, and some rough calculations.

(These are just my rough back-of-an-envelope calculations, and are not exact or adjusted figures, and there might be major mistakes.)

And, like earlier, I might have misunderstood major issues.

Below, I compare the difference in cost changes for SMC/CBT/GET for the 12 month pre-randomisation period (I get these figures by doubling the figures for the 6 month pre-randomisation period) with the 12 month post-randomisation period.


Societal Costs / Cost Benefit Analysis

My current understanding is that GET and CBT are considered to have a better cost benefit, based on 'societal costs' for GET and CBT decreasing sightly more than SMC:
(GET = extra £197 decrease in societal costs as extra to SMC £1914 decrease)
(CBT = extra £464 decrease in societal costs as extra to SMC £1914 decrease.)

At least, I think this might be the case, but I have no idea how exactly they worked it all out. The complexity is astounding.

Considering that these are mean figures, it's especially confusing because they are measuring the effects and costs on the 87% of participants who experienced no primary outcome effect for CBT/GET.


Here are my calculations:

Comparing changes in health care and societal costs from 12 month period before randomisation and 12 month period after randomisation - Difference between SMC and GET/CBT

Difference between SMC and GET (increases/decreases in means):

Extra health costs for GET (GET - SMC) 806 - (-116) = 922
(GET changes in health care costs are £922 more than SMC.)
(GET = £922 increase in health care costs, separate to SMC. SMC = £116 decrease in health care costs.)

Extra societal costs (GET - SMC) (-2111) - (-1914) = -197
(SMC societal costs are £197 more than GET.)
(GET = £197 decrease in societal costs, as extra to SMC. SMC = £1914 decrease in societal costs.)

Difference between SMC and CBT (increases/decreases in means):

Extra health care costs for CBT (CBT - SMC) 812 - (-116) = 928
(CBT changes in health care costs are £928 more than SMC.)
(CBT = £928 increase in health care costs, separate to SMC. SMC = £116 decrease in health care costs.)

Extra societal costs (CBT - SMC) (-2378) - (-1914) = -464
(SMC societal costs are £464 more than CBT.)
(CBT = £464 decrease in societal costs, as extra to SMC. SMC = £1914 decrease in societal costs.)


And these figures are interesting:

Health care costs per person improved for improvements in physical function.

SMC
Per person health savings (-116 x 148) / (148 x 58%) = £200 savings per person improved, during 12-month post-randomisation period.

GET
Per person health costs (922 x 140) / (140 x 12.6%) = £7317 health costs per person improved, separate to SMC savings, during 12-month post-randomisation period.

CBT
Per person health costs (928 x 145) / (145 x 13.4%) = £6925 health costs per person improved, separate to SMC savings, during 12-month post-randomisation period.


I might have got this wrong, but it does raise questions.
The extra spent on health care per person is about £7000 for CBT and GET, on top of spending/savings for SMC.

-----
 

biophile

Places I'd rather be.
Messages
8,977
Societal costs includes health costs

After rereading parts of the paper and crunching some more numbers, it is clearer now that societal costs includes healthcare. Societal = health costs + lost employment/production + unpaid informal care. I think I made a mistake in a previous post when calculating societal costs vs health costs.

Some of the wording is difficult to interpret at first glance. As far as I can tell, there was a significant advantage for both CBT and GET vs SMC in terms of informal care costs, but not for overall societal costs which includes informal care. However, "Net benefit values were computed for each study participant, defined as the value of a QALY multiplied by the number of QALYs gained minus the cost (from both healthcare and societal perspectives)." So the relative cost-effectiveness of CBT and GET was not achieved by a raw reduction in overall societal costs from Table 3, but rather by subtracting those costs from the estimated value of the gain in QALY, hence the cost-effectiveness from both a "societal perspective" and "healthcare perspective".
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
Although GET didn't do better overall with benefits, there was a (numerical) smaller increase in people starting taking Illness/disability benefits. The two GET centres were at Barts. As was posted earlier, Barts don't write letters for benefits. It would be interesting if this disadvantaged those in Barts over other centres/legs of the PACE Trial.

That's a very interesting point.

But both CBT and GET uptake of Illness/disability benefits are less than SMC. Do you know where CBT took place?

Strangely, all they say about this is: "the figures at followup were similar between groups."
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
One confounding factor that we need to consider is the long pipeline to receiving benefits. Its possible that some of the increase in benefits was simply due to this delay - people had applied and were waiting. So while this data does show that CBT/GET is of no substantive health benefit, we have to be careful in attributing all the increased financial benefits claims to the therapy. There is not sufficient information to be sure. If we had objective functional capacity data that would have been more useful and would have helped give this perspective. Bye, Alex