• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

A cost effectiveness of the PACE trial

Dolphin

Senior Member
Messages
17,567
Societal costs includes health costs

After rereading parts of the paper and crunching some more numbers, it is clearer now that societal costs includes healthcare. Societal = health costs + lost employment/production + unpaid informal care. I think I made a mistake in a previous post when calculating societal costs vs health costs.

Some of the wording is difficult to interpret at first glance. As far as I can tell, there was a significant advantage for both CBT and GET vs SMC in terms of informal care costs, but not for overall societal costs which includes informal care. However, "Net benefit values were computed for each study participant, defined as the value of a QALY multiplied by the number of QALYs gained minus the cost (from both healthcare and societal perspectives)." So the relative cost-effectiveness of CBT and GET was not achieved by a raw reduction in overall societal costs from Table 3, but rather by subtracting those costs from the estimated value of the gain in QALY.
That is only true for the healthcare costs.

CBT and GET resulted in reductions in societal cost, but the difference wasn't statistically significant.

What they did was: "If costs for one treatment were lower and outcomes better than another treatment, that treatment was defined as ‘dominant’." That was the case for all the CBT and GET vs SCM calculations involving societal cost calculation. They did then calculation some probabilities.
 

Dolphin

Senior Member
Messages
17,567
That's a very interesting point.

But both CBT and GET uptake of Illness/disability benefits are less than SMC. Do you know where CBT took place?

Strangely, all they say about this is: "the figures at followup were similar between groups."
Actually, I've edited my original comment as I'm not so sure now. I thought I recalled hearing that GET was in the two Barts centres. However, I believe I came across somebody who was doing GET in Scotland.

• Astley Ainsley Hospital, Edinburgh, working with the Regional Infectious
Diseases Unit, Western General Hospital, Edinburgh, both of NHS Lothian
• Bart’s and the London NHS Trust, East London
• East London and the City Mental Health NHS Trust
• Oxfordshire Mental Healthcare NHS Trust working with the Oxford
Radcliffe Hospitals Trust
• The Royal Free Hampstead NHS Trust
• Guy’s, King’s & St Thomas’ School of Medicine
and
- (added later) Frenchay Hospital in Bristol.

Maybe somebody else can say what they know on the issue of centres. People in the UK are closer to the centres than me and may have heard in other ways.
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
Societal costs includes health costs

After rereading parts of the paper and crunching some more numbers, it is clearer now that societal costs includes healthcare. Societal = health costs + lost employment/production + unpaid informal care. I think I made a mistake in a previous post when calculating societal costs vs health costs.

Thanks biophile. I don't think I'd registered that societal costs included health care costs.
I think that societal costs include everything listed in Table 3.
It's interesting that societal costs seem to include everything except welfare benefits/private pensions/income protection schemes.

As far as I can tell, there was a significant advantage for both CBT and GET vs SMC in terms of informal care costs, but not for overall societal costs which includes informal care.

For societal costs, the paper says: Differences between CBT, GET and SMC "were not statistically significant." (But looking at the figures presented in the paper, the societal costs for CBT and GET were lower than for SMC.)

"Societal costs (i.e. healthcare, informal care and lost production costs) were significantly lower for patients allocated to CBT compared to APT (difference £2607, 95% CI £432 to £5585). Other differences were not statistically significant."
 

Dolphin

Senior Member
Messages
17,567
Drat, I thought I had the hang of it. However, I'm not sure why there are differences in the same column in Table 6 for the same comparisons. Differences are small so I'm guessing maybe due to missing values i.e. the figures might be averages based on slightly different numbers of patients?
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
I haven't done any work on QALYs yet, so the following disregards QALYs.

Thinking further about my earlier post, it seems that CBT/GET didn't help reduce welfare benefits costs (according to the text of the paper), or private pension costs, or health care costs. (In fact I think that CBT/GET increased medical costs and private pension costs.) I think that the only thing that CBT/GET helped to reduce 'substantially' was societal costs, but if the very convenient 'informal care' costs are not included, then even societal costs look no different between groups.

Does anyone agree with this?

Dolphin has reminded me that the paper says that, for societal costs (i.e. healthcare, informal care and lost production costs), the differences between CBT, GET and SMC "were not statistically significant."

"Societal costs (i.e. healthcare, informal care and lost production costs) were significantly lower for patients allocated to CBT compared to APT (difference £2607, 95% CI £432 to £5585). Other differences were not statistically significant."

I think that significant/substantial differences for societal costs might appear when they calculate 'per person improved' societal costs for Chalder fatigue and SF-36 physical function. Also, the per person savings per QALY gained.
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
OK, so if I've got this right, according to the paper all of the main figures for SMC/CBT/GET are similar, except for 'informal care', and 'health care', and QALYs gained (which I haven't looked into properly yet). (And seeing as 'informal care' is included in 'societal care' costs, then it can be ignored as a separate figure.)

So all the following are similar for SMC/CBT/GET: Societal costs, welfare benefits, private pensions, lost production costs, employments costs etc.

So, apart from QALYs, the only significant difference is with health care costs which were lower for SMC than for CBT/GET. (Something else to consider is that SMC was more a effective treatment overall.)

So am I understanding it correctly that the conclusions, in favour of CBT/GET, that have been heavily promoted, are based entirely on the results for QALYs?
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
One confounding factor that we need to consider is the long pipeline to receiving benefits. Its possible that some of the increase in benefits was simply due to this delay - people had applied and were waiting. So while this data does show that CBT/GET is of no substantive health benefit, we have to be careful in attributing all the increased financial benefits claims to the therapy. There is not sufficient information to be sure. If we had objective functional capacity data that would have been more useful and would have helped give this perspective. Bye, Alex

If SMC is used as a control group, as was originally intended, then we can see the relative increases in claims for CBT/GET. The authors say that there are not 'substantial' (whatever that means) differences between SMC and CBT/GET, or that there were 'similar' figures in each group. They've used completely non-scientific wording.

I don't understand why they haven't analysed this more accurately, except if they wanted to hide the relative increase in private pension claims and income protection schemes in the CBT/GET groups. (Relative unadjusted increases of 4 to 6 percentage points for CBT/GET, by my calculations.)

Edit: Also, I think that GET increased numbers on 'income benefits' slightly (but maybe not significantly), and CBT made no difference. Maybe this is also embarrassing for them that CBT and GET made no improvements.

CBT and GET do seem to relatively decrease illness/disability benefits claims, but they've strangely dismissed this with the following: "the figures at followup were similar between groups." Maybe it was also embarrising for them that absolute claimant numbers increased in the GET and CBT groups.
 

Dolphin

Senior Member
Messages
17,567
OK, so if I've got this right, according to the paper all of the main figures for SMC/CBT/GET are similar, except for 'informal care', and 'health care', and QALYs (which I haven't looked into properly yet). (And seeing as 'informal care' is included in 'societal care' costs, then it can be ignored as a separate figure.)

So all the following are similar for SMC/CBT/GET: Societal costs, welfare benefits, private pensions, lost production costs, employments costs etc.

So, apart from QALYs, the only significant difference is with health care costs which were lower for SMC than for CBT/GET. (Something else to consider is that SMC was more a effective treatment overall.)

So am I understanding it correctly that the conclusions, in favour of CBT/GET, that have been heavily promoted, are based entirely on the results for QALYs?
It was possible for a treatment to come out on top even if differences in costs weren't statistically different using the normal sense of the word i.e. a simple t-test (hence the probability models).
 

Dolphin

Senior Member
Messages
17,567
I was just looking at Simon's file of comparisons.
Using the raw figures for informal care, CBT and GET don't involve savings that are that big compared to SMC: £237 and £408 respectively. However, when they adjust for baseline, these jump to £1165 (95% CI: £239-2894) and £1173 (95% CI: £740 to £1569). I wonder did they have options in terms of what baseline factors they could adjust for and maybe chose factors that helped increase this gap?
 

Simon

Senior Member
Messages
3,789
Location
Monmouth, UK
One confounding factor that we need to consider is the long pipeline to receiving benefits. Its possible that some of the increase in benefits was simply due to this delay - people had applied and were waiting. . . we have to be careful in attributing all the increased financial benefits claims to the therapy. There is not sufficient information to be sure.
A good point, particularly given that these patients hadn't been ill that long:

SMC: 25 months (25% less than 15 months)
CBT: 36 months (25% less than 16 months)
GET: 35 months (25% less than 18 months)

Then consider how long it can take from the onset of illness to benefits being awarded. Many people struggle on at work or go part-time for a while. In the UK, Statutory Sick Pay (paid by employers) lasts for up to 8.5 months once people have stopped workin, and some employers are considerably more generous than this. Only then would people apply for formal benefits.

For Disability Living Allowance (help with mobility/personal care) there is also a 6 month qualification period before you can be awarded benefits.

Added to that, i think Esther12 had a point in saying the Trial might help with getting benefits. While Bart's might be unhelpful to regular patients, I bet each Trial participant had paperwork showing they were part of a large clinical trial for CFS (cofunded by the DWP). That kind of thing is likely to help convince benefits assessors the patient is 'genuine'.

The authors could run analyses that would throw light on this: were people with a shorter illness duration more likely to be awarded benefits during the trial?
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
Drat, I thought I had the hang of it. However, I'm not sure why there are differences in the same column in Table 6 for the same comparisons. Differences are small so I'm guessing maybe due to missing values i.e. the figures might be averages based on slightly different numbers of patients?

I think they might be adjusted figures (?)
 

Dolphin

Senior Member
Messages
17,567
I think they might be adjusted figures.
I know the figures in Table 6 don't match Table 3 because of adjustments due to baseline factors. But within Table 6, why would there be different adjustments to the same measure for the same group/group comparison? Maybe you're right but you will need to explain your reasoning a bit to show me why that could be the case. Otherwise I think I'll continue to think it's likely because there are little variations in the numbers due to slightly different numbers filling in the EQ5d/SF-36 PF/CFQ (I might have been premature posting my last comment because I'm "content" now in thinking it is due to some missing scores).
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
I know the figures in Table 6 don't match Table 3 because of adjustments due to baseline factors. But within Table 6, why would there be different adjustments to the same measure for the same group/group comparison? Maybe you're right but you will need to explain your reasoning a bit to show me why that could be the case. Otherwise I think I'll continue to think it's likely because there are little variations in the numbers due to slightly different numbers filling in the EQ5d/SF-36 PF/CFQ (I might have been premature posting my last comment because I'm "content" now in thinking it is due to some missing scores).

I don't know the answer Dolphin...

I can't find the details, but somewhere in the paper it tells us how many missing scores there were, and if I remember correctly, it says there were quite a large number missing. So maybe you are correct in saying that they had to recalculate the means for a different set of scores.

And I imagine that if they recalculate all the means for a different number of patients, then they'd have to make fresh baseline adjustments as well.
 

Dolphin

Senior Member
Messages
17,567
I don't know the answer Dolphin, but I think we might both be correct here...

I can't find the details, but somewhere in the paper it tells us how many missing scores there were, and if I remember correctly, it says there were quite a large number. So maybe you are correct in saying that they had to recalculate the means for a different set of scores.

And I imagine that if they recalculate all the means for a different number of patients, then they'd have to make fresh baseline adjustments as well.
Well, what I would see as different "baseline adjustments" are different factors.
 

Dolphin

Senior Member
Messages
17,567
Does anyone know if we know what the cost per QALY for SMC alone is?
(If that is a legitimate and meaningful question. :confused: )
If there was another arm of the trial, with a group that did not do SMC, one could see how many QALYs* SMC added and one could see how much it cost per QALY. But we don't have such an arm. If one assumed that a control group wouldn't change in terms of QALYs as well as costs (relatively big assumptions), one could calculate it now but I very much doubt the authors have included it in the paper.

*it would be less than 1
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
Table 6.

Fatigue and Disability sections:

CBT and GET have greater health care costs than SMC.
CBT and GET have less societal costs than SMC.

They've concluded that CBT and GET were 'dominant', but I don't know why because they say:

"If costs for one treatment were lower and outcomes better than another treatment, that treatment was defined as ‘dominant’."

Does anyone understand this?
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
If there was another arm of the trial, with a group that did not do SMC, one could see how many QALYs* SMC added and one could see how much it cost per QALY. But we don't have such an arm. If one assumed that a control group wouldn't change in terms of QALYs as well as costs (relatively big assumptions), one could calculate it now but I very much doubt the authors have included it in the paper.

*it would be less than 1

Thanks for explanation Dolphin.

But I think they carried out the QALY questionnaires at baseline and 52 weeks for all therapy groups.

So I would have thought that they would have a QALY calculation for SMC, based on the difference at baseline and 52 weeks, don't you think?
 

Dolphin

Senior Member
Messages
17,567
Table 6.

Fatigue and Disability sections:

CBT and GET have greater health care costs than SMC.
CBT and GET have less societal costs than SMC.

They've concluded that CBT and GET were 'dominant', but I don't know why because they say:

"If costs for one treatment were lower and outcomes better than another treatment, that treatment was defined as ‘dominant’."

Does anyone understand this?
Yes. The dominant only applies to the calculations using societal costs not those just choosing healthcare costs (one doesn't see a full column showing "dominant"). The societal cost of those who did GET or CBT (and SMC) is less than SMC alone (largely due to the savings in informal care). Then the GET and CBT participants had better outcomes in QALYs/percentages improving by two or more on the Chalder fatigue questionnaire/percentages improving by eight or more on the SF 36 physical functioning questionnaire.

 

So GET and CBT are dominant for those three (out of the six) comparisons each i.e. GET versus SMC and CBT versus SMC.

 

 
 

Dolphin

Senior Member
Messages
17,567
Thanks for explanation Dolphin.

But they carried out the QALY questionnaires at baseline and 52 weeks for all therapy groups.

So I would have thought that they would have a QALY calculation for SMC, based on the difference at baseline and 52 weeks, don't you think?
They found the average improvement for the SMC group over baseline was 0.02 QALYs. This is based on taking measurements from a few points. Basically if one had a graph with a line across at y=0.50, one marks in the values in table 3, join the dots using straight lines (they say that as far as I recall), calculate the area above the y=0.50 line and get the average value over the 52 weeks and that is the QALYs accrued.