1. Patients launch $1.27 million crowdfunding campaign for ME/CFS gut microbiome study.
    Check out the website, Facebook and Twitter. Join in donate and spread the word!
The ePatient Revolution
Ryan Prior shares his experience and his thoughts from attending the Stanford Medicine X Conference as he contemplates the rising of the ePatient Revolution ...
Discuss the article on the Forums.

A cost effectiveness of the PACE trial

Discussion in 'Latest ME/CFS Research' started by user9876, Aug 1, 2012.

  1. Bob

    Bob

    Messages:
    8,811
    Likes:
    12,207
    South of England
    Has anyone got any insight into how the first bullet-point in the following quote fits in with all the reports that CBT and GET were the most cost effective?:

     
  2. Dolphin

    Dolphin Senior Member

    Messages:
    6,868
    Likes:
    6,146
    One can't bash them for that: any extra treatment is going to cost more money (all other things being equal i.e. it doesn't decrease other healthcare costs during the course of using it).

    What they then looked at was how much it cost for any health gains got (in terms of QALYs, improvement of 2 or more in Chalder Fatigue score (likert scoring), improvement of 8 or more in SF-36 physical functioning).
     
  3. Esther12

    Esther12 Senior Member

    Messages:
    5,375
    Likes:
    5,862
    I've heard the same about the East Sussex CFS service too.

    The one useful thing they could do....!
     
  4. Purple

    Purple Bundle of purpliness

    Messages:
    1,400
    Likes:
    2,311
    Doctor Christian Jessen, a UK TV doctor who presents Embarrassing Bodies and Supersize v Superskinny, has tweeted twice about this study (he is a prolific tweeter with a large Twitter following of about 128k)

    Friday 3 August 6:49 PM
    Dr Christian Jessen @DoctorChristian
    Saturday 4 August 12:29 PM
    Dr Christian Jessen @DoctorChristian

    His Twitter page: https://twitter.com/DoctorChristian
    And his Facebook page where his tweets also appear: https://www.facebook.com/pages/Doctor-Christian/112190008831149

    Information about Doctor Christian:
    http://en.wikipedia.org/wiki/Christian_Jessen
    http://www.drchristianjessen.com/
     
  5. Bob

    Bob

    Messages:
    8,811
    Likes:
    12,207
    South of England
    Misleading quotes from the MRC press release:
    http://www.meassociation.org.uk/?p=12343


    "Two treatments found previously to be the most effective for patients with Chronic Fatigue Syndrome or Myalgic Encephalomyelitis (CFS/ME) have now been found to be the most cost-effective treatments evaluated in a large clinical trial."

    The press release starts off with a load of nonsense. If they are going to treat the SMC control group as an extra therapy then, in fact, SMC was more effective than CBT and GET, by all primary measures. The PACE Trial wasn't set up to assess SMC as a separate therapy.


    "Adaptive pacing therapy (APT) was not cost-effective."

    This assertion is not strictly misleading in itself, untill they start making conclusions about 'pacing'.
    The cost effectiveness of APT tells us nothing about the benefits of teaching patients about pacing.
    The reason that APT was not cost-effective was because it was a failed novel attempt at a 'therapy'.


    "There is now a strong case for the NHS to invest in providing these therapies."

    If the therapies don't work for 87% of patients, and CBT is clinically ineffective at reducing physical disability, then I don't understand what the 'strong' case is.
    And we still don't have the deterioration rates. Once those have been published, then the therapies will probably look even weaker than they do now.


    "Professor Michael Sharpe from Oxford University and a co-author of the paper said:
    In the PACE trial we found that the rehabilitative treatments CBT and GET improve the fatigue and disability of people with CFS/ME."

    This is misleading, because CBT does not improve disability of people with CFS/ME. It failed to meet the threshold of a clinical useful outcome for SF-36 PF.


    "These promising findings demonstrate how MRC funding can help to identify treatments that are not only effective, but are financially viable for the health service."

    The treatments were not found to be 'effective', but only 'moderately effective' for the primary outcomes measures, and clinically ineffective for one primary outcome measure.


    "In 2011, the first findings from the PACE trial showed that CBT and GET benefit around 60 per cent of patients with CFS/ME, for whom fatigue was the main symptom."

    This is the biggest lie of all.

    The proportion of participants who achieved a clinically useful outcome (clinically useful difference from SMC) were as follows:

    CBT physical function 13% (NNT = 1 in 8)
    CBT fatigue 11% (NNT = 1 in 9)

    GET physical function 12% (NNT = 1 in 8)
    GET fatigue 15% (NNT = 1 in 7)

    By contrast, 58% (physical function) and 65% (fatigue) of the SMC control group achieved a clinically useful outcome.
     
  6. Bob

    Bob

    Messages:
    8,811
    Likes:
    12,207
    South of England
    Can anyone explain the following to me please?

    "Specialist medical care was the cheapest option in terms of absolute cost to provide, but when the benefit of treatment on quality of life was taken into account, CBT became the most cost-effective option (likelihood of 62.7 per cent). There was a 26.8 per cent likelihood that GET was the most cost effective, while the likelihood for APT and standard care alone were 2.6 and 7.9 per cent, respectively."
    http://www.meassociation.org.uk/?p=12343

    If SMC was the cheapest option, in terms of absolute cost, and it vastly outperformed the other therapies, then how can it possibly be least cost effective? Surely that means that it was the most cost effective?

    Have they completely ignored the primary outcome results for the cost benefit analysis?
     
  7. Dolphin

    Dolphin Senior Member

    Messages:
    6,868
    Likes:
    6,146
    SMC didn't "vastly outperformed the other therapies, in terms of effectiveness".

    I don't think it's that hard to understand. Replace a drug for CBT or GET: it might cost more but if it improves quality of life, it can be considered "cost-effective" (if the cost required to give 1 QALY is less than £30,000 (say)). A health service is willing to spend money it collects from taxes to help people's quality of life, within certain limits.
     
  8. Dolphin

    Dolphin Senior Member

    Messages:
    6,868
    Likes:
    6,146
    What happens in Sussex seems to come from Peter White, just as it does in Essex. If I was in such areas, I'd be pretty annoyed at him.
     
  9. Sam Carter

    Sam Carter Guest

    Messages:
    297
    Likes:
    192
    Hi Bob,

    Here are the percentage of participants who improved by at least 2 points for fatigue and 8 points for physical function (ie. who achieved an (apparent) 'clinically useful difference') at 52 weeks:

    42% APT
    59% CBT
    61% GET
    45% SMC
     
  10. Bob

    Bob

    Messages:
    8,811
    Likes:
    12,207
    South of England

    Thank you Sam, they are the figures for the post-hoc analysis for improvements in both fatigue and physical function. My figures are for the improvements considered separately, for each primary outcome measure, as per the primary results in Table 3 of the original PACE Trial paper.

    The figures that you give, Sam, are the ones that the authors prefer to quote, as they are slightly more favourable to CBT and GET. But I think that they add to the general obfuscation of the data, and they are just a quick post-hoc analysis, designed to make the results look better.
     
  11. Bob

    Bob

    Messages:
    8,811
    Likes:
    12,207
    South of England
    So the cost benefit analysis is purely based on QALYs, and not on the primary outcomes of the main paper?
     
  12. Dolphin

    Dolphin Senior Member

    Messages:
    6,868
    Likes:
    6,146
    They did one set of calculations based on QALYs (which just use the EQ5d).
    They also did the same set of calculations using (i) whether an improvement of 2+ on the Chalder Fatigue Questionnaire was achieved and (separately) (ii) whether an improvement of 8+ on the SF-36 PF was achieved.

    They didn't use the thresholds for the CFQ or SF-36 PF in the original protocol but these really weak/easy to achieve post-hoc ones.

    They looked at each question in too ways: just looking at healthcare costs (GET/CBT/APT cost more - is it value for money in terms of the general philosophy that a health service will spend money to help patients) and then looking at societal costs (again therapies might cost more society wise but a health service can spend money to help patients within certain limits).
     
  13. user9876

    user9876 Senior Member

    Messages:
    789
    Likes:
    1,930
    I believe that is true there seems to be a few ways of converting EQ-5d into QALYs. Found this paper talking a bit about alternative techniques for generating QALYs.

    http://www.ces-asso.org/sites/default/files/H5-Edwards.pdf

    I'm still thinking about it and how errors propogate through. So far I have failed to understand the probability graphs in the paper.

    It would be interesting to see how the QALYs values and improvement they come up with compare with other diseases and treatments. My guess is there is a relatively small community of health economists that deal with these things and it makes little sense outside of that group. Surprisingly for an economics paper it is not full of equations in fact I was feeling the need for an equation or 2 to explain what they are doing.
     
  14. Bob

    Bob

    Messages:
    8,811
    Likes:
    12,207
    South of England
    Dolphin, thanks for the info.
    I've started reading it now, so I'll be better informed soon.
     
  15. Bob

    Bob

    Messages:
    8,811
    Likes:
    12,207
    South of England
    An attempt to summarise some of the main outcomes, excluding the QALY analysis.


    Differences in Overall Costs and Totals


    Table 3: Service costs at baseline and follow-up.

    Total Health Costs:

    "SMC patients had significantly lower healthcare costs than those receiving APT, CBT and GET."

    "Controlling for baseline, healthcare costs after randomisation were significantly lower for SMC than for APT (difference £840, 95% CI £637 to £1117), CBT (difference £904, 95% CI £613 to £1205) and GET (difference £829, 95% CI £534 to £1165). The differences between the APT, CBT and GET groups were small and non-significant."

    Total Societal costs (consists of: healthcare, informal care and lost employment/production costs)

    Differences between CBT, GET and SMC "were not statistically significant."

    "Societal costs (i.e. healthcare, informal care and lost production costs) were significantly lower for patients allocated to CBT compared to APT (difference £2607, 95% CI £432 to £5585). Other differences were not statistically significant."

    There is, however, a difference between the totals, even if it is not significant. CBT and GET show (statistically insignificant) total societal savings. It seems that it is this (insignificant) difference from SMC that the cost benefit analysis is partly based on.

    This is all very confusing because some of the cost benefit analysis is based primarily on societal costs/savings (the difference between CBT/GET and SMC.) So, even though health care costs are greater for CBT/GET than for SMC, and there is no 'significant' difference for societal costs between CBT/GET and SMC, the costs per QALY gained, and costs per patient improved, are considered worth the costs. (I'll try and get my head around this later.)

    For CBT and GET there are overall health costs, not savings, and so the societal savings are made up of purely out of informal care and lost employment/production costs.

    For Total Societal Costs, there would be very little cost benefit, if any, for CBT and GET, if the 'informal care' costs had not been included. So, in Table 3, I think the 'informal care' costs, alone, make GET better than SMC. CBT is still slightly better than SMC without informal care costs included.
    It seems to me that including these 'informal care' costs was optional, just as excluding welfare benefits was optional, so they were very handily included by the authors to make their case.
    If they hadn't been included, then I think they might not have been able to say that CBT and GET are significantly more cost effective than SMC, but I'm not certain about that.
    So it looks like the whole basis of the paper, could rest entirely on the informal care costs.
    Very clever.

    Informal Care costs

    "Informal care costs were substantial for each group and significantly lower after receiving CBT and GET when compared to APT and SMC."

    Lost employment.

    Lost employment 'days' are given in Table 2, and lost employment 'costs' are given in Table 3.

    The paper says that CBT and GET did not improve employment prospects:
    "There was no clear difference between treatments in terms of lost employment."

    I don't understand this because the differences between the changes for SMC vs CBT and SMC vs GET for employment costs, look highly substantial. The unadjusted figures are a relative saving of £1157 for CBT and £711 for GET. (Savings compared to the SMC control group).

    Note that there were lost employment improvements in the CBT, GET and SMC groups, but the differences between CBT/GET and SMC were not significant, so CBT/GET did not improve outcomes.

    Lost Production costs

    I think the paper says that lost production costs are the same as lost employment costs:
    "The human capital approach was used with the value of lost work-time to society assumed to be reflected by national mean age and gender-specific wage rates and combined with the lost employment data to generate lost production costs."

    I think there was no difference between groups, apart from between APT and CBT. All it says about production costs is:
    "Lost production costs were significantly higher for APT compared to CBT (difference £1279, 95% CI £141 to £2772)."





    Table 4: Welfare Benefits and Other Financial Payments

    http://www.plosone.org/article/info...RI=info:doi/10.1371/journal.pone.0040808.t004

    Note that these figures are not included in the cost effectiveness evaluations.
    I think they are the only cost data in the paper that are excluded from the final analysis.

    For the overall differences, for welfare and private 'benefits' taken as a whole, I think there were no significant difference between CBT/GET vs SMC, although it's not very clear what exactly they mean by 'benefits' in the following text:
    "However, with the exception of a difference between CBT and APT, there were no significant differences in either lost work time or benefits between the treatments during follow up. In fact, benefits increased across all four treatments."


    Income Benefits:
    The proportion of participants claiming Income Benefits increased in every therapy group.
    Looking at the unadjusted figures, compared with SMC, CBT made no significant difference to income benefits claims, and GET increased numbers on income benefits slightly, but I don't know if it is a significant difference. So there is little difference between SMC, CBT and GET.

    The text says:
    "Relatively few patients were in receipt of income-related benefits or payments from income protection schemes and differences between groups were not substantial."

    Illness/disability benefits:
    The proportion of participants claiming illness/disability benefits increased in each therapy group.
    Compared with SMC, CBT/GET resulted in a relatively lower increase in numbers on illness/disability benefits, by my rough estimation using the unadjusted figures, of about 12/13 percentage points lower increase for CBT/GET than for SMC.

    Although, this doesn't tally with the text of the paper, so maybe I've got something wrong here:
    "Receipt of benefits due to illness or disability increased slightly from baseline to follow-up (Table 4). Patients in the SMC group had the lowest level of receipt at baseline but the figures at followup were similar between groups."

    They seem to be looking at absolute numbers claiming benefits in each group, rather than the relative changes in numbers claiming benefits in each group over time. So they completely fail to comment on the relative changes in illness/disability benefits. I don't understand this. Maybe there's no statistical significance but that sort of minor detail doesn't usually stop them from making outlandish claims of success.

    So for Illness/disability benefits, there were absolute increases for CBT and GET, but relative lower increases for CBT/GET than for SMC.

    Income protection schemes or private pensions:
    The proportion of participants claiming for income protection schemes or private pensions were higher in every therapy group.
    CBT and GET resulted in relative increases, compared with SMC, (but I don't know if they are statistically significant increases), in payments from income protection schemes or private pensions. (Roughly 4 to 6 percentage points more participants made claims in the CBT and GET groups, compared with SMC, using the unadjusted figures.)

    Keeping in mind that at least one of the authors works for an insurance company, the paper avoids commenting on the increase in payments from income protection schemes and private pensions, as a result of CBT and GET:
    "Relatively few patients were in receipt of income-related benefits or payments from income protection schemes and differences between groups were not substantial."
    (Note, that they do not say that the differences were not 'significant'. Crafty!)



    So, in all benefits categories, there was an absolute increase in the proportion of participants making claims, in each of the therapy groups.

    For both 'income-related benefits' and 'income protection schemes or private pensions', the increases in claimants for CBT/GET are higher (worse) than for SMC, but they just say that the differences were not 'substantial'. They don't say that the differences are not significant.

    For illness/disability benefits, there was a lower increase in the CBT/GET groups in relation to the SMC group. The paper doesn't seem to comment about whether the differences are significant in this category.

    For the overall differences, for all welfare and private 'benefits', taken as a whole, the paper seems to assert that there were no significant difference between CBT/GET vs SMC, although it's not very clear what exactly they mean by 'benefits' in the following text:
    "However, with the exception of a difference between CBT and APT, there were no significant differences in either lost work time or benefits between the treatments during follow up. In fact, benefits increased across all four treatments."


    Interestingly, there is no separate data specifically about 'private medical insurance' claims. I don't know if they collected it. If they did, then the data can't have been good because they didn't include it.




    Well, that's my lot for now. When I look at it again, I might find that I've made some major mistakes.
    I haven't yet worked out how QALYs relate to the above info.
     
    WillowJ and Dolphin like this.
  16. currer

    currer Senior Member

    Messages:
    1,324
    Likes:
    774
    I'm sorry you're finding this so upsetting, Bob. I know you worked really hard to analyse the original Pace trial.
    It is necessary to do this but - it is also, in a way dignifying a simple piece of propaganda to treat it as a scientific paper.

    @ dolphin post 81
    I remember when our CFS/ME service was taken over by the present psychiatrically led service. They wrote out the requirement to provide specialist reports from the clinic regulations. Normally in the NHS specialists have a duty to provide these reports. So this duty had to be specifically excluded before the clinic started up. It was clear at the time they were going to operate on a basis of no disability reports - after all they believe we are not ill - we just think we are.
    Our previous clinic provided vary helpful disability reports for benefit purposes - until it was taken over by the current St Bartholomew's team.
    This has been forseen and planned for a long time. The government itself is behind this lie. It is the government's lie.

    We must reiterate the truth because eventually people will realise that it is the truth. many people are loosing faith in our governments because of this falsity and abuse of their own citizens.

    If your specialist team will not give you a report on your medical condition, which it is their duty to provide under their NHS contract, you should go to your MP.
     
  17. Bob

    Bob

    Messages:
    8,811
    Likes:
    12,207
    South of England
    Yes, it's confusing isn't it.
     
  18. Esther12

    Esther12 Senior Member

    Messages:
    5,375
    Likes:
    5,862
    Just had another read through of biophiles post on their figures - thanks for that B.

    So they didn't account for the cost of additional benefit payments in their calculations on social cost? It does seem strange that they want to take credit for all positive changes, but then ignore the negative changes!


    Also: does anyone have any idea what a normal way of calculating cost effectiveness is? eg: If a drug kept someone alive for an extra year, would they also attempt to account for the additional medical and care costs this would lead to? Or any additional economic benefits this extra year of life would tend to bring?

    These could be side issues. Really the validity of their methods for assessing their actual impact upon quality of life are more key, and I'm afraid that's an area I'm totally at sea with for the moment.


    Sorry about the rest of this post... it's little more than my own notes, as I've started scrabbling about to try to better understand the of the EQ-5D, and what sort of results we should expect were the therapists nice, charming, caring and useless. It's getting late, so I've just collated stuff here... this was my day of work to get my head around it, and I've realised I'm only scratching the surface.

    There's a 100 page report from NICE looking at criticisms that have been made of NICE's use of EQ-5d here (I didn't read much, so can't say whether it's likely to be helpful or not): http://www.nicedsu.org.uk/PDFs of reports/DSU EQ5D final report - submitted.pdf

    I've started looking for EQ-5D results from placebo and alternative treatments. To me, a big potential concern is that the much of the reported improvement could be down to politeness, positive thinking, etc. As there was no placebo control group in PACE, it might be interesting if we could find our own (I've not found anything great yet).

    This was for back pain and acupuncture (not ideal as there's some reason to think acupuncture could well have a genuinely positive affect on back pain):

    http://www.ncbi.nlm.nih.gov/pubmed/16095547

    This study looked at homeopathy and FMS, and used EQ-5d scores according to it's protocol, but I've not been able to find them yet:

    http://apps.who.int/trialsearch/trial.aspx?TrialID=ISRCTN74040048
    http://www.ncbi.nlm.nih.gov/pubmed/19358959

    This looked at a number of results for an NHS acupuncture centre, and could suggest EQ-5D is prone to distortion:

    http://acmac.net/box/Gateway Outcomes.pdf

    It also mentioned something which seemed particularly problematic if the CBT treatment was emphasising the extent to which fatigue is a normal part of human life... all people feel bad sometimes etc... if this brought down the patient's conception of what the 'best imaginable health state' was, then this would have a direct impact on EQ-5D scores.

    This one mention eq-5d, but one needs to sign up to access the paper - looks like it could be a dodgy journal (I've not yet) - it's a placebo RCT on homeopathy for some sinus thing:

    http://www.explorejournal.com/article/S1550-8307(06)00563-5/abstract

    Homeopathy for skin conditions (not seen full paper yet):

    http://www.sciencedirect.com/science/article/pii/S1876382009000766

    Here's a paper with EQ-5d scores for placebo RCT onankylosing spondylitis (?), it also includes figures for a measure of fatigue:

    http://rheumatology.oxfordjournals.org/content/46/6/999.full

    Here are graph of results, with placebo group being the least good performer:

    EQ-5d utility:

    http://rheumatology.oxfordjournals.org/content/46/6/999/F2.expansion.html

    Eq-5d vas:

    http://rheumatology.oxfordjournals.org/content/46/6/999/F1.expansion.html

    BASDAI fatigue item:

    http://rheumatology.oxfordjournals.org/content/46/6/999/F5.expansion.html

    My googling also led me to this thread from Dolphin, about a paper critical of using EQ-5d for CFS:

    http://forums.phoenixrising.me/inde...ported-outcome-measures-used-in-cfs-me.10592/
     
    biophile and Dolphin like this.
  19. Bob

    Bob

    Messages:
    8,811
    Likes:
    12,207
    South of England
    Yes, this is what I have taken from it all. The health costs are more for CBT and GET than for SMC. But the health costs are considered worth it for the QALYs gained per person and societal savings per person.
     
  20. Dolphin

    Dolphin Senior Member

    Messages:
    6,868
    Likes:
    6,146
    I think you make some useful points.
    On Lost Production costs
    "Lost production costs were significantly higher for APT compared to CBT (difference £1279, 95% CI £141 to £2772)."

    The paper doesn't mention what significance is set at. I imagine 0.05. However, there are an awful lot of comparisons so many would adjust for these multiple comparisons.
     

See more popular forum discussions.

Share This Page