Davis, Racaniello et al: An open letter to PLoS One

[Dolphin]

I'm sorry i must be really stupid but there still seems to be something missing from this explanation. If the more expensive care (1.) is considered cost effective how could lowering the cost not be considered cost effective?

Because one is comparing it with specialist medical care alone which costs less as one doesn't have to pay for all those therapist hours. The difference between the extra costs and extra savings decreased to a negligible level when one lowers the value of the saving.

 

[Dolphin]

The paper reported that CBT and GET were cost effective under all assumptions. Three different assumptions were made for "informal care", which would be care provided by family members:

1. That informal care was valued at the cost of a homecare worker.

They said:

Alternative methods exist for valuing informal care, with the opportunity cost and replacement cost approaches being the most recognised. We adopted the former and valued informal care at £14.60 per hour based on national mean earnings

Office for National Statistics (2010) Annual survey of hours and earnings (ASHE) - 2010 results. Available: http://www.statistics.gov.uk/statbase/Product.asp?vlnk=1951. Accessed 2011 Oct 3

I think £14.60 is national mean earnings for everyone, not homecare workers.

Mean earnings are higher than median earnings as they include millionaires, etc.

By way of comparison, the minimum wage was £5.93. Homecare workers don't tend to be that well paid.

I re-read what the investigators said and understand it a bit better now compared to when I wrote earlier. Opportunity cost is the cost you miss out on. So the authors estimated the wages the person who provided the care lost out on, using
national mean earnings in the UK.

So in effect it doesn't make any reference to the wages of a care worker.

 

[Esther12]

I re-read what the investigators said and understand it a bit better now compared to when I wrote earlier. Opportunity cost is the cost you miss out on. So the authors estimated the wages the person who provided the care lost out on, using
national mean earnings in the UK.

So in effect it doesn't make any reference to the wages of a care worker.

Still though, the mean hourly wage does not account for the cost/time of getting to/finding/training for work.

It seems a strange way of justifying cost-effectiveness to me... but then, this is in addition to using self-reported QoL to assess interventions like CBT and GET in an unblinded trial, so it's far from the most serious problem. Also things like this could discourage self-reporting care/support:

There's this from p95 of the CBT Patients Manual. (There might be more elsewhere, I've just had a skim.)

""""""
The "wrong" kind of social support

This may seem a contradiction in terms! The examples below illustrate how the wrong kind of support can make it more difficult for you to move forward for the following reasons:

• If you have a very supportive family member (partner, parent or child) who is used to doing everything for you, it may be difficult for you to increase your activity levels. Your relative may feel that they have your best interest at heart and discourage you from doing more. They may have difficulty accepting that in order to make progress, you need to do things at regular times even if you are feeling very fatigued. If family members have been your "carer" during your illness, they can sometimes feel that they no longer have a role when you are getting better which can sometimes lead them to be critical of your CBT programme or suggest that you are making yourself worse. This may then lead you to question the validity of the programme and deter you from persevering with it particularly when you have a lot of symptoms.
""""""

From p102 of the GET Patients Manual.

""""
What happens if I don't like exercise?

No problem. The important thing to know is that you can chose (sic) any form of activity - for
example DIY, household jobs, craft work or gardening.

""""
(ie. if you were being encouraged to count housework / chores as part or all of your 'exercise' routine this could affect how you report your need for 'informal care'.)

 

[Bob]

I'm sorry i must be really stupid but there still seems to be something missing from this explanation. If the more expensive care (1.) is considered cost effective how could lowering the cost not be considered cost effective?

That's a very good but confusing question! You had me beat for a while!

I think the answer must be that, with reference to wages, we're not talking about the costs associated with the therapy but we're talking about the costs saved by the therapy. i.e. long-term care costs are saved if the therapy is successful. If there are lower savings then that compares less favourably to the alternative intervention, which is SMC in this case.

 

[Dolphin]

The point about valuing informal care at 0 is that is how they promised to analyse the data.

With different results, it could have suited them to value informal care at 0.

In the statistical analysis plan, they said they would analyse the data in 3 ways.

In the cost effectiveness paper they said:

"Fourth, we made assumptions regarding the value of unpaid care from family and friends and lost employment. However, sensitivity analyses revealed that the results were robust for alternative assumptions."

which simply is not true for the other 2 assumptions.

 

[user9876]

I re-read what the investigators said and understand it a bit better now compared to when I wrote earlier. Opportunity cost is the cost you miss out on. So the authors estimated the wages the person who provided the care lost out on, using
national mean earnings in the UK.

So in effect it doesn't make any reference to the wages of a care worker.

Using the mean is still not a good idea because it is so skewed to a few very well paid people in the city. They would of course be able to hire carers so if they want to look at opportunity costs associated with those needing to perform care then they should use the median. But often caring becomes an additional thing that families do rather than one of loosing work time so if they are looking at the opportunity costs they need to look at reduced work hours due to care commitments.

 

[BurnA]

I thought it was a mistake to emphasise that access to the data was not needed in the way they did.

I actually think it was very smart of them to say that. The author has already contradicted his paper so asking for the data would only be a distraction and they could hide behind data protection claims. This way, there is no dispute over the data they have to correct the paper immediately.

 

[Esther12]

I actually think it was very smart of them to say that. The author has already contradicted his paper so asking for the data would only be a distraction and they could hide behind data protection claims. This way, there is no dispute over the data they have to correct the paper immediately.

Yeah, I can see that, but personally I'd have done something to emphasise that while the release of data is not needed for this point, it is important.

 

[BurnA]

Yeah, I can see that, but personally I'd have done something to emphasise that while the release of data is not needed for this point, it is important.

They did suggest it as an option.
So if he doesn't provide the data he cant really justify the claims...its like their worst nightmare, the only way they can justify their claims is to provide data without being explicitly asked for it.

so he can either provide a reasonable explanation, produce the actual sensitivity analyses demonstrating “robustness” under all three assumptions outlined in the statistical analysis plan, or correct the paper’s core finding that CBT and GET are “cost-effective” no matter how informal care is valued

 

[BurnA]

I'm sorry i must be really stupid but there still seems to be something missing from this explanation. If the more expensive care (1.) is considered cost effective how could lowering the cost not be considered cost effective?

My interpretation is that they are comparing CBT / GET cost effectivness to informal care

1. That informal care was valued at the cost of a homecare worker.
2. That informal care was valued at the minimum wage.
3. That informal care was valued at zero.

Readers, including Tom Kindlon, pointed out that under assumption 2 and 3, CBT and GET would not be cost effective.

Given that CBT/GET have considerable costs, if they are compared to the most expensive alternative the conclusion might be that they are cost effective treatments. However if you compare them to the cheapest alternatives they wouldn't necessarily be cost effective.

The cheapest alternatives in this case were suggested by the paper authors, and while the costs associated with these alternatives may not reflect reality, that is what they proposed so that is what they should have reported against.

 

[Dolphin]

http://retractionwatch.com/2016/05/...ow-common-is-failure-to-reproduce/#more-40641

Retraction Watch
Tracking retractions as a window into the scientific process

Weekend reads: Improper influence by NFL; dissertations for sale; how common is failure to reproduce?

[..]

A group of researchers is pressing PLOS ONE for action on a 2012 paper reporting findings from the controversial PACE trial of chronic fatigue syndrome. (Virology Blog)