Implementing CBT for CFS in a mental health center: a benchmarking evaluation (Scheeres et al, 2008)

[Dolphin]

I'm posting this now mainly because of the second message:

J Consult Clin Psychol. 2008 Feb;76(1):163-71. doi: 10.1037/0022-006X.76.1.163.
Implementing cognitive behavioral therapy for chronic fatigue syndrome in a mental health center: a benchmarking evaluation.
Scheeres K1, Wensing M, Knoop H, Bleijenberg G.


Abstract

OBJECTIVE:

This study evaluated the success of implementing cognitive behavioral therapy (CBT) for chronic fatigue syndrome (CFS) in a representative clinical practice setting and compared the patient outcomes with those of previously published randomized controlled trials (RCTs) of CBT for CFS.

METHOD:

The implementation interventions were the following: spreading information about the new treatment setting to general practitioners and CFS patients; training mental health center (MHC) therapists in CBT for CFS; and organizing changes in the MHC patient workflow.

Patient outcomes were documented with validated self-report measures of fatigue and physical functioning before and after treatment.

The comparison of the treatment results with RCT results was done following the benchmark strategy.

RESULTS:
One-hundred forty-three CFS patients were referred to the MHC, of whom 112 started treatment. The implementation was largely successful, but a weak point was the fact that 32% of all referred patients dropped out shortly after or even before starting treatment.

Treatment effect sizes were in the range of those found in the benchmark studies.

CONCLUSIONS:
CBT for CFS can successfully be implemented in an MHC.

Treatment results were acceptable, but the relatively large early dropout of patients needs attention.

PMID:

18229994

[PubMed - indexed for MEDLINE]

 

[Dolphin]

Abstract: Treatment effect sizes were in the range of those found in the benchmark studies.

Here's what the results section reported:

"Comparison of treatment effects.

The noncontrolled effect sizes of the implementation study and the benchmark studies are given in Table 4 and in Figures 1 and 2.

The mean pre-post treatment effect size of the four benchmark studies for fatigue was {1.02 (Sharpe et al., 1996) + 2.05 (Deale et al., 1997) + [3 * 1.25 (Prins et al., 2001)] + 1.83 (Stulemeijer et al., 2005)} / 6 = 1.44 (95% confidence interval [CI] = 0.97, 1.89).

This is somewhat higher than the effect size of fatigue in the MHC, which was 1.12 (95% CI + 0.85, 1.38).

For physical functioning, the mean pre- post treatment effect size of the benchmark studies was {1.93 (Deale et al., 1997) = [3 * 0.71 (Prins et al., 2001)] + 1.19 (Stulemeijer et al., 2005)} / 5 = 1.04 (95% CI = 0.63, 1.44).

This is again somewhat higher than the effect size at the MHC for physical functioning, which was 0.64 (95% CI = 0.38, 0.89)."

Does anyone know if either or both of these two sets are different?

 

[Dolphin]

They used a lax definition of recovery:

The recovery rate was analyzed by calculating the percentage of patients clinically significantly improved. Patients were defined as clinically significantly improved at posttreatment if they had a reliable change index + 1.96 on the CIS-20 Fatigue Severity subscale, a Fatigue Severity score <= 35, and a Rand-36 Physical Functioning score >= 65 (Vercoulen et al., 1999)."

 

[Simon]

Here's what the results section reported:

"Comparison of treatment effects.

The noncontrolled effect sizes of the implementation study and the benchmark studies are given in Table 4 and in Figures 1 and 2.

The mean pre-post treatment effect size of the four benchmark studies for fatigue was {1.02 (Sharpe et al., 1996) + 2.05 (Deale et al., 1997) + [3 * 1.25 (Prins et al., 2001)] + 1.83 (Stulemeijer et al., 2005)} / 6 = 1.44 (95% confidence interval [CI] = 0.97, 1.89).

This is somewhat higher than the effect size of fatigue in the MHC, which was 1.12 (95% CI + 0.85, 1.38).

For physical functioning, the mean pre- post treatment effect size of the benchmark studies was {1.93 (Deale et al., 1997) = [3 * 0.71 (Prins et al., 2001)] + 1.19 (Stulemeijer et al., 2005)} / 5 = 1.04 (95% CI = 0.63, 1.44).

This is again somewhat higher than the effect size at the MHC for physical functioning, which was 0.64 (95% CI = 0.38, 0.89)."

Does anyone know if either or both of these two sets are different?

The first thing to say is that this is a weird way to look at effectiveness - pre-post treatment rather than treatment vs control. Much of the pre-post 'effect' could be placebo or natural improvement, which is why Cochrane Reviews, for instance, don't use it.

To add to the problem, you can't compare different patient groups eg if one clinic had patients who had been ill for less time and were younger, they would be expected to perform better than those from a clinic with older patients who had been ill for longer. Comparisons of treatment vs control helps take account of such differences.

So I don't think the comparisons are valid, and I certainly don't think you can apply formal statistical tests to the data without knowing if the patient pops are similar.

If the figures were comparable...
fatigue
MHC: 1.12 (95% CI + 0.85, 1.38).
Benchmarks: 1.44 (95% confidence interval [CI] = 0.97, 1.89).

Physical function
MHC: 0.64 (95% CI = 0.38, 0.89)
Benchmarks: 1.04 (95% CI = 0.63, 1.44).
In both cases the 95% confidence intervals of the benchmark scores include the mean MHC score, so it would be reasonable to say they weren't different. I certainly wouldn't risk a formal stats test on them (& would need n and SD for that)

 

[Dolphin]

Thanks Simon