Leonard Jason study - Changes in Activity & Perceived/Available Energy

[Dolphin]

Thanks for going to some trouble.
One point about any one actigraphy instrument not being fully accurate is that one is interested in comparing it with itself i.e. one isn't using one device at the start of the trial and another device at the end of the trial.

Another point about the actigraph figures is that one is dealing with big numbers. Unfortunately different studies use different figures but if one takes the Friedberg 2009 study: 224696.90 before the treatment; 203916.67 after. The smallest change on the SF-36 is 5 points or 5% of the score (which would translate to a change of 10% or more on initial scores of 50 or less).

 

[WillowJ]

Regarding how the Brits get cures out of their CBT/GET data (which indicates nothing close to cured for most of the people tested), I'm pretty sure they do something called extrapolation in science and math. That is, they get an improvement of, say 8% in 10 weeks and they figure 10 more weeks will get a total of 16% improvement, and so on. So just give them enough time and the patient will be cured!

It doesn't work that way even for many of their Oxford inclusion patients, because most psychiatric disease has unrecognized physical components. Bad coping illness exists pretty much only in the minds of a few psychiatrists and bureaucrats.

Which is why extrapolation is bad science and bad math when you go very far beyond the data you have.

 

[WillowJ]

The article is in the Library. Those in their energy envelope who decrease their activity levels also had a decrease in fatigue severity and increase in physical functioning scores as measured by the questionnaire.
It seems SF-36 isn't completely correlated with actigraph measurements, I believe the data Bleijenberg et al. suggests this also (from memory).

Library? Could you point me to that?

 

[Dolphin]

Library? Could you point me to that?

One needs 100 posts (and then a fairy godmother appears).

 

[Dolphin]

Regarding how the Brits get cures out of their CBT/GET data (which indicates nothing close to cured for most of the people tested), I'm pretty sure they do something called extrapolation in science and math. That is, they get an improvement of, say 8% in 10 weeks and they figure 10 more weeks will get a total of 16% improvement, and so on. So just give them enough time and the patient will be cured!

It doesn't work that way even for many of their Oxford inclusion patients, because most psychiatric disease has unrecognized physical components. Bad coping illness exists pretty much only in the minds of a few psychiatrists and bureaucrats.

Which is why extrapolation is bad science and bad math when you go very far beyond the data you have.

Even if the researchers themselves don't make such an extrapolation, it's what many people will automatically think if they hear that Graded Exercise Therapy worked - gradually doing more will get you there. Especially if the authors/proponents don't talk about an activity ceiling (which they nearly to a man (or woman) don't). It's such a pernicious concept.

But there have been some claims that CBT can lead to recovery or even full recovery in a percentage - all based on questionnaire data which I am suspicious of on its own (but not necessarily extrapolations like you refer).

 

[Sean]

EDIT

Question already answered by Dolphin.

 

[Sean]

Yes, all four groups reported decreases in fatigue severity and improvements in physical functioning despite two of the four groups having decreased actigraphy scores.

Indeed, in total, there was a decrease in average actigraphy scores across the whole cohort:

And this is the central point. The subjectively measured results (on which the CBT/GET school base their claims) do not correlate with the objectively measured ones. This is a serious problem for therapies based on the CBT/GET model. Even if the underlying conceptual model (de-conditioning, faulty somatic perceptions, etc) is correct*, the therapies based on it that they have so far offered simply do not work.

(*And there is no evidence that it is.)

The problem is that there doesn't seem to be any clear evidence as to wether SF-36 PF or actigraphs provide the most accurate measure of physical activity in these types of therapy trials, which makes it hard to interpret the data. Certainly, we can make a judgement as to which is best, but we can't be sure.

It is a fair question about exactly what actigraphs (or any similar measure) are actually measuring, and how accurate how they are. But I will choose the far more objective actigraph (actometers, etc) over any subjective self-reported measures, which are notoriously unreliable. At the very least any treatment studies should using these objective measures in addition to the subjective measures. It stuns and angers me that this is not the minimum requirement for these studies. Not one for the conspiracy theories, but on this issue it is hard to avoid the conclusion that the failure to use them in most behavourial studies (and where they do use them, the frequent failure to report the objective data in a timely manner, and interpret it fairly) is because the study authors are deliberately avoiding them as they don't want to properly test their model.

 

[Snow Leopard]

As another example, walking on the flat would presumably give a similar reading as walking up a hill, though walking uphill would probably be slower and possibly produce a lower reading than for walking on the level. I'd love to see some evidence that says actigraphs really do correlate with work done in the real world, especially in a rehab situation.

That is indeed a problem.
http://www.ncbi.nlm.nih.gov/pubmed/11214898

I still agree with Sean and Dolphin though - I still prefer actigraph measurements over a single questionaire. Now if patients actually wrote down exactly what they did every day (good luck getting that kind of compliance!), then self reported measures might be more accurate. But that is purely hypothetical.

Library? Could you point me to that?

There is a rumor going around that those who ask one of the Admin nicely might be granted access. (I can't confirm it though)

 

[alex3619]

Hi

Something else has been bothering me about the UK CBT/GET studies for years. In most research, a subject drop-out rate of 5% is considered acceptable. I don't recall seeing any of their studies that had a drop-out rate of less than 5%, and my memory may be faulty but I recall one at something like 50%. If, in the interest of fairness, we deduct 5% from their drop-out rate, and then count the remainder as fails, what would happen to their "success" rate, even if you believed their other results? I wonder if there is someone out there with the resources to go back and recalculate the stats for some of their studies - I am fairly sure that many if not most of the CBT/GET studies would be abject failures if we applied this standard.

Bye
Alex

 

[biophile]

CBT/GET "success" appears to be very fragile

Something else has been bothering me about the UK CBT/GET studies for years. In most research, a subject drop-out rate of 5% is considered acceptable. I don't recall seeing any of their studies that had a drop-out rate of less than 5%, and my memory may be faulty but I recall one at something like 50%. If, in the interest of fairness, we deduct 5% from their drop-out rate, and then count the remainder as fails, what would happen to their "success" rate, even if you believed their other results? I wonder if there is someone out there with the resources to go back and recalculate the stats for some of their studies - I am fairly sure that many if not most of the CBT/GET studies would be abject failures if we applied this standard.

The following information from systematic reviews may help, keeping in mind that no severely affected patients were included and most trials use Oxford 1991, CDC 1994, or even just a fatigue scale as criteria. I will mostly stick to the issue of dropouts, and to a lesser extent, adverse effects.

Cochrane 2008 (CBT)

Despite all the hyperbole we hear about recoveries and substantial improvements in the majority of patients, the analysis revealed that at post-treatment there was a small effect on reported "fatigue", and for a minority of patients (ie 40% for CBT vs 26% no therapy) a "clinical response", but no significant effect on physical functioning etc, inconclusive evidence for any long term benefits on any measurement (including anxiety and depression), mostly non-CDC criteria studies used to calculate the reported benefits.

"The mean aggregate reported dropout rate of all 15 studies was 16.4%." Range was 0%-40%, with 6 studies reporting dropout rates of over 20%. Dropout rates and reasons for dropout were broadly similar between groups, and dropout was also defined differently between studies, but on average patients assigned to CBT were significantly more likely to dropout than those assigned to usual care. Adverse effects were poorly reported.

"At follow-up, 1-7 months after treatment ended, people who had completed their course of CBT continued to have lower fatigue levels, but when including people who had dropped out of treatment, there was no difference between CBT and usual care." This finding, and that of a sub-group analysis ("there was no significant difference between CBT and waiting list control") seem to suggest fragility of the results. Limited evidence also suggests no advantage of CBT over exercise, which undermines the relevance of a targeted cognitive component.

Cochrane 2004 (GET)

Significant improvements to "fatigue" and physical functioning which did not diminish but became non-significant within several months when compared to the control group. Very limited evidence base. Notes that higher exercise intensity may explain higher dropout rates and poorer outcome. Higher rates of dropout (almost twice as high as control group) but deemed non-significant. Also, "no evidence that exercise therapy may worsen outcomes on average" (but no data was reported for adverse effects?).

Chambers et al 2006 (CBT/GET etc)

Concludes that CBT and GET may reduce symptoms and improve physical functioning for some patients. Discusses several methodological problems regarding outcome measures, differing CFS criteria, etc. On the issue of dropouts and adverse effects: "Our review did not find any new evidence of adverse effects (sufficient to cause withdrawal from treatment) associated with GET or CBT. However, reasons for withdrawals were often poorly reported and should be investigated in more detail in future studies." ... "There is limited evidence about adverse effects associated with behavioural interventions. Withdrawals from treatment in RCTs suggest that there may be an issue but the evidence is often difficult to interpret because of poor reporting."

Malouff et al 2008 (CBT)

Uses a somewhat different evidence base with more optimistic conclusions than Cochrane 2008 (eg higher effect sizes, 50% of patients no longer reporting clinical levels of fatigue after CBT at last followup). Notes that Oxford 1991 criteria has a trend towards significantly higher effect sizes than CDC 1994 criteria." (0.83 [0.40-1.26] vs 0.40 [0.02-0.78]) but is deemed non-significant. Similar dropout rates and range as Cochrane 2008, and mentions that "dropout with chronic fatigue clients is not unusually high" (compared to psychotherapy for anxiety disorders).

Surprisingly, an earlier (non-CFS) meta-analysis (Wierzbicki & Pekarik 1993) of 125 studies on psychotherapy dropout in general reports a mean dropout rate of 46.86%. I found this at the "Criticisms and questions regarding effectiveness" section of the Wikipedia article on Psychotherapy.

 

[Angela Kennedy]

Thanks for all of this information Biophile. That's very useful!

 

[Snow Leopard]

Good work Biophile.

There is also the Belgian study, which is the largest CBT study so far with published data, although not in a scientific journal.

Based on available epidemiological data for other countries, it can be estimated that about 20 000 Belgian adults suffer from CFS.
Between April 1st, 2002 and December, 31st 2004, 1,655 patients entered the reference centres, but in three centres, long waiting lists exist. For more than 90% of the patients, the diagnosis of CFS was confirmed. Most referred patients had been fatigued since a long time (average of 4.10 years)

Of the patients with confirmed diagnosis of CFS (N=862), 79% were considered to be candidates for an interdisciplinary rehabilitation program in the reference centre. For 30%, advice and education of the patient and/or his caregivers including first- and second level professionals, was proposed (additionally). At least 25% received a special referral to the physiotherapist, for 60 sessions at a reduced tariff (“F-list”); this could be
after the interdisciplinary treatment was finished.

Treatment was ended in only 2.8% of the cases by the patient himself, so the motivation of the patients for the treatment seemed to be high. In 71% the team considered the patient to have reached his maximal capacity — although no patient had been cured. Therapy provided systematically included CBT and GET. After treatment duration of 41 to 62 hours of rehabilitation per patient of which 83% group based, spread over 6 to 12 months, patients’ subjective feelings of fatigue were improved, but results concerning quality of life were equivocal. Psychological problems or psychiatric co-morbidities improved, but still fell outside the range of healthy adults. Physical capacity did not change; employment status decreased at the end of the therapy. It is difficult however, to judge these results, since no control group had been included.

http://www.kce.fgov.be/Download.aspx?ID=1222

This is the reality of so called rehabilitation of CFS patients without trying to treat the underlying cause(s).

 

[oceanblue]

"At follow-up, 1-7 months after treatment ended, people who had completed their course of CBT continued to have lower fatigue levels, but when including people who had dropped out of treatment, there was no difference between CBT and usual care."

Thanks for the elegant summary of the evidence on CBT, biophile, and I'd missed that quote on dropouts when I read the Cochrane review.

As far as I know, most of these studies used the SF36 scale to measure functioning so, using the imperfect SF36 scale, CBT is of neglible benefit.

 

[oceanblue]

[actigraphy can't tell the difference between walking uphill and downill] is indeed a problem.
http://www.ncbi.nlm.nih.gov/pubmed/11214898

Thanks for the link.

An objective measure of activity is better, in theory, than a subjective measure like a questionnaire. However, we're still lacking hard evidence on whether the imperfect actigraphy data is a better measure of activity than the imperfect SF36 PF. I think what we really need here is a better mousetrap.

 

[WillowJ]

One needs 100 posts (and then a fairy godmother appears).

I could really use a fairy godmother! :Sign giggle:

 

[WillowJ]

There is a rumor going around that those who ask one of the Admin nicely might be granted access. (I can't confirm it though)

Hmm, it couldn't hurt to try.

 

[WillowJ]

Even if the researchers themselves don't make such an extrapolation, it's what many people will automatically think if they hear that Graded Exercise Therapy worked - gradually doing more will get you there. Especially if the authors/proponents don't talk about an activity ceiling (which they nearly to a man (or woman) don't). It's such a pernicious concept.

But there have been some claims that CBT can lead to recovery or even full recovery in a percentage - all based on questionnaire data which I am suspicious of on its own (but not necessarily extrapolations like you refer).

I can see how that would be a problem, too. Especially since some people diagnosed with CFS have other conditions that may eventually resolve without intervention (or may fluctuate from normal to sub-normal).

 

[WillowJ]

Thanks for the link.

An objective measure of activity is better, in theory, than a subjective measure like a questionnaire. However, we're still lacking hard evidence on whether the imperfect actigraphy data is a better measure of activity than the imperfect SF36 PF. I think what we really need here is a better mousetrap.

What about that beeping watch that Natleson et al. used to get better compliance with questionnaires, I think in addition to an actigraph?

 

[WillowJ]

Hi

Something else has been bothering me about the UK CBT/GET studies for years. In most research, a subject drop-out rate of 5% is considered acceptable. I don't recall seeing any of their studies that had a drop-out rate of less than 5%, and my memory may be faulty but I recall one at something like 50%. If, in the interest of fairness, we deduct 5% from their drop-out rate, and then count the remainder as fails, what would happen to their "success" rate, even if you believed their other results? I wonder if there is someone out there with the resources to go back and recalculate the stats for some of their studies - I am fairly sure that many if not most of the CBT/GET studies would be abject failures if we applied this standard.

Bye
Alex

I have seen where they compare their drop-out rates to (CBT?) programs for anxiety, say they are comparable [not sure of the veracity of this claim-edit: looks right from biophile's post], and strongly imply that the reason for dropping out is the same between the two groups (totally ignoring the controvery over harm and ineffectiveness). They seem to consider this an acceptable and complete method of addressing drop-outs.

I've also read studies where they make the assumption that drop-outs had no change (rather than checking to see if, oh, I dunno, maybe they got worse) and actually factor that assumption of no change into their final analysis.

 

[WillowJ]

"The mean aggregate reported dropout rate of all 15 studies was 16.4%." Range was 0%-40%, with 6 studies reporting dropout rates of over 20%. ...

Surprisingly, an earlier (non-CFS) meta-analysis (Wierzbicki & Pekarik 1993) of 125 studies on psychotherapy dropout in general reports a mean dropout rate of 46.86%. I found this at the "Criticisms and questions regarding effectiveness" section of the Wikipedia article on Psychotherapy.

Rule of thumb is that drop-out rate of more than 20% makes the entire study unreliable, although I can't find my reference for that right now.