Leonard Jason study - Changes in Activity & Perceived/Available Energy

[Dolphin]

Good work Biophile.

Based on available epidemiological data for other countries, it can be estimated that about 20 000 Belgian adults suffer from CFS.

Between April 1st, 2002 and December, 31st 2004, 1,655 patients entered the reference centres, but in three centres, long waiting lists exist. For more than 90% of the patients, the diagnosis of CFS was confirmed. Most referred patients had been fatigued since a long time (average of 4.10 years)

Of the patients with confirmed diagnosis of CFS (N=862), 79% were considered to be candidates for an interdisciplinary rehabilitation program in the reference centre. For 30%, advice and education of the patient and/or his caregivers including first- and second level professionals, was proposed (additionally). At least 25% received a special referral to the physiotherapist, for 60 sessions at a reduced tariff (F-list); this could be after the interdisciplinary treatment was finished.

Treatment was ended in only 2.8% of the cases by the patient himself, so the motivation of the patients for the treatment seemed to be high. In 71% the team considered the patient to have reached his maximal capacity although no patient had been cured. Therapy provided systematically included CBT and GET. After treatment duration of 41 to 62 hours of rehabilitation per patient of which 83% group based, spread over 6 to 12 months, patients subjective feelings of fatigue were improved, but results concerning quality of life were equivocal. Psychological problems or psychiatric co-morbidities improved, but still fell outside the range of healthy adults. Physical capacity did not change; employment status decreased at the end of the therapy. It is difficult however, to judge these results, since no control group had been included.

There is also the Belgian study, which is the largest CBT study so far with published data, although not in a scientific journal.

http://www.kce.fgov.be/Download.aspx?ID=1222

This is the reality of so called rehabilitation of CFS patients without trying to treat the underlying cause(s).

Yes. The piece on the Belgian rehab (CBT/GET) clinics is pages 77-83. They spin it slightly.

There was a full audit of these clinics which was published in French and Dutch. This report is over 200 pages long. These were Fukuda patients.
There is some information (in English) on the data in this thread: http://www.forums.aboutmecfs.org/sh...ehab-clinics-(large-sample)&highlight=belgium . The audit is really fascinating.

 

[Sean]

What about that beeping watch that Natleson et al. used to get better compliance with questionnaires, I think in addition to an actigraph?

Compliance is an issue, but it is not the main issue.

The issue is that it is fairly easy to get people to change their subjective reports, which is all the questionnaires are measuring. They are not measuring actual changes, but people's views or beliefs on changes.

As imperfect and limited as actigraphs, actometers, etc, are, at least they are genuinely objective measures. If they report a change we can be vastly more certain those changes are real. What the implications of those changes are is a different issue, but they are reliable measures of change.

An objective measure of activity is better, in theory, than a subjective measure like a questionnaire.

It is always better (more reliable). This is the definition of objective.

I simply cannot agree that we do not know if the subjective questionnaires are more reliable than the objective actigraphs. Obviously the objective measures are more reliable, with all their limits. I will take the objective measures over the subjective ones any day.

 

[oceanblue]

As imperfect and limited as actigraphs, actometers, etc, are, at least they are genuinely objective measures. If they report a change we can be vastly more certain those changes are real. What the implications of those changes are is a different issue, but they are reliable measures of change.

> An objective measure of activity is better, in theory, than a subjective measure like a questionnaire

It is always better (more reliable). This is the definition of objective.

The actigraph is objective in the sense that it is independent of user opinion, as opposed to a questionnaire that is by definition subjective.

However, that does not itself make an 'objective' measure more reliable than a subjective measure. Does the actigraph reliably/accurately measure energy expenditure? Not particularly according to the published research. It reliably measures acceleration at the body but takes not account of the amount of work being done as in walking on hilly ground vs walking on the flat. And presumably the actigraph reliably measures changes in acceleration, but that doesn't necessarily translate into changes in physical activity.

If it's objective but inaccurate then it isn't necessarily better.

Presumably we are all agreed that what matters in measuring these CBT trials is whether or not patients are able to do more as a result of the intervention? Deciding that an acitgraph is a better measure of this simply because it is independent of the patient (who, lets face it, might have some insight in how much they can do) remains a personal judgement unitl there is better research evidence that actigraphy accurately measures physical activity. I still think we need a better mousetrap.

 

[oceanblue]

Suggestion for a better (or calibrated) mousetrap

I've just had a thought about how to resolve this situation: measure patient energy expenditure directly using the DLW, 'doubly-labelled water' method which researchers seem to agree is the most reliable way to meausre energy used. Then compare this data with actigraphy and SF36 data.

Next time there's a CBT (or other therapy) trial using actigraphy and SF36, some participants could be selected for the DLW measurements at baseline and at the end of the trial. The method isn't arduous for patients as it involves ingesting a sample of Heavy water (ie doubly labelled) then providing some blood or urine samples. Then the changes in energy expenditure measured by DLW can be directly compared with the actigraphy and SF36 data for the same patients to reveal the relative accuracy of SF36 PF vs actigraphy.

Measuring energy expenditure directly with DLW is expensive so wouldn't be feasible for all patients, but would be ideal for calibration work, and the actigraph manufacturers (and maye the SF36 licensors too) would fund such a study.

And the results would be more conclusive than our discussions here!

 

[Dolphin]

The actigraph is objective in the sense that it is independent of user opinion, as opposed to a questionnaire that is by definition subjective.

However, that does not itself make an 'objective' measure more reliable than a subjective measure. Does the actigraph reliably/accurately measure energy expenditure? Not particularly according to the published research. It reliably measures acceleration at the body but takes not account of the amount of work being done as in walking on hilly ground vs walking on the flat. And presumably the actigraph reliably measures changes in acceleration, but that doesn't necessarily translate into changes in physical activity.

If it's objective but inaccurate then it isn't necessarily better.

Presumably we are all agreed that what matters in measuring these CBT trials is whether or not patients are able to do more as a result of the intervention? Deciding that an acitgraph is a better measure of this simply because it is independent of the patient (who, lets face it, might have some insight in how much they can do) remains a personal judgement unitl there is better research evidence that actigraphy accurately measures physical activity. I still think we need a better mousetrap.

I have little confidence that people can accurately recall all the little bits of activity they do across the day.
I know of one severely affected person who does strict pacing (so only does tiny activities within her house generally) who showed up as 1500-2000 steps a day.

And I can well believe that a treatment like CBT or GET could mislead people in how they answer questions like how limited they are in walking certain distances - because people are generally encouraged to go for walks. Friedberg used diaries and was confident enough that substitution was what was happening i.e. how patients could increase the length of the walk but not be doing anymore in total. Another person might similarly use their energy to walk around a supermarket but say they are limited a lot with regard to walk a mile or half a mile (as they haven't cut out all the activities the person who goes for a regular walk has). So I think the SF-36 has particular problems for CBT and GET studies and maybe other studies.

Also, when comparing the figures with people with other adults - who might be working 40 hour weeks - there are problems when using a questionnaire like the SF-36 (when asking about ability to walk distances: one isn't necessarily making a like-for-like comparison). There are probably going to be lots of other circumstances like this e.g. somebody with young children. Part of the attraction with a measure like the SF-36 is that one is supposed to compare to the general population. But I don't think that is that easy with ME/CFS where people may not be doing a lot of the normal things other people do as well as any walk.

So I think the actigraph is a lot better for it. In some areas, one might need to know actual calorie use e.g. on a weight loss program. But with trials in ME/CFS, one is only interested in change so (as I have said before) one is comparing like for like, there is less of a problem if an actigraph is not 100% accurate.

 

[oceanblue]

I have little confidence that people can accurately recall all the little bits of activity they do across the day.

Nor do I, but I'm pretty sure that SF-36 doesn't require them to, it only asks what kind of activity your illness interferes with.

Also, when comparing the figures with people with other adults - who might be working 40 hour weeks - there are problems when using a questionnaire like the SF-36

The SF36 is designed for studying ill people across a wide range of disabilities; yes it can be used on the general population for measuring norms but is widely used across a whole range of illnesses. I thought that was the point of it.

I certainly agree there are reasons why the SF36 could give misleading results. The same is true of actigraphs.

In particular, the actigraphs seem to give different readings for different types of activity so if someone just does more or less of the same sort of activity, they probably will measure that change fairly well. But if someone switches to more or less arduous activity then the actigraph could misrepresent that. This summer, when my health improved for a while, I was able to venture to the bottom of my relatively steep garden. I didn't walk any further than when I just wandered around in the house or at the top of the garden, but I did use a lot more energy. I don't think the actigraph would have spotted it.

I would still dearly like to see some evidence on the accuracy of different ways of measuring physical activity. That's why I think it would be sensible for someone to do some proper Energy Expenditure measurements alongside actigraphs, questionnaires or whatever else people want to use as a proxy for physical activity. Give me data!

 

[oceanblue]

For the record, I do think that the conflicting actigraphy data raises doubts over the SF36 findings of CBT etc studies, particuarly as SF36 is a subjective measure and susceptible to bias

However, I don't think actigraphs are quite the definitive answer that researchers and the manufacturers imply, and think they merit further scrutiny so that we can be sure of what's really happening - in the spirit of robust science.

 

[Dolphin]

I've just had a thought about how to resolve this situation: measure patient energy expenditure directly using the DLW, 'doubly-labelled water' method which researchers seem to agree is the most reliable way to meausre energy used. Then compare this data with actigraphy and SF36 data.

Next time there's a CBT (or other therapy) trial using actigraphy and SF36, some participants could be selected for the DLW measurements at baseline and at the end of the trial. The method isn't arduous for patients as it involves ingesting a sample of Heavy water (ie doubly labelled) then providing some blood or urine samples. Then the changes in energy expenditure measured by DLW can be directly compared with the actigraphy and SF36 data for the same patients to reveal the relative accuracy of SF36 PF vs actigraphy.

Measuring energy expenditure directly with DLW is expensive so wouldn't be feasible for all patients, but would be ideal for calibration work, and the actigraph manufacturers (and maye the SF36 licensors too) would fund such a study.

And the results would be more conclusive than our discussions here!

I would like to spend more money spent on objective outcome measures in ME/CFS research so certainly wouldn't have any problem with such a study.
If you look at the 5m PACE Trial, the exercise tests aren't going to use gases. So for the 6-minute walk test for example, I could see one group pushing their body harder than another group (say the pacing group might not push themselves too hard). So the exercise test isn't necessarily going to give us good data about level of fitness. I am going to have little confidence in the outcome measures of this trial which is going to cost 5m of UK taxpayers' money.

 

[Snow Leopard]

Here is the SF-36:
http://www.wsib.on.ca/wsib/wsibobj.nsf/LookupFiles/POCMTBIRand/$File/MTBI_RAND.pdf

It lacks both specificity and objectivity in my opinion. I don't think it can be considered reliable for a treatment, who's major component is to change beliefs. All you are going to measure is a change towards overestimating capability.

As for DLW, that is extremely invasive. I doubt it would be possible to do more than a limited case study of a handful of people with such a method.

 

[oceanblue]

I would like to spend more money spent on objective outcome measures in ME/CFS research so certainly wouldn't have any problem with such a study.
If you look at the 5m PACE Trial, the exercise tests aren't going to use gases. So for the 6-minute walk test for example, I could see one group pushing their body harder than another group (say the pacing group might not push themselves too hard). So the exercise test isn't necessarily going to give us good data about level of fitness.

I'm suggesting a Doubly-labelled Water test, no gases or exercise tests involved. Instead it measures total energy use over a given period, which is what makes it such a powerful technique. http://www.answers.com/topic/double-labelled-water-technique. Appears to be the best available method for measuring energy expenditure outside lab situations, which is why it is used to evaluate actigraphs, but equally can be compared with SF36 data. As you say, the PACE trial is using none of this (and I can't see why they use actigraphs at baseline if they didn't use them at follow-up).

 

[oceanblue]

As for DLW, that is extremely invasive. I doubt it would be possible to do more than a limited case study of a handful of people with such a method.

Not that invasive, in that it's a one-off dose of heavy water, and (non ME/CFS) studies regularly use DLW. As i said above, the idea would be to only apply this to a section of the patients and only do it once, as a way of calibrating SF36 and actigraphs, not to use it as the principal measure - which would indeed be impractical.

 

[Dolphin]

As you say, the PACE trial is using none of this (and I can't see why they use actigraphs at baseline if they didn't use them at follow-up).

I'm not sure if it has been said before in this thread but the cynic in mean things they were scared about what it would show. The initial protocol did have it - that was the protocol for which AfME sent a letter of support to the MRC. It seems to me quite likely that at least one of the four funders (DWP, NHS, Scottish Chief Scientists Office and MRC) could have approved a grant based on that protocol. Anyway, DLW is interesting - I have come across it before (it was an article about compliance to diets) but the way they talked it seemed to be expensive so never thought about it in terms of ME/CFS studies but maybe it could have a value. Anyone know how much it might cost?

 

[oceanblue]

I'm not sure if it has been said before in this thread but the cynic in mean things they were scared about what it would show. The initial protocol did have it - that was the protocol for which AfME sent a letter of support to the MRC. It seems to me quite likely that at least one of the four funders (DWP, NHS, Scottish Chief Scientists Office and MRC) could have approved a grant based on that protocol.

Would be interesting to know if the Trial Steering Committee had a formal rationale for accepting the change, not least because including actigraphy at baseline but not follow-up is simply a waste of money, as well as time.

Anyway, DLW is interesting - I have come across it before (it was an article about compliance to diets) but the way they talked it seemed to be expensive so never thought about it in terms of ME/CFS studies but maybe it could have a value. Anyone know how much it might cost?

I read somewhere that the Heavy Water used costs over 500 per person, though I think that is the major cost by some way. But looking at the PACE trial, to do it for 100 participants might have cost 100,000 so pretty affordable relative to their budget. Even more so given that actigraphy results elsewhere has cast doubt on the SF36 data, and such DLW work could provide a better answer to the key question: "does this therapy actually help patients to do more, as opposed to believing they are doing more?"

 

[WillowJ]

the key question: "does this therapy actually help patients to do more, as opposed to believing they are doing more?"

Yes, that would be the key. Or, as opposed to the researchers believing the patients are doing more (based on results from whatever measurement is being used).

I don't see any reason to expect why any exercise (other than merely to empty lymphs and maybe light stretching, both of which could be accomplished with careful light massage, and perhaps some orthopedic mobilization for the comparatively well patients, to preserve the patient's energy for activities of daily living) should help in the absense of medical intervention targeting ion channel dsyfunction, nerve conduction dysfunction, mitochondrial dysfunction, immune dysfunction, hypovalemia, cardiac issues when present, and probably other as-yet-undiscovered problems (we don't think we fully understand all the contributors to PEM).

I have a question for someone with access to the full text. What inclusion criteria was used? Jason has talked a lot about how there are lots of different kinds of patients in the CFS construct, many of which don't have the actual disease that was supposed to have been being studied under that name [ME]. Is he saying exercise is useful to some of the non-ME patients in the CFS construct? Or did he use his new ME/CFS revised Canadian definition?

 

[Snow Leopard]

This is a new paper based on the results of an older study "Nonpharmacologic Interventions for CFS A Randomized Trial" (2007). The 1994 Fukuda definition was used as well as the 2007 "CFS questionaire" (Jason et al.) - the 'revised CCC' guidelines did not exist then.

 

[oceanblue]

Questions over use of SF-36

On this post, the use of the SF-36 has been ciriticised as a subjective measure. I've been looking at the SF-36 more closely and have some separate concerns both over its wording and overall suitability for measuring trial outcomes.

Wording issues

Question 3 of the SF-36, which underlies the SF-36 Physical Functioning (PF) subscale score, asks if your health limits in 10 different types of activiites. The second of these focuses on:
Moderate activities, such as moving a table, pushing a vacuum cleaner, bowling, or playing golf

Pushing a vacuum cleaner and playing a round of golf in the same option? This doesn't make sense to me.

Question 4, which gives rise to the Role Physical, RP, subscale asks
During the past 4 weeks, have you had any of the following problems with your work or other regular daily activities as a result of your physical health?
The first of 4 sub-questions is
Cut down on the amount of time you spent on work or other activities

The problem with the wording is that it seems to be concerned with how things have changed recently, so if you have a chronic illness you might not have 'cut down' at all but still be very limitied in what you can do. Such amibiguous wording could affect how patients respond.

Overall Suitablility

First off, the SF-36 was developed as a generic, rather than illness-specific measure. This makes it good for comparing between illnesses and between healthy and ill populations but doesn't necessarily make it ideal for looking at changes within a given group of ill people.

Secondly, the SF stands for Short Form: the SF-36 used simplified questioning that was intended to be quick and easy to complete (increasing completion rates by respondents) but at a cost of 10-20% loss of accuracy of the longer questionnaires it was based on. That may be a compromise well worth making in, say, a huge survey of households, but is harder to justify in a therapy trial where the SF36 may be the primary outcome measure.

Finally, and perhaps most importantly, there doesn't seem to be much evidence on the validity of the SF36 in treatment trials. This is from the SF36 website:

Results from clinical studies comparing scores for patients before and after treatment have largely supported hypotheses about the validity of SF-36 scales based on psychometric studies. For example, clinical studies have shown that three of the scales (PF, RP, and BP) with the most physical factor content (Figure 2) tend to be most responsive to the benefits of knee replacement (Kantz et al., 1992), hip replacement (Kantz et al., 1992; Lansky, Butler, & Waller, 1992), and heart valve surgery (Phillips & Lanky, 1992).

Which is not exactly compelling evidence.

Of course, none of this makes actigraphy more reliable, but it does cast further douby on the suitablity of the SF36 for measuring changes in physical activity in treatment trials.

I still believe a Doubly Labelled Water measure of Energy Expenditure is the best way to validate/cailbrate whatever measure is used, but as well as testing actigraphy and SF36, researchers might also consider whether there are better questionnaires to use than the SF36.

 

[WillowJ]

This is a new paper based on the results of an older study "Nonpharmacologic Interventions for CFS A Randomized Trial" (2007). The 1994 Fukuda definition was used as well as the 2007 "CFS questionaire" (Jason et al.) - the 'revised CCC' guidelines did not exist then.

very good, thanks. I feel better now.

 

[Dolphin]

I'm just making my way through this thread - didn't get to read all the messages at the time. Have just read the full paper.

Krupp Fatigue Severity Scale

8. Fatigue is among my three most disabling symptoms.

It has occured to me that there could be a problem with this. If one does an intervention and it makes some symptoms worse (or even if one deteriorated for another reason), fatigue may no longer be one of the three most disabling symptoms - the fatigue score may improve but that doesn't mean the fatigue actually improved!

 

[Dolphin]

On this post, the use of the SF-36 has been ciriticised as a subjective measure. I've been looking at the SF-36 more closely and have some separate concerns both over its wording and overall suitability for measuring trial outcomes.

Wording issues

Question 3 of the SF-36, which underlies the SF-36 Physical Functioning (PF) subscale score, asks if your health limits in 10 different types of activiites. The second of these focuses on:
Moderate activities, such as moving a table, pushing a vacuum cleaner, bowling, or playing golf

Pushing a vacuum cleaner and playing a round of golf in the same option? This doesn't make sense to me.

Question 4, which gives rise to the Role Physical, RP, subscale asks
During the past 4 weeks, have you had any of the following problems with your work or other regular daily activities as a result of your physical health?
The first of 4 sub-questions is
Cut down on the amount of time you spent on work or other activities

The problem with the wording is that it seems to be concerned with how things have changed recently, so if you have a chronic illness you might not have 'cut down' at all but still be very limitied in what you can do. Such amibiguous wording could affect how patients respond.

Overall Suitablility

First off, the SF-36 was developed as a generic, rather than illness-specific measure. This makes it good for comparing between illnesses and between healthy and ill populations but doesn't necessarily make it ideal for looking at changes within a given group of ill people.

Secondly, the SF stands for Short Form: the SF-36 used simplified questioning that was intended to be quick and easy to complete (increasing completion rates by respondents) but at a cost of 10-20% loss of accuracy of the longer questionnaires it was based on. That may be a compromise well worth making in, say, a huge survey of households, but is harder to justify in a therapy trial where the SF36 may be the primary outcome measure.

Finally, and perhaps most importantly, there doesn't seem to be much evidence on the validity of the SF36 in treatment trials. This is from the SF36 website:



Which is not exactly compelling evidence.

Good points. The Chalder fatigue scale, the other primary outcome measure in the PACE Trial, also in the FINE Trial and other trials has:

For each question, patients have to say whether they have the symptom:

0. Less than usual ____
1. No more than usual ____
2. More than usual ____
3. Much more than usual ____

so again not good for people with a chronic illness.

I remember reading once that a particular study told the patients to interpret it as compared to their pre-illness level. But there were lots and lots of times there was no mention of this in a paper.

 

[Dolphin]

Hi

Something else has been bothering me about the UK CBT/GET studies for years. In most research, a subject drop-out rate of 5% is considered acceptable. I don't recall seeing any of their studies that had a drop-out rate of less than 5%, and my memory may be faulty but I recall one at something like 50%. If, in the interest of fairness, we deduct 5% from their drop-out rate, and then count the remainder as fails, what would happen to their "success" rate, even if you believed their other results? I wonder if there is someone out there with the resources to go back and recalculate the stats for some of their studies - I am fairly sure that many if not most of the CBT/GET studies would be abject failures if we applied this standard.

Bye
Alex

In the Cochrane review of GET, which can be downloaded for free at:
http://www.cochrane.org/reviews/en/ab003200.html (press full text button)

Analysis 1.3. Comparison 1 Exercise Therapy vs Control (treatment as
usual or relaxation + flexibility), Outcome 3 Acceptability of Treatment.

Test for overall effect: Z = 1.71 (P = 0.088)

The normal test of significance is p<0.05. Authors will often then refer to p values where 0.10<=p<0.05 as a trend. (occasionally, p<0.1 may even be defined as a significant result e.g. De Lange 2008 (Grey Matter Volume, CBT and CFS study)

The text plays it down:

"D - Drop-out: Drop-out was more common among exercise therapy participants (23/161) than among control participants (13/154) (RR 1.73, 95% CIs 0.92 to 3.24,), although the difference was not significant."

but the authors thank Peter White for his assistance and we know Peter White likes to say GET is say and effective.