The utility of patient-reported outcome measures among patients with me/cfs

[snowathlete]

Qual Life Res. 2016 Sep 6. [Epub ahead of print]
The utility of patient-reported outcome measures among patients with myalgic encephalomyelitis/chronic fatigue syndrome.
Murdock KW1, Wang XS2, Shi Q3, Cleeland CS3, Fagundes CP1,3, Vernon SD4.
Author information

Abstract
PURPOSE:
Debilitating fatigue is a core symptom of myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS); however, the utility of patient-reported symptom outcome measures of fatigue for ME/CFS patients is problematic due to ceiling effects and issues with reliability and validity. We sought to evaluate the performance of three patient-reported symptom measures in a sample of ME/CFS patients and matched controls.

METHODS:
Two hundred and forty ME/CFS patients and 88 age, sex, race, and zip code matched controls participated in the study. Participants completed the Multidimensional Fatigue Inventory-20, DePaul Symptom Questionnaire, and RAND SF-36.

RESULTS:
The general and physical fatigue subscales on Multidimensional Fatigue Inventory-20, as well as the role of physical health on the RAND SF-36, demonstrated questionable or unacceptable internal consistency and problematic ceiling effects. The DePaul Symptom Questionnaire demonstrated excellent internal reliability, and less than 5 % of participants were at the ceiling on each subscale. The post-exertional malaise subscale on the DePaul Symptom Questionnaire demonstrated excellent clinical utility as it was able to differentiate between ME/CFS patients and controls (OR 1.23, p < .001) and predicted ceiling effects on other patient-reported outcome subscales. A score of 20 on the post-exertional malaise subscale of the DePaul Symptom Questionnaire optimally differentiated between patients and controls.

CONCLUSIONS:
Significant ceiling effects and concerns with reliability and validity were observed among Multidimensional Fatigue Inventory-20 and RAND SF-36 subscales for ME/CFS patients. The DePaul Symptom Questionnaire addresses a number of concerns typically identified when using patient-reported outcome measures with ME/CFS patients; however, an improved multidimensional patient-reported outcome tool for measuring ME/CFS-related symptoms is warranted.

 

[Bob]

This seems like a useful study. I'm making a note of it for future reference. It's a shame that they didn't include the Chalder fatigue scale so it could get a good trashing.

 

[Sean]

One dodgy subjective self-report primary outcome measure down, one to go.

 

[user9876]

This seems like a useful study. I'm making a note of it for future reference. It's a shame that they didn't include the Chalder fatigue scale so it could get a good trashing.

I think the CFQ is so bad they may have thought it wasn't even worthy of study.

 

[Dolphin]

Ceiling effects, defined as a proportion of patients reporting maximum scores on a given measure more than 40 % of the time [23, 25],

I wonder is this a common definition of ceiling effects? I've never seen it before. I think if even a lower percentage are at the maximum it can still be a problem.

 

[Dolphin]

The vast majority of patient-reported outcome measures targeting fatigue were not designed to measure post-exertional malaise, a hallmark of ME/CFS [20]. The DePaul Symptom Questionnaire [29] was designed to evaluate symptoms of ME/CFS, including post-exertional malaise and has demonstrated excellent convergent and discriminant validity [19], as well as test–retest reliability [10].

Indeed, methodological concerns such as measurement of a variety of different symptoms were noted in a review of the effectiveness of interventions for treating ME/CFS symptoms [30].

30. Whiting, P., et al. (2001). Interventions for the treatment and management of chronic fatigue syndrome—A systematic review. Journal of the American Medical Association, 286(11), 1360–1368.

 

[Dolphin]

ME/CFS participants were eligible for inclusion in the study if they had been previously diagnosed with ME/CFS using the Fukuda [3] or Canadian criteria [11]. Participants with ME/CFS also had to have post-exertional malaise lasting[24 h and significant cognitive impairment in short-term memory and concentration.

 

[user9876]

I wonder is this a common definition of ceiling effects? I've never seen it before. I think if even a lower percentage are at the maximum it can still be a problem.

Really a ceiling effect is a clipped distribution. There is some maths around clipped normal distributions but the equations are quite hideous.

 

[Dolphin]

Although a widely utilized measure evidencing strong psychometric characteristics [33], the RAND SF-36 has demonstrated problems measuring substantial reductions in functioning among ME/CFS patients [13]. Scores were reverse coded such that higher scores indicate more severe impairment in functioning.

Important to note if reading this paper

 

[Dolphin]

Probably not of interest to that many people, but these are commonly used questionnaires in ME/CFS especially the SF 36.

Internal consistency was questionable for the reduced motivation subscale (a = .69) on the Multidimensional Fatigue Inventory-20 for the ME/CFS sample. For the control group, internal consistency on the physical fatigue scale was in the questionable range on the Multidimensional Fatigue Inventory-20 (a = .69). All other scales were within the acceptable to good range on the Multidimensional Fatigue Inventory-20 (a = .72–.81) for ME/CFS and control participants. On the RAND SF-36, internal consistency was unacceptable for the general health subscale (a = .56), and the energy/fatigue subscale was within the questionable range (a = .68), for the ME/ CFS sample. In the control sample, two RAND SF-36 subscales were within the unacceptable range (i.e., role limitations due to emotional problems (a = .48) and emotional well-being (a = .46), while the general health subscale was within the questionable range (a = .66).

 

[Dolphin]

Table 2 Means, standard deviations, and ceiling effects for subscales among the ME/CFS and control samples
Measure

ME/CFS patients (n = 240)

Controls (n = 88)

Number (%) of ME/CFS patients at ceiling

Number (%) of controls at ceiling

Again probably not of interest to that many people but I thought it was interesting to see the percentages at the ceilings for the various subscales. Again remember that the SF 36 scores are reversed to the normal scoring method.

 

[Dolphin]

Differentiating ME/CFS patients and control participants

To determine if measure subscales were able to effectively differentiate between ME/CFS and control participants, a series of logistic regression analyses were conducted. Separate analyses were run for each measure (see Table 3). For the Multidimensional Fatigue Inventory-20, the physical fatigue subscale significantly predicted group membership. In regard to the RAND SF-36, the role limitations due to physical health and general health subscales were significant predictors of whether or not participants were in the ME/CFS and controls samples. For the DePaul Symptom Questionnaire, the cognitive, post-exertional malaise, and sleep subscales significantly differentiated between the ME/CFS and control participants.

The only thing about this was that patients were required to have postexertional malaise so this finding is not that surprising (though healthy controls were not required not to have postexertional malaise).

ME/CFS participants were eligible for inclusion in the study if they had been previously diagnosed with ME/CFS using the Fukuda [3] or Canadian criteria [11]. Participants with ME/CFS also had to have post-exertional malaise lasting[24 h and significant cognitive impairment in short-term memory and concentration.

 

[Snow Leopard]

Really a ceiling effect is a clipped distribution. There is some maths around clipped normal distributions but the equations are quite hideous.

Beauty is in the eye of the beholder!

 

[RogerBlack]

Not forgetting of course:
https://www.ncbi.nlm.nih.gov/pubmed/24064428
Effects of Time Frame on the Recall Reliability of CFS Symptoms.

Results suggested that the optimal time frame for reliably reporting CFS symptoms was six months for sore throat, lymph node pain, muscle pain, post-exertional malaise, headaches, memory/concentration difficulties, and unrefreshing sleep. For joint pain, the optimal time frame was one month. Researchers who are interested in the assessment of CFS symptoms need to take recall time frame into account, especially when the intended goal is to standardize and improve the methods used to reliably and accurately diagnose this complex illness.

Meaning several things, but amongst them a distortion of what 'normal' is is likely after a few months.