Assessing severity and self-report treatment outcomes in children with CFS/ME

[Simon]

Child Care Health Dev. 2014 Mar 24. doi: 10.1111/cch.12135. [Epub ahead of print]
Assessing severity of illness and outcomes of treatment in children with Chronic Fatigue Syndrome/Myalgic Encephalomyelitis (CFS/ME): a systematic review of patient-reported outcome measures (PROMs).
Haywood KL1, Collin SM, Crawley E.

Abstract

Chronic Fatigue Syndrome or Myalgic Encephalomyelitis (CFS/ME) in children is characterized by persistent or recurrent debilitating fatigue which results in a substantial reduction in activity.

There is a growing interest in the use of questionnaires, or patient-reported outcome measures (PROMs), to assess how patients function and feel in relation to their health and associated healthcare. However, guidance for PROM selection for children with CFS/ME does not exist.

We reviewed the quality and acceptability of PROMs used with children with CFS/ME to inform recommendations for practice. We conducted a systematic review of PROMs completed by children with CFS/ME. The quality of the evaluative studies and the reviewed measures were assessed against recommended criteria using an appraisal framework and the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. We sought evidence of measurement (reliability, validity, responsiveness, interpretability, data quality) and practical properties (acceptability, relevance, feasibility).

Sixteen articles were included in the review, providing evidence of reliability and/or validity for 13 PROMs. Of these, five were child-specific (one health-related quality-of-life; four emotional well-being) and eight were not (four emotional well-being, three fatigue-specific; and one generic).

All measures had limited evidence of measurement properties and no evidence of practical properties. Recommendations for patient-reported assessment are difficult to make because of limited evidence of the quality and acceptability of PROMs for children with CFS/ME.

The appraisal method highlighted significant methodological and quality issues which must be addressed in future research.

There is a lack of qualitative evidence describing the outcomes of healthcare that are important to children with CFS/ME, and the relevance or appropriateness of available measures.

Future PROM development and evaluation in this group must seek to involve children collaboratively to ensure that the outcomes that children care about are assessed in an acceptable way.
--------------

I like this study for two reasons. First, using outcome measures that matter to patients is very important in my view and rarely happens. For example, the Chalder Fatigue Scale uses questions dreamt up by Simon Wessely and did not attempt to involve patients to check the scale actually measured fatigue in a way that was relevant to people with ME/CFS. That's not the only problem with the Chalder scale, but it's a big one.

Secondly, Kirstie Haywood has already published a brilliant piece of work assessing ME/CFS PROMS (patient-reported outcome measures) across the board (ie inc for adults), that was exceptionally rigorous. This concluded that not only did most measures fail to ensure they were measure outcomes that matter to patient, they were also poorly designed scales (ie didn't measure the right thing, and measured the wrong thing badly).

While I also believe we need to use objective measures to assess treatment outcomes, eg activity-measuring devices like actomters, I think focusing on outcomes chosen by patients is important too eg fatigue and cognitive functioning might be captured by a social functioning outcome measure.

 

[Cheshire]

There is a growing interest in the use of questionnaires, or patient-reported outcome measures (PROMs), to assess how patients function and feel in relation to their health and associated healthcare.

They use the very same tools that will never make them change their point of view, it's the snake that bites its own tail. A "growing interest", (sight...) we're not finsihed with all that crap...

 

[Simon]

Continuing to plough my lonely furrow here. The user-friendly summary at the end says:

Key messages

• The poor quality and acceptability of available measures
(PROMs) limits current recommendations for the patient reported
assessment of childhood CFS/ME of relevance to
routine practice, service evaluation or research settings.

• Existing measures have significant methodological flaws
which must be addressed in future research.

• Little is known about the outcomes of healthcare that
really matter to children with CFS/ME, and the relevance
of available measures to this group has not been evaluated.

• Future PROM development and evaluation must seek to
involve children with CFS/ME more actively in the process
to ensure that the outcomes that really matter to children
are assessed in an acceptable way.

Further highlights from the full text:

To clarify, this study focuses on studies that evaluated the PROMs in children with CFS, while the previous review had looked at studies using adults. Though in many cases the same scales in both studies eg the Chalder Fatigue scale has been tested in both children and adultts.

No evidence that current PROMs measure the right thing
This is the killer line in the review:

PROM content or face validity, relevance, or appropriateness to children with CFS/ME was not assessed in any of the reviewed studies.

In other words, the research reviewed on these questionnaires hadn't shown they are measuring what is actually meaningful or relevant - which you might actually expect to be the starting point for choosing a PROM. If it isn't measuring the right thing, the statistical qualities of a questionnaire is immaterial.

The paper comments on the emphasis in literature on emotional measures ahead of anything else eg functioning:

... It is unclear if aspects of emotional well-being reflect the outcomes of healthcare most valued by people (adults and children) with CFS/ME [very dry comment]. Although healthcare should seek to deliver the outcomes that matter most to patients (Department of Health 2010), there is a dearth of qualitative evidence which seeks to understand what really matters to adults or children with CFS/ME. This evidence is crucial to informing the appropriate evaluation of health and social care, and to enhancing our understanding of the relevance of available PROMs.

Pointing the way ahead?:

Moreover, a child-centred, collaborative approach to developing a paediatric PROM for CFS/ME is essential to enable assessment of the true burden of CFS/ME and the relative success of healthcare.

There's a lot of detail about specific scales evaluated eg CIS-20 and Chalder Fatigue Scale but unless there is a clamour for more I will leave it there.

 

[Esther12]

Seems like a decent paper that points out a lot of things that have already been said on here.

Rather wish these problems had been explored before 20 years of research and clinical practice was based on poorly thought through outcome measures though.

re clamour for more: I don't want to give you chores, but if you thought that they had anything interesting to say about the more commonly used scales (Chalder Fatigue), I'd be interested.

 

[Dolphin]

My guess is that a child would be unlikely to consider themselves recovered unless they could take part in high intensity activities like sports and dancing.

(I don't think adults are really recovered until they do them either but I think a child should be able to do them even more)

 

[Dolphin]

Recognition of what children with CFS/ME have to say about their health and health needs is essential for the provision of tailored health and social care, and to inform appropriate evaluation (Fitzpatrick et al. 1998).

Fitzpatrick, R., Davey, C., Buxton, M. J. & Jones, D. R. (1998) Evaluating patient-based outcome measures for use in clinical trials. Health Technology Assessment, 2, i–74.

The growing focus of many healthcare systems on patient outcomes supports the use of high-quality patient-reported outcome measures (PROMs) to provide a patient-centred assessment of how they function, feel and survive with respect to their health and associated healthcare (Devlin & Appleby 2010). Well-developed PROMs are often self-completed or interview-administered questionnaires which contain a number of questions, or items. Items are selected to provide a structured assessment of health across a range of concerns from symptoms and physical functioning, to well-being and quality-of-life.

Devlin, N. & Appleby, J. (2010) Getting the most out of PROMs. Putting health outcomes at the heart of NHS decision-making.

 

[Dolphin]

Despite identifying a recent growth in the number of studies which have sought to evaluate PROM application in children with CFS/ME (Haywood et al. 2007), clear recommendations for PROM selection cannot be made because of the poor quality and limited evidence of relevance or acceptability to children.

High-quality measures require robust evidence of a range of essential measurement and practical properties (Nunnally & Bernstein 1994; Streiner & Norman 2008). However, evidence of test–retest reliability, measurement error, structural validity, data quality and precision, and score interpretation was not identified for any of the 13 reviewed measures; only 7/13 had evidence of internal consistency reliability, identified from just 2/16 included articles. None of the studies examined whether the PROMs used were relevant or appropriate for children.

The majority of evaluations included relatively small numbers of subjects hence, results should be interpreted with caution (Mokkink et al. 2010). All the included studies included mainly older children and adolescents hence and our results are not necessarily applicable to younger children. Neither are our results necessarily generalizable to the general population as most populations studied were identified from specialist clinics.

 

[Dolphin]

Qualitative research to explore the perspective of key groups, including children and their parents, is essential to support greater involvement in PROM development and evaluation, to identify the outcomes of healthcare that really matter, question structure that is relevant and acceptable, and to work towards more collaborative ways of working (Staniszewska et al. 2012).

 

[Dolphin]

re clamour for more: I don't want to give you chores, but if you thought that they had anything interesting to say about the more commonly used scales (Chalder Fatigue), I'd be interested.

Three fatigue-specific measures were reviewed. All were developed for adult populations; only the CIS-20 was developed for adults with CFS/ME (Vercoulen et al. 1994). Although limited, the Chalder Fatigue Scale (Chalder et al. 1993) and the CIS-20 have promising evidence of measurement properties: only the Chalder Fatigue Scale has evidence of reliability, validity (uncertain evidence) and responsiveness following completion by UK-based groups. However, the Chalder Fatigue Scale asks respondents to compare their symptoms with the ‘usual’. Children with profound fatigue may change their view, or internal standards, of what ‘usual fatigue’ is (Schwartz et al. 2007), and hence report relatively low levels of fatigue. This ‘response shift’ and potential limitation of the response scale has not been specifically explored in children, and the resulting underestimation of fatigue is unknown. Although the CIS-20 has limited to moderate evidence of validity, there is limited or uncertain evidence of reliability and responsiveness respectively. It has only been evaluated in a Dutch population. However, alternative fatigue measures exist (Hewlett et al. 2007; Elbers et al. 2012), which should be assessed for content, relevance and appropriateness to children with CFS/ME. For example, the Profile of Fatigue-Related Symptoms (PFRS) (Ray et al. 1992) was originally developed for adults with CFS/ME. It assesses fatigue across four distinct domains and has evidence of good content validity in this group (Haywood et al. 2012); further evaluations in children with CFS/ME would be helpful.

 

[Dolphin]

Aside: I don't recall noticing changes to the SF-36 PF between version 1 and version 2. Anyone know what the changes were?

There is insufficient evidence to support a recommendation for use of the SF-36 (domains or summary scales) in children with CFS/ME.No alternative generic measures have been evaluated in this group. The revised SF-36 (version 2) has not been evaluated in this group, but the revised response options for the PF domain should improve data quality and performance (Ware et al. 2001). Given that the combined use of generic and specific measures is recommended (Fitzpatrick et al. 1998; Fitzpatrick 2009), the concurrent evaluation of the SF-36v2 against alternative generic and specific measures in this group is required. However, evidence of the appropriateness and acceptability of the SF-36 to children is essential.

 

[Esther12]

However, the Chalder Fatigue Scale asks respondents to compare their symptoms with the ‘usual’. Children with profound fatigue may change their view, or internal standards, of what ‘usual fatigue’ is (Schwartz et al. 2007), and hence report relatively low levels of fatigue. This ‘response shift’ and potential limitation of the response scale has not been specifically explored in children, and the resulting underestimation of fatigue is unknown.

I don't see why that would just be a problem for children, especially if adults were going through a course of CBT intended to encourage them to view their symptoms as normal.