Jason/DePaul: The implications of using a broad versus narrow set of criteria in research

[mango]

The implications of using a broad versus narrow set of criteria in research

Leonard A. Jason, Kristen Gleason, Pamela Fox

Journal of Medicine and Therapeutics, 1(2), 1-6. doi: 10.15761/JMT.1000111

Abstract
The Fukuda et al. criteria is the most widely used clinical case definition for diagnosing patients with chronic fatigue syndrome (CFS).

Despite the frequency with which the Fukuda criteria are applied, the list of symptoms outlined in this case definition were not well enough specified to be easily applied to research settings.

In 2005, Reeves et al. laid out a set of standards for operationalizing the Fukuda definition, specifying scales and cutoff scores for measuring the symptom criteria. This operationalization, often known as the empirical criteria, has been shown to identify an unexpectedly large number of patients, seemingly widening the net of inclusion for CFS diagnostic criteria.

However, in a recent study in 2016 by Unger and colleagues it has been suggested that the 2005 Reeves et al. 2005 operationalization of the Fukuda criteria does not over-identify the number of patients with CFS as had been previously reported.

This article reviews prior studies which provide context for these findings and offers a possible explanation for the discrepancies.

Clearly, determining what case definition to use and how to operationalize it remains an important activity for scientists in this field, as it will influence work in multiple domains, including etiology, pathophysiology, epidemiology and treatment.

Key words
chronic fatigue syndrome, epidemiology, case definitions

https://oatext.com/the-implications...arrow-set-of-criteria-in-research.php#Article

 

[Tom Kindlon]

A lot of this paper is about the CDC's empiric criteria (Reeves et al 2005) and this particular study:

3. Unger ER, Lin JM, Tian H, Gurbaxani BM, Boneva RS, et al. (2016) Methods of applying the 1994 case definition of chronic fatigue syndrome - impact on classification and observed illness characteristics. Popul Health Metr 14: 5. [Crossref]

Here is a comment I previously wrote on this study:

Depression scores in this follow-up study are very different to scores in original study (looking solely at the Reeves et al. (2005) operationalization)

Leonard Jason and colleagues previously raised concerns about the Reeves et al. (2005) chronic fatigue syndrome (CFS) criteria [which have also been described as an operationalisation of the Fukuda et al (1994) criteria] (1-4). In particular, Jason and colleagues were concerned that some people who did not have CFS might get diagnosed with CFS using this new set of criteria. They found some evidence to support this concern in a study of those with major depressive disorder who did not have CFS: 38% were found to satisfy these new criteria for CFS(4).

Looking solely at the current study, it would look like there might have been little basis for these concerns. Of 71 people classified with CFS in the current study, only one (1.4%) had a Zung self-rating depression scale (SDS) (5) score of >=60. The mean SDS score for the 71 CFS participants was 44.78 (calculated from the data in Table 4) (6).

However, it should be noted that the SDS (depression) scores in the follow-up study are very different from the scores in the original Georgia cohort(7). Of the 113 people diagnosed with CFS in the original Georgia cohort, data for 112 (99.1%) was published(7). The average SDS score was considerably higher at 56.2. Possibly more revealingly, 40.2% had a SDS score of >=60. As described in the paper, the SDS scale provides an index score and categories reflecting no (<50), mild (50-59), moderate (60-69), and severe (>=70) depression.

I am not sure why there should be such a large difference in a cohort between the initial and follow-up studies in the rate of those with moderate or severe depression (40.2% vs 1.4%). But it does mean that caution should be used in terms of interpreting the findings reported in the current paper and their significance regarding the Reeves et al. (2005) criteria (1,6).

References:

[1]. Reeves WC, Wagner D, Nisenbaum R, Jones JF, Gurbaxani B, Solomon L, Papanicolaou DA, Unger ER, Vernon SD, Heim C. Chronic fatigue syndrome--a clinically empirical approach to its definition and study. BMC Med. 2005 Dec 15;3:19.

[2]. Fukuda K, Straus SE, Hickie I, Sharpe MC, Dobbins JG, Komaroff A. The chronic fatigue syndrome; a comprehensive approach to its definition and study. Ann Int Med 1994, 121:953-959.

[3]. Jason LA, & Richman JA. How science can stigmatize: The case of chronic fatigue syndrome. Journal of CFS 2007;14:85-103.

[4]. Jason LA, Najar N, Porter N, Reh C. Evaluating the Centers for Disease Control's empirical chronic fatigue syndrome case definition. Journal of Disability Policy Studies 2009;20;93.

[5]. Zung WW, Richards CB, Short MJ. Self-rating depression scale in an outpatient clinic: further validation of the SDS. Arch Gen Psychiatry.1965;13(6):508-515.

[6]. Unger ER, Lin JM, Tian H, Gurbaxani BM, Boneva RS, Jones JF. Methods of applying the 1994 case definition of chronic fatigue syndrome - impact on classification and observed illness characteristics. Popul Health Metr. 2016 Mar 12;14:5.

[7]. Heim C1, Nater UM, Maloney E, Boneva R, Jones JF, Reeves WC. Childhood trauma and risk for chronic fatigue syndrome: association with neuroendocrine dysfunction. Arch Gen Psychiatry. 2009 Jan;66(1):72-80

 

[Tom Kindlon]

From the current paper:

It is possible that Unger and colleagues [3] were successful in eliminating psychiatric confounds, as they incorporated a number of methods to screen out these exclusionary conditions. Indeed, only one of the CFS cases they identified was diagnosed with moderate to severe depression. If this most recent iteration of the empirical criteria was successful in screening out those with exclusionary illnesses, then this might help explain why the two diagnostic methods (i.e., the traditional approach and the empirical approach) in the Unger et al. [3] study resulted in the identification of a fairly similar number of CFS cases. However, many, if not most, ongoing CFS studies do not include structured clinical interviews aimed at identifying exclusionary psychiatric problems and ruling out the other potential causal factors.

 

[Tom Kindlon]

Good point I thought:

Additionally, there is likely to be a good amount of variability in how this case definition is used. In particular, the potential for variation in the methods used to assess substantial reduction has not yet been adequately explored. Operationalizing key concepts outlined in the Fukuda criteria is important. For example, it would be useful to find a reproducible way to specify fatigue as outlined in Fukuda [1]: “chronic fatigue that is of new or definite onset (i.e., not lifelong). The fatigue is not the result of ongoing exertion. The fatigue is not substantially alleviated by rest.” To this end, others have outlined a way to define “lifelong,”3 which is indeed a challenging task [23].

Let’s examine how Unger and colleagues [3] operationalized “not substantially alleviated by rest.” First the person would need to answer “no” to fatigue was made a lot better by rest to fulfill this requirement. But if they responded “yes” to fatigue was made a lot better by rest, they could be included if their fatigue was relieved by rest “some of the time,” “a little of the time,” “or hardly ever.” They would be not included if they said that their fatigue was relieved by rest “all of the time” or “most of the time.” The problem with this approach stems from the fact that much of the time, rest does relieve fatigue symptoms for many patients with CFS. However, for these patients, rest is not fully curative and does not increase the stamina and endurance necessary to carry on life tasks. Therefore, while it is important to operationalize this part of the Fukuda case definition, it is critical to do so in a way that distinguishes between those who’s rest fully eliminates the symptom complex and those form whom this does not occur (e.g., patients with CFS). It is equally important to determine if CFS induced fatigue is result of ongoing exertion. The failure of the Unger et al. article [3] and the empiric criteria to address this key issue of ongoing exertion causing the fatigue is problematic. In other words, unless questions have been carefully crafted and validated, a person could meet the CFS diagnosis whose fatigue is mainly due to excessive exertion, and with lifestyle issues such as being over-committed.