Exploring Case Definition for CFS/ME (Leonard Jason, 2013)

[Simon]

Examining case definition criteria for chronic fatigue syndrome and myalgic encephalomyelitis

Background: Considerable controversy has transpired regarding the core features of myalgic encephalomyelitis (ME) and chronic fatigue syndrome (CFS). Current case definitions differ in the number and types of symptoms required. This ambiguity impedes the search for biological markers and effective treatments.

Purpose: This study sought to empirically operationalize symptom criteria and identify which symptoms best characterize the illness.

Methods: Patients (n = 236) and controls (n  = 86) completed the DePaul Symptom Questionnaire, rating the frequency and severity of 54 symptoms. Responses were compared to determine the threshold of frequency/severity ratings that best distinguished patients from controls. A Classification and Regression Tree (CART) algorithm was used to identify the combination of symptoms that most accurately classified patients and controls.

Results:
A third of controls met the symptom criteria of a common CFS case definition when just symptom presence was required; however, when frequency/severity requirements were raised, only 5% met the criteria.

Employing these higher frequency/severity requirements, the CART algorithm identified three symptoms that accurately classified 95.4% of participants as patient or control: fatigue/extreme tiredness, inability to focus on multiple things simultaneously, and experiencing a dead/heavy feeling after starting to exercise.

Conclusions:

Minimum frequency/severity thresholds should be specified in symptom criteria to reduce the likelihood of misclassification.

Future research should continue to seek empirical support of the core symptoms of ME and CFS to further progress the search for biological markers and treatments.
----

The basic finding that symptom frequency and severity are important in separating patients and controls is interesting. In the PACE recovery paper, the authors argued that most of the population had several mecfs symptoms at any time and implied that patients shouldn't get too hung up on such things - though the population study PACE cited was flawed and in any case showed the symptoms only affected daily activity for a small minority of the healthy.

 

[wdb]

Good to see some research being done in that area, it's a shame there are not more detailed results, it would be interesting to look for any sub-groups exclusive to the patients, and also to see if any commonly used criteria are ineffective at discriminating.

Would also be good to know the criteria used to diagnose the patients as that would presumably have to a large extent dictated the results.

 

[caledonia]

I'm confused. How were the controls picked out? Aren't controls usually healthy controls? If so, why would a few of them show up with ME/CFS?

 

[wdb]

I'm confused. How were the controls picked out? Aren't controls usually healthy controls? If so, why would a few of them show up with ME/CFS?

They were healthy controls, I think that is one of the points that they are making is that some diagnostic criteria are so loose particularly when only symptom presence is measured that even healthy people sometimes meet those criteria.

 

[Simon]

Commentary on full text

The stuff in large quote boxes can be skipped if you don't want too much detail.

Geeky stuff about patients and diagnosis
All the patient data came from the CFIDS SolveCFS BioBank using the "DePaul Symptom Questionnaire" devised by Leonard Jason and colleagues. All the Biobank patients were 'diagnosed by a licensed physician' according to Fukuda or CCC criteria:no more details are given, but the biobank patients all came from 'expert physician clinics' so presumably Klimas, Peterson etc.

Note that all 3 diagnoses: Fukuda, CCC or ME-ICC in the study were based on replies to the DePaul questionnaire, not the original physician diagnosis (though the DePaul questionnaire was designed to diagnose so should be OK given that patients also had an expert physician diagnosis). However, the version of the DePaul questionnaire used didn't have questions on 2 ICC symptoms: intolerance to temperature extremes and 'susceptible to viral infections with prolonged recovery', so the ICC diagnosis in this study was described as ICC-adjusted.

Symptom severity matters: many healthy controls have enough mild symptoms for ME/CFS
Jason points out the the three case definitions don't define individual symptom severity or frequency (though ICC requires overall symptom severity to require a 50% reduction in activity levels) and it turns out that a good chunk of healthy controls have enough mild, infrequent symptoms to meet the case definitions. Requiring symptoms to be at least moderate, present for at least half the time mean only a small minority of healthy controls would meet the case definitions:

% controls meeting case definition with mild v moderate symptoms
Fukuda: 34% mild; 5% >= moderate
CCC: 21% mild; 4%
ICC: 15% mild; 4%

The authors acknowledge a significant problem with the DSQ approach, ie that it doesn't take account of acitivity levels:

symptoms were not asked about in relation to activity; as some patients may only experience certain symptoms in response to activity, the prevalence of these symptoms may be underrepresented in this study’s results.

Given that most patients use pacing specifically to minimise symptom severity and frequency this is an important issue.

The paper also points out that at this higher symptom severity/frequency threshold less than half of CCC and ICC patients have most of the individual immune, neuroendocrine and autonomic symptoms. In contrast, most have almost all of the sleep, PEM and neurocognitive symptoms.

end of part 1

 

[Simon]

At the heart of this paper is a data mining technique called CART: Classification and Regression Tree, which was used to classify individuals as "CFS" or "Controls" based on symptom data. A good explanation of how CARTs work comes from Wikipedia:

CART prediction example: Surviving The Titanic
OK, using data mining to 'predict' who will survive a historical event might seem a bit odd but it illustrates the point about how a few pieces of data on a passenger: age, sex and number of siblings/spouse on board can be used to accurately predict an outcome. For Jason's study the 'outcome' is CFS or control, but the principle is the same as for these Titanic passengers and uses a simple decision tree.
click on the image to enlarge

A "CART" tree showing survival of passengers on the Titanic
As with the Jason paper, just three pieces of data are needed for the prediction:
1. Sex:
Basically, if you are male you are probabl going to die
2. Age:
If you a male younger than 9.5 years you are going to die
3. If you are a male aged 10 or more, you might well survive IF you also have several siblings/spouse aboard

Essentially, a computer sifts through all the data looking for which are the key few questions that will sort those that lived from those that died for the Titanic. For Jason's study, the computer looked at the few specific symptoms that would divide CFS cases from healthy controls.

CART model predicts CFS using three symptoms
Using a CART, and couting symptoms only when they were at least moderate and present for at least half the time, just 3 symptoms (out of 54) were able to predict if a person was a CFS case or a heathy control:
- fatigue or extreme tiredness
- inability to focus on more than one thing at a time [neurocognitive problem]
- experiencing a dead or heavy feeling after starting to exercise ["Post-exertional Malaise" problem]

The CART model classified 95% of patients and controls correctly (nb these cases had originally been diagnosed as either Fukuda or ICC patients). There is always a risk of 'overfitting' a model to any particular dataset so the authors randomly split the sample. 66% of patients and controls made up a "Training Set" that was used to develop the model. The 95% accuracy rate was on the remaining "Test" set. This reduces the influence of any overfitting, but ideally the CART model would also be tested on a completely independent sample too.

A "dead or heavy feeling after starting exercise" and even "inability to focus on more than one thing at a time" might not seem like the most obvious cardinal symptoms to define mecfs. However, the dead/heavy feeling is one of several symptoms in the DSQ that captures PEM and not multi-tasking is one of several neurocognitive symptoms, and a feature of CART models is that the precise symptom they pick can be a bit quirky (and perhaps is down to overfitting). But very similar results would probably come from using a model with different PEM or neurocognitive symptoms, because the individual PEM/neurocogitive symptoms are likely to be highly correlated with one another.

Case for a few core symptoms?
Leonard Jason has long argued for focusing on a few core symptoms ie creating a tighter definition by insisting on a few core symptoms (none are mandatory in Fukuda so you can be diagnosed without PEM and without neurocog symptoms) - as opposed to more mandatory symptoms. CCC and ICC effectively do both. This paper revisits the argument:

Future refinement of case definitions may wish to focus on requiring a small set of core symptoms, such as post-exertional malaise, neurocognitive symptoms, and possibly unrefreshing sleep.

The danger with adding a requirement for many symptoms is that it may inadvertently select pateints with more psychiatric symptoms. A recent study by Jason et al found just this, but that study also found a higher level of disability - and more psychiatric symptoms may just be a response to more impairment.

Intriguingly, a submitted article from Jason (presumably using the same data as this new study) supports a case definition based on PEM, Neurocognitive problems and a third factor encompassing a host of other symptoms covered by the CCC and ICC to various degrees:

A factor analysis by Brown and Jason [28] resulted in a three-factor solution that supports such a case definition structure. The analysis resulted in one factor comprised of post-exertional malaise items, one factor of neurocognitive items, and one larger factor that encompassed pain, immune, neuroendocrine, and autonomic items.

Factor analysis basically mines the data to find common cluster, where each 'factor' is made up of a group of symptoms that cluster together.

Congratulations to those who made it this far, but these posts are a lot shorter than the full text ...

 

[Firestormm]

Thanks very much @Simon

So, then. Jason at least would say that there is a need for a new definition. Interesting in light of the IOM study. I do hope they take his stuff on board. Shame he isn't on the panel.

I reviewed his earlier paper on one of the IOM threads. I think this move towards frequency and severity of core symptoms has been something he's been exploring for a while. It might have been the paper you linked to - I'd have to check.

My question then is this: in the UK we have 3 or 4 (depending) severities once you have received a diagnosis - I wonder if this would impact on those?

Mild, Moderate, Severe, Very Severe​

I mean Jason's method must surely help to categorise the degree to which a patient is affected by their condition - although he didn't use it for that in this study the results must indicate who is at the top end and who at the bottom, in terms of how their are affected.

Edit: That 2011 paper was the one I looked at. I looked because he seemed to be addressing issue of operationalising the definitions which is something I believe IOM has been charged with. I am still unsure though if this is what they mean by operationalising - but still it is an interesting means of determining suitability of diagnosis. I am not though sure how reported symptoms might stack up against eventual biomarkers.

 

[WillowJ]

Jason also has a 2010 paper which discusses the importance of symptom frequency and severity. The full text is free at this time.

 

[Simon]

Thanks very much @Simon

... I am not though sure how reported symptoms might stack up against eventual biomarkers.

That might be the most important issue. Although I like the idea of focusing on a few mandatory core symptoms, I'm not convinced there is any good reason to create a new case definition (IOM included) until we have a better handle on the biology.

The argument goes that without the right case definiton we won't have 'pure' cohorts and so have no chance with the biology anyway. But personally I think the most successful approach is likely to be with Big Data ('omics including genomics, gene expression, proteomics, metabolomics, and huge cohorts of the sort that Stephen Holgate has talked about. I think only such a Big approach is likely to reveal the real subtypes of me cfs -it's an approach already being used in cancer, eg this recent Nature paper.

My question then is this: in the UK we have 3 or 4 (depending) severities once you have received a diagnosis - I wonder if this would impact on those?

Mild, Moderate, Severe, Very Severe​

I mean Jason's method must surely help to categorise the degree to which a patient is affected by their condition - although he didn't use it for that in this study the results must indicate who is at the top end and who at the bottom, in terms of how their are affected.

We do have these categories? I didn't realise there was anything so fixed. But the best way to assess overall severity is probably with overall measures eg SF36 Functioning scales or Charles Shepherd's or AYME's (developed with patients). Rather than, say, summing up the individual symptoms and trying to get a total symptom burden.