1. Patients launch $1.27 million crowdfunding campaign for ME/CFS gut microbiome study.
    Check out the website, Facebook and Twitter. Join in donate and spread the word!
Knitting Equals Pleasure, Despite ME/CFS
Jody Smith loves knitting. Again. She thought her days of knitting and purling were long over but ... she's back ...
Discuss the article on the Forums.

Jason - Sensitivity and Specificity of the CDC Empirical CFS Case Definition

Discussion in 'Latest ME/CFS Research' started by Jerry S, May 4, 2010.

  1. Jerry S

    Jerry S Senior Member

    Messages:
    422
    Likes:
    13
    Chicago
    I don't think this has been posted yet. The full paper can be found at:

    http://www.scirp.org/journal/psych/

    This is the abstract:

    Journal: Psychology, 2010, 1: 9-16, doi:10.4236/psych.2010.11002 Published Online April 2010
    Authors:Leonard A. Jason, Meredyth Evans, Abigail Brown, Molly Brown,
    Nicole Porter, Jessica Hunnell, Valerie Anderson, Athena Lerch

    Affiliation: DePaul University, Chicago, American. Email: <Ljason@depaul.edu>

    Received February 10th, 2010; revised March 7th, 2010; accepted March 8th, 2010.

    ABSTRACT

    In an effort to bring more standardization to the chronic fatigue syndrome (CFS) Fukuda et al. case definition [1], the Centers for Disease Control and Prevention (CDC) has developed an empirical case definition [2] that specifies criteria and instruments to diagnose CFS.

    The present study investigated the sensitivity and specificity of this CFS empirical case definition with diagnosed individuals with CFS from a community based study that were compared to non-CFS cases. All participants completed questionnaires measuring disability (Medical Outcome Survey Short-Form-36) [3], fatigue (the Multidimensional Fatigue Inventory) [4], and symptoms (CDC Symptom Inventory) [5].

    Findings of the present study indicated sensitivity and specificity problems with the CDC empirical CFS case definition.

    The paper concludes:

    "The sensitivity and specificity outcomes for the Reeves et al. criteria suggest that these recommended scales and cutoff points would not be considered a good diagnostic tool for selecting CFS cases from the general population."
  2. Jerry S

    Jerry S Senior Member

    Messages:
    422
    Likes:
    13
    Chicago
    Some comments on the discussion section of the study (my bolds):

    JS Note: The Reeves 2007 study [29] found a prevalence rate of 2540 per 100,000. Part of the explanation for this stunningly high rate was greater sensitivity. "The 6- to 10-fold greater prevalence estimates also reflect application of more sensitive and specific measures of the CFS diagnostic parameters specified by the 1994 case definition." Yet this study by Jason finds the empirical case definition lacks sensitivity - 35% of the "true" CFS cases were missed. The only other explanation, if the claim Georgia might be a CFS hotspot is dismissed, is the empirical definition is grossly non-specific.

    JS Note: This is extremely likely.

    JS Note: Bottom line - After 25 years, the CDC still hasn't come up with an adequate case definition of CFS.

    Partial References:

  3. Dolphin

    Dolphin Senior Member

    Messages:
    6,585
    Likes:
    5,187
    Thanks for posting the link for the full paper, Jerry. Haven't seen it posted it elsewhere. Will link to it from the petition.

    This paper really vindicates everyone who had doubts about the definition.
  4. mezombie

    mezombie Senior Member

    Messages:
    324
    Likes:
    19
    East Coast city, USA
    Bumping...for others to read. Jason's article is great and Jerry's comments are astute.
  5. Dolphin

    Dolphin Senior Member

    Messages:
    6,585
    Likes:
    5,187
    In my opinion, the main finding of this study can be summarised as follows from Table 1:

    ----------------
    For the 3 parts of the CDC's empiric criteria for CFS:

    MFI (fatigue criteria)
    Sensitivity: 0.95 (i.e. 95% of the true CFS cases will satisfy the criteria)
    Specificity: 0.27 (i.e. 73% of the non-CFS cases will satisfy the criteria)


    SI (core symptoms criteria)
    Sensitivity: 0.59 (i.e. 59% of the true CFS cases will satisfy the criteria)
    Specificity: 0.73 (i.e. 27% of the non-CFS cases will satisfy the criteria)


    SF-36 (substantial reductions criteria)
    Sensitivity: 0.96 (i.e. 96% of the true CFS cases will satisfy the criteria)
    Specificity: 0.17 (i.e. 83% of the non-CFS cases will satisfy the criteria)


    Meet Criteria (overall)
    Sensitivity: 0.65 (i.e. 65% of the true CFS cases will satisfy the criteria for CFS, 35% will not)
    Specificity: 0.76 (i.e. 24% of the non-CFS cases will satisfy the criteria for CFS)

    [Aside: I checked with the authors and the following is going to be added to the data: "Some of the participants did not complete all three questionnaires, and were thus excluded from the overall sensitivity and specificity figures."]

    -----------------

    These are not good figures for a set of criteria. They are so "broad" that 24% of the non-CFS cases will satisfy the criteria for CFS, yet even with these "broad" criteria, 35% of the true CFS cases will be missed.
    Each of the three parts of the criteria are very poor ("bad") in one way i.e. either the specificity or sensitivity is very poor.
  6. Mithriel

    Mithriel Senior Member

    Messages:
    564
    Likes:
    37
    Scotland
    Thanks for that analysis Tom, I have great trouble getting my head round these papers.

    Mithriel
  7. Dolphin

    Dolphin Senior Member

    Messages:
    6,585
    Likes:
    5,187
    SF-36 questions

    Thanks Mithriel.

    As I mentioned on another thread, many people would benefit from filling in a SF-36 questionnaire to know what is being talked about.

    This can be done at:
    http://www.sf-36.org/demos/SF-36.html
    or
    http://www.qualitymetric.com/demos/sf-36.aspx

    If one goes to (near end of) https://www.fstrf.org/apps/cfmx/apps/actg/html/QOLForms/manualql601-2799.pdf one can see how they are scored. Except for the pain scale (and the composite scores which Lenny Jason may not talk about), once you know the questions, it is just a question of expanding them to fit 100 (in other words, getting a percentage). So the SF-36 physical functioning subscale is 10-30 (which is the same as 0-20) - so one just multiplies each score by five (if you are doing the 10-30 scale, subtract by 10 first). Anyway with the sample tests, they do the scoring for you.
  8. Jerry S

    Jerry S Senior Member

    Messages:
    422
    Likes:
    13
    Chicago
    Thanks, Tom, for breaking down the numbers and and providing links to the SF-36 questionnaire. Thanks, mezombie, for the kind comment.

    Another quote from the Jason paper:

    I don't know what numbers you'd get using 65% sensitivity and 76% specificity, but Jason's example gives one an idea of how far off the mark the empirical definition is.
  9. Dolphin

    Dolphin Senior Member

    Messages:
    6,585
    Likes:
    5,187
    I pointed out to Lenny they made a slight error with Bayes' theorem and a correction will be placed with the article.
    If 4,200 people have CFS, they shouldn't be included in figures to do with the rest.
    So one calculates the other figures from 1,000,000-4,200=995,800.
    So there would be 49,790 cases who did not have CFS but were identified as having CFS in the scenario he gave.

    65% sensitivity and 76% specificity would give:
    Number satisfying the criteria from the CFS cases: 1,000,000*0.0042=4200*.65=2730
    Number satisfying the criteria from the non-CFS cases: (1,000,000-4200)=995,800*.24=238992
    So if it was left to the questionnaires, the number of proper cases would only be: 2730/(2730+238992)=0.0112939 of the number they say have CFS (1.1%)!

    Of course, this was done in two stages so many of the healthies were excluded by the phone call stage.
  10. Jerry S

    Jerry S Senior Member

    Messages:
    422
    Likes:
    13
    Chicago
    Amazing!! Thanks, Tom!
  11. rebecca1995

    rebecca1995 Apple, anyone?

    Messages:
    359
    Likes:
    20
    Northeastern US
    Exactly. The numbers from the Jason paper, along with Tom's analysis, scare the daylights out of me.

    It's so frustrating that much of the CAA's literature, like the Patient Brochure found here, endorses the empirical definition/Reeves Criteria.

    The brochure says, "More than 4 million people in the United States have chronic fatigue syndrome (CFS)..." A number that high can only be obtained using the empirical definition.

    Why would the nation's largest ME charity (as fred would say :Retro wink:) support a definition that's wildly and demonstrably inaccurate?
  12. Dolphin

    Dolphin Senior Member

    Messages:
    6,585
    Likes:
    5,187
    Thanks Jerry.

    Regarding the CAA and the empiric criteria, they did come out against it last year.
    They were in a sort of "no man's land" before that, neither really supporting it, nor criticising it.
    They were in a difficult situation as (i) they were tied to the CDC for the awareness etc contract (ii) Suzanne Vernon's name is on the criteria.

    However, with a bit of a nudge, they appear to be more against it now although they may still feel 4 million is a useful figure for lobbying (I don't like that figure myself and as you say, rebecca1995, it comes from the empiric criteria).
  13. Jerry S

    Jerry S Senior Member

    Messages:
    422
    Likes:
    13
    Chicago
    Yes, Rebecca. Based on an estimated US population of 310 million the various prevalence rates for CFS give:

    Jason, 1999. A community-based study in Chicago - 422 per 100,000

    1.3 million

    Reyes, 2003. A CDC study in Wichita, KA - 235 per 100,000

    730,000

    Reeves, 2007. A study using the empirical definition - 2540 per 100,000

    7.9 million

    I don't know where the 4 million figure comes from, but it certainly isn't based on any study using Fukuda. The empirical definition is the only one which gives figures that high.

    ETA: The CAA has come out against the empirical definition, as Tom noted, yet they seem reluctant to abandon the high CFS numbers the empirical definition generated.
  14. Dolphin

    Dolphin Senior Member

    Messages:
    6,585
    Likes:
    5,187
    (Not very exciting)

    http://www.pophealthmetrics.com/content/5/1/6/comments

  15. Mithriel

    Mithriel Senior Member

    Messages:
    564
    Likes:
    37
    Scotland
    So it might be your fault Tom :Retro smile:

    With all these false positives, any intervention could be shown to "significantly" help CFS without a single person with ME/CFS getting any better. Papers using the Oxford definition come to mind.

    I am trying to work out the maths, but I keep getting muddled, very annoying!

    Mithriel
  16. Dolphin

    Dolphin Senior Member

    Messages:
    6,585
    Likes:
    5,187
    No, the CDC had used the 4 million figure before that (by accident in an ad before the study came out). I just thought it was easier to show how they might have arrived at the figure.

    Yes, exactly.
  17. Bellyhill

    Bellyhill Guest

    hi
    This is Gagan.

    I am from NJ i just an account on this forum

See more popular forum discussions.

Share This Page