Systematic Overreaching
March 6th, 2014
Jennie Spotila Leave a comment Go to comments
A
study published last month* by a group from Norway reports a systematic review of ME/CFS case definitions. It’s important because the Institute of Medicine panel is likely conducting a similar process as part of its work, but the study is also significant for the startling and overreaching conclusions drawn by the authors.
Quick Summary:
- This study collects and examines case definitions for ME/CFS, and extracts prevalence data. This will be a helpful future reference.
- The authors then step far beyond their own evidence and draw a variety of overreaching conclusions.
- Future analysis must be wary of cognitive bias in order to avoid the same mistakes.
What They Did
Bruberg
et al began by searching the literature for case definitions of ME/CFS (a term encompassing CFS, ME, and combinations of the two). The group then looked for validation studies, meaning studies that applied one or more case definitions to patient populations. Of the twenty case definitions identified in the first step, seven have never been validated in a patient population. Thirty-eight validation studies were identified that test one or more case definition, but the authors were very critical of the quality of those studies saying, “We did not identify any study which rigorously assessed the reproducibility or feasibility of the different case definitions.”
The authors categorized the validation studies, and the summaries and Tables 1-4 are actually quite helpful. Five studies independently applied several case definitions to a single patient population, and thirteen studies took patients diagnosed under one definition (usually Fukuda) and sequentially applied case definitions assumed to be more specific. The authors caution that this second group of studies “should be interpreted with great caution” because there were so many differences in diagnostic method and definition application, including different thresholds for exclusion of psychiatric comorbidity. Finally, twenty-one studies presented prevalence estimates and the authors extracted data on prevalence rates for Fukuda, Oxford, and the Australia 1990 case definitions.
Personally, I find the collection of these studies and extraction of data to be both interesting and important for future reference. I think it is very likely that the IOM panel is conducting a similar review of the case definition literature, as well. Rigorous examination of case definitions and their applications is essential for the IOM and for the ME/CFS field in general. Bruberg
et al found this body of evidence to be “methodologically weak and heterogeneous, making it questionable to compare the case definitions.” What I found both startling and disturbing is that the authors then go on to draw a number of conclusions that are simply not supported by the evidence they examined...
Read more:
http://www.occupycfs.com/2014/03/06/systematic-overreaching/