Coyne: Bad stats, non sequitur conclusions in Lancet chronic fatigue syndrome/suicide study

[Cheshire]

A tale of a convenience sample with inconvenient serious limitations.

The study has too small a sample to warrant a manuscript submitted to a peer-reviewed journal. If you keep looking at it, its problems get compounded.


I recommend downloading this open access article and following me as I explained its flaws.


What, too small a sample? With 2147 patients, the study is probably the largest ever of mortality among patients with chronic fatigue syndrome. But the adequacy of sample size is determined not by the total number of participants, but the number having particular events, in this case, the relatively rare events of mortality and suicide. In the seven-year follow-up, there were 17 instances of all-cause mortality, 11 among women and six among men. There were five suicides, three among women and two among men.

https://jcoynester.wordpress.com/20...lancet-chronic-fatigue-syndromesuicide-study/

 

[John Mac]

Does the total number of deaths not seem unusually low?

17 in 7 years is 2.4 deaths per year.

2.4/2147 is 1.1 deaths per thousand per year

Typical death rates in Europe are 8 - 12 per thousand per year.

And as we are talking about an increased risk of suicide that 1.1 looks remarkably low.

The World Bank's table of death rates:

http://data.worldbank.org/indicator/SP.DYN.CDRT.IN

 

[Esther12]

I was feeling that maybe I'd been too casually critical of this paper, having only given it a brief read, so comforting to read Coyne show that my hunches weren't too unreasonable.

I thought it was weird that this got in the Lancet too. I wondered if maybe it had more exciting spin in it when first submitted, and then post-Tuller the Lancet decided to ensure it got toned down during review?

The other thing is that sometimes nothing papers like this end up being politically useful for the authors in behind-the-scenes ways that we do not understand.

 

[Scarecrow]

Typical death rates in Europe are 8 - 12 per thousand per year.

Would those by all age deaths? The patient population in the study is likely to have a younger age profile than the general population.

 

[John Mac]

Would those by all age deaths? The patient population in the study is likely to have a younger age profile than the general population.

Yes I think the tables cover all ages including old people. The ME patients would be a subset of that, I imagine not including many older people which might explain the difference.

 

[anciendaze]

While I was explaining the problems with this paper, and the resulting newspaper headlines, to a friend, I realized two things: 1) the number of actual suicides on which those headlines depend is comparable to anecdotal evidence also seen in newspapers, including suicides apparently triggered by the threat of involuntary commitment to locked wards; 2) the frightening suicide rate reported in those patients is comparable to suicide rates among psychiatrists and brain surgeons, though solid data on this is harder to acquire.

If this quality of evidence is grounds for committing CFS patients to psychiatric care what does it say about committing psychiatrists?

Added: "I think the numbers speak for themselves; these are obviously seriously disturbed people."