(General) "Two Dark Side Statistics Papers" (blog post)

[Dolphin]

A January 2 blog post summarising two papers which have nothing specifically to do with ME/CFS or related conditions:

(i) False Positive Psychology: Undisclosed Flexibility In Data Collection And Analysis Allows Presenting Anything As Significant

(ii) How To Have A High Success Rate In Treatment: Advice For Evaluators Of Alcoholism Programs

http://slatestarcodex.com/2014/01/02/two-dark-side-statistics-papers/

 

[alex3619]

The great thing about this table isn’t just that it shows that seemingly impressive results are exactly the same as placebo. The great thing it shows is that results in the placebo groups in the four studies could be anywhere from a 22.5% success rate to an 87% success rate. These aren’t treatment differences – all four groups are placebo! This is one hundred percent a difference in study populations and in success measures used. In other words, depending on your study protocol, you can prove that there is a 22.5% chance the average untreated alcoholic will achieve remission, or an 87% chance the average untreated alcoholic will achieve remission.

You can bet that rehabs use the study protocol that finds an 87% chance of remission in the untreated. And then they go on to boast of their 90% success rate. Good job, rehab!

I thought this part was worth emphasizing. Choice of protocols can massively boost apparent remission, without actually doing any better than placebo, or even when worse than using placebo. The choice of controls is critical. The choice of protocol is critical. What is being measured? How objective is that measure?