Why fMRI inferences for spatial extent have inflated false-positive rates

[Effi]

https://www.sciencedaily.com/releases/2016/06/160627160927.htm

Common statistical methods used to analyse brain activity through images taken with MRI scanners cannot be trusted, as shown by Anders Eklund and Hans Knutsson of Linköping University, and Thomas Nichols of the University of Warwick, in the highly-ranked journal PNAS.
(...)
How may of the studies done with fMRI need to be redone?

"I can't answer that; many of them were done 10 to 15 years ago and it's not certain that the basic data still exists. The important thing is that researchers think about what method they use in future," Dr Eklund says.

full journal article: http://www.pnas.org/content/early/2016/06/27/1602413113

Functional MRI (fMRI) is 25 years old, yet surprisingly its most common statistical methods have not been validated using real data. Here, we used resting-state fMRI data from 499 healthy controls to conduct 3 million task group analyses. Using this null data with different experimental designs, we estimate the incidence of significant results. In theory, we should find 5% false positives (for a significance threshold of 5%), but instead we found that the most common software packages for fMRI analysis (SPM, FSL, AFNI) can result in false-positive rates of up to 70%. These results question the validity of some 40,000 fMRI studies and may have a large impact on the interpretation of neuroimaging results.

 

[Simon]

Softwares for fMRI yield erroneous results: Cluster failure: Why fMRI inferences for spatial extent have inflated false positive rates -- ScienceDaily

Dr Eklund now used the current analysis methods and compared 20 healthy people with 20 other healthy people. In other words, there should not have been any differences -- or, in any case only the five percent that chance provides. In total he made three million comparisons of randomly selected groups with data from 499 healthy persons.

"The differences were considerably greater than five percent, up to 60 percent in the worst case," Dr Eklund says.

This means that the analyses could have shown positive results where there shouldn't have been any, thereby indicating brain activity where there was no activity.

He also analysed the same data set with his more calculation-heavy method and obtained a considerably better correspondence, with differences in the expected five percent of cases.

 

[adreno]

Ouch.