"Scientific method: Statistical errors" (Nature, 2014) On problems with p-values and ways forward

[Dolphin]

For the more academically minded or at least, if you don't know what p values are, this is probably a bit complicated. But it's not an indepth statistical article that requires reading a lot of maths or equations.

Scientific method: Statistical errors
P values, the 'gold standard' of statistical validity, are not as reliable as many scientists assume.
Regina Nuzzo
12 February 2014

http://www.nature.com/news/scientific-method-statistical-errors-1.14700

 

[Valentijn]

These are sticky concepts, but some statisticians have tried to provide general rule-of-thumb conversions (see 'Probable cause'). According to one widely used calculation5, a P value of 0.01 corresponds to a false-alarm probability of at least 11%, depending on the underlying probability that there is a true effect; a P value of 0.05 raises that chance to at least 29%.

If I'm understanding the figures in another paper correctly (the one directly cited by that passage is pay-to-read), the probability of a false-alarm with a p-value of 0.10 is around 44%.