Useful phrase? "HARKing: Hypothesizing After the Results are Known"

[Dolphin]

I just came across the phrase "harking" in a James C. Coyne blog:

The authors’ failure to single out one or two of these variables a priori (ahead of time) sets them up to pick-the-best hypothesizing after results are known or HARKING. We do not actually know, but there is a high risk of bias.

from: http://blogs.plos.org/mindthebrain/...dynamic-therapy-better-routine-care-anorexia/

HARKing: Hypothesizing After the Results are Known
Pers Soc Psychol Rev August 1998

1. Norbert L. Kerr
   1. Department of Psychology, Michigan State University

Abstract
This article considers a practice in scientific communication termed HARKing (Hypothesizing After the Results are Known).

HARKing is defined as presenting a post hoc hypothesis (i.e., one based on or informed by one's results) in one's research report as if it were, in fact, an a priori hypotheses.

Several forms of HARKing are identified and survey data are presented that suggests that at least some forms of HARKing are widely practiced and widely seen as inappropriate.

I identify several reasons why scientists might HARK.

Then I discuss several reasons why scientists ought not to HARK.

It is conceded that the question of whether HARKing's costs exceed its benefits is a complex one that ought to be addressed through research, open discussion, and debate.

To help stimulate such discussion (and for those such as myself who suspect that HARKing's costs do exceed its benefits), I conclude the article with some suggestions for deterring HARKing.

Free full text at: http://www.sozialpsychologie.uni-frankfurt.de/wp-content/uploads/2010/09/kerr-1998-HARKing.pdf

 

[biophile]

PACE could be guilty of this? The recovery paper's introduction mentions defining operationalized criteria for recovery on relevant domains and then calculating the proportion of participants meeting each criteria and the composite criteria.

They reveal they had a previous definition of recovery, but abandoned it and changed it in light of new population data (which they previously admitted was suggested to them by a peer reviewer of the Lancet paper).

They claimed to have changed the definition of recovery before doing the analysis for this paper, but then fail to admit that some of the new criteria were already derived and described as post-hoc in a previous paper 2 years before.

 

[Sean]

The physicist Richard Feynman insisted his students and colleagues (on the theorist side of physics) calculated results and made firm predictions before they saw the experimental data, precisely because it was too easy to do a post-hoc fit.

His view was that this was the only way to keep scientists honest and properly test their theorising.

Which is why I regard the huge discrepancy between the predictions made by PACE and the actual results, plus independently the post-hoc manipulations, as both central to any assessment of PACE, and very damning of it.

In general we look for a new law by the following process. First we guess it. Then we compute the consequences of the guess to see what would be implied if this law that we guessed is right. Then we compare the result of the computation to nature, with experiment or experience, compare it directly with observation, to see if it works. If it disagrees with experiment it is wrong. In that simple statement is the key to science. It does not make any difference how beautiful your guess is. It does not make any difference how smart you are, who made the guess, or what his name is – if it disagrees with experiment it is wrong. That is all there is to it.

Feynman Lectures on Physics.