Fudging hell
Are results in top journals to be trusted?
PUBLICATION bias in academic journals is nothing new. A finding of no correlation between sporting events and either violent crime or property crime may be analytically top class, but you couldn’t be blamed, frankly, for not giving a damn. But if journal editors are more interested in surprising or dramatic results, there is a danger that the final selection of published papers offer a distorted vision of reality.
This should skew the distribution of published results, towards more 'significant' findings. But a paper just published in the American Economic Journal finds evidence of a different sort of bias, closer to the source. Called "Star Wars, the empirics strike back", it analyses 50,000 papers published between 2005 and 2011 in three top American journals. It finds that the distribution of results (as measured by z-score, a measure of how far away a result is from the expected mean) has a funny double-humped shape (see chart). The dip between the humps represents "missing" results, which just happen to be in a range just outside the standard cut-off point for statistical significance (where significance is normally denoted with stars, though the name may also be something to do with a film recently released—file under 'economists trying to be funny'). Their results suggest that among the results that are only just significant, 10-20% have been fudged.
Mais
This should skew the distribution of published results, towards more 'significant' findings. But a paper just published in the American Economic Journal finds evidence of a different sort of bias, closer to the source. Called "Star Wars, the empirics strike back", it analyses 50,000 papers published between 2005 and 2011 in three top American journals. It finds that the distribution of results (as measured by z-score, a measure of how far away a result is from the expected mean) has a funny double-humped shape (see chart). The dip between the humps represents "missing" results, which just happen to be in a range just outside the standard cut-off point for statistical significance (where significance is normally denoted with stars, though the name may also be something to do with a film recently released—file under 'economists trying to be funny'). Their results suggest that among the results that are only just significant, 10-20% have been fudged.
Mais
Nenhum comentário:
Postar um comentário