8/12/2019 Publish and Be Wrong

1/1

The EconomistOct 9th 2008

Publish and be wrong - One group of researchers thinks headline-grabbing scientific reports are the

most likely to turn out to be wrong

IN ECONOMIC theory the winners curse refers to the idea that someone who places the winning id in an

auction may ha!e paid too much" Consider# for e$ample# ids to de!elop an oil field" Most of the offers are

li%ely to cluster around the true !alue of the resource# so the highest idder proaly paid too much"

&he same thing may e happening in scientific pulishing# according to a new analysis" 'ith so many scientificpapers chasing so few pages in the most prestigious (ournals# the winners could e the ones most li%ely to

o!ersell themsel!es)to trumpet dramatic or important results that later turn out to e false" &his would

produce a distorted picture of scientific %nowledge# with less dramatic *ut more accurate+ results either

relegated to oscure (ournals or left unpulished"

In Public Library of Science (PloS) Medicine# an online (ournal# ,ohn Ioannidis# an epidemiologist at Ioannina

-chool of Medicine# .reece# and his colleagues# suggest that a !ariety of economic conditions# such as

oligopolies# artificial scarcities and the winners curse# may ha!e analogies in scientific pulishing"

/r Ioannidis made a splash three years ago y arguing# uite con!incingly# that most pulished scientific

research is wrong" Now# along with Neal 1oung of the National Institutes of ealth in Maryland and Omar 3l4

5aydli# an economist at .eorge Mason 5ni!ersity in 6airfa$# 7irginia# he suggests why"

It starts with the nuts and olts of scientific pulishing" undreds of thousands of scientific researchers are

hired# promoted and funded according not only to how much wor% they produce# ut also to where it gets

pulished" 6or many# the ultimate accolade is to appear in a (ournal li%e Natureor Science" -uch pulications

oast that they are !ery selecti!e# turning down the !ast ma(ority of papers that are sumitted to them"

Picking winners

&he assumption is that# as a result# such (ournals pulish only the est scientific wor%" ut /r Ioannidis and his

colleagues argue that the reputations of the (ournals are pumped up y an artificial scarcity of the %ind that

%eeps diamonds e$pensi!e" 3nd such a scarcity# they suggest# can ma%e it more li%ely that the leading (ournals

will pulish dramatic# ut what may ultimately turn out to e incorrect# research"

/r Ioannidis ased his earlier argument aout incorrect research partly on a study of 9 papers in leading

(ournals that had een cited y more than :#000 other scientists" &hey were# in other words# well4regarded

research" ut he found that# within only a few years# almost a third of the papers had een refuted y other

studies" 6or the idea of the winners curse to hold# papers pulished in less4well4%nown (ournals should e more

reliale; ut that has not yet een estalished"

&he groups more general argument is that scientific research is so difficult)the sample si