THE WISDOMOF CROWDS

WHY THE MANY ARE SMARTER THAN THE FEW

AND HOW COLLECTIVE WISDOM SHAPES BUSINESS,

ECONOMIES, SOCIETIES, AND NATIONS

JAMES SuRowIEcKi

I) 0 U B L E U A Y

New York London Toronro Sydney Auckknd

SPUBLiSHED BY DOUBLEDAYa division of Random House, Inc.

DOUBLEDAY and the portrayal of an anchorwith a dolphin areregistered trademarks of Random House, Inc.

Some of the material in this book was originally published indifferent form in The New Yorker.

Library of Congress Cataloging-in-Publication DataSurowiecki, James, 1967—

The wisdom of crowds why the many are smarter than the few and how collectivewisdom shapes business, economies, societies, and nations / James Surowiecki.

p. cm.Includes bibliographical references.

I. Consensus (Social sciences) 2. Common good. I. Title.JC328.2.S87 2003303.3’8—dc222003070095

ISBN 0-385-50386-5

Copyright © 2004 by James Surowiecki

All Rights Reserved

PRINTED IN THE UNITED STATES OF AMERICA

June 2004

5 7 9 10 8 6 4

INTRODUCTION

one div ii) the hill üí I 9O~,the British scientist francs Galtorlleh his borne iii I lie wtt’n of Pl~iTlOtitO Bad head cI for a COtintfl’fair. ( ;alton was eighlv-h\’e years old anti beginning to feel uk age.hIJI he was still brimming wIth 11)5’ curiosity taut hao won himrenown —-anti DIII I)YletV for his ivork DI) slatjsl ics and Ihe scienceof llcreditv. Anti OIi [hal parlicular day. \~hai(alton was curious,lI)OtIt \Vas ii ye stock.

(Jalton’s destination was the annual \~st of Englanc: I atStock and Ponil iv I :xhihition a regional fair where the local farm-ers and townspeople gathered U. appraist. (lie quaiit~of eachothers cattle, sheep, chicke ns, horse arId pigs. \\hadering t hrotlghrows td stalls examining isnrkhorses and prit.e hogs may Seem 10have been a sirange way for a scientist les1,et-iall~an elderly one)IC) spend an altt’rI)OOn, 1)111 there was a certain logic 10 is.. (~dlton

was a man obsessed \~ith two I hiogs: the measurement of physicaland mental qualities, and Urceding. And whai. alter all - is a live-stock shots bum a big showcase br the effect:; of good aI id badbreeding?

l3reetling mattered to Galton because lie believed that on) ai cry Few penile had the characteris~mcsnecessary to keep socielies[ICiltin: I le had deo aeti much ot his career to measuring thosecharacteristics n fact, in order to prove thai tIM’ VaSl IbflhiorItV of

XII INYflODUCYION

people did not have them. At the International Exhibition of 1884in London, for instance, he set up an “Anthropometric Laborator)ç”where he used devices of his own making to test exhibition-goerson, among other things, their “Keenness of Sight and of Hearing,Colour Sense, Judgment of Eye, [and] Reaction Time.” His exper-iments left him with little faith in the intelligence of the averageperson, “the stupidity and wrong-headedness of many men andwomenbeing so great as to be scarcely credible.” Only if power andcontrol stayed in the hands of the select, well-bred fet~Galton be-lieved, could a society remain healthy and strong.

As he walked through the exhibition that day, Galton cameacrossa weight-judging competition. A fat ox had been selected andplacedon display, andmembers ofa gatheringcrowd were lining upto place wagers on theweight of theox. (Or rathe~they were plac-big wagers on what the weight ofthe ox would be after it had been“slaughtered and dressed.”) For sixpence, you could buy a stampedand numbered ticket, where you filled in your name,your address,and your estimate. The best guesses would receive prizes.

Eight hundred people tried their luck. Theywere a diverse lot.Many of themwere butchers and farmers, whowere presumably ex-pert atJudging the weight oflivestock, but there were also quite afew people who had, as it were, no insider knowledge of cattle.“Many non-experts competed,” Galton wrote later in the scientificjournal Nature, “like those clerks and otherswho have no expertknowledge of hones, but who bet on races, guided by newspapers;friends, and their own fancies.” The analogy to a democraqc inwhich people of radically different abilities and interests each getone vote, had suggested itself to Galton immediately “The averagecompetitorwas probably as well fitted formaking a just estimateofthe dressed weight of the ox, as an average voter is ofjudging themerits ofmost political issues on which he votes,” he wrote.

Gakon was interested in figuring out what the “average vote?was capable of because he wanted to prove that the average voterwas capable of venj little. So he turned the competition into an im-

)PITRODUCTION XIII

promptu experiment. When the contest was over and the prizeshad been awarded, Galton borrowed the tickets from the organiz-ers and ran a series of statistical testson them. Galton arranged theguesses (which totaled 787 in all, after he had to discard thirteenbecause they were illegible) in order from highest to lowest andgraphed them to see if they would form a bell curve. Then, amongother things, he added all the contestants’ estimates, and calcu-lated the mean of the group’s guesses. That number represented,you could say, thecollective wisdom ofthe Plymouth crowd. If thecrowd were a single person, that was how much it would haveguessed the ox weighed.

Galton undoubtedly thought that the average guess of thegroup would be way off the mark. After all, mix a few very smartpeople with some mediocre people and a lot ofdumb people, andit seems likely you’d end up with a dumb answet But Galton waswrong. The crowd had guessed that the ox, after it had beenslaughtered and dressed, would weigh 1,197 pounds. After it hadbeen slaughtered and dressed, the ox weighed 1,1% pounds. Inother words, the crowd~sjudgmentwas essentially perfect. Perhapsbreeding did not mean so much after all. Calton wrote latet “Theresult seemsmore creditable to the trustworthinessofa democraticjudgment than might have been expected.” That was, to say theleast, an understatement.

II

What Francis Galton stumbled on that day in Plymouth was thesimple, but powerful, truth that is at the heart ofthis book. underthe right circumstances, groups are remarkably intelligent, and areoften smarter than the smartest people in them. Groups do notneed to be dominated by exceptionally intelligent people in order tobe smart. Even if most of the people within a group are not espe-cially welJ-info~edor rational, it can still reach a collectively wise

XIV INTRODUCTION

decision. This is a good thing, since human beings are not perfectlydesigned decision makers. Instead, we are what the economist Her-bert Simon called “houndedly rational.” We generally have less in-formation than we’d like. We have limited foresight into the future.Most of us lack the ability—and the desire—to make sophisticatedcost-benefit calculations. Instead of insisting on finding the bestpossible decision, we will often accept one that seems good enough.And we often let emotion affect our judgment. Yet despite all theselimitations, when our imperfect judgments are aggregated in theright way our collective intelligence is often excellent.

This intelligence, or what I’ll call the wisdom of crowds,” is atwork in the world in many different guises. It’s the reason the Inter-net search engine Google can scan a billion Web pages and find theone page that has the exact piece of information you were lookingfor It’s the reason it’s so hard to make money betting on NFLgames, and it helps explain why for the past fifteen years, a fewhundred amateur traders in the middle of iowa have done a betterjob of predicting election results than Gallup polls have. The wis-dom of crowds has something to tell us about why the stock marketworks (and about why every so often, it stops working). ‘The idea ofcollective intelligence helps explain why when you go to the con-venience store in search of milk at two in the morning, there is acarton of milk waiting there for you, and it even tells us somethingimportant about why people pay their taxes and help coach LittleLeague. It’s essential to good science. And it has the potential tomake a profound difference in the way companies do business.

in one sense, this book tries to describe the world as it is,looking at things that at first glance may not seem similar but thatare ultimately very much alike. But this book is also about theworld as it might be. One of the striking things about the wisdomof crowds is that even though its effects are all around us, it’s easyto miss, and, even when it’s seen, it can he hard to accept. Most ofus, whether as voters or investors or consumers or managers, be-lieve that valuable knowledge is concentrated in a very few hands

INTRODUCTION XV

(or, rather, in a very few heads). We assume that the key to solvingproblems or making good decisions is finding that one right personwho will have the answer. Even when we see a large crowd of peo-ple, many of them not especially well-informed, do somethingamazing like, say predict the outcomes of horse races, we are morelikely to attribute that success to a few smart people in the crowdthan to the crowd itself. As sociologists Jack B. SoIl and RichardLarrick put it, we feel the need to “chase the expert.” The argumentof this book is that chasing the expert is a mistake, and a costly oneat that. We should stop hunting and ask the crowd (which, ofcourse, includes the geniuses as well as everyone else) instead.Chances are, it knows.

III

Charles Mackay would have scoffed at the idea that a crowd ofpeople could know anything at all. Mackay was the Scottish jour-nalist who, in 1841, published Extraordinaiy Popular Delusions andthe Madness of Crowds, an endlessly entertaining chronicle of massmanias and collective follies, to which the title of my book payshomage. For Mackay crowds were never wise. They were nevereven reasonable. Collective judgments were doomed to be ex-treme. “Men, it has been well said, think in herds,” he wrote. “Itwill he seen that they go mad in herds, while they only recover theirsenses slowly and one by one.” Mackay’s take on collective mad-ness is not an unusual one. In the popular imagination, groups tendto make people either dumb or crazy or both. The speculatorBernard Baruch, for instance, famously said: “Anyone taken as anindividual is tolerably sensible and reasonable—---as a member of acrowd, he at once becomes a blockhead.” Henry David Thoreaulamented: “The mass never comes up to the standard of its bestmember, but on the contrary degrades itself to a level with the low-est.” Friedrich Nietzsche wrote, “Madness is the exception in mdi-

XVI INTRODUCTION

viduals but the rule in groups,” while the English historian ThomasCarlyle put it succinctly: “1 do not believe in the collective wisdomof individual ignorance.”

Perhaps the most severe critic of the stupidity of groups wasthe French writer Gustave Le Eon, who in 1895 published thepolemical classic The Crowd: A Study of the Popular Mind. Le Eonwas appalled by the rise of democracy in the West in the nine-teenth centur)ç and dismayed by the idea that ordinary people hadcome to wield political and cultural power. But his disdain forgroupswent deeper than that.A crowd, Le Eon argued, was morethan just the sum of its members. Instead, it was a kind of inde-pendent organism. ft had an identity and a will of its own, and itoften acted in ways that no one within the crowd intended. Whenthe crowd did act, Le Eon argued, it invariably acted foolishly. Acrowd might be brave or cowardly or cruel, but it could never besmart. As he wrote, “In crowds it is stupidity and not mother witthat is accumulated.” Crowds “can neveraccomplish acts demand-ing a high degree of intelligence,” and they are “always intellectu-ally inferior to the isolated individual.” Strikingl)c for Le Eon, theidea of“the crowd” included not just obvious examples of collectivewildness, like lynch mobs or rioters. It also included just about anykind of group that could make decisions.

So Le Eon lambasted juries, which “deliver verdicts ofwhicheach individual juror would disapprove.” Parliaments, he argued,adopt laws that each of theft members would normally reject. Infact, if you assembled smart people who were specialists in a hostof different fields and asked them to “make decisions affectingmatters ofgeneral interest,” the decisions they would reachwouldbe no better, on the whole, than those “adopted by a gathering ofimbeciles.”

Over the course of this book I Ibilow Le Ron’s lead in givingthe words “group” and “crowd” broaddefinitions, using thewords torefer to everything from game-showaudiences tomultibillion-dollarcorporations to a crowd of sports gamblers. Some of the groups in

INtRODUCTiON XVII

this book, like the management teams in Chapter 9. are tightly or-ganized and very much aware of their identities as groups. Othercrowds, like the herds of cars caught in traffic that I write about inChapter 7, have no formal organization at all. And still others, likethe stock market, exist mainly as an ever-changing collection ofnumbers and dollars. These groups are all different, but they havein common the ability to act collectively to make decisions andsolve prohlems—even if the people in the groups aren’t alwaysaware that’s what they’re doing. And what is demonstrahly true ofsome of these groups—namely, that they are smart and good atproblem solving—is potentially true of most, if not all, of them. Inthat sense, Gustave Le l3on had things exactly backward. if you puttogether a big enough and diverse enough group of people and askthem to “make decisions affecting matters of general interest,” thatgroup’s decisions will, over time, be “intellectually (superiorj to theisolated individual,” no matter how smart or well-informed he is.

‘V

Judging the weight of an ox is hardly a complex task. But, as I sug-gested above, collective intelligence can be brought to bear on awide variety of problems, and complexity is no bar. in this book, Iconcentrate on three kinds of problems. The first are \.vhat I’ll callcognition problems. ‘Fhese are problems that have or will have de-finitive solutions. For example, “Who will win the Super Bowl thisyear?” and ‘How many copies of this new ink-jet rinter will we sellin the next three months?” are cognition problems. So, too, is “Howlikely is it that this drug will he approved by the FDA?” Questionsto which there may not he a single right answer, but to which someanswers are certainly better than others—such as, “What would hethe best place to build this new public swimming pool?”—are cog-nition problems, too.

The second kind of problem is what’s usually called a coordi-

XVIII INTRODUCTION

nation problem. Coordination problems require members of agroup (market, subway riders, college students looking for a party)to figure out how to coordinate their behavior with each other,knowing that everyone else is trying to do the same. How do buy-ers and sellers find each other and trade at a fair price? How docompanies organize their operations? How can you drive safely inheavy traffic? These are all problems of coordination.

The final kind of problem is a cooperation problem. As theirname suggests, cooperation problems involve the challenge of get-ting self-interested, distrustful people to work together, even whennarrow self-interest would seem to dictate that no individualshould take part. Paying taxes, dealing with pollution, and agreeingon definitions of what counts as reasonable pay are all examples ofcooperation problems.

A word about structure. The first half of this book is, youmight say, theory although leavened by practical examples. There’sa chapter for each of the three problems (cognition, coordination,and cooperation), and there are chapters covering the conditionsthat are necessary for the crowd to he wise: diversity, indepen-dence, and a particular kind of decentralization. The first half be-gins with the wisdom of crowds, and then explores the threeconditions that make it possible, before moving on to deal with co-ordination and cooperation.

The second part of the book consists of what are essentiallycase studies. Each of the chapters is devoted to a different way oforganizing people toward a common (or at least loosely common)goal, and each chapter is about the way collective intelligence ei-ther flourishes or flounders. In the chapter about corporations, forinstance, the tension is between a system in which only a few peo-ple exercise power and a system in which many have a voice. Thechapter about markets starts with the question of whether marketscan be collectively intelligent, and ends with a look at the dynam-ics of a stock-market bubble.

There are many stories in this book of groups making bad

INTRODUCTION XIX

decisions, as well as groups making good ones. Why? Well, onereason is that this is the way the world works. The wisdom ofcrowds has a far more important and beneficial impact on oureveryday lives than we recognize, and its implications for the fu-ture are immense. But in the present, many groups struggle tomake even mediocre decisions, while others wreak havoc withtheir had judgment. Groups work well under certain circum-stances, and less well under others. Groups generally need rulesto maintain order and coherence, and when they’re missing ormalfunctioning, the result is trouble. Groups benefit from mem-bers talking to and learning from each other, hut too much com-munication, paradoxically, can actually make the group as a wholeless intelligent. While big groups are often good for solving cer-tain kinds of problems, big groups can also he unmanageable andinefficient. Conversely, small groups have the virtue of being easyto run, but they risk having too little diversity of thought and toomuch consensus. Finally Mackay was right about the extremes ofcollective behavior: there are times—think of a riot, or a stock-market bubble—svhen aggregating individual decisions producesa collective decision that is utterly irrational. The stories of thesekinds of mistakes are negative proofs of this books argument, un-derscoring the importance to good decision making of diversityand independence by demonstrating what happens when they’remissing.

Diversity and independence are important because the bestcollective decisions are the product of disagreement and contest,not consensus or compromise. An intelligent group, especiallywhen confronted with cognition problems, does not ask its mem-bers to modify their positions in order to let the group reach a de-cision everyone can be happy with. Instead, it figures out how to usemechanisms----like market prices, or intelligent voting systems—to aggregate and produce collective judgments that represent notwhat any one person in the group thinks but rather, in some sense.what they all think. Paradoxically the best way for a group to he

XX INYflODUCTION

smart is for each person in it to think and act as independently aspossible.

V

I began this Introduction with an example ofa group solving a sim-ple problem: figuring out the weight ofan ox. Ill end it with an a-ample of a group solving an incredibly complexproblem: locating alost submarine. The differences between the two cases are im-mense. But the principle in each is the same.

In May 1968, the U.S. submarine Scorpion disappeared on itsway back to Newport News after a tour of duty in the North At-lantic. Although the navy knew the sub’s last reported location, ithad no idea what had happened to the Scorpion, and only thevaguest sense ofhow far it might have traveled after it had last maderadio contact. As a result, the area where the navybegan searchingfor theScorpionwas a circle twentymiles wide andmany thousandsof feet deep. You could not imagine a morehopeless task. The onlypossible solution, one might have thought, was to track down threeor four top experts on submarines and ocean currents, ask themwhere they thought the Scorpion was, and search there. But, asSherry Sontag and Christopher Drew recount in their book BlindMans Bluff a naval officer named JohnCraven had a different plan.

First, Craven concocted a series of scenarios—alternative ex-planations for what might have happened to theScorpion. Then heassembleda team ofmen with a wide range of knowledge, includ-ing mathematicians, submarine specialists, and salvage men. In-stead ofasking them to consultwith each other to come upwith ananswer, he asked each of them to offer his best guess about howlikely each of the scenarios was. To keep things interesting, theguesseswere in theform ofwagers, with bottles ofChivas Regal asprizes. And so Craven’s men bet on why the submarine ran into

INTRODUCTION XXI

trouble, on its speed as it headed to the ocean bottom, on thesteepness of its descent, and so forth.

Needless to say no one of these pieces of information couldtell Craven where the Scorpion was. But Craven believed that if heput all the answers together, building a composite picture of howthe Scorpion died, he’d end up with a pretty good idea of where itwas. And that’s exactly what he did. He took all the guesses, andused a formula called Bayes’s theorem to estimate the Scorpion’s fi-nal location. (Bayes’s theorem is a way of calculating how new in-formation about an event changes your preexisting expectations ofhow likely the event was.) When he was done, Craven had whatwas, roughly speaking, the group’s collective estimate of where thesubmarine was.

The location that Craven came up with was not a spot thatany individual member of the group had picked. In other words,not one of the members of the group had a picture in his head thatmatched the one Craven had constructed using the informationgathered from all of them. The final estimate was a genuinely col-lective judgment that the group as a whole had made, as opposedto representing the individual judgment of the smartest people init. It was also a genuinely brilliant judgment. Five months after theScorpion disappeared, a navy ship found it. It was 220 yards fromwhere Craven’s group had said it would he.

What’s astonishing about this story is that the evidence thatthe group was relying on in this case amounted to almost nothing.It was really just tiny scraps of data. No one knew why the subma-rine sank, no one had any idea how fast it was traveling or howsteeply it fell to the ocean floor. And yet even though no one inthe group knew any of these things, the group as a whole knewthem all.

1.

THE WISDOM OF CROWDS

hf, years hence, people remember anything about the TV gameshowWho Wants to & a Millionaire?, theywill probably rememberthe contestants’ panicked phone calls to Mends and relatives. Orthey may have a faint memory of that short-lived moment whenRegis Philbin became a fashion icon for his willingness towear adark blue tiewith a dark blue shirt. What people probably won’t re-member is that every weekWho Wants to & a Millionaire? pittedgroup intelligence against individual intelligence, and that everyweek, group intelligence won.

Who Wants to Be aMillionaire? was a simple show in termsof structure: a contestant was asked multiple-choice questions,which got successively more difficult, and if she answered fifteenquestions in a row correctl)c she walked away with $1 million. Theshow’s gimmickwas that if a contestant got stumped by a question,she could pursue three avenuesofassistance. First, she could havetwo of the four multiple-choice answers removed (so she’d have atleast a fifty-fifty shot at the right response). Second, she couldplace a call to aMend or relative, a person whom, before the show,shehad singled out as one ofthe smartestpeople she knew, and askhim or her for the answet And third, she could poll the studio au-dience, which would immediately cast its votes by computet

4 JAMES suRowlEcKi

Everything we think we know about intelligence suggests that thesmart individual would offer the most help. And, in fact, the “ex-perts” did okay offering the right answer—under pressure—almost65 percent of the time. But they paled in comparison to the audi-ences. Those random crowds of people with nothing better to doon a weekday afternoon than sit in a TV studio picked the right an-swer 91 percent of the time.

No\~çthe results of Who Wants to Be a Millionaire? wouldnever stand up to scientific scrutiny We don’t know how smart theexperts were, so we don’t know how impressive outperformingthem was. ,And since the experts and the audiences didn’t alwaysanswer the same questions, it’s possible, though not likely that theaudiences were asked easier questions. Even so, it’s hard to resistthe thought that the success of the Millionaire audience was amodern example of the same phenomenon that Francis Cakoncaught a glimpse of a century ago.

As it happens, the possibilities of group intelligence, at leastwhen it came to judging questions of fact, were demonstrated by ahost of experiments conducted by American sociologists and psy-chologists between 1920 and the mid-1950s, the heyday of re-search into group dynamics. Although in general, as we’ll see, thebigger the crowd the better, the groups in most of these earlyexperiments—which for some reason remained relatively unknownoutside of academia—were relatively small. Yet they nonethelessperformed very well. The Columbia sociologist Hazel Knightkicked things offwith a series of studies in the early I 920s, the firstof which had the virtue of simplicity In that study Knight asked thestudents in her class to estimate the room’s temperature, and thentook a simple average of the estimates. The group guessed 72.4 de-grees, while the actual temperature was 72 degrees. This was not,to be sure, the most auspicious beginning, since classroom tem-peratures are so stable that it’s hard to imagine a class’s estimatebeing too far off base. But in the years that followed, far more con-vincing evidence emerged, as students and soldiers across America

THE WISDOM OF CROWDS S

were subjected to a barrage of puzzles, intelligence tests, andwordgames. The sociologist Kate H. Gordon asked two hundred stu-dents to rank items by weight, and found that the group’s “esti-mate” was 94 percent accurate, which was better than all but fiveof the individual guesses. In another experiment students wereasked to look atten piles of buckshot—each a slightly different sizethan the rest—that had been glued to a piece of white cardboard,and rank them by size. This time, the group’s guess was 94.5 per-cent accurate. A classic demonstration of group intelligence is thejelly-beans-in-the-jar experiment, in which invariably the group’sestimate is superior to the vast majorityof the individual guesses.When finance professor Jack Treynor ran the experiment in hisclass with a jar that held 850 beans, the group estimate was 871.Only one ofthe fifty-six people in the class made a better guess.

There are two lessons to draw from these experiments. First,in most ofthem the members ofthegroup were not talking to eachother or working on a problem togethet They were making indi-vidual guesses, which were aggregated and then averaged. This isexactly what Galton did) and it is likely to produce excellent re-sults. (In a later chapter, we’ll see how having members interactchanges things, sometimes for the better, sometimes for theworse.) Second, the group’s guess will not be better than that ofeveiy single person in the group each time. In many (perhaps most)cases, therewill be a few peoplewho do better than thegroup. Thisis, in some sense, a good thing, since especially in situations wherethere is an incentive for doingwell (like, say, the stock market) itgives people reason to keep participating. But there is no evidencein these studies that certain people consistently outperform thegroup. In other words, if you run ten different jelly-bean-countingexperiments, it’s likely that each time one or two students willout-perform the group. But they will not be the same students eachtime. Over the ten experiments, the group’s performance will al-most certainly be the bestpossible. The simplest way to get reliablygood answers is just to ask the group each time.

S JAMES SUROWIECICI

A similarlyblunt approach also seems to work when wrestlingwith other kinds ofproblems. The theoretical physicist Norman L.Johnson has demonstrated this using computer simulations of in-dividual “agents” making their way through a maze. Johnson, whodoes his workat the LosAlamos National Laborator)ç was interestedin understanding how groups might be able to solve problems thatindividuals on their own found difficult. So he built a maze—onethat could be navigated via many different paths, some shorter, andsome longer—and sent a group ofagents into themaze one by one.The first time through, they just wandered around, the way youwould ifyou were looking for a particular café in a city where you’dnever been before. Whenever they came to a turning point—whatJohnson called a “node”—they would randomly choose to go rightor left. Therefore some people found their way, by chance, to theexit quicld)c others more slowly. Then Johnson sent theagents backinto the maze, but this time he allowed them to use the informa-tion they’d learned on their first trip, as if the~ddropped breadcrumbs behind them the first time around. Johnson wanted toknow how well his agents would use their new information. Pre-dictably enough, they used it well, andwere much smarter the sec-ond time through. The average agent took 34.3 steps to find theexit the first time, and just 12.8 steps to find it the second.

The key to the experiment, though, was this: Johnson tookthe results of all the trips through the maze and used them to cal-culate what he called the group’s “collective solution.” He figuredout what a majority ofthe group didat each node ofthe maze, andthenplotted a path through the maze based on themajority’s deci-sions. (If more people turned left than right at a given node, thatwas the direction he assumed the group took. ‘fle votes were bro-ken randomly.) The group’s path was just nine steps long, whichwas not only shorter than the path of the average individual (12.8steps), but as short as the path that even the smartest individualhad been able to come up with. It was also as good an answer asyou could find.There was noway to get through the maze in fewer

THE WISDOM OF CROWDS 7

than nine steps, so the group had discovered the optimal solution.The obvious question that follows, though, is: The judgment ofcrowds may be good in laboratory settings and classrooms, butwhat happens in the real world?

II

At 11:38 AM on January 28, 1986, the space shuttle Challengerlifted off from its launch pad at Cape Canaveral. Seventy-four sec-onds later, it was ten miles high and rising. Then it blew up. Thelaunchwas televised, so news of the accident spread quickly Eightminutes after theexplosion, the first story hit the Dow Jones NewsWire.

The stock market did not pause to mourn. Within minutes,investors started dumping the stocks ofthe fourmajor contractorswho had participated in the Challenger launch; Rockwell Interna-tional, which buik the shuttle and its main engines; Lockheed,whichmanaged ground support; Martin Marietta, which manufac-tured the ship’s external fuel tanlq and MortonThiokol, which builtthe solid-fuel booster rocket. Twenty-one minutes after the explo-sion, Lockheed’s stock was down 5 percent, Martin Marietta’s wasdown 3 percent, and Rockwell was down 6 percent.

Morton Thiokol’s stock was hit hardest of all. As the financeprofessors Michael t Maloney and J. Harold Mulherin report intheir fascinating study of the market’s reaction to the Challengerdisastet so many investors were trying to sell Thiokol stockand sofew people were interested in buying it that a trading halt wascalled almost immediately. When the stock started trading again,almost an hour after the explosion, it was down 6 percent. By theend of the day, its decline had almost doubled, so that at marketclose, Thiokol’s stockwas down nearly 12 percent. By contrast, thestocks of the three other firms started to creep back up, and by theend of the day their value had fallen only around 3 percent.

S JAMES SUROWIECk~

What this means is that the stock market had, almost imme-diately labeled Morton Thiokol as the company that was responsi-ble for the Challenger disaster. The stock market is, at least intheory a machine for calculating the present value of all the “freecash flow” a company will earn in the future, (Free cash flow is themoney that’s left over after a company has paid all its bills and itstaxes, has accounted for depreciation, and has invested in the busi-ness- It’s the money you’d get to take home and put in the hank ifyou were the sole owner of the company) The steep decline inThiokol’s stock price—especially compared with the slight declinesin the stock prices of its competitors—was an unmistakable signthat investors believed that Thiokol was responsible, and that theconsequences for its bottom line would he severe.

As Maloney and Mulherin point out, though, on the day ofthe disaster there were no public comments singling out Thiokol asthe guilty party While the New York ‘Times article on the disasterthat appeared the next morning did mention two rumors that hadbeen making the rounds, neither of the rumors implicated Thiokol,and the Times declared, There are no clues to the cause of the ac-cident.”

Regardless, the market was right. Six months after the explo-sion, the Presidential Commission on the Challenger revealed thatthe 0-ring seals on the hooster rockets made by Thiokol—sealsthat were supposed to prevent hot exhaust gases from escaping--became less resilient in cold weather, creating gaps that allowedthe gases to leak out. (The physicist Richard Feynman famouslydemonstrated this at a congressionalhearing by dropping an 0-ringin a glass of ice water. When he pulled it out, the drop in temper-ature had made it brittle.) In the case of the Challenger, the hotgases had escaped and burned into the main fuel tank, causing thecataclysmic explosion. Thiokol was held liable for the accident.The other companies were exonerated.

In other words, within a half hour of the shuttle blowing up,the stock market knew what company was responsible. To be sure,

tHE WISDOM OF CROWDS S

this was a single event, and it’s possible that themarke?s singlingout of Thiokol was just luck Or perhaps the company’s businessseemed especially susceptible to a downturn in the space program.Possibly the trading halt had sent a signal to investors to be waryThese all are important cautions, but there is still something eerieabout what the market did. Thaes especially true because in thiscase the stock market was working as a pure weighing machine,undistorted by the factors—media speculation, momentum trad-ing, and Wall Street hype—that make it a peculiarly erraticmech-anism for aggregating the collective wisdom of investors. That day,it was just buyers and sellers trying to figure out what happenedand getting it right.

How did they get it right? That’s the question that Maloneyand Muiherin found so vexing. First, they looked at the records ofinsider trades to see ifThiokol executives, who might have knownthat their companywas responsible, had dumped stock on January28. They hadn’t. Nor had executives atThiokol’s competitors, whomight have heard about the 0-rings and sold Thiokol’s stock short.There was no evidence that anyone had dumped Thiokol stockwhile buying the stocks of the other three contractors (whichwould have been the logical trade for someone with inside infor-mation). Savvy insiders alone did not cause that first-day drop inThiokol’s price. It was all those investors—most of them relativelyuninformed—who simply ~efused to buy the stock.

But why did they not want Thiokol’s stock? Maloney andMulherin were finally unable to come up with a convincing answerto that question. In the end, they assumed that insider informationwas responsible for the fail in Thiokol’s price, but they could notexplain how. Tellingly, they quoted the Cornell economistMaureenO’Hara, who has said, “While markets appear to work in practice,we are not sure how they work in theory”

Maybe. But it depends on what you mean by “theory” Ifyoustrip the story down to its basics, after all,what happened that Jan-uary day was this: a large group of individuals (the actual and po-

‘0 JAMES SUROWIECKI

tential shareholders of Thiokol’s stock, and the stocks of its com-petitors) was asked a question—”l-low much less are these fourcompanies worth now that the Challenger has exploded?”—thathad an objectively correct answer. Those are conditions underwhich a crowds average estimate—which is, dollar weighted, whata stock price is—h likely to be accurate. Perhaps someone did, infact, have inside knowledge of what had happened to the 0-rings.But even if no one did, ifs plausible that once you aggregated allthe bits of information about the explosion that all the traders inthe market had in their heads that day, it added up to somethingclose to the truth. As was true of those who helped John Cravenfind the Scorpion, even if none ofthe traders was sure that Thiokolwas responsible, collectively theywere certain it was.

The market was smart that day because it satisfied the fourconditions that characterize wise crowds: diversity of opinion (eachperson should have someprivate information, even if it’s just anec-centric interpretation of the known facts), independence (people’sopinions are not determined by the opinions of those aroundthem), decentralization (people are able to specialize and draw onlocal knowledge), and aggregation (some mechanism exists forturning private judgments into a collective decision). If a group sat-isfies those conditions, its judgment is likely to be accurate. Why?At heart, the answer rests on a mathematical truism, If you ask alarge enough group of diverse, independent people to make a pre-diction or estimate a probabilit~çand then average those estimates,the errors each of them makes in coming up with an answer willcancel themselves out. Each person’s guess, youmight say, has twocomponents: information and errot Subtract the error, and you’releftwith the infonnation.

Now, even with the errors canceled out, it’s possible that agroup’s judgment will be bad. For the group to be smart, there hasto be at least some information in the “information” part of the “in-formation minus error” equation. (If you’d asked a large group of

iNS WISDOM OF CROWDS It

children to buy and sell stocks in the wake of the Challenger dis-aste it’s unlikely they would have picked out Thiokol as the cul-prit.) What is striking, though—and what makes a phrase like “thewisdom of crowds” meaningful—is just how much information agroup’s collective verdict so often contains. In cases like FrancisGalton’s experiment or the Challenger explosion, thecrowd is hold-ing a nearly complete picture of the world in its collective brain.

F~rhapsthis isn’t surprising. After all, we are the products ofevolution, and presumablywe have been equipped to make sense ofthe world around us. But who knew that, given the chance, we cancollectivelymake so much sense ofthe world. After all, think aboutwhat happens if you ask a hundred people to run a 100-meter race,and then average their times. The average time will not be betterthan the time of the fastest runners. It will be worse. It will be amediocre time. But ask a hundred people to answer a question orsolve a problem, and the average answerwilloften be at least as goodas the answerofthe smartest member SMithmost things, the averageis mediocrity With decision making, it’s often excellence. You couldsay it’s as ifweve beenprogrammed tobe collectively smart.

III

Truly successful decision making, of course, demands more thanjust a picture of the world as it is. It demands in addition a pictureofthe world as it will (or at least as it may) be.Any decision-makingmechanism therefore has to be good under conditions of uncer-tainty. And whafs more uncertain than the future? Group intelli-gence may be good at telling how many jelly beans are in a jar orremembering the year Nirvana released Nevennind. But how doesit perform under conditions of true uncertainty when the right an-swer is seemingly unknowable—.because it hasn’t happened yet?

Robert Walker’s entire career depends on the answer to that

12 JAMES SUROWIECKI

question. Walker is the sports book director at the Mirage Hoteland Casino in Las Vegas, which means that every week he fieldsthousands ofbets in sports ranging from pro football to Ivy Leaguebasketball. For all those games, Walker has to offer a line (or pointspread), which lets bettors knowwhich team is favored to win andby how many points. The way the tine works is simple. Say the Gi-ants are favored this week by three and a half points over the Rams.If you bet on the Giants, they have to win by four points or morefor you to win the bet. Conversely, if you bet on the Rams, theyhave to lose by three points or less (or win), for you to walk awaywith the casino’s money. In other sports, bets are framed in termsof odds: if you bet on the favorite, you might have to put down$150 to get $100 back, while if you bet on the underdog, you’dhave to lay down $75 towin $100.

As a bookmaker Walker~sjob is not to try to pick what teamwill win. He leaves that to the gamblers, at least in theory Instead,his job is to make sure that the gamblers bet roughly the sameamount ofmoney on one team ason the othet If he does that, thenhe knows that he will win half the bets he’s taken in and lose theother half.Why wouldWalker be satisfied with just breaking even?Because bookiesmake more money on everybet theywin than theylose on every bet they get wrong. If you place a point-spread betwith a bookie, you have to put up $11 to win $10. Imagine thereare only two bettors, one who bets on the favorite and the otherwho bets on the underdog. Walker takes in $22 ($11 from each ofthem). He pays out $2) to the winner. The $1 he keeps is hisprofit. That slim advantage, which is known as the vigorish, or thevig, is what pays the bookie’s bills. And the bookie keeps that ad-vantage only when he avoids having toomuch money ridingon oneside of a bet.

To keep that from happening, Walker needs to massage thepoint spread so that bets keep coming in for both teams. ‘The linewe want is the line that’ll split the public, because that’s when you

THE WISDOM or CROWDS 13

start earning that vig,” he said. In the weekbefore the 2001 SuperBowl, for instance, the Mirage’s opening line had the BaltimoreRavens favored by two and a half points, But soon after the line wasposted, the Mirage booked a couple ofearly $3,000 bets on Balti-more. That’s not much money, but it was enough to convinceWalker to iaise thepoint spread to three. If everyone wanted tobeton Baltimore, chances were the line wasn’t right. So the linemoved. The opening line is set by the bookmaker, but it shiftslargely in response to what bettors do—much as stock prices riseand fall with investor demand.

In theory you could set the opening line wherevei and sim-ply allow it to adjust from there automatically, so that the pointspread would rise or fall anytime there was a significant imbalancebetween the amounts wagered on each side. The Mirage wouldhave no problem doing this; its computerized database tracks thebets as they come in. But bookies place a premium on making theopening line as accurate as possible, because if they set it badlythey’re going to get stuck taking a lot of bad bets. Once a lineopens, though, it’s out of the bookie’s hands, and a game’s pointspread ends up representing bettors’ collective judgment of whatthefinal outcome of that game will be. As Bob Martin, whowas es-sentially the country’soddsmaker in the 1970s, said, “Once you puta number on the board, it becomes public property”

The public, it turns out, is pretty smart. It does not have acrystal ball: point spreads only weakly predict the final scores ofmost NFL games, for instance. But it is very hard for even well-informed gamblers to beat the final spread consistently. In abouthalf the games, favorites cover the spread, while in the other halfunderdogs beat the spread. This is exactlywhat a bookie wants tohave happen. And there are no obvious mistakes in the market’sjudgment—like, say, home teams winning more than the crowdpredicts they will, or road underdogs being consistently under-valued. Flaws in the crowd’s judgment are found occasionaIl~çbut

$4 JAMES SUROWIECKI

when they are they’re typically like theone documented in a recentpaper that found that in weeks fifteen, sixteen, and seventeen ofthe NFL season, home underdogs have historically been a goodbet. So you have to search hard to outperform the betting crowd.Roughly three-quarters of the time, the Mirage’s final line will bethe most reliable forecast ofthe outcomes ofNFL games that youcan find.

The sameis true in many other sports. Because sports bettingis a lund of ready-made laboratory to study predictions and theiroutcomes, a host of academics have perused gambling markets tosee how efficient—that is, how good at capturing all the availableinformation—they are. The results of their studies are consistent:in general, in most major sports the market is relativelyefficient. Insome cases, the crowd’s performance is especially good: in horseracing, for instance, the final odds reliably predict the race’s orderof finish (that is, the favorite wins most often, the horse with thesecond-lowest odds is the second-most-often winner, and so on)and also provide, in economist Raymond D. Sauer’s words, “rea-sonably good estimates of the probability of winning.” In otherwords, a three-to-one horsewill win roughly a quarter of the time.There are exceptions: odds are less accurate in those sports andgaines where the betting market is smaller and less liquid (mean-ing that the odds can change dramatically thanks to only a fewbets), like hockey or golf or small-college basketball games. Theseare often the sports where professional gamblers can make realmone~cwhich makes sense given that we know the bigger thegroup, the more accurate it becomes. And there are also some in-teresting quirks: in horseracing, for instance, people tend to betonlong shots slightly more often than they should and beton favoritesslightly less often than they should. (This seems to be a case ofrisk-seeking behavior: bettors, especially bettors who have beenlosing, would rather take a flyer on a long shot that offers the pos-sibi(ity of big returns than grind it out by betting on short-odds fa-

THE WISDOM OF CROWDS 15

vorites.) But on the whole, if bettors aren’t collectively foreseeingthe future, they’re doing the next best thing.

Iv

Recently I decided I needed—this minutel—the exact text of BillMurray’s Caddyshack riff about toting the Dalai Lama’s golf bag.The punch line of the riff is “So I got that going for me, which isnice” and the Dalai Lama, in MurraVs telling, likes to say “Cungagalunga.” So I went to Google, the Internet search engine, typed in“going for me” and “gunga,” and hit the search button. A list of 695Web pages came hack. First on the list was an article FromGolfOnline, which included the second half of the nh. That wasokay, but third on the list was a Web site for something called thePenn State Soccer Club. The goalie, a guy named David heist, hadposted the entire monologue. The search took 0.18 seconds.

Then I needed to check out the Mulherin paper on the Chal-lenger that I discuss above. I couldn’t remember the author’s name,so I typed in “‘stock market’ challenger reaction”: 2,370 pagescame back. The first one was an article by Slate’s Daniel Grossabout the Mulherin paper. The third was Mulherin’s ownWeb site,with a link to his paper. That search—which, remember, did notinclude Mulherin’s name—took 0.10 seconds, A few minutes Eater.my search for the lyrics to a Ramones song about Ronald Reaganvisiting the Bithurg cemetery took 0.23 seconds, and the First itemon the list had what I needed.

If you use the Internet regularly, these examples of Google’sperformance will not surprise you. This is what we have come toexpect from Google: instantaneous responses with the exact pagewe need up high in the rankings. But if possible, it’s worth lettingourself be a little amazed at what happened during those routinesearches Each time, Google surveyed billions of Web pages and

IS JAMES SUROWIECKI

picked exactly the pages that I would find most useful. The cumu-lative time for all the searches: about a minute and a half.

Google started in 1998, at a time whenYahoo! seemed to havea stranglehold on the search business—and if Yahoo! stumbled,then AltaVista or Lycos looked certain to be the last man standing.But within a couple of years, Google had become the defaultsearch engine for anyone who used the Internet regularly, simplybecause it was able to do a better job of finding the right pagequickly. And the way it does that—and does it while surveyingthree billion Web pages—is built on the wisdom of crowds.

Google keeps the details of its technology to itself, but thecore of the Google system is the PageRank algorithm, which wasfirst defined by the company’s founders, Sergey Brin and LawrencePage, in a now-legendary 1998 paper called ‘The Anatomy of aLarge-Scale Hypertextual Web Search Engine.” PageRank is an al-gorithm—a calculating method—that attempts to let all the Webpages on the Internet decide which pages are most relevant to aparticular search. Here’s how Google puts it:

PageB.ankcapitalizes on the uniquelydemocratic charac-teristic ofthe web byusing its vast link structure as an or-ganizational tool. In essence, Google interprets a link frompageA topage B as a vote, bypage A, for page B. (oogleassessesa page’s importance by the votes it receives. ButGoogle looks at more than sheer volume ofvotes, or links;it also analyzes the page that casts thevote. Votes cast bypages that are themselves “important”weig~imore heavilyand help to make other pages Important.”

In that0.12 seconds, whatGoogle is doing is asking theentire Webto decide which page contains the most useful information, andthe page that gets the most votes goes first on the list. And thatpage, or the one immediately beneath it, more often than not is infact the onewith the most useful information.

tHE WISDOM OP CROWDS 17

Now, Google is a republic, not a perfect democracy As thedescription says, the more people that have linked to a page, themore influence that page has on the final decision. The final voteis a “weighted average”—just as a stock price or an NFL pointspread is—rather than a simple average like the ox-weighers’ esti-mate. Nonetheless, the big sites that have more influence over thecrowd’s final verdict have that influence only because of all thevotes that smaller sites have given them. If the smaller sites weregiving the wrong sites too much influence, Google’s search resultswould not be accurate. In the end, the crowd still rules. To besmart at the top, the system has to be smart all the way through.

V

If allowing people to bet on sportingevents effectively createsa kindofmachine that’s good atpredicting the outcome ofthose events, anobvious question follows: Wouldn’t people betting on other kinds ofevents be equallygood, as a group, at predicting them?Why confineourselves to knowing what the chances are of Los Angeles beatingSacramento if there’s a way we could know what the chances are of,say, George W. Bush beating John Kersy?

We do have a well-established way ofknowing what GeorgeW.Bush’s chances are: thepoll. If you want to know how people are go-ing to vote, you just ask them. Polling is, relatively speaking, accu-rate. It has asolid methodology behind it, and is statistically rigorous.But there’s reason to wonder if a market such as the betting mar-ket—one that allowed the people participating in it to rely on manydifferentkinds ofinformation, including but not limited to polls—might at the very least offer a competitive alternative to Gallup.That’s why the Iowa Electronic Markets (IEM) project was created.

Fbunded in 1988 and run by the CollegeofBusiness atthe Uni-versity of Iowa, the IEM features a host of markets designed topredict the outcomes of elections—presidential, congressional, gu-

18 JAMES SUROWIECKI

bernatorial, and foreign. Open to anyone who wants to participate,the ibM allows people to buy and sell futures contracts” based onhow they think a given candidate will do in an upcoming election.While the IbM offers many different types of contracts, two are mostcommon. One is designed to predict the winner of an election. In thecase of the California recall in 2003, for instance, you could havebought an “Arnold Schwarzenegger to win” contract, which wouldhave paid you $1 when Schwarzenegger won. Had he lost, you wouldhave gotten nothing. The price you pay for this kind of contract re-flects the market’s judgment of a candidate’s chances of victory If acandidate’s contract costs 50 cents, it means, roughly speaking, thatthe market thinks he has a 50 percent chance of winning. If it costs80 cents, he has an 80 percent chance of winning, and so on.

The other major kind of IbM contract is set up to predictwhat percentage of the final popular vote a candidate will get. Inthis case, the payoffs are determined by the vote percentage: ifyou’d bought a George W Bush contract in 2000, you would havereceived 48 cents (he got 48 percent of the vote) when the electionwas over.

If the IbM’s predictions are accurate, the prices of these dif-ferent contracts will be close to their true values. In the market topredict election winners, the favorite should always win, and big-ger favorites should win by bigger margins. Similarly in the vote-share market, if George W Bush were to end up getting 49 percentof the vote in 2004, then the price of a George \V. Bush contractin the run-up to the election should be close to 49 cents.

So how has the IbM done? Well, a study of the IbM’s per-formance in forty-nine different elections between 1988 and 2000found that the election-eve prices in the [EM were, on average, offby just 1.37 percent in presidential elections, 3.43 percent in otherU.S. elections, and 2.12 percent in foreign elections. (Those num-bers are in absolute terms, meaning that the market would havebeen off by 1.37 percent if, say, it had predicted that Al Gore wouldget 48.63 percent of the vote when in reality he got 50 percent.)

THC WISDOM or CROWDS IS

The IEM has generally outperformed themajor national polls, andhas beenmore accurate than them evenmonths in advance of theactual election. Over the course of the presidential elections be-tween 1988 and 2000, for instance, 596 different polls were re-leased. Three-fourths of the time, the IEM~smarket price on theday each of those polls was released was more accurate. Polls tendto be very volatile, with vote shares swinging wildly up and down.But the IEM forecasts, though ever-changing, are considerably lessvolatile, and tend to change dramatically only in response to newinformation. That makes themmore reliable as forecasts.

What’s especially interesting about this is that the IEM isn’tvery big—there have never been more than eight hundred or sotraders in the market—and it doesn’t, in any way, reflect themakeup of the electorate as a whole. The vast majority of tradersare men, and a disproportionate—though shrinking—number ofthem are from Iowa. So the people in the market aren’t predictingtheir own behavior. But their predictions of what the voters of thecountrywill do are better than the predictions you get when youask thevoters themselves what they’re going to do.

The IEM’s success has helped inspire other similar markets,including the Hollywood Stock Exchange (HSX), which allowspeople to wager on box-office returns, opening-weekend perfor-mance, and theOscars. The HSX enjoyed itsmost notable successin March of 2000. That was when a team of twelve reporters fromThe Wail Street Journal assiduously canvassed members of theAcademy ofMotion PicturesArtsand Sciences in order to find outhow they had voted. The Academy was not happy about this. Theorganization’s president publicly attacked theJournal for trying to8scoop us before Oscar night,” and the Academy urged membersnot to talk to reporters. But with theJournal promising anonymit~cmore than a fewpeople—356, or about 6 percent of all members—disclosed how they had filled out their ballots. The Friday beforethe ceremony, theJournal published its results, forecasting thewin-ners in the six major Oscar categories—Best Picture, Best Direc-

20 JAMES SOROWIECKI

tor, Best Actor and Best Actress, Best Supporting Actor and BestSupporting Actress. And when the envelopes were opened, theJournal’s predictions--—-much to the Academy’s dismay—turned outto he pretty much on target, with the paper picking five of the sixwinners. The I ISX, though, had (lone even better, getting all six ofthe six right. In 2002, the exchange, perhaps even more inipres-sivelv picked thirty-five of the eventual forty Oscar nominees.

The HSX’s hox-office forecasts are not as impressive or as ac-curate as the IBM’s election forecasts. But Anita Elberse, a profes-sor of marketing at Harvard Business School, has compared theHSX’s forecasts to other Hollywood prediction tools, and foundthat the ElSX’s closing price the night hefore a movie opens is thesingle best available forecast of its weekend box office. As a result,the HSX’s ou’net Cantor index Holdings, is now marketing its datato Hollywood studios.

One of the interesting things about markets like the IBM andthe I-ISX is that they work fairly well without much—or any—money at stake. The IBM is a real-money market, hut the most youcan invest is $500, and the average trader has only $50 at stake. Inthe HS~,the wagering is done entirely with play money All the ev-idence we have suggests that people focus better on a decisionwhen there are financial rewards attached to it (which may help ex-plain why the IBM’s forecasts tend to he more accurate). ButDavid Pennock—a researcher at Overture who has studied thesemarkets closely--——found that, especially for active traders in thesemarkets, status and reputation provided incentive enough to en-courage a serious investment of time and energy in what is, afterall, a game.

As the potential virtues of these decision markets have be-come obvious, the range of subjects they cover has grown rapidly.At the Newshutures and TradeSports exchanges, people could bet,in the fail of 2003. on whether or not Kohe Bryant would he con-victed of sexual assault, on whether and ‘~..‘henweapons of mass de-struction would be found in Iraq, and on whether Arid Sharon

1HZ WISDOM OF CROWDS at

would remain in power longer than \~sirArafat. Ely Dahan, a pro-fessor atUCLA, has experimented with a classroom-decision mar-ket in which students bought and sold securities representing avariety of consumer goods and services, including SUVs, ski re-sorts, and personal digital assistants. (In a real-life market of thiskind, the value of a security might depend on the first-year sales ofa particular SUM) The market’s forecasts were eerily similar to thepredictions that conventional market research had made (but theclassroom research was muchcheaper). In the fall of 2003, mean-while, MITs Technology Review set up a site called InnovationFutures, where people could wager on future technological devel-opments. And Robin Hanson, an economics professor at GeorgeMason University whowas one of the first to write about the pos-sibility ofusing decision markets in myriad contexts, has suggestedthat such markets could be used to guide scientific research andeven as a tool to help governments adopt better policies.

Some of these markets will undoubtedly end up being of lit-de use, either because they’ll fail to attract enough participants tomake intelligent forecasts or because they’ll be trying to predict theunpredictable. But given the right conditions and the right prob-lems, a decision market’s fundamental characteristics—diversit)cindependence, and decentralization—are guaranteed to make forgood group decisions. And because such markets represent a rela-tively simple and quick means of transformingmanydiverse opin-ions into a single collective judgment, they have the chance toimprove dramatically the way organizations make decisions andthink about the future.

In that sense, the most mystifying thing about decision mar-kets is how little interest corporate America has shown in them.Corporate strategy is all about collecting information from manydifferent sources, evaluating the probabilities of potential out-comes, and making decisions in the face of an uncertain future.These are tasks for which decision markets are tailor-made. Yetcompanies have remained, for the most part, indifferent to this

2* JAMES SUNOWIECKI

source ofpotentially excellent information, and have been surpris-ingly unwilling to improve their decision making by tapping intothe collective wisdom of their employees. We’ll look more closelyat people’s discomfort with the idea of the wisdom of crowds, butthe problem is simple enough: just because collective intelligenceis real doesn’t mean that it will be put to good use.

A DECISION MARKET is an elegant and well-designed method forcapturing the collective wisdom. But the truth is that the specificmethod that one uses probably doesn’t matter very much. In thischapter, we’ve looked at a host of different ways of tapping intowhat a group knows: stock prices, votes, point spreads, pari-mutuelodds, computer algorithms, and futures contracts. Some of thesemethods seem to work better than others, but in the end there’snothing about a futures market that makes it inherently smarterthan, say, Google or a pari-mutuel pool. These are all attempts totap into the wisdom of the crowd, and that’s the reason they workThe real ke)c it turns out, is not so much perfecting a particularmethod, but satisfying the conditions—diversit~ç independence,and decentralization—that a group needs to be smart. As well seein the chapters that follo~that’s the hardest, but also perhaps themost interesting, part of the story