wisdom or madness? comparing crowds with expert evaluation in funding the arts

Download Wisdom or Madness?  Comparing Crowds with  Expert Evaluation in Funding  the Arts

If you can't read please download the document

Post on 27-Dec-2015

885 views

Category:

Documents

0 download

Embed Size (px)

DESCRIPTION

‘Wisdom or Madness? Comparing Crowds with Expert Evaluation in Funding the Arts’ is a working paper by Ethan Mollick and Ramana Nanda, which came out in late June. In it, the two authors examine the viability of crowd wisdom, as it relates specifically to theater projects on Kickstarter.

TRANSCRIPT

  • Electronic copy available at: http://ssrn.com/abstract=2443114

    Copyright 2014 by Ethan Mollick and Ramana Nanda

    Working papers are in draft form. This working paper is distributed for purposes of comment and discussion only. It may not be reproduced without permission of the copyright holder. Copies of working papers are available from the author.

    Wisdom or Madness? Comparing Crowds with Expert Evaluation in Funding the Arts Ethan Mollick Ramana Nanda

    Working Paper

    14-116 June 20, 2014

  • Electronic copy available at: http://ssrn.com/abstract=2443114

    1

    Wisdom or Madness? Comparing Crowds with Expert Evaluation in Funding the Arts*

    Ethan Mollick The Wharton School University of Pennsylvania

    Ramana Nanda

    Harvard Business School

    June 20, 2014

    Abstract Infieldsasdiverseastechnologyentrepreneurshipandthearts,crowdsofinterestedstakeholdersareincreasinglyresponsiblefordecidingwhichinnovationstofund,aprivilegethatwaspreviouslyreservedforafewexperts,suchasventurecapitalistsandgrantmakingbodies.Littleisknownaboutthedegreetowhichthecrowddiffersfromexpertsinjudgingwhichideastofund,and,indeed,whetherthecrowdisevenrationalinmakingfundingdecisions.Drawingonapanelofnationalexpertsandcomprehensivedatafromthelargestcrowdfundingsite,weexaminefundingdecisionsforproposedtheaterprojects,acategorywhereexpertandcrowdpreferencesmightbeexpectedtodiffergreatly.Weinsteadfindsubstantialagreementbetweenthefundingdecisionsofcrowdsandexperts.Wherecrowdsandexpertsdisagree,itisfarmorelikelytobeacasewherethecrowdiswillingtofundprojectsthatexpertsmaynot.Examiningtheoutcomesoftheseprojects,wefindnoquantitativeorqualitativedifferencesbetweenprojectsfundedbythecrowdalone,andthosethatwereselectedbyboththecrowdandexperts.Ourfindingssuggestthatthedemocratizationofentrythatisfacilitatedbythecrowdfundinghasthepotentialtolowertheincidenceoffalsenegatives,byallowingprojectstheoptiontoreceivemultipleevaluationsandreachouttoreceptivecommunitiesthatmaynototherwiseberepresentedbyexperts.

    *emollick@wharton.upenn.edu and RNanda@hbs.edu Authorship is alphabetical; both authors contributed equally in developing this paper. We are grateful to Ajay Agrawal, Teresa Amabile, Ronnie Chatterji, Jing Chen, Chuck Eesley, Avi Goldfarb, Mukti Khaire, Matt Marx, Matthew Rhodes-Kropf, Antoinette Schoar and Tavneet Suri for helpful conversations and feedback, and to the participants of the Economics of Entrepreneurship and Innovation Conference at Queens School of Business. We thank John Galvin and Andrew Marder at HBS for their help in implementing the survey. We gratefully acknowledge financial support from the Kauffman Foundations Junior Faculty Fellowship and Harvard Business Schools Division of Research and Faculty Development.

  • Electronic copy available at: http://ssrn.com/abstract=2443114

    2

    1. IntroductionOneofthecentralinsightsofSchumpeter(1942)wastohighlightthedistinctrolesplayedby

    entrepreneursandinvestorsinbringingnewideastofruition,sinceitallowedforthepossibilitythatgoodideasmightbeprecludedfromthemarketbecausetheywereunabletoattractfinancingfrominvestors.Otherscholarshaveshownthattheopinionsoffundingauthoritiessimilarlyaffectthedirectionofinnovationinothercriticalfields,includingscientificresearch(Bourdieu,1975;Goldfarb,2008;Nelson,1959)andcreativeindustries(Caves,2000,2003).Ineachsetting,scholarshavenotedthatconstrainedfinancingenvironmentsandthecontrolexertedbyasmallgroupofexpertsmeansthatprojectsthatareultimatelyfundedarenotnecessarilyobjectivelybestbutaresubjecttothespecificinformation,agency,andorganizationalconstraintsfacedbythesmallnumberofexpertsmakingfundingdecisions.(e.g.Caves,2000;Ferrary&Granovetter,2009;Goldfarb,2008;Kerr,Nanda,&RhodesKropf,2013;Kortum&Lerner,2000).

    Rapidtechnologicaladvancesoverthepastfewyearshavemadeitsignificantlyeasierforinnovatorstocircumventexperts,andinsteadcommunicatedirectlywithalargenumberofinterestedstakeholders,leadingtothegrowingrelianceoncrowdstomakedecisionsthatoncerestedsolelyinthehandsofasmallnumberexperts.Forexample,thecrowdfundingplatformKickstarterhasraisedmorethan$1billionforover60,000projectsfromoversixmillionbackerssinceitsfoundingin2009,projectsthatmayotherwisehaveneededtoseekfundingfromventureinvestorsorgrantmakingbodies.Infact,since2012,thisplatformaloneraisesmoremoneyfortheartsannuallythanthetotalfundingprovidedthroughtheUSgovernmentrunNationalEndowmentfortheArts.Theimportanceofcrowdbaseddecisionsaregrowinginawiderangeoffields,includingthefundingoftechnologybasedstartups(Agrawal,Catalini,&Goldfarb,2013;Mollick,2014),thedevelopmentofnewproducts(Afuah&Tucci,2012;Poetz&Schreier,2012;VonHippel,1986)andscientificresearch(Franzoni&Sauermann,2014).

    Despitethegrowingroleofcrowdsinmakingdecisionsoncelefttoexperts,littleisknownabouthowcrowdsandexpertsmaydifferintheirabilitytojudgeprojects,orevenwhethercrowddecisionmakingisbasedonanyrationalcriteriaatall.Indeed,twopopularbooksseparatedby150yearsofferstarklydifferentviewsofthevalueofcrowds,withCharlesMackay(1852)warningaboutthemadnessofcrowdswhileJamesSurowiecki(2004)morerecentlyextolledthewisdomofcrowds.Studiestodateattemptingtounderstandthedifferencesbetweencrowdsandexpertshavefocusedonpredictiontasksormarkets,whereaggregateddecisionscanbemoreaccuratethanindividualactions(Budescu&

  • 3

    Chen,2014;Larrick,Mannes,&Soll,2012;Ray,2006;Tetlock,2005),orelseoncaseswhereorganizationscangetpositiveresultsbyreachingouttocrowdsundercarefullycontrolledcircumstances(Bogers,Afuah,&Bastian,2010;Terwiesch&Ulrich,2009).Crowdfunding,ontheotherhand,doesnothaveanysuchcontrols,anddiffersfromthesepreviousphenomenainmanyways:thereisnomarketrewardmechanismforcorrectanswers;noclearcriteriadeterminingwhatisbeingpredicted;noexpertoversightofthecrowddecisionmakingprocess;andnodirectcoordinationbetweencrowdmembers.Inthiscase,thecrowdcanbesubjecttomanygroupdecisionmakingfallacies,asIsenberg(2012)arguedaboutcrowdfunding:groupirrationalityiswelldocumentedcrowdsarewiseonlyinaverylimitedsetofcircumstances.Asoftenasnot,crowdsbringustulipcrazes,subprimemeltdowns,theKittyGenovesescandal,Salemwitchtrials,andothertragedies.Thus,thereisnoclearevidenceonwhethercrowdsarerationalatall,and,iftheyare,whatdifferencesmightexistbetweenthejudgmentofcrowdsandthatofexperts.

    Inthispaper,weaimtoprovidethefirstrobustcomparisonbetweencrowdandexpertevaluationofprospectiveprojectsthatrelyontasteandjudgment,ratherthanprediction,usingrichdatafromthecrowdfundingplatformKickstarter.Severalfeaturesofourresearchdesignaimtoaddressthespecificchallengesassociatedwithmakingsuchacomparison.First,wehavecompletedataoncrowdfundingcampaigns(thatis,theapplications)forallprojects,includingthosethatwereunsuccessful.Second,werestrictourstudytotheaterprojects,whichhaveanumberofappealingfeatures:evaluatingthementailsbothasubjectiveartisticcomponentandanunderstandingofthecommercialbottomline;itispossibletodevelopasomewhatobjectivesenseofthelongrunsuccessoftheproject;andtheaterfundingisaonetimeevent,notsubjecttothechallengesassociatedwithmultistagefinancingthatmightleadcrowdstochoosedifferentinvestmentsfromdeeppockedprofessionalinvestorseveniftheyratedprojectsthesame(Nanda&RhodesKropf,2012).Finally,werecruitexpertevaluatorswhohavedeepexpertisejudgingtheaterapplicationsforinstitutionssuchastheNationalEndowmentfortheArts.TheprojectswererenderedinthesameformatastheKickstartercampaigns,hadsimilarfinancialrequirementstotraditionalgrantapplicants,andwerestrippedofanyreferencestotheoutcomeofthecampaign.Werandomlyassignsetsofprojectstotheseexpertstoevaluate,whereeachsetconsistsofthreefailedandthreesuccessfulprojects.Thisapproachallowsustoalsoincludeexpertfixedeffectstoexaminetherobustnessofourresults.

    Wehavethreemainfindings.First,onaverage,wefindaremarkabledegreeofcongruencebetweentherealizedfundingdecisionsbycrowdsandtheevaluationofthosesameprojectsbyexperts.

  • 4

    Projectsthatwerefundedbythecrowdsreceivedconsistentlyhigherscoresfromexperts,weremorelikelytoberatedasbeingthebestofthesixintheset,andweremuchmorelikelytohavereceivedfundingfromtheexperts.ThispatternwasequallytrueforprojectsthatwereoverfundedonKickstarterthatis,projectsthatraisedmoremoneythatwasaskedfor.Giventhestrongpositivecorrelationsweobserveincrowdandexpertevaluations,eveninacategorywithasubjectivenaturesuchastheater,weareabletostronglyrejectthenotionofthemadnessofcrowdsinoursetting.

    Oursecondfindingisthatdespitethebroadcongruenceinevaluation,weseeasystematicpatternintermsofthedisagreement.Oftheprojectswherethereisnoagreement,thecrowdismuchmorelikelytohavefundedaprojectthatthejudgedidnotlike.Around75%oftheprojectswherethereisadisagreementareoneswherethecrowdfundedaprojectbuttheexpertwouldnothavefundedit.Wealsoseeaclearpatternintermsofthecharacteristicsofprojectsthatarelikedbythecrowds:theyoffermultipletiersofrewardsandprovidemoreupdates,whilethosethatarerankedhighlybytheexpertsbutnotfundedbycrowdshavesystematicallyfewerrewardlevels,fewerpicturesandfewervideos.Thatis,weseemtofindthatthereisanarttoraisingmoneyfromcrowds,onethatmaybesystematicallydifferentfromthatofraisingmoneyfromexperts.Thecrowdsseemtoplaceemphasison,orextractinformationcontentfromdifferentattributesoftheprocessthanexperts.

    Ourthirdfindingrelatestothelongertermoutcomesoftheseprojects.Anattractivefeatureofthefactthatmostofthedisagreementsarerelatedtoprojectsfundedbythecrowd,weareabletocomparethelongtermoutcomesofprojectsthatwerelikedbybothcrowdsandexpertsandthoselikedonlybythecrowds.Despitethefactthatthesewereratedsignificantlyworsebytheexperts,wefindnoqualitativeorquantitativedifferencesinthelongtermoutcomesoftheseprojects.Theyareequallylikelytohavedeliveredonbudget,resultinorganizationsthatcontinuetooperate,and,althoughthemetricsforsuccessaresubjective,wefindstrongqualitativeevidencethatmanyoftheseshowshadbothcommercialsuccessandreceivedpos

Recommended

View more >