harnessing the power of the crowd –the next generation for...

©2013PRICESystems,LLCAllRightsReserved|DecadesofCostManagementExcellence 1©2013PRICESystems,LLCAllRightsReserved|DecadesofCostManagementExcellence

HarnessingthePoweroftheCrowd– theNextGenerationforSoftwareOutsourcing?

ArleneMinkiewicz,ChiefScientist

©2013PRICESystems,LLCAllRightsReserved|DecadesofCostManagementExcellence 2

Agenda

§ Introduction

§ CrowdsourcingDefined– Crowdsourcingingeneral– CrowdsourcingSoftwareDevelopment– BenefitsandChallenges

§ CrowdsourcingExamples

§ CrowdsourcingSoftwareDevelopmentPlatforms

§ Cost,ScheduleandQualityImplicationsofCrowdSourcing


Introduction

§ Crowdsourcing– thepracticeofusingthecrowdtosolveaproblemoraccomplishcertaintasks– hasbeenaroundforaverylongtime

§ Recenttechnologyadvanceshavecreatedanenvironmentwhereitiseasyandeffectivetoengageacommunityofinterestedpersonstosolvemanytypesofproblems– Designofsoftware,logos,businesscards,websites,etc

– Testingofsoftwareapplications

– Documentation

– Fundingandsupportinginnovativeprojects

– Solvingcomplexmathematicalandalgorithmicchallenges

– Creatingnewandinnovativeproductswithinanexistingproductline

§ Manyarguethateffectiveandsensibleapplicationsofsoftwaredevelopmentcrowdsourcingcreateopportunitiestoimprovecost,scheduleandqualityofsoftwareapplications

§ Thispaperdiscussescrowdsourcing,softwaredevelopmentinacrowdsourcingplatformanddiscussescostandqualityconsiderationswhilecrowdsourcing


CrowdsourcingDefined

§ TheexpressionCrowdsourcingwascoinedasaportmanteau(alinguisticblendofwords)ofthewordscrowdandoutsourcing

§ Crowdsourcingisaparticipativeonlineactivityinwhich:– Individual,institutions,nonprofitsorcompaniesproposethevoluntaryundertakingofatask

ortasksofvaryingcomplexityandmodularityto….– Agroupofindividualsofvaryingknowledge,heterogeneityandnumberviaa….– OpencallissuedonlinegenerallythroughaCrowdsourcingplatform…

§ Somewellknownexamplesinrecenthistory– Wikipedia– Linux– Lays‘Doyourselfaflavorcontest’

§ Crowdsourcingcanbe….– Competitive– onlyselectedparticipantswillwin,inhighlycompetitivesituationsmanywill

competebutonlyasfewwillwin– Non-CompetitiveorCollaborative– groupsofindividualsworktogethertoaccomplishtasks


CrowdsourcingDefined(Continued)

§ Typesofcrowdsourcing:– Crowdfunding– projectsarefundedbyalargegroupofpeoplewhoareinterestedintheproject

– Crowdsourceddesign– projectsarefundedtoengagethecrowdinanefforttodesignsomething(likeawebsite)

– Crowdwisdom – Usersaskquestionstothecrowd(likeYahooAnswers)orcorporationsposecomplexscientificortechnicalproblemstotakeadvantageoftheknowledgeofthecrowd(likeInnoCentive)


CrowdsourcingDefinition(Continued)

§ Definitions– Crowdworker – onewhoacceptsanonlinecrowdsourcingtasksandcollaboratesorcompetesforsuccessfulcompletion

– Crowdsourcer – anorganizationorindividualthatproposedtohavesomeoralloftheirtasksaccomplishedthroughcrowdsourcing.

– Crowdsourcingmarketplace– environmentorplatformthatactsasanintermediarybetweenthecrowdsourcer andthecrowdworker throughaplatformforintegrationbetweentheparties

§ CrowdsourcingvsOutsourcing– Outsourcinginvolvesoutplacingofspecificcorporatetaskstoathirdpartycontractorororganizationforanegotiatedfee

– Crowdsourcinginvolvestasksbeingallocatedtoanundefinedgroupofunknownindividualswho,whensuccessful,arerewardedfortheireffortinperformingthetask


Crowdsourcingexamples

§ Waze.com– themassescontributerealtimeinformationabouttrafficconditions,locationofspeedtraps,generaltraveltime– thisinformationisbroadcasttoallusers

§ Wikipedia– anencyclopediawrittenbypeopleacrosstheglobe– bothexpertsandamateurs.

§ Kickstarter– morethan$2.7Bpledgedtoprojectsbyacommunityofpeopleinterestedintheprojects– novels,artprojects,movies,etc.

§ Lego– usersdesignnewproductsthatotherusersvoteon.Theideawiththemostamountofvotesisproducedandthecreatedreceives1%royaltyonnetrevenue

§ AirBnB – peoplewhoneedaplacetostayfindpeoplewhohaveaplaceforthemtostayviaanonlineapplication


CrowdsourcingBenefitsforCrowdsourcers

§ Accesstothecollectiveknowledgeofalargegroupofindividualswithawidediversityandbreadthofideasandknowledge

§ Innovationonsteroids– Optimizescreativity

§ Costsavings– generallylessexpensivethanhiringdedicatedprofessionals

§ Reduceoverhead

§ RapidAcquisition– findingsolutionsthatalreadyexist

§ SolutionDiversity

§ SolutionScalability


CrowdsourcingBenefitsforCrowdworkers

§ Economic– winnersarepaidfortheirsolutions

§ SocialRecognitionorStatuswithinthecommunity

§ Acquisitionofnewknowledgeinthefield– Attemptingcontestswheretheyneedtolearnnewtechnologies– Throughdiscussionboardswithfellowcontestparticipants

§ Experienceintheirfield

§ Thrillofcompetition


CrowdsourcingChallenges

§ Openingthedoortothecrowdwillnotonlyresultincreative,innovativeideas,therewillalsolikelybesomehalf-bakedideaswhichrequiretimeandefforttoreviewandevaluate

§ Toomanychefsmaycreatechallengestoeffectivedecisionsandtimelyprogress

§ Cheaplabormaynotbethebestlabor

§ Intellectualpropertyissues

§ Nocontract– noconsequencesfornondelivery

§ Scheduleissuesifnoonesignsupforcertaintasksortheygounfinished

§ QualityControl


SoftwareDevelopmentCrowdsourcing

§ Opencallforparticipationinsoftwaredevelopmenttasksincluding:– Concept– Requirements– Design– Coding– Testing– Documentation

§ Insteadofbeingaccomplishedbyanon-premise oroutsourcedsoftwaredevelopmentteam– thesetaskscanbeassumedbymembersofthegeneralpublicwhoarepartoftheassociatedcrowdsourcingplatformcommunity



§ Technologyadvancescryfornewwaystodevelopsoftware…– Cloudcomputing– BigDataanalysis– MobileComputing– Increasedsupportforcollaborationinsoftwaredevelopment

§ Paradigmshiftinthesoftwaredevelopmentareaofpeerproductionsoftware– Likelytobecomethenextgenerationofoutsourceoroff-shoring



§ Howdoesitwork?– Crowdsourcer developsaspecificationfortheworkdesiredandtheassociatedtasksanddeliverables

– ThesetasksarepublishedonlineviatheCrowdsourcingplatformtothatplatform’scommunity

– Inadditiontheamountofrewardandrequiredtimeframefordeliveryarepublishedaswell

– Communitymembersaccepttasksandbeginwork(potentiallysomeintermediatebackandforthforclarification)

– Uponcompletiontasksaresubmitted– Theone(ormore)bestsolutionsareselectedandrewardsareissuetothecrowdworkers responsibleforthewinningsolutions


SoftwareDevelopmentCrowdsourcingExamples

§ Harvard– a2weeklongTopCoder Marathonmatchresultedina976xincreasetocomputationspeedforalgorithmsfordifficultgeneticsbasedchallenges

§ NASA– turningtothepublicforout-of-the-boxthinkingontopicssuchasanewalgorithmtoimproveRobonaut’s tooltotakeadvantageofmaterialsfoundonasteroids,themoonorMars

§ Humana– usedTopCoder todeliveranASP.Net applicationrevampingtheirwebsitewhichhadfallenbehindindustrystandards– within12weekstheywereabletobetterprovidetheircustomerswithneededpharmaceuticalservices


SoftwareDevelopmentCrowdsourcingPlatforms

§ Mostsoftwaredevelopmentcrowdsourcingplatformsfocusonasingleareaofsoftwaredevelopment

§ ForDesignthereareplatformssuchas– DesignCrowd– CrowdSpring– 99Designs

§ ForTestingthereareplatformssuchas– CrowdsourceTesting– bugFinder– Mob4-hire– appStori (Betatestingandcrowdfundingfornewapps)

§ TopCoder andGetACoder appearstobeonofthefewplatformsthattranscendstheentiredevelopmentprocess


TopCoder


CostandQualityConsiderations

§ Crowdsourcingcreatesanenvironmentwheresomepercentageofcostsandschedulecanbecontainedforaproject

§ Organizationstakingadvantageofcrowdsourcingopportunitiescancontrolcostandscheduleforcrowdsourcetasks– Prizeamountsaregivenbythecrowdsourcer– Deadlinesforcompleteddeliverablesaregivenbythecrowdsourcer

§ Smallprojectswheremostorallofthetasksarecrowdsourcedwillbeabletodetermineprojectcostsfairlywell– Thoughthereshouldbeconsiderationformanagingtheproject,developingspecifications,

reviewingsubmissions,assessingquality,finalintegrationorintegrationwithothercapabilities,etc.

– Thereshouldalsobeconsiderationsforsituationswherenooneacceptsataskoracceptersfailtodeliverasolution– theseshouldbeconsideredpartoftheriskofacostassessment

§ Largeprojectsshoulddeterminecostbasedonthecapabilitytobedeliveredanddiscountforcrowdsourcingtasksbasedonhistoricalknowledge– ReviewofcompletedtasksonTopCoder orsomeotherplatformprovidesgoodhistoricalinformation

aboutpreviouscrowdsourcedtasks


CostandQualityConsiderations

§ Whenusingcheaplabor– thereisalwaysaconcernforquality.– Ingeneralcrowdworkers doitfortheexperienceandcompetitionaswellasthemoney,sohighqualitysolutionsarequitelikelyhowever….• Clearqualitystandardshouldbedefinedandenforcedbythecrowdsourcerwithrewardsonlygoingtothehighestqualitysolutions• Peerreviewsofsolutionsshouldbeanessentialpartofthereviewprocess• Historicaldataonqualityismoreillusivethanforcostandschedule–organizationsshouldstrivetocreatetheirownqualitymetricscollectionstobetterunderstandthequalityriskswhencrowdsourcingonaparticularcrowdsourcingplatform


Conclusions

§ Crowdsourcing– whileit’sbeenaroundforalongtime– isstillgainingtraction– especiallyinthesoftwaredevelopmentarena

§ Crowdsourcingplatformsexistforvarioussoftwaredevelopmenttasks,andhavebeenusedsuccessfullyinvariouscontexts– Lotsofsuccesseswithwebdesignandsoftwaretesting– Lotsofsuccesseswithsolvingcomplextechnicalandalgorithmicproblems

§ Careshouldbetakenwhendecidingtousecrowdsourcingbasedon:– Criticalityoftheprojecttothebusiness– Criticalityoftimetomarketfortheproject– Organization’sexperiencewithcrowdsourcing

§ Aswithallsoftwaredevelopmentendeavorsitisimportanttoassesstherisksoftheprojectinthecontextofhistoryandexperience

©2013PRICESystems,LLCAllRightsReserved|DecadesofCostManagementExcellence 20©2013PRICESystems,LLCAllRightsReserved|DecadesofCostManagementExcellence

Q&ASESSION

[email protected]

ArleneF.MinkiewiczChiefScientist

PRICE® Systems,LLC.17000CommerceParkway– SuiteA

Mt.Laurel,NJ 08054Office856-608-7222 Mobile856-630-9408


References[1]Codabux,Z.,Williams,B.,”Managing TechnicalDebt:AnIndustrialCaseStudy,2013,InternationalWorkshoponManagingTechnicalDebt,SanFrancisco,CA,May2013,p8-15

[2]Ergin,L.,“TechnicalDebt- DonotUnderestimatetheDanger”,availableathttp://www.slideshare.net/lemiorhan/technical-debt-do-not-underestimate-the-danger (Retrieved3/12/2015)

[3]Curtis,B.,Sappidi,J.,Szynkarski,A.,“EstimatingthePrincipalofanApplication’sTechnicalDebt”,IEEESoftware,vol 29,no.6,pp34-42,Dec2012

[4]Curtis,B.,Sappidi,J.,Szynkarski,“Estimatingthesize,codeandtypesofTechnicalDebt”,ThirdInternationalWorkshoponManagingTechnicalDebt2012,2012

[5]Curtis,B,“MeasuringandManagingTechnicalDebt”,availableathttp://omg.org/news/meetings/tc/tx-14/special-events/cisq-presentations/CISQ-Seminar-2014-6-17-BILL-CURTIS-Measuring-and-Managing-Technical-Debt.pdf,(Retrieved3/12/2015)

[6]SWEBOKVersion3andGuidetoSWEBOKavailableathttp://www.computer.org/web/swebok/v3


BackupInformation


CrowdsourcingDesignPlatforms

§ DesignCrowd – logoorwebsiteoranycreativeprojects– Postadesigncontest….

• Postsabriefdescribingwhattheywantdesigned–alongwithwhattheywanttopayforthedesign

• Designersreadbriefandstartworking• Getfeedback,eliminatedesignsyoudon’tlike,suggestchanges

• Selectadesignandthesubmitter(s)arecompensated

§ CrowdSpring – graphicdesign,websiteandmobileappdesign,productdesign


CrowdsourcingTestPlatforms

§ CrowdsourcedTesting

§ BugFinder– DefineProjects– ProjectsareManagedbyBugFinder– ReviewBugs– BugsarefixedbyBugFinder Community– Softwareisreleased


SoftwareDevelopmentCrowdsourcingPlatforms

§ TopCoder.Com– Projectsfallintooneofthefollowingtracks:Design,DevelopmentandDataScience.Withinthesetracksprojectscanbefurthercharacterized:• AlgorithmsandAnalytics• AppDesignandDevelopment• AppModernization• CommunityBuilding• InnovationPrograms• StaffAugmentation

– TopCoder presentschallengesfromtheircustomersinoneofthethreetracks• RealworldsystemsarebrokenintotaskareassotheCrowdworker canworkinareastheyknowbest• Crowdworkers competetodevelopthebestsolutioninthegiventimeframe• Alsoregularcompetitions(SingleRoundMatches)forfunandskilltesting/development

– Despitethecompetition,anenvironmentofcollaborationexistsasacommunityisdeveloped– tipsaresharedamongcompetitorsviadiscussionforums

– Peerreviewensuresthatthesolutionsdevelopedmeetthehighestqualitystandards


TopCoder


Aportmanteauisalinguisticblendofwords!

harnessing the power of the crowd –the next generation for...

Documents