Harnessing the Power of the Crowd

Using Crowd-Based Challenges as Tools for Engineering Projects

NASA’s Center of Excellence for Collaborative Innovation (CoECI)

Steve Rader steven.n.rader@nasa.gov @NASA_NTL

ChallengesFood Supplies

Clean Water

Rising Sea Levels Global WarmingAsteroids

3

Nanotechnology

Remote Hyper Spectral

SensingGenomics

Technology Explosion Moore’s Law

4

How do we take advantage of this explosion of technology

advancement from around the world to solve our Problems?

How do we stay current on everything that is going on in technology that could help us

our problem?

5

We are used to the model of “the expert”, that keeps up with all of the tech advancements that we need for a given field. In the past, our biggest struggle has been to find and keep these experts!But now, the expert you need today is different than the expert you need tomorrow….

The pace of technology development is just too much.

Individuals cannot

possibly keep up with the technology

advancements

Even if you could find the right expert for your problem, could you afford them?

Would you be able to work through the red tape to get them to work with

you?

What would you do with him after you have your answer and you need to move to the next expert area?What about all of the technology advancement

that is going on out there that doesn’t

NEED an expert… it just needs you to find it!

It is all like looking for a

needle in a

haystack!

7

But we now have an advantage!

A global population thatis becoming increasingly CONNECTED by the INTERNET.

8

Internet User Growth from 2000-2012 and Potential

Asia1.1B

28%

Europe519M

63%

Africa

167M

16%

North Americ

a274M

79%

Latin Americ

a255M

43%

Middle

East90M

40% Oceana/Austrialia

24M68%

The World2.4B34%

Users in 2000

Additional users by 2012

Population Not on the

Internet YET

Current # of Users

% of Population

that is Online

566% Growth from 2000-2012

LegendBased on table from: http://www.internetworldstats.com/stats.htm

9

With more and more of the world’s population on the Internet, the ability to access people with unique skills, expertise, and experience is rapidly increasing.

Academic students researchers

faculty

Retired scientists, engineers, & programmers

Current “under-employed”

scientists, engineers, & programmers

The Tip of the Iceberg! Not only are the Billions of people around the world rapidly getting online. Right now, the 2.8B people that ARE online are just barely aware of these platforms and what they mean to them…

10

11

Cognitive Surplus

This “interest” in using their skills to make a difference or win a prize outside of normal employment (at night or on the weekends) can be described as….

People seem genuinely interested in engaging in problem solving and being “the one” to make a difference!

12

If two heads are

better than

one….

Now consider how this cognitive surplus begins to become a powerful

force as more of the current 2.8 billion connected individuals join in the

efforts and the remaining 4.2 billion become connected on the Internet.

The possibilities to harness the creativity of the human mind are enormous!

Image: Illumination Entertainment, Universal Studios

But just because 2.8 Billion people are connected does NOT mean that they are organized ….

AT ALL…

In fact it can be chaos.

14

In fact… it’s fascinating what global connectivity has meant to many.

15

Networks & Communities

Curated Communities

InventorsSoftware Coders

Film-Makers

Photographers

Graphic Artists

Engineers & Designers

800,000*

84,000*1,081,000

745,000

50,000

1,860,000*estimate

Well formulated crowd-based platforms actively work tobuild a community of users that are passionate enthusiasts.

16

Curated CommunitiesResources and

Tools for Members

Incentives for Members to Participate

User Agreements for

Privacy and Payment

Mechanism for Handling IP

Licensing and/or Transfer

Community Building &

Communication

Curated communities are built around enabling people to pursue their passion and create a win-win for the company and its community members.

They provide structure and incentives.

Communities do NOT like to be exploited!

17

Gold G

uts Good G

loryWhy Does The Crowd Contribute?

Earn Money (real or virtual)Have Fun (or pass the time)

Socialize with OthersObtain Recognition or

Prestige (leaderboards, badges)

Do Good (altruism)Learn Something New

Obtain Something ElseCreate Self-Serving Resource

Crowdsourcing & Human Computation Labeling Data & Building Hybrid Systemsby Matthew Lease, Assistant Professor at University of Texas at Austin on May 03, 2013http://www.slideshare.net/mattlease/crowdsourcing-human-computation-labeling-data-building-hybrid-systems

Multiple Incentives can often operate in parallel

Effectively Using Communities

Create an Innovative New

Solution

Apply an Existing

Technology (in an innovative

way)

Access Best Possible

Product or Service

(competition winner)

Find an Existing Solution

(you didn’t know existed)

Access Very Specific

Expertise(found through

competition)

18

Solve a Problem

Develop a Product

Provide a Service

Innovation from Diversity found via Challenges (Experience, Context/Perspective, Expertise)

Expert or Domain Focused MembershipDiverse Membership

High Quality Products/Services(via Competition to get Best in Domain)

19

The Crowd Just Gave Me 3500

Solutions…

Now What Do I Do?

20

If you aren’t careful, you end up sorting through a heap of potential solutions … …most of which are

NOT what you need!

21

Crowdsourcing innovative solutions is really a lot more like mining…. You need special processes and tools to extract the gold from the ore.

22Executing a Crowd-Sourced Challenge

Formulate the Problem Statement

Design the Challenge

Execute the Challenge

Pick the Winner(s)Judging

Get Your Solution IP licensing and/or transfer

Solution Filtering (optional)

Knowing how to do all of these steps really helps tomitigate the issues associated with this “too many solutions” problem.

ALL of these steps can help to minimize the number of solutions you end up needing to judge.

A well formulated problem statement (with good success criteria)

A well designed challenge (including setting the right prize

amount)

Solution filtering mechanisms are

offered by some platforms

We Can Use the Crowd as a Force

Multiplier…

23

…To Do MORE with LESS

www.infographically.com

This is an established and GROWING INDUSTRY!

25

Crowdsourcing is Mainstream

What is NASA Doing with Crowdsourcing?

NASA Innovation Pavilion

28

“One Stop Shop” Website which Lists ALL NASA Challenges

http://www.nasa.gov/solve

Centennial Challenges

NTL Curated Community Challenges

Citizen Science Challenges

Space Apps ChallengeSoftware

Apps

Scientific Observations/C

ontributions

Tech Dev & DemoYears

Months

Months

Days/Weeks

Duration Prizes Product Participation

$M+

$1K-$100Ks

Recognition

Recognition

Ideas, Designs, Software

29

30

First Things First!

NASA has some of the most amazing employees on the

PLANET!

Tap into that crowd FIRST!

31

Over 15,000 Registered Members(25% of NASA’s 60,000 CS & Contractor Workforce)

15-20 Challenges per Year

Growth of the NASA@work Community since October 2012

2-4 Active challenges posted at any one time

New challenge posts every ~2-3 weeks

People that work at NASA want to

make a difference!

Innovation & Problem Solving

Using Challenges with Diverse Communities to develop unique and innovative approaches to unsolved problems

NASA Innovation Pavilion on

33

Global Communityof 350,000+ Solvers

Diverse Member Base

High Success Rate in Solving Hard

Problems with Innovative Solutions Map showing participation of 2900 solvers

from 80 countries in NASA’s pilot challenges

34

“Diverse perspectives and tools enable collections of people to find more and better solutions.”

Diversity Trumps Ability Theorem: “The people you’d least expect to solve

a problem were exactly the ones most likely to crack it.”

Scott PageAuthor of Difference:

How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies

35

Diversity is the Key to Innovation

One MIT study into InnoCentive revealed that solvers were more successful when they had less

experience in the relevant discipline.

Some data suggests that as much as 70% of successful InnoCentive challenge solutions are solved by

individuals outside of the challenge’s specific technical

domain.

36

“In 60 days, Roche was able to solve a problem that it and its partner have been tinkering with and optimizing for the last 15 years. The solutions provided actually mirrored the entire history of Roche’s R&D programme. All of the solutions Roche had tried came in. “

Swiss company with 80,000 employees, Roche operates in 150 countries and has R&D operations in Europe, North America and Asia-PacificRoche is a world leader in in vitro diagnostics.

Julian Birkinshaw, MLabnotes, University of London Business School

Roche ran an InnoCentive challenge

37

$20,000 in prizesOver 2800 registrants219 Entries

Winning Submission: Barium tracers for atmospheric analysis

Winner was Ted Ground from Rising Star,

Texas (population 799)

38

Algorithm CompetitionsLeverage Competition to Optimize Complex

Algorithmic Problems

Case Study

CODERS SUBMITTED SOLUTIONS DIFFERENT APPROACHES TOSOLVE PROBLEM IDENTIFIED

WINNING COUNTRIESRUSSIA, FRANCE, EGYPT, BELGIUM & US

122 654 89 5

Improve on NIH MegaBlast algorithmfor nucleotide sequence alignment

Winning solution performs 120x faster

ANTIBODY SEQUENCE ANNOTATION

Source:

47 min. 16 sec.4.3 hours

$2M+ Multi-yearDevelopment

15% Improvement!Over current method of identifying asteroids in the main belt of Asteroids that orbit between Mars & Jupiter

1241 Registrants625 Solutions Submitted$74,124 in Prizes AwardedAlgorithm AND App

42

Using Competitions for Software Development

Conceptualization

Specification

Wireframes

Storyboards

Architecture

Assembly

Component Dev

Bug Hunt

Bug Race

Idea Generation

Image Credit: Wikipedia, Systems development life-cycle, http://en.wikipedia.org/wiki/Systems_development_life-cycle (as of Mar. 27, 2013, 05:48 GMT).

Source:

44

Leverage Competition to Optimize Engineering Designs

Design Challenges

47

GrabCAD 30 Day challenge for $3000: 492 CAD Designs Submitted

Using New Methods to Search for New and Emerging Technology to Meet NASA’s Needs

Searching for Technologies

52

Nanotechnology

Remote Hyper Spectral

SensingGenomics

Technology Explosion Moore’s Law

Yet2.com

Provides a “matching” service that finds technologies and solutions from industry, academia, and/or individuals for a given need/challenge.

Includes a 130,000 member community and links to over 16,000 commercial entities.

Very effective (and cost effective) in searching for existing products or development efforts.

Yet2.com ChallengesBone Density Measurement

Monitoring of Water and Biocides

Non-Invasive Intra-Cranial Pressure Measurement“Much more than we expected! Very pleasantly surprised that this process exposed so many potential solutions with such wide breadth and depth”“Learned that we should have revisited technologies that we rejected earlier 81 Leads Identified

63 Rejected3 High Interest Solutions5 Other Interesting Solutions6 Potential Complementary Technologies

2 Potential Solutions

Non-

129 Challenges

4 Tech Surveys

2 Videos

1 Design

14 Algorithms

15 Software

2 Ideation

2 Graphics

76 Challenges

NASA Innovation Pavilion

9 Theoretical

2 Ideation

2 Reduction to Practice

NASA Challenge Experience with Curated

Communities

Includes NASA and other U.S. Gov’t Agency Challenges executed under NASA’s CoECI

Crowd-Based Challenges are valuable tools that should be considered for future engineering

projects

Effective Method to Find Truly Innovative Solutions

Key Method to Find the Right Existing Technologies (out of the sea of R&D work across a variety of domains)

Efficient Method to Augment Project Teams with Specialized Skills as Needed

Using Crowds as Engineering Tools

Steve Radersteven.n.rader@nasa.gov@steverader

AcknowledgementsReferences:

Crowdsourcing Landscape: www.crowdsourcingresults.com, www.infographically.comGlobal Internet Use: Based on table from: http://www.internetworldstats.com/stats.htmHowe, Jeff (2008-08-18). Crowdsourcing: Why the Power of the Crowd Is Driving the

Future of Business, Doubleday Religious Publishing Group.

“Diverse perspectives and tools enable collections of people to find more and better solutions.”

Scott Page. Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies.

Karim R. Lakhani et al., “The Value of Openness in Scientific Problem Solving.”

Graphics: experts - http://directoryofexcelexperts.com/needle in haystack - http://siliconbeachclearly.com/global network - http://www.emirates.com/iceberg - http://www.weirdoptics.com/iceberg-illusion/minions - Illumination Entertainment, Universal Studioslolcats – http://background-pictures.picphotos.net/farmville – http://farmville.startpagina.nltinder – http://apptiinder.comthumbsdown – http://commons.wikimedia.orgcommunity – http://www.blogworld.comminivan – http://vzyalslidingdoor.blogspot.commilking machine - greenfusestock.photoshelter.comrocket scientists – LIFE magazineinnovation – http://abovethecrowd.comproblem solving – http://creativerealities.comequations – http://gallaryhip.com, http://imaginenvision.comdesigns – http://www.grabcad.com