Human Computation

Steven EmoryCS 575

Overview

What is Human Computation?

History of Human Computation

Examples of Human Computation

Bad Example

Good Example

Challenges in Human Computation

Definition of Computation

Normally we rely on computers to do all the work

On input:

Step #1

Step #2

...

Step #n

Ouput

Definition of Human Computation

Some or all steps are solved by human(s)

On input:

Step #1

Step #2 (ask a human)

Step #3 (ask a human)

...

Step #n

Ouput

History of Human Computation

1980's: Interactive Genetic Algorithms

2000's: Human-Based Genetic Algorithms

2000's: Outsourced Human Spam

2000's: Interactive Guessing Games

Quicksort Example

Bad Example: Quicksort

Input: Unsorted array

Select a pivot (human selection)

Swap last element with pivot element

Partition array using pivot element

Insert pivot element into correct position

Repeat above steps for left/right side partitions

Output: Sorted array

Quicksort Problems

Solved faster by computer alone

Not rewarding

Boring

Painful

Photomosaic Example

Photomosaic Algorithm

Can be solved by computer alone.

On input “image gallery,” “source image”1.) Tile source image.2.) From left-to-right, top-to-bottom, compare each image in the image gallery to each tile in the source image, inserting the gallery image with the lowest error.3.) Output photomosaic.

Photomosaic Problems

Noticable visual artifacts

Could use a randomized algorithm instead

Can we do better using Human Computation?

Of course!

Photomosaic Solutions

Optimize important features first

Objects (eyes, nose, mouth)

Edges (chin, facial edges, hair, body)

Revised algorithm:

On input “image gallery,” “source image”1.) Tile source image.2.) Select important features (ask human).3.) Optimize (randomly) important features.4.) Optimize (randomly) unimportant features.5.) Output photomosaic.

When to Use

When the problem:

is hard for a computer, but easy for a human

can be done better by a human

needs a human-like (artistic/creative) solution

is not boring to a human (music, art, games)

is rewarding (financially or emotionally)

is one and done

Human Computation Challenges

User interface design

Coordinating many human participants

Analagous to distributed computing

Honesty

Prolonged computation

reCAPTCHA Example

Used to digitize old books (make e-books)

OCR normally works 99% of the time

OCR accuracy drops for older books

Old paper

Old printing techniques

Solution: Ask humans to determine words OCR fails to classify

When enough humans agree, consider it solved

reCAPTCHA Example

Metadata Example

Algorithm: Assign metadata to images

Useful for content/multimedia management systems (i.e. Istockphoto)

No algorithm exists for image labeling

Luis Von Ahn's solution: The ESP Gamehttp://www.espgame.org

ESP Game Demo

Metadata Example

Problem:

Getting humans to agree correctly

Electric Sheep Example

Brief fractal explanation:

Iterative process based on chaos, dynamical systems

Newton's Method FractalSolve az3 + bz2 + cz + d = 0 for complex numbersa, b, c, d are fractal parameters

Cubic equation has 3 rootsRed = converges to root #1Green = converges to root #2Blue = converges to root #3Black = fails to converge

Electric Sheep Example

Electric Sheep Example

Interactive genetic algorithm

Humans evaulate fitness

Animated fractal parameters are mutated

Algorithm has been running for years

Implemented as a screensaver

http://www.electricsheep.org/

Electric Sheep Example

Conclusions

There many hard/impossible to solve problems

Nothing shameful about using Human Computation

Applications in art, music, computer vision, security, content management

References

Luis Von Ahn

Google Talk lecture on Human Computation

http://www.cs.cmu.edu/~biglou/

http://www.espgame.org

The Art of Artificial Evolution (2008)

http://www.springerlink.com/content/r68831/?p=d04c460aa17749eb8153fec3d0507f68&pi=9