Robust Real-Time Object DetectionAuthor: Paul Viola, Michael J. JonesInternational Journal of Computer Vision 57(2), 137–154, 2004

Presented by ERLI – M10115801

Background & Motivation• Detecting object on real time

• Detecting face

Outline

IntroductionImage Representation – Integral Image

Learning Algorithm – AdaBoost

Cascade ClassifierExperiment and Result

IntroductionObject DetectionDetection System

Object Detection>> Face Detection

Paul Viola Michael J. Jones

Introduction

Detection System

24x24

Detection Window

The size of detection window

will be enlarged with a certain scale

Introduction

Image Representation

Integral ImageIntegral Image Feature

Integral Image• Image representation

• Feature

• The integral image at location x,y contains the sum of the pixels above and to the left of x,y

Integral Image | AdaBoost | Cascade

Sum of pixel value in this area

(x,y)

Integral Image• Image representation

• Feature

• The integral image at location x,y contains the sum of the pixels above and to the left of x,y

Integral Image | AdaBoost | Cascade

Sum of pixel value in this area

(x,y)

How to calculate D ?

A B

C D

1 2

3 4

Integral Image• Image representation

• Feature

Integral Image | AdaBoost | Cascade

How to calculate D ?D = (1+4) – (2+3)

A B

C D

1 2

3 4

1234

Integral Image Feature

Integral Image | AdaBoost | Cascade

Detection window

A

B

C

D

Detection window

Detection window Detection window

The value of integral image feature is the difference between the sum of the pixels within the two rectangular regions.

S1 S2 |S1-S2|

Integral Image Feature

Integral Image | AdaBoost | Cascade

The size and position of feature can be different.

• The eye region is darker than the upper-cheeks.

• The nose bridge region is brighter than the eyes.

Integral Image• Different size and position of integral image give so many possible

features

• How do we obtain the best representing features possible?

• How can we refrain from wasting time on image background? (i.e. non-object)

Integral Image | AdaBoost | Cascade

Good Feature Not Good Feature

AdaBoost…

AdaBoostWeak ClassifierStrong ClassifierAdaBoost

Weak Classifier • Consist of just 1 feature

• 1 feature that can separate the image data with error rate less than 0.5

Feature 1

Feature 2

Feature 3

θ 1

θ 2

θ 3

Weak Classifier 1

Weak Classifier 2

Weak Classifier 3

1/10 = 0.1

6/10 = 0.6

3/10 = 0.3

Integral Image | AdaBoost | Cascade

Strong Classifier• Set of features/weak classifier

Integral Image | AdaBoost | Cascade

1 1 1strong

11 ( ) ( )

( ) 20 otherwise

n n nh hh

x xx

Weak Classifier

How to build the strong classifier?

AdaBoost

AdaBoost Pseudo Code

• T = number of feature we choose

• h(x, f,p,θ)) = A weak classifier

• f = feature

• θ = threshold

• p = polarity indicating the direction of the inequality (less than or greater than θ)

Integral Image | AdaBoost | Cascade

AdaBoost starts with a uniform distribution of “weights” over training examples.

Select the classifier with the lowest weighted error (i.e. a “weak” classifier)

Increase the weights on the training examples that were misclassified.

(Repeat)

At the end, carefully make a linear combination of the weak classifiers obtained at all iterations.

AdaBoost Ilustration

1 1 1strong

11 ( ) ( )

( ) 20 otherwise

n n nh hh

x xx

Slide taken from a presentation by Qing Chen, Discover Lab, University of Ottawa

Integral Image | AdaBoost | Cascade

Classifier• By increasing the number of features per classifier, we:

◦ Increase detection accuracy.◦ Decrease detection speed.

• Experiments showed that a 200 feature classifier makes a good face detector:◦ Takes 0.7 seconds to scan an 384 by 288 pixel image.

• Problem: Not real time! (At most 0.067 seconds needed).

Integral Image | AdaBoost | Cascade

Cascade of Classifier…

Cascade Classfifier

Cascade

Cascade of Classifier• Classifier is structured as several layer/hierarchy

• Each layer consist of 1 strong classifier (set of weak classifiers/features)

• Evaluating all input window

• Ignore non-face window

• Continue for processing “suspected” face window

Integral Image | AdaBoost | Cascade

Cascade of Classifier• The number of feature/weak classifier in every layer is based on the

value of false positive rate and detection rate determined by user

• Previous layers have simpler classifier than next layer

Integral Image | AdaBoost | Cascade

Stage 1

Stage 2

Stage 3

…Furth

er Processing

Input Image Window

Ignore/Reject Input

Ignore/Reject Input

Ignore/Reject Input

Ignore/Reject Input

Y

N

Y

N

Y

N

Y

N

Cascade of Classifier

Integral Image | AdaBoost | Cascade

Stage 1(2 feature)

N

Y Stage 2(2++ feature)

Cascade of Classifier Pseudo Code

Integral Image | AdaBoost | Cascade

Experiment• Training set• Face: 4916 labeled frontal faces

•Testing / Real-time test• Face: MIT + CMU frontal face test set• Contain: 130 images with 507 labeled frontal faces

Result

Cascade Classifierv.s

Non Cascade Classifier

Cascade classifier nearly 10x faster.

Result

They use dataset MIT+CMU with 5 images removed

Voting (Using 3 Viola-Jones Classifier with different parameter)++- : Face--+ : Non Face

Failure Mode• Informal observation suggests that the face detector can detect

faces that are tilted up to about ±15 degrees in plane and about ± 45 degrees out of plane

• Harsh backlighting in which the faces are very dark while the background is relatively light sometimes causes failures

• Proposed algorithm will also fail on significantly occluded faces