face detection ying wu [email protected] electrical and computer engineering northwestern...
TRANSCRIPT
![Page 1: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/1.jpg)
Face Detection
Ying [email protected]
Electrical and Computer EngineeringNorthwestern University, Evanston, IL
http://www.ece.northwestern.edu/~yingwu
![Page 2: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/2.jpg)
Face Detection: The Problem
The Goal:
Identify and locate faces in an image
The Challenges:
Position
Scale
Orientation
Illumination
Facial expression
Partial occlusion
![Page 3: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/3.jpg)
Outline The Basics Visual Detection
– A framework– Pattern classification– Handling scales
Viola & Jones’ method– Feature: Integral image– Classifier: AdaBoosting– Speedup: Cascading classifiers– Putting things together
Other methods Open Issues
![Page 4: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/4.jpg)
The Basics: Detection Theory
Bayesian decision Likelihood ratio detection
![Page 5: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/5.jpg)
Bayesian Rule
iii
iiiii pxp
pxp
xp
pxpxp
)()|(
)()|(
)(
)()|()|(
posterior likelihoodprior
![Page 6: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/6.jpg)
Bayesian Decision
Classes {1, 2,…, c}
Actions {1, 2,…, a}
Loss: (k| i)
Risk: Overall risk:
Bayesian decision
c
kkkii xpxR
1
)|()|()|(
x
dxxpxxRR )()|)((
)|(minarg* xR kk
![Page 7: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/7.jpg)
Minimum-Error-Rate Decision
ki
kiki 1
0)|(
c
k ikikkkii xpxppxR
1
)|(1)|()()|()|(
ikxpxp kii )|()|( if decide
![Page 8: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/8.jpg)
Likelihood Ratio Detection
x – the data H – hypothesis
– H0: the data does not contain the target– H1: the data contains the target
Detection: p(x|H1) > p(x|H0) Likelihood ratio
tHxp
Hxp
)0|(
)1|(
![Page 9: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/9.jpg)
Detection vs. False Positive
“+” “-”
false positive miss detection
threshold
“+” “-”
false positive miss detection
threshold
![Page 10: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/10.jpg)
Visual Detection
A Framework Three key issues
– target representation– pattern classification– effective search
![Page 11: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/11.jpg)
Visual Detection
Detecting an “object” in an image– output: location and size
Challenges– how to describe the “object”?– how likely is an image patch the image of the
target?– how to handle rotation?– how to handle the scale?– how to handle illumination?
![Page 12: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/12.jpg)
A Framework
Detection window Scan all locations and scales
![Page 13: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/13.jpg)
Three Key Issues
Target Representation Pattern Classification
– classifier– training
Effective Search
![Page 14: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/14.jpg)
Target Representation
Rule-based – e.g. “the nose is underneath two eyes”, etc.
Shape Template-based– deformable shape
Image Appearance-based– vectorize the pixels of an image patch
Visual Feature-based– descriptive features
![Page 15: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/15.jpg)
Pattern Classification
Linear separable
Linear non-separable
![Page 16: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/16.jpg)
Effective Search
Location– scan pixel by pixel
Scale– solution I
keep the size of detection window the same use multiple resolution images
– solution II: change the size of detection window
Efficiency???
![Page 17: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/17.jpg)
Viola & Jones’ detector
Feature integral image Classifier AdaBoosting Speedup Cascading classifiers Putting things together
![Page 18: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/18.jpg)
An Overview
Feature-based face representation AdaBoosting as the classifier Cascading classifier to speedup
![Page 19: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/19.jpg)
Harr-like features
Q1: how many features can be calculated within a detection window?Q2: how to calculate these features rapidly?
![Page 20: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/20.jpg)
Integral Image
![Page 21: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/21.jpg)
The Smartness
![Page 22: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/22.jpg)
Training and Classification
Training– why?– An optimization problem– The most difficult part
Classification– basic: two-class (0/1) classification– classifier– online computation
![Page 23: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/23.jpg)
Weak Classifier
Weak?– using only one feature for classification– classifier: thresholding
– a weak classifier: (fj, j,pj)
Why not combining multiple weak classifiers? How???
![Page 24: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/24.jpg)
Training: AdaBoosting
Idea 1: combining weak classifiers
Idea 2: feature selection
![Page 25: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/25.jpg)
Feature Selection
How many features do we have? What is the best strategy?
![Page 26: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/26.jpg)
Training Algorithm
![Page 27: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/27.jpg)
The Final Classifier
This is a linear combination of a selected set of weak classifiers
![Page 28: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/28.jpg)
Learning Results
![Page 29: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/29.jpg)
Attentional Cascade
Motivation– most detection windows contain non-faces– thus, most computation is wasted
Idea?– can we save some computation on non-faces?– can we reject the majority of the non-faces
very quickly?– using simple classifiers for screening!
![Page 30: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/30.jpg)
Cascading classifiers
![Page 31: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/31.jpg)
Designing Cascade Design parameters
– # of cascade stages– # of features for each stage– parameters of each stage
Example: a 32-stage classifier– S1: 2-feature, detect 100% faces and reject 60% non-faces– S2: 5-feature, detect 100% faces and reject 80% non-faces– S3-5: 20-feature– S6-7: 50-feature– S8-12: 100-feature– S13-32: 200-feature
![Page 32: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/32.jpg)
Comparison
![Page 33: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/33.jpg)
Comments
It is quite difficult to train the cascading classifiers
![Page 34: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/34.jpg)
Handling scales
Scaling the detector itself, rather than using multiple resolution images
Why?– const computation
Practice– Use a set of scales a factor of 1.25 apart
![Page 35: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/35.jpg)
Integrating multiple detection
Why multiple detection?– detector is insensitive to small changes in
translation and scale Post-processing
– connect component labeling– the center of the component
![Page 36: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/36.jpg)
Putting things together
Training: off-line– Data collection
positive data negative data
– Validation set– Cascade AdaBoosting
Detection: on-line– Scanning the image
![Page 37: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/37.jpg)
Training Data
![Page 38: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/38.jpg)
Results
![Page 39: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/39.jpg)
ROC
![Page 40: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/40.jpg)
Summary
Advantages– Simple easy to implement– Rapid real-time system
Disadvantages– Training is quite time-consuming (may take
days)– May need enormous engineering efforts for
fine tuning
![Page 41: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/41.jpg)
Other Methods
Rowley-Baluja-Kanade
![Page 42: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/42.jpg)
Rowley-Baluja-Kanade
Train a set of multilayer perceptrons and arbitrate a decision among all the inputs, and search among different scales,
[Rowley, Baluja and Kanade, 1998]
![Page 43: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/43.jpg)
RBK: Some Results
Courtesy of Rowley et al., 1998
![Page 44: Face Detection Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL yingwu](https://reader036.vdocuments.site/reader036/viewer/2022081513/56649f335503460f94c4fb66/html5/thumbnails/44.jpg)
Open Issues
Out-of-plane rotation Occlusion Illumination