face detection slides adapted grauman & liebe’s tutorial bleibe/teaching/tutorial-aaai08/ also...
TRANSCRIPT
![Page 1: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/1.jpg)
Face detection
Slides adapted Grauman & Liebe’s tutorial • http://www.vision.ee.ethz.ch/~bleibe/teaching/tutorial-aaai08/
Also see Paul Viola’s talk (video)• http://www.cs.washington.edu/education/courses/577/04sp/contents.html#DM
![Page 2: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/2.jpg)
2
Limitations of Eigenfaces
Eigenfaces are cool.
But they’re not great for face detection.
Chief Limitations• not very accurate• not very fast
To make it work on the camera, we need ~30fps, and near-perfect accuracy.
2
![Page 3: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/3.jpg)
3
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Rectangle filters
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1
P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. CVPR 2001.
![Page 4: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/4.jpg)
4
Why rectangles?
Answer: very very fast to compute• Trick: integral images (aka summed-area-tables)
(x,y)
Value at (x,y) is sum of pixels above and to the left of (x,y)
integral image
![Page 5: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/5.jpg)
5
Integral images
What’s the sum of pixels in the blue rectangle?
input image integral image
A B
C D
![Page 6: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/6.jpg)
6
Integral images
integral image
A B
C D
![Page 7: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/7.jpg)
7
Integral images
integral image
A
![Page 8: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/8.jpg)
8
Integral images
integral image
B
![Page 9: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/9.jpg)
9
Integral images
integral image
C
![Page 10: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/10.jpg)
10
Integral images
integral image
D
![Page 11: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/11.jpg)
Integral imagesWhat’s the sum of pixels in the rectangle?
DA CB
![Page 12: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/12.jpg)
Computing sum within a rectangle
• Let A,B,C,D be the values of the integral image at the corners of a rectangle
• Then the sum of original image values within the rectangle can be computed as: sum = A – B – C + D
• Only 3 additions are required for any size of rectangle!
D B
C A
Lana Lazebnik
![Page 13: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/13.jpg)
Filter as a classifierHow to convert the filter into a classifier?
Outputs of a rectangle feature on faces and non-
faces.
Resulting weak classifier:
![Page 14: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/14.jpg)
14
Finding the best filters...
Considering all possible filter parameters: position, scale, and type:
180,000+ possible filters associated with each 24 x 24 window
Which of these filters(s) should we use to determine if a window has a face?
![Page 15: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/15.jpg)
Boosting
Weak Classifier 1
Slide credit: Paul Viola
![Page 16: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/16.jpg)
Boosting
WeightsIncreased
![Page 17: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/17.jpg)
Boosting
Weak Classifier 2
![Page 18: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/18.jpg)
Boosting
WeightsIncreased
![Page 19: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/19.jpg)
Boosting
Weak Classifier 3
![Page 20: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/20.jpg)
Boosting
Final classifier is a combination of weak classifiers
![Page 21: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/21.jpg)
Boosting: training
• Initially, weight each training example equally
• In each boosting round:– find the weak classifier with lowest weighted training error
– raise weights of training examples misclassified by current weak classifier
• Final classifier is linear combination of all weak classifiers
– weight of each learner is directly proportional to its accuracy)
• Exact formulas for re-weighting and combining weak classifiers depend on the particular boosting schemeSlide credit: Lana Lazebnik
![Page 22: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/22.jpg)
Perc
eptu
al and S
enso
ry A
ugm
ente
d C
om
puti
ng
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
AdaBoost AlgorithmStart with uniform weights on training examples
Evaluate weighted error for each feature, pick best.Re-weight the examples:Incorrectly classified -> more weightCorrectly classified -> less weight
Final classifier is combination of the weak ones, weighted according to error they had.
Freund & Schapire 1995
{x1,…xn}For T rounds
![Page 23: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/23.jpg)
Perc
eptu
al and S
enso
ry A
ugm
ente
d C
om
puti
ng
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
First two features selected
Viola-Jones Face Detector: Results
![Page 24: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/24.jpg)
• Even if the filters are fast to compute, each new image has a lot of possible windows to search.
• How to make the detection more efficient?
![Page 25: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/25.jpg)
Cascading classifiers for detection
• Form a cascade with low false negative rates early on
• Apply less accurate but faster classifiers first to immediately discard windows that clearly appear to be negative Kristen Grauman
![Page 26: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/26.jpg)
Viola-Jones detector: summary
•Train with 5K positives, 350M negatives•Real-time detector using 38 layer cascade•6061 features in all layers•[Implementation available in OpenCV: http://www.intel.com/technology/computing/opencv/]
Faces
Non-faces
Train cascade of classifiers with
AdaBoost
Selected features, thresholds, and weights
New image
App
ly to
each
subw
indo
w
Kristen Grauman
![Page 27: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/27.jpg)
Perc
eptu
al and S
enso
ry A
ugm
ente
d C
om
puti
ng
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Viola-Jones Face Detector: Results
![Page 28: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/28.jpg)
Perc
eptu
al and S
enso
ry A
ugm
ente
d C
om
puti
ng
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Viola-Jones Face Detector: Results
![Page 29: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/29.jpg)
Perc
eptu
al and S
enso
ry A
ugm
ente
d C
om
puti
ng
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Viola-Jones Face Detector: Results
![Page 30: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/30.jpg)
Perc
eptu
al and S
enso
ry A
ugm
ente
d C
om
puti
ng
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Detecting profile faces?
Can we use the same detector?
![Page 31: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/31.jpg)
Perc
eptu
al and S
enso
ry A
ugm
ente
d C
om
puti
ng
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Paul Viola, ICCV tutorial
Viola-Jones Face Detector: Results
![Page 32: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/32.jpg)
Everingham, M., Sivic, J. and Zisserman, A."Hello! My name is... Buffy" - Automatic naming of characters in TV video,BMVC 2006. http://www.robots.ox.ac.uk/~vgg/research/nface/index.html
Example using Viola-Jones detector
Frontal faces detected and then tracked, character names inferred with alignment of script and subtitles.
![Page 33: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/33.jpg)
Application: streetview
![Page 34: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/34.jpg)
3434
Application: streetview
![Page 35: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/35.jpg)
3535
Application: streetview
![Page 36: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/36.jpg)
Consumer application: iPhoto 2009
http://www.apple.com/ilife/iphoto/Slide credit: Lana Lazebnik
![Page 37: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/37.jpg)
Consumer application: iPhoto 2009
Things iPhoto thinks are faces
Slide credit: Lana Lazebnik
![Page 38: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/38.jpg)
What other categories are amenable to window-based representation?
![Page 39: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/39.jpg)
Perc
eptu
al and S
enso
ry A
ugm
ente
d C
om
puti
ng
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Pedestrian detection• Detecting upright, walking humans also possible using
sliding window’s appearance/texture; e.g.,
SVM with HoG [Dalal & Triggs, CVPR 2005]
Kristen Grauman
![Page 40: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/40.jpg)
Perc
eptu
al and S
enso
ry A
ugm
ente
d C
om
puti
ng
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Window-based detection: strengths
• Sliding window detection and global appearance descriptors:
Simple detection protocol to implement Good feature choices critical Past successes for certain classes
Kristen Grauman
![Page 41: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/41.jpg)
Perc
eptu
al and S
enso
ry A
ugm
ente
d C
om
puti
ng
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Window-based detection: Limitations
• High computational complexity For example: 250,000 locations x 30 orientations x 4 scales =
30,000,000 evaluations! If training binary detectors independently, means cost
increases linearly with number of classes
• With so many windows, false positive rate better be low
Kristen Grauman
![Page 42: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/42.jpg)
Perc
eptu
al and S
enso
ry A
ugm
ente
d C
om
puti
ng
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Limitations (continued)
• Not all objects are “box” shaped
Kristen Grauman
![Page 43: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/43.jpg)
Perc
eptu
al and S
enso
ry A
ugm
ente
d C
om
puti
ng
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Limitations (continued)
• Non-rigid, deformable objects not captured well with representations assuming a fixed 2d structure; or must assume fixed viewpoint
• Objects with less-regular textures not captured well with holistic appearance-based descriptions
Kristen Grauman
![Page 44: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/44.jpg)
Perc
eptu
al and S
enso
ry A
ugm
ente
d C
om
puti
ng
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Limitations (continued)
• If considering windows in isolation, context is lost
Figure credit: Derek Hoiem
Sliding window Detector’s view
Kristen Grauman
![Page 45: Face detection Slides adapted Grauman & Liebe’s tutorial bleibe/teaching/tutorial-aaai08/ Also see Paul Viola’s talk (video)](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ed45503460f94be511f/html5/thumbnails/45.jpg)
Perc
eptu
al and S
enso
ry A
ugm
ente
d C
om
puti
ng
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Vis
ual O
bje
ct
Recog
nit
ion
Tu
tori
al
Limitations (continued)
• In practice, often entails large, cropped training set (expensive)
• Requiring good match to a global appearance description can lead to sensitivity to partial occlusions
Image credit: Adam, Rivlin, & Shimshoni Kristen Grauman