computer science department detection, alignment and recognition of real world faces erik...
TRANSCRIPT
![Page 1: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/1.jpg)
Computer Science Department
Detection, Alignment and Recognitionof Real World FacesErik Learned-Miller
with Vidit Jain, Gary Huang, Andras Ferencz, et al.
Faces in the Wild
![Page 2: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/2.jpg)
2Computer Science
Is Face Recognition Solved?
![Page 3: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/3.jpg)
3Computer Science
Is Face Recognition Solved?
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
“100% Accuracy in Automatic Face Recognition” [!!!]
Science 25 January 2008
![Page 4: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/4.jpg)
4Computer Science
Is Face Recognition Solved?
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
“100% Accuracy in Automatic Face Recognition” [!!!]
Science 25 January 2008
A history of overstated results.
![Page 5: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/5.jpg)
5Computer Science
The Truth
Many different face recognition problems• Out of context, accuracy is meaningless!
Many problems are REALLY HARD!• For some problems
state of the art is 70% or worse!
We have a long way to go!
![Page 6: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/6.jpg)
6Computer Science
Face Recognition at UMass
Labeled Faces in the Wild The Detection-Alignment-Recognition pipeline Congealing and automatic face alignment Hyper-features for face recognition New directions in recognition
![Page 7: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/7.jpg)
7Computer Science
Labeled Faces in the Wild
http://vis-www.cs.umass.edu/lfw/
![Page 8: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/8.jpg)
8Computer Science
The Many Faces of Face Recognition
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.QuickTime™ and a
TIFF (Uncompressed) decompressorare needed to see this picture.
Labeled Faces in the Wild
![Page 9: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/9.jpg)
9Computer Science
The Many Faces of Face Recognition
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.QuickTime™ and a
TIFF (Uncompressed) decompressorare needed to see this picture.
Labeled Faces in the Wild
![Page 10: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/10.jpg)
10Computer Science
The Many Faces of Face Recognition
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.QuickTime™ and a
TIFF (Uncompressed) decompressorare needed to see this picture.
Labeled Faces in the Wild
![Page 11: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/11.jpg)
11Computer Science
The Many Faces of Face Recognition
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.QuickTime™ and a
TIFF (Uncompressed) decompressorare needed to see this picture.
Labeled Faces in the Wild
![Page 12: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/12.jpg)
12Computer Science
The Many Faces of Face Recognition
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Labeled Faces in the Wild
![Page 13: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/13.jpg)
13Computer Science
Labeled Faces in the Wild
13,233 images, with name of each person 5749 people 1680 people with 2 or more images
Designed for the “unseen pair matching problem”.• Train on matched or mismatched pairs.• Test on never-before-seen pairs.
Distinct from problems with “galleries” or training data for each target image.
Best accuracy: currently about 73%!
![Page 14: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/14.jpg)
14Computer Science
Detection-Alignment-Recognition Pipeline
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
DetectionRecognitionAlignment
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
“Same”
![Page 15: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/15.jpg)
15Computer Science
Detection-Alignment-Recognition Pipeline
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
DetectionRecognitionAlignment
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
“Same”
Parts should work together.
![Page 16: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/16.jpg)
16Computer Science
Labeled Faces in the Wild
All images are output of a standardface detector.
Also provides aligned images. Consequence: any face recognition algorithm
that works well on LFW can easily be turned into a complete system.
![Page 17: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/17.jpg)
17Computer Science
Congealing (CVPR 2000)
QuickTime™ and a decompressor
are needed to see this picture.
![Page 18: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/18.jpg)
18Computer Science
Criterion of Joint Alignment
Minimize sum of pixel stack entropies by transforming each image.
A pixel stack
![Page 19: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/19.jpg)
19Computer Science
Congealing Complex Images
Window around pixel SIFT vector and clusters
SIFT clusters
vector representingprobability of each cluster,or “mixture” of clusters
![Page 20: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/20.jpg)
QuickTime™ and aMicrosoft Video 1 decompressorare needed to see this picture.
![Page 21: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/21.jpg)
21Computer Science
Crash Course on Martian Identification
?
Test: Find Bob after one meeting
Martian training set
=
=
=
Bob
![Page 22: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/22.jpg)
22Computer Science
Training Data
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
“same”
“different”
![Page 23: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/23.jpg)
23Computer Science
General Approach to Hyper-feature method
Carefully align objects Develop a patch-based model of
image differences. Score match/mismatch based on patch
differences.
![Page 24: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/24.jpg)
24Computer Science
Three Models
1. Universal patch model:P(patchDistance|same)P(patchDistance|different)
2. Spatially dependent patch model:P(patchDistance |same,x,y)P(patchDistance |different,x,y)
3. Hyper-feature dependent model:1. P(patchDistance |same,x,y,appearance)2. P(patchDistance |different,x,y,appearance)
![Page 25: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/25.jpg)
25Computer Science
Universal Patch Model
A single P(dist | same) for all patches
Different blue patches are evidence against a match!
![Page 26: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/26.jpg)
26Computer Science
Spatial Patch Model
P(dist|same,x1,y1) estimated separately from P(dist|same,x2,y2)
Greatly increases discriminativeness of model.
![Page 27: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/27.jpg)
27Computer Science
Hyper-Feature Patch Model
Is the patch from a matching face going tomatch this patch?
![Page 28: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/28.jpg)
28Computer Science
Hyper-Feature Patch Model
Is the patch from a matching face going tomatch this patch? Probably yes
![Page 29: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/29.jpg)
29Computer Science
Hyper-Feature Patch Model
What about this patch?
![Page 30: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/30.jpg)
30Computer Science
Hyper-Feature Patch Model
What about this patch?Probably not.
![Page 31: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/31.jpg)
31Computer Science
Ridiculous Errors from the World’s Best Unconstrained Face Recognition System
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.QuickTime™ and a
TIFF (Uncompressed) decompressorare needed to see this picture.
![Page 32: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/32.jpg)
32Computer Science
Ridiculous Errors from the World’s Best Unconstrained Face Recognition System
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
![Page 33: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/33.jpg)
33Computer Science
The New Mission: Estimate Higher Level Features
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
![Page 34: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/34.jpg)
34Computer Science
The New Mission: Estimate Higher Level Features
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Can we guesspose?
![Page 35: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/35.jpg)
35Computer Science
The New Mission: Estimate Higher Level Features
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Can we guessgender?
![Page 36: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/36.jpg)
36Computer Science
The New Mission: Estimate Higher Level Features
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Can we guessdegree of balding,
beardedness,moustache?
![Page 37: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/37.jpg)
37Computer Science
The New Mission: Estimate Higher Level Features
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Can we say thatnone of these individuals are
the same person?
![Page 38: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/38.jpg)
38Computer Science
What can we do with a good segmentation?
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
![Page 39: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/39.jpg)
39Computer Science
CRF Segmentations
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
![Page 40: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/40.jpg)
40Computer Science
CRF Segmentations
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
![Page 41: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/41.jpg)
41Computer Science
Who’s This?
![Page 42: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/42.jpg)
42Computer Science
Who’s This?
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
![Page 43: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/43.jpg)
43Computer Science
Who’s This?
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
from www.coolopticalillusions.com
![Page 44: Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et](https://reader036.vdocuments.site/reader036/viewer/2022062800/56649de55503460f94add0ac/html5/thumbnails/44.jpg)
Computer Science Department
Thanks