face recognition: an introduction. face recognition face is the most common biometric used by humans...
TRANSCRIPT
Face Recognition:An Introduction
Face Recognition
• Face is the most common biometric used by humans
• Applications range from static, mug-shot verification to
a dynamic, uncontrolled face identification in a cluttered
background
• Challenges:
• automatically locate the face
• recognize the face from a general view point under
different illumination conditions, facial expressions,
and aging effects
Authentication vs Identification• Face Authentication/Verification (1:1 matching)
• Face Identification/Recognition (1:N matching)
www.viisage.com
Access Control
Applications
www.visionics.com
Video Surveillance (On-line or off-line)
Applications
Face Scan at Airports
www.facesnap.de
Face Recognition Application
Why is Face Recognition Hard?
Many Faces of the same person
Face Recognition Difficulties
• Identify similar faces (inter-class similarity)
• Accommodate intra-class variability due to:
• head pose
• illumination conditions
• expressions
• facial accessories
• aging effects
• Cartoon faces
Inter-class Similarity
• Different persons may have very similar appearance
Twins Father and son
www.marykateandashley.com news.bbc.co.uk/hi/english/in_depth/americas/2000/us_elections
Intra-class Variability• Faces with intra-subject variations in pose, illumination,
expression, accessories, color, occlusions, and brightness
Sketch of a Pattern Recognition Architecture
FeatureExtraction
ClassificationImage
(window)ObjectIdentityFeature
Vector
Example: Face Detection
• Scan window over image
• Classify window as either:– Face– Non-face
ClassifierWindowFace
Non-face
Example: Finding skin
• Skin has a very small range of (intensity independent) colors, and little texture– Compute an intensity-independent color measure, check if
color is in this range, check if there is little texture (median filter)
– See this as a classifier - we can set up the tests by hand, or learn them.
– get class conditional densities (histograms), priors from data (counting)
• Classifier is
Face Detection
Face Detection Algorithm
Face Localization
Lighting Compensation
Skin Color Detection
Color Space Transformation
Variance-based Segmentation
Connected Component &Grouping
Face Boundary Detection
Verifying/ WeightingEyes-Mouth Triangles
Eye/ Mouth Detection
Facial Feature Detection
Input Image
Output Image
Face Recognition: 2-D and 3-D
2-D
Face Database
Time(video)
2-D
RecognitionData
3-D 3-D
RecognitionComparison
Prior knowledgeof face class
Pose-dependent
Algorithms
Pose-invariant
Pose-dependency
Matching features
Appearance-based (Holistic)
Feature-based (Analytic)
Hybrid
Viewer-centered Images
Object-centered Models
Face representation
Taxonomy of Face Recognition
Image as a Feature Vector
• Consider an n-pixel image to be a point in an n-dimensional space, x Rn.
• Each pixel value is a coordinate of x.
x1
x2
x3
Nearest Neighbor Classifier { { RRjj } } are set of training images.
x1
x2
x3RR11
RR22
II
),(minarg IRdistID jj
Comments• Sometimes called “Template Matching”• Variations on distance function (e.g. L1,
robust distances)• Multiple templates per class- perhaps many
training images per class.• Expensive to compute k distances, especially
when each image is big (N dimensional).• May not generalize well to unseen examples
of class.• Some solutions:
– Bayesian classification– Dimensionality reduction
EigenfacesEigenfacesA set of eigenfaces can be generated by performing a mathematical process called principal component analysis (PCA) on a large set of images depicting different human faces.
Informally, eigenfaces can be considered a set of "standardized face ingredients", derived from statistical analysis )(covariance matrix ) of many pictures of faces. Any human face can be considered to be a combination of these standard faces.
EigenfacesEigenfacesFor example, one's face might be composed of the average face plus 10% from eigenface 1, 55% from eigenface 2, and even -3% from eigenface 3.
The eigenfaces that are created will appear as light and dark areas that are arranged in a specific pattern.
EigenfacesEigenfacesThese eigenfaces can now be used to represent both existing and new faces: we can project a new (mean-subtracted) image on the eigenfaces and thereby record how that new face differs from the mean face.
The eigenvalues associated with each eigenface represent how much the images in the training set vary from the mean image in that direction.
EigenfacesEigenfacesWe lose information by projecting the image on a subset of the eigenvectors, but we minimize this loss by keeping those eigenfaces with the largest eigenvalues.
For instance, if we are working with a 100 x 100 image, then we will obtain 10,000 eigenvectors. In practical applications, most faces can typically be identified using a projection on between 100 and 150 eigenfaces, so that most of the 10,000 eigenvectors can be discarded.
How do you construct Eigenspace?
[ ] [ ][ x1 x2 x3 x4 x5 ] W
Construct data matrix by stacking vectorized images and then apply Singular Value Decomposition (SVD)
EigenfacesEigenfaces
Eigenfaces• Modeling
1. Given a collection of n labeled training images,2. Compute mean image and covariance matrix.3. Compute k Eigenvectors (note that these are
images) of covariance matrix corresponding to k largest Eigenvalues.
4. Project the training images to the k-dimensional Eigenspace.
• Recognition1. Given a test image, project to Eigenspace.2. Perform classification to the projected training
images.
Eigenfaces: Training Images
Eigenfaces
Mean Image Basis Images
Difficulties with PCA
• Projection may suppress important detail– smallest variance directions may not be
unimportant
• Method does not take discriminative task into account– typically, we wish to compute features that allow
good discrimination– not the same as largest variance
Fisherfaces: Class specific linear projection
• An n-pixel image xRn can be projected to a low-dimensional feature space yRm by
y = Wx
where W is an n by m matrix.
• Recognition is performed using nearest neighbor in Rm.
• How do we choose a good W?
PCA & Fisher’s Linear Discriminant
• Between-class scatter
• Within-class scatter
• Total scatter
• Where– c is the number of classes
i is the mean of class i
– | i | is number of samples of
i..
Tii
c
iiBS ))((
1
c
i x
TikikW
ik
xS1
))((
WB
c
i x
TkkT SSxS
ik
1
))((
1
2
12
PCA & Fisher’s Linear Discriminant
• PCA (Eigenfaces)
Maximizes projected total scatter
• Fisher’s Linear Discriminant
Maximizes ratio of projected between-class to projected within-class scatter
WSWW TT
WPCA maxarg
WSW
WSWW
WT
BT
Wfld maxarg
12
PCA
FLD
Four Fisherfaces Database
Eigenfaces and Fisherfaces