a review of face recognition...– face recognition is a challenging problem because of the...
TRANSCRIPT
2010-04-10 Department of Electronics and Information Engineering, HUST
1
A Review of Face Recognition
problem-oriented analysis of face recognition techniques
Qiang [email protected]
2010-04-10 Department of Electronics and Information Engineering, HUST
2
Outline
• Introduction• Face recognition across pose• Illumination invariant face recognition• Face recognition from a single image per person• Appendix• Reference
2010-04-10 Department of Electronics and Information Engineering, HUST
3
Introduction
Automaticface analysis
Face detection Face recognition Facial expressionrecognition
2010-04-10 Department of Electronics and Information Engineering, HUST
4
Introduction• Backgrounds of face recognition
– A key issue in face analysis is finding efficient descriptors for face appearance [11].
– Face recognition is one of the most important biometric techniques.
– Characteristics:• Easily collectable, universal and non-intrusive.• It is natural and passive over other biometric techniques
requiring cooperative subjects such as fingerprint recognition and iris recognition. It is ideal for applications in scenarios where fingerprinting or iris scanning are impractical (for example, surveillance) or undesirable due to problems of social acceptance.
2010-04-10 Department of Electronics and Information Engineering, HUST
5
Introduction
• Basic concepts– Biometrics are physiological (for example, fingerprints
and face) and behavioral (for example, voice and gait) characteristics used to determine or verify an individual’s identity.
– Verification (or Authentication) is performed by matching an individual’s biometric with the template of the claimed identity only.
– Identification (or Recognition), on the other hand, is performed by matching an individual’s biometric with the template of every identity in the database [16].
2010-04-10 Department of Electronics and Information Engineering, HUST
6
Introduction
• Problems involved with face recognition– Face recognition is a challenging problem because of
the diversity in faces and variations caused by pose, illumination, expressions, occlusions, gender, and makeup [16].
– This report provided some recent advances in face recognition across pose and illumination. In addition, face recognition from a single image per person is also reviewed.
2010-04-10 Department of Electronics and Information Engineering, HUST
7
Face recognition across pose
• Backgrounds– Though many face recognition approaches reported
satisfactory performances, their successes are limited to the conditions of controlled environment, which are unrealistic in many real applications.
– In recent surveys of face recognition techniques, pose variation was defined one of the prominent unsolved problems in the research of face recognition and it gains great interest in the computer vision and pattern recognition research community [2].
2010-04-10 Department of Electronics and Information Engineering, HUST
8
Face recognition across pose
• Problem definition, challenges– Face recognition across pose refers to recognising face images
in different poses by computers. – It is of great interest in many face recognition applications, most
notably those using indifferent or uncooperative subjects, such as surveillance systems. For example, face recognition is appealing in airport security to recognise terrorists and keep them from boarding plane.
– The most natural solution for this task might be to collect multiple gallery images in all possible poses to cover the pose variationsin the captured images, which requires a fairly easy face recognition algorithm.
– In many real situations, however, it is tedious and/or difficult to collect these multiple gallery images in different poses and therefore the ability of face recognition algorithm to tolerate pose variations is desirable.
2010-04-10 Department of Electronics and Information Engineering, HUST
9
Face recognition across pose
• Problem definition, challenges– If a face recognition does not have a good pose tolerance, given
a frontal passport photo, the system appears to require cooperative subjects who look directly at the camera and face recognition is no longer passive and non-intrusive.
– Therefore, pose invariance or tolerance is a key ability for face recognition to achieve its advantages of being non-intrusive over other biometric techniques requiring cooperative subjects such as fingerprint recognition and iris recognition.
• Basic concepts about pose variation– Tilt: vertical rotations– Yaw: horizontal rotations, i.e., to turn about the vertical axis.
2010-04-10 Department of Electronics and Information Engineering, HUST
10
Face recognition across pose
• Pose variation in the PIE database
2010-04-10 Department of Electronics and Information Engineering, HUST
11
Face recognition across pose• Evaluations
– As many pose-invariant face recognition approaches have been proposed recently, the need of evaluating different algorithms on a fair basis increased. A number of face image database have been established for the purpose to compare performances of different face recognition algorithms across pose.
2010-04-10 Department of Electronics and Information Engineering, HUST
12
Face recognition across pose• Evaluations
2010-04-10 Department of Electronics and Information Engineering, HUST
13
Face recognition across pose• Evaluations
2010-04-10 Department of Electronics and Information Engineering, HUST
14
Face recognition across pose
• Techniques Categorization
2010-04-10 Department of Electronics and Information Engineering, HUST
15
Face recognition across pose
2010-04-10 Department of Electronics and Information Engineering, HUST
16
Face recognition across pose
2010-04-10 Department of Electronics and Information Engineering, HUST
17
Face recognition across pose
2010-04-10 Department of Electronics and Information Engineering, HUST
18
Face recognition across pose
2010-04-10 Department of Electronics and Information Engineering, HUST
19
Face recognition across pose
2010-04-10 Department of Electronics and Information Engineering, HUST
20
Face recognition across pose
2010-04-10 Department of Electronics and Information Engineering, HUST
21
Face recognition across pose• Conclusions and further discussion
– Prior knowledge• Techniques which learn image variations caused by pose
changes often require extensive training and the performance is dependent on training data.
• Techniques without prior knowledge usually require more than one gallery image to successfully compensate pose variations.
– The 3D face recognition approaches can generally handle larger pose variations than 2D techniques.
• The existing 3D face reconstruction methods made suboptimal surface assumptions on human faces, which affects the reconstruction results. (e.g. Lambertian assumption)
2010-04-10 22
Face recognition across pose• Conclusions and further discussion
– Comprehensive consideration• On the other hand, a comprehensive consideration of the
complicated face surface reflection mechanism and external lighting parameters brings serious ill-conditions to 3D face modeling, because the number of unknown parameters is excessive and the problems are intractable.
– Non-linear mapping • The strategy of non-linear mapping has the promise to find a
feature space best suitable to pose variations, while the current research stage is preliminarily limited to fundamental mapping functions (e.g., radial basis functions).
• The question of whether there is a feature space where rotated faces are separable is still open. An answer to this question may lead to a clearer understanding of pose-invariant face recognition problem, similar to the findings of linear subspaces in illumination-invariant face recognition.
Department of Electronics and Information Engineering, HUST
2010-04-10 Department of Electronics and Information Engineering, HUST
23
Face recognition across pose
• Eigen light field (ELF) [12]– Method introduction
• This method unified all possible appearances of faces in different poses within a framework of light field, which is in a 4D space (two viewing directions and two pixel positions).
• Assuming human faces as convex Lambertian objects, this light-field was highly redundant and consequently the light field coefficients were associated in different poses for the same identity.
2010-04-10 Department of Electronics and Information Engineering, HUST
24
Face recognition across pose• Eigen light field (ELF) [12]
– In training stage• PCA was performed on those concatenated
vectors from different training identities. • Because of the redundancy of the light-field, face
images in different poses were represented using a single set of eigen vectors and eigen values to capture the variations due to identity changes.
warped Pose variant imagesrepresentation
A set of face images in different poses of different IDs
A uniform shape:feature points correspond to the light field
Concatenated vector (one per ID)
PCA
2010-04-10 Department of Electronics and Information Engineering, HUST
25
Face recognition across pose
warped Projection ontothe eigen space
Recognition byComparing the projected
Eigen coefficients
Input image
gallery and/or probe image
• Eigen light field (ELF) [12]– In recognition stage
• The recognition procedure is as below.• The dimensionality of input images is usually
smaller than that of the light field (image dimension times number of poses).
2010-04-10 Department of Electronics and Information Engineering, HUST
26
Face recognition across pose
• Locally linear regression [13]– Method introduction
• The basic idea lies in generating virtual frontal view from any given nonfrontal view to obtain a virtual gallery/probe face in order to solve pose problem.
• Following this idea, this paper proposed a simple, but efficient, novel locally linear regression (LLR) method, which generates the virtual frontal view from a given nonfrontal face image.
2010-04-10 Department of Electronics and Information Engineering, HUST
27
Face recognition across pose
• Locally linear regression [13]– Assumption and idea
• The basic assumption of the paper is there exists an approximate linear mapping between a nonfrontal face image and its frontal counterpart.
• With the assistance of a generic cylindrical face model, the author proposed to generate virtual frontal views from single horizontally rotated views through local linear regression (LLR).
2010-04-10 Department of Electronics and Information Engineering, HUST
28
Face recognition across pose
• Locally linear regression [13]– Virtual frontal view generation
• The author first perform dense sampling in the nonfrontal face image to obtain many overlapped local patches.
• Then, the linear regression technique is applied to each small patch for the prediction of its virtual frontal patch.
• Through the combination of all these patches, the virtual frontal view is generated.
2010-04-10 Department of Electronics and Information Engineering, HUST
29
Face recognition across pose• Locally linear regression [13]
– Procedure• In training stage, the face image was first divided into 10–30
evenly distributed patches in terms of an average cylindrical face model. In each patch, linear regression was performed to minimize the sum-square of image differences between frontal and non-frontal face images under a linear transformation.
• Then in testing stage, the input non-frontal image was also divided into patches in the same manner and each patch was transformed using the trained linear transformation matrix to form the appearance in the frontal view.
• Finally, all reconstructed patches were combined with a intensity averaging of overlapped pixels to form holistic frontal virtual views for recognition.
2010-04-10 Department of Electronics and Information Engineering, HUST
30
Face recognition across pose
• Tied factor analysis (TFA) [14]– Introduction
• The algorithm has the following distinctive characteristics:• It is probabilistic and provides a posterior probability for the
matching to a gallery (identification) or for whether the two faces match or belong to different people (verification).
• The algorithm is based on a generative model that describes how an underlying pose-invariant representation created the (pose-varying) observed data (see Fig. 1). This is in contrast to most existing algorithms, where the direction of information flow is from the observed image to the pose-invariant representation.
2010-04-10 Department of Electronics and Information Engineering, HUST
31
Face recognition across pose
• Tied factor analysis (TFA) [14]– Introduction
• In matching, one may think about the question: “What is the probability that two images were created from the same underlying representation?” However, the fact is this underlying representation is uncertain and never form an explicit point estimate.
• It is admitted that modeling the relationship between the entire frontal and nonfrontal faces (the global approach) is too challenging. Instead, this paper builds several local models describing how each individual facial feature (nose, eye, etc.) changes with pose. Then, the information from each model are combined by using naïve Bayes to make a final recognition decision.
2010-04-10 Department of Electronics and Information Engineering, HUST
32
Face recognition across pose
• Tied factor analysis (TFA) [14]
2010-04-10 Department of Electronics and Information Engineering, HUST
33
Face recognition across pose
• Tied factor analysis (TFA) [14]– Observed and Identity spaces
• Observed Image Data– It is assumed that the observed data is vectorized to form
an observed data vector.– The subspace to which faces commonly project is termed
the face manifold. – The manifold has two key characteristics that must be
captured by the proposed model. First, the mean positionin the manifold changes systematically with the pose of the face. Second, for a given individual, the position of the observation vector, relative to this mean, varies.
2010-04-10 Department of Electronics and Information Engineering, HUST
34
Face recognition across pose
• Tied factor analysis (TFA) [14]– Observed and Identity spaces
• Observed Image Data
2010-04-10 Department of Electronics and Information Engineering, HUST
35
Face recognition across pose
• Tied factor analysis (TFA) [14]– Observed and Identity spaces
• Identity Space Representation– At the core of the algorithm is the notion that there
genuinely exists a multidimensional variable h that represents the identity of the individual, regardless of the pose. We term the space of possible values for this variable as identity space, and the variable itself is termed a latent identity variable.
– Latent identity variables (LIVs) have this key property: If two LIVs take the same value, they represent the same person. If they take different values, they represent different people. LIV may be discrete or continuous.
2010-04-10 Department of Electronics and Information Engineering, HUST
36
Face recognition across pose
• Tied factor analysis (TFA) [14]– Observed and Identity spaces
• From Identity Space to Observed Space– 1. Choose the point in the identity space that
corresponds to the individual for which we create image data from some prior distribution.
– 2. Choose a pose (also from a prior distribution).– 3. Transform this identity variable to the observation
space by using a deterministic function. This function depends on the pose.
– 4. Add noise to the resulting observation vector.
2010-04-10 Department of Electronics and Information Engineering, HUST
37
Face recognition across pose• Tied factor analysis (TFA) [14]
– Observed and Identity spaces• Tied Factor Analysis
– We denote the jth image of individual i in the kth pose by xijk. We assume that this data was generated from an underlying latent identity variable, which we denote hi. The deterministic mapping between the identity and the observed spaces is affine. It comprises a set of offsets m1,...,K and a set of linear functions (matrices) F1,...,K. The generative process can be described as follows
– Formally write the model in terms of conditional probabilities as below
2010-04-10 Department of Electronics and Information Engineering, HUST
38
Face recognition across pose
• Tied factor analysis (TFA) [14]– Observed and Identity spaces
• Tied Factor Analysis– To complete the definition of the generative model, a
prior on the latent identity variables h is defined. – The prior is assumed to be a zero-mean Gaussian with
identity covariance I. – This is required to ensure that the learning process (see
next section) converges.
2010-04-10 Department of Electronics and Information Engineering, HUST
39
Face recognition across pose
• Tied factor analysis (TFA) [14]
2010-04-10 Department of Electronics and Information Engineering, HUST
40
Face recognition across pose
• Tied factor analysis (TFA) [14]– Learning System Parameters
• Learning parameters• To maximize
2010-04-10 Department of Electronics and Information Engineering, HUST
41
Face recognition across pose
• Tied factor analysis (TFA) [14]– Learning System Parameters
• E-step
2010-04-10 Department of Electronics and Information Engineering, HUST
42
Face recognition across pose
• Tied factor analysis (TFA) [14]
2010-04-10 Department of Electronics and Information Engineering, HUST
43
Face recognition across pose
• Tied factor analysis (TFA) [14]– Learning System Parameters
• Considering (2) and (3), since all terms on the right-hand side of (5) are normally distributed, the left-hand side is also normally distributed (with a mean vector and a covariance matrix). The first two moments of this distribution equal
2010-04-10 Department of Electronics and Information Engineering, HUST
44
Face recognition across pose
• Tied factor analysis (TFA) [14]– Learning System Parameters
• M-step (GP prior: )
2010-04-10 Department of Electronics and Information Engineering, HUST
45
Face recognition across pose
• Tied factor analysis (TFA) [14]– Learning System Parameters
• M-step– These derivative expressions are equated to zero and
are rearranged to provide the following update rules:
2010-04-10 Department of Electronics and Information Engineering, HUST
46
Face recognition across pose
• Tied factor analysis (TFA) [14]– Learning Results
• Data extraction from FERET for training and testing;• Face image segmentation from the background;• Image registration using 21 image features on the face;• Concatenation of the pixel values from the red, green, and
blue (RGB) channels to form a long observation vector.• Then build six models, each describing the variation between
one of the six non-frontal poses and the frontal pose. (according to the specific experiment)
• Use the model to predict how a face will look at a different pose. (uncertainty!)
2010-04-10 Department of Electronics and Information Engineering, HUST
47
Face recognition across pose
• Tied factor analysis (TFA) [14]– Recognition
• The model is fundamentally Bayesian in nature and describes a probability distribution over the predicted images. It is not clear how we can exploit knowledge about the uncertainty in the predicted image.
• This paper employed a probabilistic approach to face identification.
2010-04-10 Department of Electronics and Information Engineering, HUST
48
Face recognition across pose
• Tied factor analysis (TFA) [14]– Recognition
• Characteristics of the probabilistic approach:• The criteria for a gallery and probe face matching are that the
observed data vectors are explained by exactly the same value of the identity variable.
• Since our observations are noisy, we can never be sure which value the identity variable takes. Hence, we integrate out (evidence framework) the hidden identity variable to give a final formulation that does not depend on an estimate of h.
• The final decision is based on a calculation of the relative likelihood that the observed vectors were explained by different configurations of the underlying set of identity variables.
2010-04-10 Department of Electronics and Information Engineering, HUST
49
Face recognition across pose
• Tied factor analysis (TFA) [14]– Recognition
In order to calculate this integral, we reformulate the generative equation as a standard factor analyzer, for which the solution is known:
2010-04-10 Department of Electronics and Information Engineering, HUST
50
Face recognition across pose
• Tied factor analysis (TFA) [14]– Recognition
• Note that the terms Pr(Mn) are the prior probability for each model. In the specific experiments, this is set to the uniformvalue of 1/N for each model.
• The final recognition decision is made by choosing the maximum a posteriori model (MAP).
2010-04-10 Department of Electronics and Information Engineering, HUST
51
Face recognition across pose
2010-04-10 Department of Electronics and Information Engineering, HUST
52
Face recognition across pose
• Tied factor analysis (TFA) [14]
2010-04-10 Department of Electronics and Information Engineering, HUST
53
Face recognition across pose
2010-04-10 Department of Electronics and Information Engineering, HUST
54
Face recognition across pose
• Tied factor analysis (TFA) [14]
2010-04-10 Department of Electronics and Information Engineering, HUST
55
Face recognition across pose
• Tied factor analysis (TFA) [14]– Experiments
• Experiment 1: Face Identification Using Raw Pixel Data • Experiment 2: Face Identification with Local Gabor Data• Experiment 3: Face Verification• Experiment 4: Approximation of Evidence Term• Experiment 5: Automated versus Manual Keypoint Detection
2010-04-10 Department of Electronics and Information Engineering, HUST
56
Face recognition across pose
• Tensor-Based AAM with Continuous Variation Estimation[15]
2010-04-10 Department of Electronics and Information Engineering, HUST
57
Illumination invariant face recognition
• Background– The illumination problem is one of the well-known
problems in face recognition in uncontrolled environment [3].
• Techniques Categorization– Passive approaches
• Based on given intensity images
– Active approaches• Using additional devices to get more information
2010-04-10 Department of Electronics and Information Engineering, HUST
58
Illumination invariant face recognition
• Passive approaches– Illumination Variation Modeling (based on a statistical
or physical model)• Linear Subspaces
– Photometric alignment, 3D linear subspace, segment linear subspace
• Illumination Cone– Cone-attached, Cone-casted
• Spherical Harmonics– Recent advances of 3D spherical harmonic basis morphable
model (SHBMM) in [9].• Nine point lights• Generalized Photometric Stereo
2010-04-10 Department of Electronics and Information Engineering, HUST
59
Illumination invariant face recognition
• Passive approaches– Illumination Invariant Features
• Features Derived from Image Derivatives– Gradient angle, and recent advance of Gradientfaces [7]– Symmetric shape from shading (A prototype image with
normalized illumination can be obtained from a single training image under unknown illumination.)
– Statistical shape from shading model (to recover face shape from a single image and to synthesize the same face under new illumination)
• Quotient Image• Retinex approach• Transformation domain features• Local Binary Pattern
2010-04-10 Department of Electronics and Information Engineering, HUST
60
Illumination invariant face recognition
• Passive approaches– Illumination Invariant Features
• Features Derived from Image Derivatives• Quotient Image
– Quotient illumination relighting– Generic intrinsic illumination subspace
• Retinex approach (luminance estimation by smoothed image)– A single Gaussian function is applied to smooth the image in
the single scale retinex approach, – multi-scale retinex (MSR) approach (using isotropic smoothing)– Self-Quotient Image (essentially a MSR, anisotropic smoothing)– Morphological Quotient Image (use mathematical morphology
operation to smooth the original image)• Transformation domain features• Local Binary Pattern
2010-04-10 Department of Electronics and Information Engineering, HUST
61
Illumination invariant face recognition• Passive approaches
– Illumination Invariant Features• Features Derived from Image Derivatives• Quotient Image• Retinex approach• Transformation domain features (frequency domain)
– Eigenphase approach (perform PCA in the phase domain)– Apply SVM directly on phase– Quaternion correlation method in a wavelet domain– A series of work based on advance correlation filters
• Local Binary Pattern– LBP is unaffected by any monotonic grayscale transformation
in that the pixel intensity order is not changed after such a transformation. Recent advance LGBPHS [27], HGPP [28].
2010-04-10 Department of Electronics and Information Engineering, HUST
62
Illumination invariant face recognition• Passive approaches
– Photometric Normalization• Histogram equalization• Gamma Intensity Correction for illumination normalization• Homomorphic filtering• Illumination normalization in the wavelet domain
– Histogram equalization is applied to subband image.• Local Normalization• DCT to compensate for illumination variation
– 3D Morphable Model• 3D Morphable Model describes the shape and texture
separately (PCA analysis on database of 3D scans); To fit a face image under unknown illumination to the model, an optimization process is needed to optimize shape and texture coefficients along with 22 rendering parameters.
2010-04-10 Department of Electronics and Information Engineering, HUST
63
Illumination invariant face recognition
2010-04-10 Department of Electronics and Information Engineering, HUST
64
Illumination invariant face recognition
• Active approaches– 3D information
• 3D information can be represented as range image, profile, surface curvature, Extended Gaussian Image (EGI), Point Signature, and etc.
• The 3D modality can be fused with 2D modality, i.e. texture, to achieve better performance.
• Attention: 2D face images used above are captured in a controlled environment.
2010-04-10 Department of Electronics and Information Engineering, HUST
65
Illumination invariant face recognition
• Active approaches– Infrared
• The infrared spectrum ranges from 0.7�m -10mm. It can be divided into 5 bands, namely:
• Near-Infrared (Near-IR) (0.7-0.9�m), the Short-Wave Infrared (SWIR) (0.9-2.4�m), the Mid-Wave Infrared (MWIR) (3.0-8.0�m), the Long-Wave Infrared (LWIR) (8.0-14.0�m), and Far-Infrared (FIR) (14.0�m-10mm). Near-IR and SWIR belong to reflected infrared (0.7-2.4�m), while MWIR and LWIR belong to thermal infrared (2.4�m-14mm).
• Thermal Infrared– A survey on visual and infrared face recognition in [32].
• Active Near-IR Illumination• Active Differential Imaging
2010-04-10 Department of Electronics and Information Engineering, HUST
66
Illumination invariant face recognition
• Gradientfaces [7]– Reflectance Model
• where I(x,y) is image pixel value, R(x,y) is the reflectance and L(x,y) is the illuminance at each point (x,y) .
• A “common” assumption is that L varies very slowly while Rcan change abruptly, which means L is approximately smooth.
• It should be pointed out that the conclusion (L be approximately smooth) may be violated in shadow boundaries.
2010-04-10 Department of Electronics and Information Engineering, HUST
67
Illumination invariant face recognition
• Gradientfaces [7]– Gradient Domain Versus Pixel Domain
• the pixel points are not completely independent of each other, there are some relationships between neighboring pixel points.
• the conventional face recognition methods, such as PCA and LDA, which are implemented in pixel domain such that they ignore the underlying relationships between neighboring pixel points.
• While the gradient domain explicitly considers such relationships between neighboring pixel points such that it is able to reveal underlying inherent structure of image data.
2010-04-10 Department of Electronics and Information Engineering, HUST
68
Illumination invariant face recognition
• Gradientfaces [7]– Gradientfaces
2010-04-10 Department of Electronics and Information Engineering, HUST
69
Illumination invariant face recognition
2010-04-10 Department of Electronics and Information Engineering, HUST
70
Illumination invariant face recognition
2010-04-10 Department of Electronics and Information Engineering, HUST
71
Illumination invariant face recognition
• Narrowband Spectral Range Selection [8]– Chang et al. [8] introduced a distribution separation
measure and the selection of the optimal spectral range by ranking these separation values. The fused images from these chosen spectral ranges are verified to outperform the conventional broadband images by 3%–20%.
– They proposed using narrowband subspectral images instead of conventional broadband images to improve recognition performance. A spectral range selection algorithm was developed to choose the optimal band images under given illumination conditions.
2010-04-10 Department of Electronics and Information Engineering, HUST
72
Face recognition from a single image per person
• Introduction [4]– Many research efforts [1] have been focused on how
to improve the accuracy of a recognition system.– Most of them ignore the potential problem when the
face database at hand has only one sample image per person stored.
– Under this condition, most of the traditional methods such as eigenface and fisherface will suffer serious performance drop or even fail to work.
2010-04-10 Department of Electronics and Information Engineering, HUST
73
Face recognition from a single image per person
• The one sample per person problem [4]– Backgrounds
• Geometric-based methods (template matching): geometrical feature extraction. One image per person is not a problem at all for these methods.
• Appearance-based techniques: vectorlized representation of face image. These techniques greatly improved the effectiveness and efficiency of face recognition systems.
• However, one of the key components of appearance-based methods is their learning mechanism, whose performance is heavily affected by the number of training samples for each face.
2010-04-10 Department of Electronics and Information Engineering, HUST
74
Face recognition from a single image per person
• The one sample per person problem [4]– The challenges of one sample problem
• Broadly speaking, one sample problem is directly related to the small sample size problem in statistics and pattern recognition.
• Dimensionality reduction techniques such as principal component analysis (PCA).
• The covariance matrix for the N training samples C:
2010-04-10 Department of Electronics and Information Engineering, HUST
75
Face recognition from a single image per person
• The one sample per person problem [4]– The challenges of one sample problem
• Various extended algorithms during the last decades based on the standard eigenface technique, including
– probabilistic-based eigenface, – linear discriminative analysis (LDA) based subspace
algorithms, – support vector machine (SVM) based method, – feature line method,– Evolution pursuit, and – Laplacianfaces.
• If only one training sample per person is available, most of them will either reduce to the basic eigenface approach or simply fail to work in that case.
2010-04-10 Department of Electronics and Information Engineering, HUST
76
Face recognition from a single image per person
• The one sample per person problem [4]– The significance of the one sample problem
• The extreme case of one sample per person really commonly happens in real scenarios.
• On the other hand, storing only one sample per person in the database has several advantages desired by most real world applications:
– Easy to collect samples, either directly or indirectly– Save storage cost– Save computational cost
2010-04-10 Department of Electronics and Information Engineering, HUST
77
Face recognition from a single image per person
• Techniques Categorization [4]
2010-04-10 Department of Electronics and Information Engineering, HUST
78
Face recognition from a single image per person
2010-04-10 Department of Electronics and Information Engineering, HUST
79
Face recognition from a single image per person
• Recent advances [33, 18]
2010-04-10 Department of Electronics and Information Engineering, HUST
80
Face recognition from a single image per person
2010-04-10 Department of Electronics and Information Engineering, HUST
81
Appendix
• Categorization of face recognition techniques– These algorithms are categorized from two different
perspectives [16].– Perspective of the type of data.
• 1) 2D face recognition (which use 2D grayscale or color images),
• 2) 3D face recognition (which use 3D range images or pointclouds of faces), and
• 3) multimodal face recognition algorithms (which use both 2D and 3D facial data).
2010-04-10 Department of Electronics and Information Engineering, HUST
82
Appendix
• Categorization of face recognition techniques– Perspective of the type of approach those algorithms
use. (Zhao et al. [1] reviewed appearance-based (2D) face recognition algorithms from this perspective.)
• 1) holistic• 2) feature-based (or referred to as region based)• 3) hybrid-matching face recognition algorithms.
2010-04-10 Department of Electronics and Information Engineering, HUST
83
Appendix
• Categorization of face recognition techniques– Holistic algorithms match the faces as a whole for
recognition. • the eigenfaces of Turk and Pentland [19] that use the
Principal Component Analysis (PCA), • Fisherfaces [20] that use Linear Discriminant Analysis (LDA), • methods based on the Independent Component Analysis
(ICA) [21], • Bayesian methods [22], and • Support Vector Machine (SVM) methods [23]. • Neural networks [24] have also been used for holistic face
recognition.
2010-04-10 Department of Electronics and Information Engineering, HUST
84
Appendix
• Categorization of face recognition techniques– Feature-based structural matching approaches
• HMM [25]. • Mixture Distance [26].• The graph-matching approach [10] is one of the most
successful region-based approaches [1]. • LBP [11], LGBPHS [27], HGPP [28].• Region-based methods can prove useful in case of variations
(for example, illumination and expression) in the images [1].
2010-04-10 Department of Electronics and Information Engineering, HUST
85
Appendix
• Categorization of face recognition techniques– Hybrid approaches
• the modular eigenfaces approach [29] uses both global eigenfaces and local eigenfeatures.
• Other examples include the flexible appearance model based method in [30] and [31].
2010-04-10 Department of Electronics and Information Engineering, HUST
86
Reference• [1] Face Recognition A Literature Survey [Zhao_ACMCS2003]• [2] Face recognition across pose: A review [Xiaozheng
Zhang_PR200911]• [3] Illumination invariant face recognition: A survey [Xuan
Zou_IEEEConf2007]• [4] Face Recognition from a Single Image per Person: A Survey
[Xiaoyang Tan_PR2006]• [5] Recent advances in face biometrics with Gabor wavelets: A
review [Angel Serrano_PRL2009]• [6] A review on Gabor wavelets for face recognition [Linlin
Shen_PAA2006]• [7] Face Recognition Under Varying Illumination Using Gradientfaces
[TP Zhang_TIP200911]• [8] Improving Face Recognition via Narrowband Spectral Range
Selection Using Jeffrey Divergence [Hong Chang_TIFS200903]• [9] Face Relighting from a Single Image under Arbitrary Unknown
Lighting Conditions [Yang Wang_TPAMI200911]
2010-04-10 Department of Electronics and Information Engineering, HUST
87
Reference• [10] Face Recognition by Elastic Bunch Graph Matching
[WisFelKrue_TPAMI1997]• [11] Face description with local binary patterns: Application to face
recognition [Timo Ahonen_TPAMI2006]• [12] Appearance-Based Face Recognition and Light-fields [Ralph
Gross_TPAMI200404]• [13] Locally linear regression for pose-invariant face recognition [X.
Chai_S.Shan_TIP2007]• [14] Tied Factor Analysis for Face Recognition across Large Pose
Differences [Simon J.D. Prince_TPAMI2008] • [15] Tensor-Based AAM with Continuous Variation Estimation:
Application to Variation-Robust Face Recognition [TPAMI200906]• [16] An efficient multimodal 2D-3D hybrid approach to automatic
face recognition [Ajmal S. Mian_TPAMI07]• [17] Making FLDA applicable to face recognition with one sample
per person [SC Chen_PR2004]• [18] Face Recognition Using Combined Multiple Feature Extraction
Based on Fourier-Mellin Approach for Single Example image per Person [Yee Ming ChenPR2010M]
2010-04-10 Department of Electronics and Information Engineering, HUST
88
Reference• [19] Eigenfaces for Recognition [JCN1994]• [20] Eigenfaces vs. Fisherfaces: Recognition Using Class Specific
Linear Projection [TPAMI1997]• [21] Independent Component Representation for Face Recognition
[Proc. SPIE1998]• [22] Probabilistic Visual Learning for Object Representation
[TPAMI1997]• [23] Support Vector Machines Applied to Face Recognition
[NIPS1998]• [24] Evolutionary Pursuit and Its Application to Face
Recognition[TPAMI2000]• [25] HMM Based Architecture for Face Identification [IJIVC1994]• [26] Feature-Based Face Recognition Using Mixture Distance
[CVPR1996]
2010-04-10 Department of Electronics and Information Engineering, HUST
89
Reference• [27] Local Gabor binary pattern histogram sequence (LGBPHS): a
novel non-statistical model for face representation and recognition [Zhang_Shan_ICCV2005]
• [28] Histogram of Gabor phase patterns (HGPP): a novel object representation approach for face recognition [Zhang_Shan_TIP2007]
• [29] View-Based and Modular Eigenspaces for Face Recognition [Pentland_CVPR1994]
• [30] Automatic Face Identification System Using Flexible Appearance Models [Lanitis_IVC1995]
• [31] Component-Based Face Recognition with 3D Morphable Models [Huang_AVBPA2003]
• [32] Recent advances in visual and infrared face recognition: a review [Seong G. Kong_CVIU2005]
• [33] Sparsity preserving discriminant analysis for single training image face recognition [SC Chen_PRL2010]
2010-04-10 Department of Electronics and Information Engineering, HUST
90