Face Recognition

Rashmi Shastri

Determine the identity of a face in an image The image can be a frame from a video Processing needs to be fast Classification problem Need faces images for training

Recognize criminals◦ – In public spaces (airports, shopping centers)◦ – In stores

Verify identity to grant access in restricted areas: non-invasive Biometrics

Passport control at terminals in airports Participant identification in meetings System access control

ATMs Computers Cell phones

◦ – personalized menu Intelligent buildings

◦ Music settings personalized

As old as computer vision

The subject of face recognition is as old as computer vision, both because of the practical importance of the topic and theoretical interest from cognitive scientists. Despite the fact that other methods of identification (such as fingerprints, or iris scans) can be more accurate, face recognition has always remains a major focus of research because of its non-invasive nature and because it is people's primary method of person identification.

Perhaps the most famous early example of a face recognition system is due to Kohonen [5], who demonstrated that a simple neural net could perform face recognition for aligned and normalized face images. The type of network he employed computed a face description by approximating the eigenvectors of the face image's autocorrelation matrix; these eigenvectors are now known as `eigenfaces.'

Kohonen's system was not a practical success, however, because of the need for precise alignment and normalization. In following years many researchers tried face recognition schemes based on edges, inter-feature distances, and other neural net approaches. While several were successful on small databases of aligned images, none successfully addressed the more realistic problem of large databases where the location and scale of the face is unknown.

Kirby and Sirovich (1989) [6] later introduced an algebraic manipulation which made it easy to directly calculate the eigenfaces, and showed that fewer than 100 were required to accurately code carefully aligned and normalized face images. Turk and Pentland (1991) [7] then demonstrated that the residual error when coding using the eigenfaces could be used both to detect faces in cluttered natural imagery, and to determine the precise location and scale of faces in an image. They then demonstrated that by coupling this method for detecting and localizing faces with the eigenface recognition method, one could achieve reliable, real-time recognition of faces in a minimally constrained environment. This demonstration that simple, real-time pattern recognition techniques could be combined to create a useful system sparked an explosion of interest in the topic of face recognition.

Eigenfaces are a set of eigenvectors used in the computer vision problem of human face recognition. The approach of using eigenfaces for recognition was developed by Sirovich and Kirby (1987) and used by Matthew Turk and Alex Pentland in face classification. It is considered the first successful example of facial recognition technology.[3]

100% match to any image at any angle

Instantly recognizing any person

Tied into a “super database” that knows who everyone is

Available to and in use by law enforcement.

Affected by lighting angle & size of face, quality of captured and known images etc.

Requires a “high-end” computer for real time face capture and processing

Stand alone systems

Varying confidence of match depending on application

Multiple unique or proprietary image formats in use

Intelligence images not available to local law enforcement

Data sharing across jurisdiction is a problem

Requires human judgment on match analogy

Current Stare

1. Acquire the image of a individual face 2 ways to acquire an image

1)Digitally scan an existing photograph 2)Acquire a live picture of a subject

2. Locate image of a face software is used to locate the faces in the image that has been

obtained3. Analysis of facial image

software measures face according to is peaks and valleys (nodal points)

focuses on the inner region of the face known as the “golden triangle”

nodal points are used to make a face print4. Comparison

the face print created by the software is compared to all face prints the system has stored in its database.

5. Match or no match software decides whether or not any comparisons from step 4

are close enough to declare a possible match

Two major challenges:◦ The illumination variation problem

Images of the same face appear differently due to the change in lighting

◦ The pose variation problem Basically, the existing solution can be divided into three

types:1. multiple images in both training stage and recognition stage2. multiple images in training stage, but only one image in recognition

stage 3. single image based methods

Other Challenges◦ Accessories◦ Expression

Can be divided in three major categories

◦ Knowledge based Human specified rule (Yand and Huang 1997) Multi resolution rule based

◦ Appearance based methods Uses Holistic texture features.

◦ Feature based approach Uses geometrical facial features

◦ Model /Template based approach Uses active appearance models, shape models, fitting morphable models.

False Positives-FACIAL RECOGNITION VENDOR TEST 2000-PALM BEACH INTERNATIONAL AIRPORT-FACIAL RECOGNITION VENDOR TEST 2002

Legal Issues

[1] Guodong Guo; Li, S.Z.; Kapluk Chan “Face recognition by support vector machines”, Automatic Face and Gesture Recognition, 2000. Proceedings. Fourth IEEE International Conference on, Vol., Iss., 2000 Pages:196-201

[2] Lian Hock Koh, Surendra Ranganath and Y. V. Venkatesh, “An integrated automatic face detection and recognition system”, Pattern Recognition, Volume 35, Issue 6, June 2002, Pages 1259-1273.

[3] http://en.wikipedia.org/wiki/Eigenface

[5] T. Kohonen, Self-organization and Associative Memory, Springer-Verlag, Berlin, 1989.

[6] M. Kirby and L. Sirovich, ``Application of the karhunen-loeve procedure for the characterization of human faces,'' IEEE Pattern Analysis and Machine Intelligence, vol. 12, no. 1, pp. 103-108, 1990.

[7] M. Turk and A. Pentland, ``Eigenfaces for recognition,'' J. Cog. Neuroscience, vol. 3, no. 1, pp. 71-86, 1991.

PCA◦ Derived from Karhunen-Loeve's transformation. Given an s-dimensional vector representation of

each face in a training set of images, Principal Component Analysis (PCA) tends to find a t-dimensional subspace whose basis vectors correspond to the maximum variance direction in the original image space. This new subspace is normally lower dimensional (t<<s). If the image elements are considered as random variables, the PCA basis vectors are defined as eigenvectors of the scatter matrix.

ICA◦ Independent Component Analysis (ICA) minimizes both second-order and higher-order

dependencies in the input data and attempts to find the basis along which the data (when projected onto them) are - statistically independent . Bartlett et al. provided two architectures of ICA for face recognition task: Architecture I - statistically independent basis images, and Architecture II - factorial code representation.

LDA◦ Linear Discriminant Analysis (LDA) finds the vectors in the underlying space that best

discriminate among classes. For all samples of all classes the between-class scatter matrix SB and the within-class scatter matrix SW are defined. The goal is to maximize SB while minimizing SW, in other words, maximize the ratio det|SB|/det|SW| . This ratio is maximized when the column vectors of the projection matrix are the eigenvectors of (SW^-1 × SB).

EP◦ Aa eigenspace-based adaptive approach that searches for the best set of projection axes in

order to maximize a fitness function, measuring at the same time the classification accuracy and generalization ability of the system. Because the dimension of the solution space of this problem is too big, it is solved using a specific kind of genetic algorithm called Evolutionary Pursuit (EP).

EBGM◦ Elastic Bunch Graph Matching (EBGM). All human faces share a similar topological structure.

Faces are represented as graphs, with nodes positioned at fiducial points. (exes, nose...) and edges labeled with 2-D distance vectors. Each node contains a set of 40 complex Gabor wavelet coefficients at different scales and orientations (phase, amplitude). They are called "jets". Recognition is based on labeled graphs. A labeled graph is a set of nodes connected by edges, nodes are labeled with jets, edges are labeled with distances.

Kernel Methods◦ The face manifold in subspace need not be linear. Kernel methods are a generalization of linear

methods. Direct non-linear manifold schemes are explored to learn this non-linear manifold. Trace Transform

◦ The Trace transform, a generalization of the Radon transform, is a new tool for image processing which can be used for recognizing objects under transformations, e.g. rotation, translation and scaling. To produce the Trace transform one computes a functional along tracing lines of an image. Different Trace transforms can be produced from an image using different trace functions.

AAM◦ An Active Appearance Model (AAM) is an integrated statistical model which combines a model of

shape variation with a model of the appearance variations in a shape-normalized frame. An AAM contains a statistical model if the shape and gray-level appearance of the object of interest which can generalize to almost any valid example. Matching to an image involves finding model parameters which minimize the difference between the image and a synthesized model example projected into the image.

3-D Morphable Model◦ Human face is a surface lying in the 3-D space intrinsically. Therefore the 3-D model should be

better for representing faces, especially to handle facial variations, such as pose, illumination etc. Blantz et al. proposed a method based on a 3-D morphable face model that encodes shape and texture in terms of model parameters, and algorithm that recovers these parameters from a single image of a face.

Bayesian Framework◦ A probabilistic similarity measure based on Bayesian belief that the image intensity differences

are characteristic of typical variations in appearance of an individual. Two classes of facial image variations are defined: intrapersonal variations and extrapersonal variations. Similarity among faces is measures using Bayesian rule.

Boosting & Ensemble Solutions◦ The idea behind Boosting is to sequentially employ a weak learner on a weighted version of a

given training sample set to generalize a set of classifiers of its kind. Although any individual classifier may perform slightly better than random guessing, the formed ensemble can provide a very accurate (strong) classifier. Viola and Jones build the first real-time face detection system by using AdaBoost, which is considered a dramatic breakthrough in the face detection research. On the other hand, papers by Guo et al. are the first approaches on face recogntion using the AdaBoost methods.