Face Recognition Committee Machine:Methodology, Experiments and A System Application

Oral Defense by Sunny Tang15 Aug 2003

Outline

IntroductionFace RecognitionProblems and Objectives

Face Recognition Committee Machine

Committee MembersResult, Confidence and WeightStatic and Dynamic Structure

Outline

Face Recognition SystemSystem ArchitectureFace Recognition ProcessDistributed Architecture

Experimental ResultsConclusionQ & A

Introduction: Face Recognition

DefinitionA recognition process that analyzes facial characteristics

Two modes of recognitionIdentification: “Who is this”Verification: “Is this person who she/he claim to be?”

Face Recognition Applications

SecurityAccess control systemLaw enforcement

Multimedia databaseVideo indexingHuman search engine

Problems & Objectives

Current problems of existing algorithmsNo objective comparisonAccuracy not satisfactoryCannot handle all kinds of variations

ObjectivesProvide thorough and objectively comparisonPropose a framework to integrate different algorithms for better performanceImplement a real-time face recognition system

Face Recognition Committee Machine (FRCM)

MotivationAchieve better accuracy by combining predictions of different experts

Two structures of FRCMStatic structure (SFRCM)Dynamic structure (DFRCM)

Static vs. Dynamic

Static structureIgnore input signalsFixed weights

Dynamic structureEmploy input signal to improve the classifiersVariable weights

Committee Members

Template matching approachEigenfaceFisherfaceElastic Graph Matching (EGM)

Machine learning approachSupport Vector Machines (SVM)Neural Networks (NN)

Review: Eigenface & Fisherface

Feature spaceEigenface: Principal Component Analysis (PCA)Fisherface: Fisher’s Linear Discriminant (FLD)

Training & RecognitionProject images on feature spaceCompare Euclidean distance and choose the closest projection

Review: Elastic Graph Matching

Based on dynamic link architectureExtract facial feature by Gabor wavelet transformFace is represented by a graph consists of nodes of jets

Compare graphs by cost functionEdge similarity Se and vertex similarity Sv

Cost function

Review: SVM & Neural Networks

SVMLook for a separating hyperplane which separates the data with the largest margin

Neural NetworksAdjust neuron weights to minimize prediction error between the target and output

Result, Confidence & Weight

ResultResult of expert

ConfidenceConfidence of expert on its result

WeightWeight of expert’s result in ensemble

SFRCM Architecture

Result & Confidence (1)

Eigenface, Fisherface & EGMResult:

• Identification:

• Verification:

Confidence:• Identification:

• Verification:

Result & Confidence (2)

SVMOne-against-one approachResult:• Identification: SVM result • Verification: direct matching

Confidence:

Result & Confidence (3)

Neural networkA binary vector of size J for target representationResult:

• Identification:

• Verification:

Confidence: output value oj

Weight

Derived from performance of expert:

Amplify the difference of the performance:

Normalize in range [0, 1]:

Voting Machine

Assemble result and confidenceScore of expert’s result:

Ensemble result:

SFRCM Drawbacks

Fixed weights under all situationsThe weights of the experts are fixed no matter which images are given.

No update mechanismThe weights cannot be updated once the system is trained

DFRCM Architecture

Gating network is includedImage is involved in determination of weight

Gating Network

Keep the performance of experts on different face databasesDetermine the database of input imageGive the corresponding weights of the experts for that database

Feedback Mechanism1. Initialize ni,j and ti,j to 02. Train each expert i on different database j3. While TESTING

a) Determine j for each test imageb) Recognize the image in each expert ic) If ti,j != 0 then Calculate pi,j

d) Else Set pi,j = 0e) Calculate wi,j

f) Determine ensemble resultg) If FEEDBACK then Update ni,j and ti,j

4. End while

Implementation: Face Recognition System

Real-time face recognition systemImplementation of FRCMFace processing

Face trackingFace detectionFace recognition

System Architecture

Face Recognition Process

EnrollmentCollect face images to train the experts

RecognitionIdentificationVerification

System Snapshots

Identification Verification

Problems of FRCM on mobile device

Memory limitationLittle memory for mobile devicesRequirement for recognition

CPU power limitationTime and storage overhead of FRCM

Distributed Architecture

ClientCapture imageEnsemble results

ServerRecognition

Distributed System: Evaluation

ImplementationDesktop (1400MHz), notebook (300MHz)S: Startup, R: Recognition

Distinct servers:

Experimental Results

Databases used:ORL from AT&T LaboratoriesYale from Yale UniversityAR from Computer Vision Center at U.A.B HRL from Harvard Robotics Laboratory

Cross validation testing

1. Apply median filter to reduce noise in background

2. Apply Sobel filter for edge detection3. Covert to a binary image4. Apply horizontal and vertical projection5. Find face boundary 6. Obtain the center of the face region.7. Crop the face region and resize it

Preprocessing

ORL Result

ORL Face database

400 images40 peopleVariations

• Position• Rotation• Scale• Expression

Yale Result

Yale Face Database

165 images15 peopleVariations

• Expression• Lighting

AR Result

AR Face Database1300 images130 peopleVariations

• Expression• Lighting• Occlusions

HRL Result

HRL Face Database

345 images5 peopleVariation

• Lighting

Average Running Time & ResultsAverage running time

Average experimental results

Conclusion

Make a thorough comparison of five face recognition algorithmsPropose FRCM to integrate different face recognition algorithmsImplement a face recognition system for real-time applicationPropose a distributed architecture for mobile device

Question & Answer Section

Thanks