Face Recognition Committee Machine:Methodology, Experiments and A System Application
Oral Defense by Sunny Tang15 Aug 2003
Outline
IntroductionFace RecognitionProblems and Objectives
Face Recognition Committee Machine
Committee MembersResult, Confidence and WeightStatic and Dynamic Structure
Outline
Face Recognition SystemSystem ArchitectureFace Recognition ProcessDistributed Architecture
Experimental ResultsConclusionQ & A
Introduction: Face Recognition
DefinitionA recognition process that analyzes facial characteristics
Two modes of recognitionIdentification: “Who is this”Verification: “Is this person who she/he claim to be?”
Face Recognition Applications
SecurityAccess control systemLaw enforcement
Multimedia databaseVideo indexingHuman search engine
Problems & Objectives
Current problems of existing algorithmsNo objective comparisonAccuracy not satisfactoryCannot handle all kinds of variations
ObjectivesProvide thorough and objectively comparisonPropose a framework to integrate different algorithms for better performanceImplement a real-time face recognition system
Face Recognition Committee Machine (FRCM)
MotivationAchieve better accuracy by combining predictions of different experts
Two structures of FRCMStatic structure (SFRCM)Dynamic structure (DFRCM)
Static vs. Dynamic
Static structureIgnore input signalsFixed weights
Dynamic structureEmploy input signal to improve the classifiersVariable weights
Committee Members
Template matching approachEigenfaceFisherfaceElastic Graph Matching (EGM)
Machine learning approachSupport Vector Machines (SVM)Neural Networks (NN)
Review: Eigenface & Fisherface
Feature spaceEigenface: Principal Component Analysis (PCA)Fisherface: Fisher’s Linear Discriminant (FLD)
Training & RecognitionProject images on feature spaceCompare Euclidean distance and choose the closest projection
Review: Elastic Graph Matching
Based on dynamic link architectureExtract facial feature by Gabor wavelet transformFace is represented by a graph consists of nodes of jets
Compare graphs by cost functionEdge similarity Se and vertex similarity Sv
Cost function
Review: SVM & Neural Networks
SVMLook for a separating hyperplane which separates the data with the largest margin
Neural NetworksAdjust neuron weights to minimize prediction error between the target and output
Result, Confidence & Weight
ResultResult of expert
ConfidenceConfidence of expert on its result
WeightWeight of expert’s result in ensemble
SFRCM Architecture
Result & Confidence (1)
Eigenface, Fisherface & EGMResult:
• Identification:
• Verification:
Confidence:• Identification:
• Verification:
Result & Confidence (2)
SVMOne-against-one approachResult:• Identification: SVM result • Verification: direct matching
Confidence:
Result & Confidence (3)
Neural networkA binary vector of size J for target representationResult:
• Identification:
• Verification:
Confidence: output value oj
Weight
Derived from performance of expert:
Amplify the difference of the performance:
Normalize in range [0, 1]:
Voting Machine
Assemble result and confidenceScore of expert’s result:
Ensemble result:
SFRCM Drawbacks
Fixed weights under all situationsThe weights of the experts are fixed no matter which images are given.
No update mechanismThe weights cannot be updated once the system is trained
DFRCM Architecture
Gating network is includedImage is involved in determination of weight
Gating Network
Keep the performance of experts on different face databasesDetermine the database of input imageGive the corresponding weights of the experts for that database
Feedback Mechanism1. Initialize ni,j and ti,j to 02. Train each expert i on different database j3. While TESTING
a) Determine j for each test imageb) Recognize the image in each expert ic) If ti,j != 0 then Calculate pi,j
d) Else Set pi,j = 0e) Calculate wi,j
f) Determine ensemble resultg) If FEEDBACK then Update ni,j and ti,j
4. End while
Implementation: Face Recognition System
Real-time face recognition systemImplementation of FRCMFace processing
Face trackingFace detectionFace recognition
System Architecture
Face Recognition Process
EnrollmentCollect face images to train the experts
RecognitionIdentificationVerification
System Snapshots
Identification Verification
Problems of FRCM on mobile device
Memory limitationLittle memory for mobile devicesRequirement for recognition
CPU power limitationTime and storage overhead of FRCM
Distributed Architecture
ClientCapture imageEnsemble results
ServerRecognition
Distributed System: Evaluation
ImplementationDesktop (1400MHz), notebook (300MHz)S: Startup, R: Recognition
Distinct servers:
Experimental Results
Databases used:ORL from AT&T LaboratoriesYale from Yale UniversityAR from Computer Vision Center at U.A.B HRL from Harvard Robotics Laboratory
Cross validation testing
1. Apply median filter to reduce noise in background
2. Apply Sobel filter for edge detection3. Covert to a binary image4. Apply horizontal and vertical projection5. Find face boundary 6. Obtain the center of the face region.7. Crop the face region and resize it
Preprocessing
ORL Result
ORL Face database
400 images40 peopleVariations
• Position• Rotation• Scale• Expression
Yale Result
Yale Face Database
165 images15 peopleVariations
• Expression• Lighting
AR Result
AR Face Database1300 images130 peopleVariations
• Expression• Lighting• Occlusions
HRL Result
HRL Face Database
345 images5 peopleVariation
• Lighting
Average Running Time & ResultsAverage running time
Average experimental results
Conclusion
Make a thorough comparison of five face recognition algorithmsPropose FRCM to integrate different face recognition algorithmsImplement a face recognition system for real-time applicationPropose a distributed architecture for mobile device
Question & Answer Section
Thanks