advances in face biometrics: towards uncontrolled...
TRANSCRIPT
31/01/2018
1
1 of 19
Advances in Face Biometrics:
Towards Uncontrolled Scenarios
Prof. Julian Fierrez
http://atvs.ii.uam.es/fierrez/ (Based on the PhD Thesis by Ester Gonzalez-Sosa)
Escuela Politécnica Superior
UNIVERSIDAD AUTONOMA DE MADRID
April 2017
2 of 19
Face Recognition Scenarios
• From controlled scenarios …..
31/01/2018
2
3 of 19
Face Recognition Scenarios
• To uncontrolled scenarios
4 of 19
Face Recognition Scenarios
• To uncontrolled scenarios
31/01/2018
3
5 of 19
Outline
1. Case Study: ICB-RW 2016
2. Hand-crafted Approach
3. Deep Learning Approach
4. Experimental Results
5. Discussion
6. Conclusions
6 of 19
Case Study: ICB-RW 2016
• Realistic surveillance scenarios: challenging variability sources
E. Gonzalez-Sosa, R. Vera-Rodriguez, J. Fierrez, and J. Ortega-Garcia, “Exploring Facial Regions in Unconstrained Scenarios: Experience on ICB-RW”, IEEE Intelligent Systems, 2018. (To appear)
BLUR ILLUMINATION OCCLUSION POSE
EXPRESSION POSE &
OCCLUSION
DISTANCE &
OCCLUSIONOCCLUSION
• 2016 International Competition of Face Recognition in the Wild (ICB-RW)
J Neves, H Proença, “ICB-RW 2016: International Challenge on Biometric Recognition in the Wild”, Proc. Intl Conf. on Biometrics, 2016.
31/01/2018
4
7 of 19
Case Study: ICB-RW 2016
– QUIS CAMPI dataset
– Pan-Tilt-Zoom (PTZ) cameras: up to 50 meters
– 90 subjects
– Training each subject: 3 mug-shot images (1 frontal), 5 PTZ images
– Testing 5 PTZ images (separate from training)
– Closed-set identification
– Performance as Cumulative Match Curves (CMC)
Unconstrained conditions
8 of 19
Case Study: ICB-RW 2016
Training images from 90 subjects
Test image Who is this subject?
0.3
0.4
0.2
Score
Identification Mode (Closed Set)
31/01/2018
5
9 of 19
Hand-crafted Approach
FACE DETECTION: X. Zhu, D. Ramanan, "Face Detection, Pose Estimation, and Landmark Localization in the Wild", Proc. CVPR, 2012.FACE FRONTALIZATION: Hassner et al., “Effective Face Frontalization in Unconstrained Images”, Proc. CVPR, 2015.HISTROGRAM EQ: Struct et al., “Photometric normalization techniques for illuminance variance”, Advances in Face Image Analysis: Techniques and Technologies, IGI Global, May 2010.
10 of 19
Deep Learning Approach
Features directly learnt from data
Convolutional Neural Networks (CNN):
• A) Feature Extraction: features from a particular layer followed by a
classification stage (SVM, SoftMax, distance-based, etc.)
• B) Fine-Tuning : re-train the last layers of CNN for the target task. (Transfer Learning / Domain Adaptation / ...)
31/01/2018
6
11 of 19
Deep Learning Approach
Pre-trained model: VGG-FACE
Inspired from VGG-Very-Deep-16 CNN network
39 layers
16 convolutional layers
135 M parameters learnt
Trained with 2.6 million faces to classify 2622 classes
O. M. Parkhi, A. Vedaldi, A. Zisserman, “Deep Face Recognition”, Proc. British Machine Vision Conference, 2015.
12 of 19
Deep Learning Approach
Pre-trained model: VGG-FACE
Inspired from VGG-Very-Deep-16 CNN network
39 layers
16 convolutional layers
135 M parameters learnt
Trained with 2.6 million faces to classify 2622 classes
31/01/2018
7
13 of 19
Experimental Results
Proposed Systems
Deep
Learning
Hand-crafted
Feature Extractor
Fine Tuning
14 of 19
Experimental Results
A posteriori
31/01/2018
8
15 of 19
Discussion: Examples
Genuine ranked-1
16 of 19
Discussion: Examples
Genuine ranked-2
31/01/2018
9
17 of 19
Occlusion assessment:
Occlusion included in Training Occlusion not included in Training
Discussion: Examples
18 of 19
Conclusions
Deep Learning for ICB-RW – Pros:
• Generally performs better than hand-crafted approaches
• Good generalization capability
• Pre-trained DL to extract features (compared to fine-tuning)
Deep Learning for ICB-RW – Cons:
• Generalization is still limited
Room for improvement (e.g., training-testing mismatch)
31/01/2018
10
19 of 19
Advances in Face Biometrics:
Towards Uncontrolled Scenarios
Prof. Julian Fierrez
http://atvs.ii.uam.es/fierrez/ (Based on the PhD Thesis by Ester Gonzalez-Sosa)
Escuela Politécnica Superior
UNIVERSIDAD AUTONOMA DE MADRID
April 2017