inter-modality face sketch recognition hamed kiani
TRANSCRIPT
Inter-modality Face Sketch Recognition
Hamed Kiani
2
Outline
• Overview• Previous Works• Proposed Approach• Results• Summary
Inter-modality Face Sketch Recognition ICME'12
Inter-modality Face Sketch Recognition ICME'12
3
Overview
• Face Recognition
Known Face Images
Face Recognition
System
Identity
Input Face
Inter-modality Face Sketch Recognition ICME'12
4
Overview• Face sketch recognition
Known Face Photos
(Mug shot)
Photo-Sketch
Matching
Suspect’s identity
Viewing
Verbal description Drawin
g
Eyewitness
Police artist
Sketch
Inter-modality Face Sketch Recognition ICME'12
5
Overview
–Modality Gap: the difference of visual cues between face sketch and photo.
Intra-modality approaches
Inter-modality approaches
Matching face photos and sketches in a same modality by (photo or sketch) Synthesis
Matching photo and sketch of different modalities (direct matching).
Tang and Wang [ECCV’03] Liu et al. [CVPR’5]
Wang and Tang [PAMI’09]
Klare and Jain [SPIE ‘10]Klare et al. [PAMI’11]
Zhang et al. [CVPR’11]
Image synthesisNo modality gap
Modality gap No image synthesis
Inter-modality Face Sketch Recognition ICME'12
6
Overview
• Visual cues of face come from:– Fine texture (appearance):low contrast details, flaws, moles, wrinkles , etc.–Coarse texture (shape):high contrast boundaries of facial
components eyes, mouth, etc
Inter-modality Face Sketch Recognition ICME'12
7
Overview
• Face textures and modality gap: Fine textures of a face photo captured by camera
(true pixels) Fine texture of a sketch is rendered by artist,
depending on drawing style and tools Fine textures of face photo and sketch are not
equivalent: high amount of modality gap Coarse texture (facial component and
boundaries) exists in both sketch and photo modality gap is not affected significantly by
coarse texture
Inter-modality Face Sketch Recognition ICME'12
8
Proposed Approach
• Histogram of Averaged Oriented Gradients (HAOG): a modified version of Histogram of Oriented Gradients (HOG)
• HOG for sketch recognition: Modeling local appearance and shapeBased on fine and coarse textures. “Fine texture leads to a high amount of modality gap”
Inter-modality Face Sketch Recognition ICME'12
9
Proposed Approach
• Idea of HAOG:Emphasizing coarse texture much more than fine texture in feature extraction.
• How?By averaged gradient vector (dominant gradient) instead of pixel’s gradient vector (orientation and magnitude).
Inter-modality Face Sketch Recognition ICME'12
10
Proposed Approach
• But: Local gradients cannot directly be averaged, opposite gradient vectors cancel each other
• Solution: Doubling the angles of the gradient vectors before averaging: equal to squaring the length of gradient vectors [Bazen and Grez, 2002].
Inter-modality Face Sketch Recognition ICME'12
11
Proposed Approach
• Thus, we define squared gradient vectors
Inter-modality Face Sketch Recognition ICME'12
12
Proposed Approach
• HAOG x-gradient
y-gradient
Inter-modality Face Sketch Recognition ICME'12
13
Proposed Approach• HAOG
HAOG
Inter-modality Face Sketch Recognition ICME'12
14
Proposed Approach• Given a query sketch and a gallery of face
photos , face sketch recognition is done by:
: HAOG descriptor , :chi-square
Inter-modality Face Sketch Recognition ICME'12
15
Proposed Approach
Figure 1. (a1) Face photo, (a2) Face sketch, (b1,b2) Gradient magnitudes of (a1,a2), Squared gradient magnitudes of
(a1,a2).
Inter-modality Face Sketch Recognition ICME'12
16
Proposed Approach
Figure 2. Face sketch (top), photo (bottom), (b,c,d) local patches (first row), HAOG descriptors (second row) and HOG descriptors
(third row).
Inter-modality Face Sketch Recognition ICME'12
17
Results
• Results on CUHK dataset with 606 pairs of face photo/sketch
Inter-modality Face Sketch Recognition ICME'12
18
Summary
• Face sketch recognition vs. face recognition
• Modality gap• HOG vs. HAOG• Future work
Inter-modality Face Sketch Recognition ICME'12
19