face recognition vendor test 2006 experiment 4 covariate study
TRANSCRIPT
![Page 1: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/1.jpg)
Face Recognition Vendor Test 2006 Face Recognition Vendor Test 2006 Experiment 4 Covariate StudyExperiment 4 Covariate Study
Dr. J. Ross BeveridgeDr. J. Ross BeveridgeDr. Dr. GeofGeof H. GivensH. GivensDr. Bruce DraperDr. Bruce DraperMr. Mr. YuiYui Man Man LuiLui
Colorado State UniversityColorado State University
Dr. P.Dr. P. Jonathon Phillips Jonathon PhillipsNational Institute of Standards and TechnologyNational Institute of Standards and Technology
The work was funded in part by the Technical Support Working Group (TSWG) under Task T-1840C.
![Page 2: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/2.jpg)
MotivationMotivation
Factors that influence face recognitionFactors that influence face recognition
![Page 3: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/3.jpg)
4/21/2008 3FRVT 2006 Experiment 4 Covariate Analysis
Motivation Motivation -- Attributes of PeopleAttributes of PeopleWhat makes recognition harder/easier?What makes recognition harder/easier?
YoungYoung ……
![Page 4: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/4.jpg)
4/21/2008 4FRVT 2006 Experiment 4 Covariate Analysis
MotivationMotivation -- Attributes of PeopleAttributes of PeopleGender?Gender?
AgeAge
![Page 5: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/5.jpg)
4/21/2008 5FRVT 2006 Experiment 4 Covariate Analysis
Motivation Motivation -- Attributes of PeopleAttributes of PeopleRace?Race?
AgeAge GenderGender
![Page 6: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/6.jpg)
4/21/2008 6FRVT 2006 Experiment 4 Covariate Analysis
Motivation Motivation Smile?-- Smile?Expression?Expression?
AgeAge GenderGender RaceRace
![Page 7: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/7.jpg)
4/21/2008 7FRVT 2006 Experiment 4 Covariate Analysis
Motivation Motivation -- EnvironmentEnvironmentControl: Mugshot vs. posed indoor or outdoorControl: Mugshot vs. posed indoor or outdoor
AgeAge GenderGender RaceRace ExpressionExpression
![Page 8: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/8.jpg)
4/21/2008 8FRVT 2006 Experiment 4 Covariate Analysis
Motivation Motivation -- GlassesGlasses
AgeAge GenderGender RaceRace ExpressionExpression
Glasses in uncontrolled imagery.Glasses in uncontrolled imagery.
UncontrolledUncontrolled
![Page 9: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/9.jpg)
4/21/2008 9FRVT 2006 Experiment 4 Covariate Analysis
Motivation Motivation - Recap- Recap
AgeAge Gender
Race Expression
GlassesUncontrolled …
Gender
Race Expression
GlassesUncontrolled …
![Page 10: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/10.jpg)
4/21/2008 10FRVT 2006 Experiment 4 Covariate Analysis
But Wait, ThereBut Wait, There’’s More, Qualitys More, Quality You cannot do much aboutYou cannot do much about
Gender, Age, Race, Gender, Age, Race, …… Some control overSome control over
Setting, Glasses, Expression, Setting, Glasses, Expression, …… What about measurable image properties?What about measurable image properties?
Resolution, Focus, Resolution, Focus, ……
ISO SC 37 "Biometrics” - Factors Affecting Face Image Quality ImagingACQUISITION PROCESS AND CAPTURE DEVICE PROPERTIES
…2. physical properties (e.g. resolution and contrast)
ISO SC 37 "BiometricsISO SC 37 "Biometrics”” -- Factors Affecting Face Image Quality ImagingFactors Affecting Face Image Quality ImagingACQUISITION PROCESS AND CAPTURE DEVICE PROPERTIESACQUISITION PROCESS AND CAPTURE DEVICE PROPERTIES
……2. physical properties (e.g. resolution and contrast)2. physical properties (e.g. resolution and contrast)
![Page 11: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/11.jpg)
Covariate AnalysisCovariate Analysis
4/21/2008 11FRVT 2006 Experiment 4 Covariate Analysis
![Page 12: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/12.jpg)
4/21/2008 12FRVT 2006 Experiment 4 Covariate Analysis
For Analysis We Need For Analysis We Need …… Lots of Performance Data Lots of Performance Data
FRVT 2006FRVT 2006
MethodologyMethodology Generalized Linear Mixed Effect ModelGeneralized Linear Mixed Effect Model
Specific ProblemSpecific Problem Uncontrolled frontal still against mugshot galleryUncontrolled frontal still against mugshot gallery
![Page 13: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/13.jpg)
4/21/2008 13FRVT 2006 Experiment 4 Covariate Analysis
Introduction Introduction -- More on FRVTMore on FRVT
22 participants, 10 countries22 participants, 22 participants, 10 countries10 countries
ChinaChina DenmarkDenmark GermanyGermany IsraelIsrael JapanJapanRomaniaRomania SpainSpain South South
KoreaKoreaUnited United KingdomKingdom
United United StatesStates
10 US, 7 foreign, 5 with offices in and outside the US10 US, 7 foreign, 5 with offices in and outside the US
Executing Agency
![Page 14: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/14.jpg)
4/21/2008 14FRVT 2006 Experiment 4 Covariate Analysis
Introduction Introduction -- ProgressProgress
For controlled frontal still
images
For controlled frontal still
images2006 - Falsely turn away 1/100 people,when only admitting 1/1000 imposters.2006 2006 -- Falsely turn away 1/100 people,Falsely turn away 1/100 people,when only admitting 1/1000 imposters.when only admitting 1/1000 imposters.
![Page 15: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/15.jpg)
4/21/2008 15FRVT 2006 Experiment 4 Covariate Analysis
Our Focus Our Focus -- Uncontrolled StillsUncontrolled Stills
![Page 16: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/16.jpg)
Turn Away 20/100(at 1/1,000 FAR)
FRVT 2002 Performance(Controlled vs Controlled)
4/21/2008 16FRVT 2006 Experiment 4 Covariate Analysis
Uncontrolled to Controlled StillUncontrolled to Controlled Still
2006 2006 -- 2006 - Falsely turn away 10/100 to 40/100 people,when only admitting 1/1000 impostors.Falsely turn away 10/100 to 40/100 people,Falsely turn away 10/100 to 40/100 people,
when only admitting 1/1000 impostors.when only admitting 1/1000 impostors.
![Page 17: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/17.jpg)
4/21/2008 17FRVT 2006 Experiment 4 Covariate Analysis
FRVT Covariate AnalysisFRVT Covariate Analysis
Algorithm Algorithm -- score fusion of 3 top performers.score fusion of 3 top performers.
Uncontrolled match to Controlled.Imagery Imagery -- Uncontrolled match to Controlled.
Subset of FRVT 2006 Experiment 4Subset of FRVT 2006 Experiment 4 345 subjects and 345 subjects and 110,514110,514 match scores.match scores.
![Page 18: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/18.jpg)
4/21/2008 18FRVT 2006 Experiment 4 Covariate Analysis
Performance VariablePerformance Variable
Verification Outcome: Success / FailureVerification Outcome: Success / FailureVerifiedVerified FARFAR GenderGender RaceRace ……
YesYes 1/1001/100 FemaleFemale AsianAsian ……
VerifiedVerified FARFAR GenderGender RaceRace ……
NoNo 1/1,0001/1,000 MaleMale AsianAsian ……
VerifiedVerified FARFAR GenderGender RaceRace ……
YesYes 1/10,0001/10,000 MaleMale WhiteWhite ……
* Outcomes for illustration purposes only.
Levels 1/100 1/1,000 & 1/10,000Levels 1/100 1/1,000 & 1/10,000
FAR is a factorFAR is a factor
Distributed across pairs.Distributed across pairs.
There are 110,514 pairs like this! There are 110,514 pairs like this! There are 110,514 pairs like this!
![Page 19: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/19.jpg)
4/21/2008 19FRVT 2006 Experiment 4 Covariate Analysis
Face Region In Focus MeasureFace Region In Focus Measure
FRIFM: Sum of FRIFM: Sum of SobelSobel edge magnitude inside an edge magnitude inside an ellipse bounding the face. ellipse bounding the face.
“Active Computer Vision by Cooperative Focus and Stereo” by Eric Krotkov.
![Page 20: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/20.jpg)
4/21/2008 20FRVT 2006 Experiment 4 Covariate Analysis
Face Region In Focus MeasureFace Region In Focus MeasureLow FRIFM examples High FRIFM examples
![Page 21: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/21.jpg)
Fitting a Statistical ModelFitting a Statistical Model
4/21/2008 21FRVT 2006 Experiment 4 Covariate Analysis
Generalized Linear Mixed Model
Generalized Linear Generalized Linear Mixed ModelMixed Model
……Verification Outcomes
Verification Outcomes
AgeAge GenderGender
RaceRaceGlassesGlassesFocusFocus Head TiltHead Tilt
ResolutionResolution
FARFAR
![Page 22: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/22.jpg)
Using the Statistical ModelUsing the Statistical Model
4/21/2008 22FRVT 2006 Experiment 4 Covariate Analysis
Fitted Generalized Linear Fitted Generalized Linear Mixed Model
Fitted Generalized Linear Mixed ModelMixed Model
AgeAge GenderGender
RaceRaceGlassesGlassesFocusFocus Head TiltHead Tilt
ResolutionResolution
FARFAR
P(success | covariates)
![Page 23: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/23.jpg)
4/21/2008 23FRVT 2006 Experiment 4 Covariate Analysis
Let Let AA and and BB be 2 covariates that might influence be 2 covariates that might influence algorithm performance. For example, A=gender algorithm performance. For example, A=gender (categorical) and B=Query(categorical) and B=Query--EyeEye--Distance (continuous). Distance (continuous). Let a index levels of A.Let a index levels of A.
Let j index the FAR setting, Let j index the FAR setting, ααjj
YYpabjpabj is is 1 if Person 1 if Person pp is verified correctly, 0 otherwise. is verified correctly, 0 otherwise.
YYpabjpabj depends on:depends on: person person pp, covariates , covariates AA and and BB, and , and false alarm rate false alarm rate ααjj..
Analysis is: Mixed Effects Logistic Regressionwith Repeated Measures on People.
Generalized Linear Mixed ModelGeneralized Linear Mixed Model
![Page 24: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/24.jpg)
4/21/2008 24FRVT 2006 Experiment 4 Covariate Analysis
GLMM Model Continued GLMM Model Continued ……
![Page 25: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/25.jpg)
4/21/2008 25FRVT 2006 Experiment 4 Covariate Analysis
The outcomes, i. e. verification success/failure, are uncorrelated when testing different people but correlated when testing the same person under different configurations.
The outcomes, i. e. verification success/failure, are The outcomes, i. e. verification success/failure, are uncorrelated when testing different people but uncorrelated when testing different people but correlated when testing the same person under correlated when testing the same person under different configurations.different configurations.
This means:This means:
The Mixed in Generalized Linear The Mixed in Generalized Linear MixedMixed effect Model.effect Model.
Subject VariationSubject Variation
![Page 26: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/26.jpg)
Vendor Test Covariate Analysis FindingsVendor Test Covariate Analysis Findings
From the highly expected From the highly expected ………… to the unexpected.to the unexpected.
![Page 27: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/27.jpg)
4/21/2008 27FRVT 2006 Experiment 4 Covariate Analysis
Finding 1: False Accept RateFinding 1: False Accept Rate
Solid = IndoorsDashed = Outdoors
![Page 28: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/28.jpg)
4/21/2008 28FRVT 2006 Experiment 4 Covariate Analysis
Finding 2: GenderFinding 2: Gender
Solid = IndoorsDashed = Outdoors
![Page 29: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/29.jpg)
4/21/2008 29FRVT 2006 Experiment 4 Covariate Analysis
Finding 3: RaceFinding 3: Race
Solid = IndoorsDashed = Outdoors
![Page 30: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/30.jpg)
4/21/2008 30FRVT 2006 Experiment 4 Covariate Analysis
Finding 4: GlassesFinding 4: Glasses
Solid = IndoorsDashed = Outdoors
![Page 31: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/31.jpg)
4/21/2008 31FRVT 2006 Experiment 4 Covariate Analysis
Finding 5:Finding 5:Small Medium Large
![Page 32: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/32.jpg)
Small Medium Large
Size of query image(distance between eyes)
4/21/2008 32FRVT 2006 Experiment 4 Covariate Analysis
Finding 5:Finding 5:
![Page 33: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/33.jpg)
Query environment
4/21/2008 33FRVT 2006 Experiment 4 Covariate Analysis
Finding 5:Finding 5:Small Medium Large
![Page 34: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/34.jpg)
Boundary of observed data
4/21/2008 34FRVT 2006 Experiment 4 Covariate Analysis
Finding 5:Finding 5:Small Medium Large
![Page 35: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/35.jpg)
Large PV range~0.90 −
~0.10
4/21/2008 35FRVT 2006 Experiment 4 Covariate Analysis
Finding 5:Finding 5:Small Medium Large
![Page 36: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/36.jpg)
Low FRIFM good; even for one image
4/21/2008 36FRVT 2006 Experiment 4 Covariate Analysis
Finding 5:Finding 5:Small Medium Large
![Page 37: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/37.jpg)
4/21/2008 37FRVT 2006 Experiment 4 Covariate Analysis
FRIFM ConclusionFRIFM Conclusion
Large performance variation.Large performance variation. Indoors [>0.95, ~.0.70] Indoors [>0.95, ~.0.70] Outdoors [~0.90, ~0.10]. Outdoors [~0.90, ~0.10].
Interaction between covariatesInteraction between covariates Environments (indoors, outdoors)Environments (indoors, outdoors) Query image sizeQuery image size Target and query FRIFMTarget and query FRIFM
Low FRIFM goodLow FRIFM good Effect if control for only one imageEffect if control for only one image
Outdoors: query size very importantOutdoors: query size very important
![Page 38: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/38.jpg)
4/21/2008 38FRVT 2006 Experiment 4 Covariate Analysis
FRIFM ConclusionFRIFM Conclusion
According to this analysisAccording to this analysis Out of focus is higher qualityOut of focus is higher quality Remember, edge density surrogate for focusRemember, edge density surrogate for focus
Is this really quality, Is this really quality, …… Or other environmental factors, Or other environmental factors, …… Or algorithm aberration?Or algorithm aberration?
![Page 39: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/39.jpg)
4/21/2008 39FRVT 2006 Experiment 4 Covariate Analysis
GLMM and Quality StandardsGLMM and Quality Standards
Factors Affecting Face Image Quality Character
RICHNESS OF IDENTIFYING CHARACTERISTIC Š BIOLOGICAL CHARACTERS
Behavior SPOOFING
Imaging ACQUISITION PROCESS AND CAPTURE DEVICE PROPERTIES
Environment AMBIENT CONDITION
FACE
1. anatomical characteristic (e.g. head dimensions, eye position) 2. injuries and scars 3. ethnic group 4. impairment 5. Heavy facial wears, such as thick or dark glasses
1. closed eyes 2. (exaggerated) expression 3. hair across the eye 4. head pose 5. makeup 6. subject posing (frontal / non-frontal to camera)
1. image enhancement and data reduction process 2. physical properties (e.g. resolution and contrast) 3. optical distortions 4. static properties of the background (e.g. wallpaper) 5. camera characteristics • sensor resolution
6. scene characteristics • geometric distortion
1. dynamic characteristics of the background like moving objects 2. variation in lighting and relate potential defects as • deviation from the
symmetric lighting • uneven lighting on the
face area • extreme strong or weak
illumination
3. subject posing, e.g.: • too far (face too small),
or too near (face too big) • out of focus (low
sharpness) • partial occlusion of the
face
From Covariates to Quality Measures ….From Covariates to Quality Measures From Covariates to Quality Measures ……..
![Page 40: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/40.jpg)
4/21/2008 40FRVT 2006 Experiment 4 Covariate Analysis
ConclusionConclusion Quality is NOT in the eyes of the beholderQuality is NOT in the eyes of the beholder It is in the performance numbersIt is in the performance numbers Model quantifies performance change.Model quantifies performance change.
Turn the knob. Turn the knob. Read off the change in performance. Read off the change in performance. Interaction between covariates.Interaction between covariates.
Tells us where to put our effortsTells us where to put our efforts Indoors it is FRIFM.Indoors it is FRIFM. Outdoors it is Query Image Size.Outdoors it is Query Image Size.
These models are used in other fields.These models are used in other fields. e.g., Biomedical.e.g., Biomedical.
Studies of Biometrics should use them.Studies of Biometrics should use them.
![Page 41: Face Recognition Vendor Test 2006 Experiment 4 Covariate Study](https://reader036.vdocuments.site/reader036/viewer/2022071612/61570654a097e25c76503e22/html5/thumbnails/41.jpg)
Thank YouThank You