![Page 1: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/1.jpg)
Utilizing Posterior Probability forRace-composite Age EstimationEarly Applications to MORPH-II
Benjamin Yip
NSF-REU in Statistical Data Mining and Machine Learningfor Computer Vision and Pattern Recognition
The University of North Carolina at Wilmington
![Page 2: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/2.jpg)
Presentation Outline
1. Introduction
2. MORPH-II
3. Image Pre-processing
4. Methods
5. Preliminary Results
6. Conclusions
1
![Page 3: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/3.jpg)
Introduction
![Page 4: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/4.jpg)
The Importance of Age Estimation
The human face encodes a wealth of information about theindividual; when properly analyzed, this information becomesvaluable data that can be used in a wide array of applications [2]:
• Electronic Customer Relationship Management (ECRM)
• Surveillance monitoring• Biometrics
2
![Page 5: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/5.jpg)
The Importance of Age Estimation
The human face encodes a wealth of information about theindividual; when properly analyzed, this information becomesvaluable data that can be used in a wide array of applications [2]:
• Electronic Customer Relationship Management (ECRM)• Surveillance monitoring
• Biometrics
2
![Page 6: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/6.jpg)
The Importance of Age Estimation
The human face encodes a wealth of information about theindividual; when properly analyzed, this information becomesvaluable data that can be used in a wide array of applications [2]:
• Electronic Customer Relationship Management (ECRM)• Surveillance monitoring• Biometrics
2
![Page 7: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/7.jpg)
Previous Research
Previous research [3] has shown that age estimation is highlysensitive to race and gender categories. However, as far as we aware,no previous models have taken multiracial individuals into account.
3
![Page 8: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/8.jpg)
Project Overview
MORPH-IIPre-
processingGenderClsf.
Race ClassificationRace Classification
BM AgeEstimator
WM AgeEstimator
BF AgeEstimator
WF AgeEstimator
MaleFemale
Figure 1: The four subgroup age estimators correspond to: black females(BF), white females (WF), black males (BM) and white males (WM)
4
![Page 9: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/9.jpg)
Utilizing Posterior Probability
Let b,w ∈ [0, 1] be the posterior probabilities of belonging to eachrace, where b+ w = 1.
Race Classification
Black AgeEstimator
White AgeEstimator
YB YW
b w
Y∗ = (b ∗ YB) + (w ∗ YW)
5
![Page 10: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/10.jpg)
MORPH-II
![Page 11: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/11.jpg)
Introduction to MORPH-II
Table 1: Number of Images by Gender and Race
Black White Asian Hispanic Other Total
Male 36,821 7,958 140 1,661 64 46,644Female 5,756 2,590 13 99 32 8,490Total 42,577 10,548 153 1,760 96 55,134
Table 2: Number of Distinct Individuals
Black White Asian Hispanic Other Total
Male 8829 2056 47 507 19 11458Female 1491 628 4 28 8 2159Total 10320 2684 51 535 27 13617
6
![Page 12: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/12.jpg)
Introduction to MORPH-II
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
4.04
9 im
ages
0
1000
2000
3000
4000
5 10 15 20 25 35 50
Number of Images
Num
ber
of In
divi
dual
s
Images per Individual
Figure 2: Images per Subject
0
500
1000
1500
2000
15 20 25 30 35 40 45 50 55 60 65 70 75
Age
Fre
quen
cy
Age Distribution
Figure 3: Age Distribution
7
![Page 13: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/13.jpg)
Image Pre-processing
![Page 14: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/14.jpg)
Objectives of Pre-processing
Pre-processing is a very important operation in computer visiontasks, as valuable information is usually enveloped in meaninglessspace. Accordingly, the absence of a pre-processing step can havedrastic consequences on the results.
8
![Page 15: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/15.jpg)
Pre-processing Objectives
Given a standardmugshot,
the objective is to:
1. ensure properrotation,
2. extract only theface, and
3. standardize theimage,
yielding thepre-processed face.
9
![Page 16: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/16.jpg)
Pre-processing MORPH-II
Figure 4: Stages of Pre-processing Figure 5: Challenging Images
10
![Page 17: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/17.jpg)
Methods
![Page 18: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/18.jpg)
Subsetting
Given the heavily imbalanced nature of MORPH-II, only images ofBlack and White individuals were used in this project. These imageswere subsetted according to the below scheme:
bf_1 wf_1 bm_1 wm_1
bf_2 wf_2 bm_2 wm_2
f1
f2
m1
m2
Figure 6: Subsetting Scheme
In this reduced dataset there are a total of 20,560 images in a 3 to 1Male to Female ratio. The images for each gender are split into twohalves, and every race and gender subgroup is divided similarly (±1image). There are no common individuals between subsets.
11
![Page 19: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/19.jpg)
Feature Extraction
After undergoing histogram equalization to correct for illuminationvariation, features were extracted from each image using LocalBinary Patterns (LBPs); these feature vectors were used for all stagesof the age estimation pipeline.
Figure 7: Examples of LBPs1
A Note on Dimensionality ReductionThe dimension of the LBP feature vectors was reduced using Principal ComponentAnalysis (PCA). 400 principal components were kept.1http://scikit-image.org/docs/0.12.x/auto_examples/features_detection/plot_local_binary_pattern.html
12
![Page 20: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/20.jpg)
Race Classification
Race ClassifierSupport Vector Machine (SVM)with a linear kernel
• Used to obtain posteriorprobabilities
Table 3: Race Classification Accuracies
train : test Accuracy Cost
f1 : f2 96.03 .0002f2 : f1 95.97 .0001m1 : m2 96.01 .0001m2 : m1 98.13 .0001
Posterior probabilities are obtained by fitting a sigmoid/logisticmodel to the SVM outputs (Platt scaling) [4]:
P(y = 1 | f) = 11+ exp(Af+ B) (1)
13
![Page 21: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/21.jpg)
Age Estimation
ModelSupport Vector Regression (SVR)with a linear kernel
Race Classification
Black AgeEstimator
White AgeEstimator
YB YW
b w
Y∗ = (b ∗ YB) + (w ∗ YW)
14
![Page 22: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/22.jpg)
Preliminary Results
![Page 23: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/23.jpg)
Partial Dataset
The Partial (even) dataset contains 1,000 images of 1,000 distinctindividuals; it differs from the reduced version of MORPH-II in thatthe overall age distribution is uniform.
Table 4: Results on Partial Dataset
MAE Weighted MAE
bf_1 6.71 6.695bf_2 6.634 6.45wf_1 7.007 6.535wf_2 6.579 6.461bm_1 5.501 5.475bm_2 4.979 4.799wm_1 4.443 4.434wm_2 4.518 4.498
15
![Page 24: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/24.jpg)
Full Dataset (reduced)
Table 5: Results on Full Dataset (reduced)
MAE Weighted MAE
bf_1 6.894 6.872bf_2 6.316 6.389wf_1 5.826 5.824wf_2 5.791 5.863bm_1 5.283 5.279bm_2 5.359 5.362wm_1 4.985 4.983wm_2 4.971 4.987
16
![Page 25: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/25.jpg)
Conclusions
![Page 26: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/26.jpg)
Conclusions
While the results from the Partial dataset seem to show that thecomposite age estimate is an improvement over the straightforwardprediction, the preliminary results from the reduced MORPH-IIdataset are mixed. Much more work remains to be done in testingthe model on the full dataset.
17
![Page 27: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/27.jpg)
Future Work
The race labels do not account for multiracial individuals; therefore,there may be mixed race individuals in the training sets that arediluting the models. To correct for this:
1. Assemble training sets of individuals who are not of mixed race(i.e. they fall clearly into the Black or White race categories)
2. Use these sets to train race classifiers and age models
18
![Page 28: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/28.jpg)
Future Work
The race labels do not account for multiracial individuals; therefore,there may be mixed race individuals in the training sets that arediluting the models. To correct for this:
1. Assemble training sets of individuals who are not of mixed race(i.e. they fall clearly into the Black or White race categories)
2. Use these sets to train race classifiers and age models
18
![Page 29: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/29.jpg)
References i
G. Bingham, B. Yip, M. Ferguson, and C. Nansalo.MORPH-II: Inconsistencies and Cleaning.
Y. Fu, G. Guo, and T. S. Huang.Age synthesis and estimation via faces: A survey.IEEE Transactions on Pattern Analysis and Machine Intelligence,32(11):1955–1976, Nov 2010.
G. Guo and G. Mu.Human age estimation: What is the influence across race andgender?In 2010 IEEE Computer Society Conference on Computer Visionand Pattern Recognition - Workshops, pages 71–78, June 2010.
![Page 30: Utilizing Posterior Probability for Race-composite Age ...people.uncw.edu/chenc/STT592_Deep Learning/2017 NSF... · IntroductiontoMORPH-II Table1:NumberofImagesbyGenderandRace Black](https://reader034.vdocuments.site/reader034/viewer/2022050514/5f9ebd880c682d074956aebb/html5/thumbnails/30.jpg)
References ii
J. C. Platt.Probabilistic outputs for support vector machines andcomparisons to regularized likelihood methods.In ADVANCES IN LARGE MARGIN CLASSIFIERS, pages 61–74. MITPress, 1999.
K. Ricanek and T. Tesafaye.Morph: A longitudinal image database of normal adultage-progression.In Automatic Face and Gesture Recognition, 2006. FGR 2006. 7thInternational Conference on, pages 341–345. IEEE, 2006.