face detection and recognition using image processing
TRANSCRIPT
-
8/3/2019 Face Detection and Recognition Using Image Processing
1/43
Face detection andrecognition using image
processing
Group 11
-
8/3/2019 Face Detection and Recognition Using Image Processing
2/43
Group Members
Jam Zia-ul-Haq
Hafiz Nisar Ahmad
Muhammad Yasir
-
8/3/2019 Face Detection and Recognition Using Image Processing
3/43
contents
Introduction
Face detection
Face recognition
Implementation
-
8/3/2019 Face Detection and Recognition Using Image Processing
4/43
Introduction
Face interface
Face detection
Face recognition
Face detection Face recognitionMr.Chan
Prof..Cheng
Face database
Output:
-
8/3/2019 Face Detection and Recognition Using Image Processing
5/43
What is Face Detection?
Given an image, tellwhether there is
any human face, ifthere is, where isit(or where theyare).
Click to edit Master text stylesSecond level
Third level Fourth level
Fifth level
Click to edit Master text stylesSecond level
Third level Fourth level
Fifth level
-
8/3/2019 Face Detection and Recognition Using Image Processing
6/43
Importance of Face Detection
The first step for any automatic face recognitionsystem system
First step in many Human Computer Interactionsystems
Expression Recognition
Cognitive State/Emotional State Recogntion
First step in many surveillance systems
Tracking: Face is a highly non rigid objectA step towards Automatic TargetRecognition(ATR) or generic objectdetection/recognition
Video coding
-
8/3/2019 Face Detection and Recognition Using Image Processing
7/43
Face Detection: current state
State-of-the-art:
Front-view face detection can be done at >15
frames per second on 320x240 black-and-whiteimages on a 700MHz PC with ~95% accuracy.
Detection of faces is faster than detection ofedges!
Side view face detection remains to be difficult.
-
8/3/2019 Face Detection and Recognition Using Image Processing
8/43
Face Detection: challenges
Out-of-Plane Rotation: frontal, 45 degree, profile,upside down
Presence of beard, mustache, glasses etc
Facial Expressions
Occlusions by long hair, hand
In-Plane Rotation
Image conditions:
Size
Lighting condition
Distortion
Noise
Compression
-
8/3/2019 Face Detection and Recognition Using Image Processing
9/43
Different Approaches
Knowledge-based methods:
Encode what constitutes a typical face, e.g., therelationship between facial features
Feature invariant approaches:
Aim to find structure features of a face that exist evenwhen pose, viewpoint or lighting conditions vary
Template matching:
Several standard patterns stored to describe the face asa whole or the facial features separately
Appearance-based methods:
The models are learned from a set of training imagesthat capture the representative variability of faces.
-
8/3/2019 Face Detection and Recognition Using Image Processing
10/43
Knowledge-Based Methods
Top Top-down approach: Represent aface using a set of human-coded rulesExample:
The center part of face has uniform intensityvalues
The difference between the average intensityvalues of the center part and the upper part issignificant
A face often appears with two eyes that aresymmetric to each other, a nose and a mouth
Use these rules to guide the searchprocess
-
8/3/2019 Face Detection and Recognition Using Image Processing
11/43
Knowledge-Based Method: [Yang andHuang 94]
Level 1 (lowest resolution):
apply the rule the center part of the face has4 cells with a basically uniform intensity to
search for candidatesLevel 2: local histogram equalizationfollowed by edge equalization followedby edge detection
Level 3: search for eye and mouthfeatures for validation
-
8/3/2019 Face Detection and Recognition Using Image Processing
12/43
Knowledge-based Methods: Summary
Pros:
Easy to come up with simple rules
Based on the coded rules, facial features in an inputimage are extracted first, and face candidates are
identified
Work well for face localization in uncluttered background
Cons:
Difficult to translate human knowledge into rulesprecisely: detailed rules fail to detect faces and generalrules may find many false positives
Difficult to extend this approach to detect faces indifferent poses: implausible to enumerate all thepossible cases
-
8/3/2019 Face Detection and Recognition Using Image Processing
13/43
Feature-Based Methods
Bottom-up approach: Detect facial features(eyes, nose, mouth, etc) first
Facial features: edge, intensity, shape, texture,
color, etcAim to detect invariant features
Group features into candidates and verify them
-
8/3/2019 Face Detection and Recognition Using Image Processing
14/43
Feature-Based Methods: Summary
Pros: Features are invariant to pose andorientation change
Cons:
Difficult to locate facial features due to severalcorruption (illumination, noise, occlusion)
Difficult to detect features in complexbackground
-
8/3/2019 Face Detection and Recognition Using Image Processing
15/43
Template Matching Methods
Store a template
Predefined: based on edges orregions
Deformable: based onfacial contours (e.g.,Snakes)
Templates are hand-coded(not learned)
Use correlation to locatefaces
Click to edit Master text stylesSecond level
Third level Fourth level
Fifth level
-
8/3/2019 Face Detection and Recognition Using Image Processing
16/43
Template-Based Methods: Summary
Pros:
Simple
Cons:
Templates needs to be initialized near the faceimages
Difficult to enumerate templates for different
poses (similar to knowledge-based methods)
-
8/3/2019 Face Detection and Recognition Using Image Processing
17/43
Image Features
Rectangle filters
Rectangle_Feature_value f=
(pixels in white area) (pixels in shaded area)
-
8/3/2019 Face Detection and Recognition Using Image Processing
18/43
Example
Find theRectangle_Feature_value (f) of the box
enclosed by the dottedline
Rectangle_Feature_value f=
(pixels in white area)
(pixels in shaded area)f=(8+7)-(0+1)
=15-1= 14
1 2 3 3
3 0 1 3
5 8 7 1
0 2 3 6
E l A i l f
-
8/3/2019 Face Detection and Recognition Using Image Processing
19/43
Example: A simple facedetection method using one
feature
Result
This is a face:T he eye-part is dark,the nose-part is brightSo f is large, hence it is face
This is not a face.Because f is small
qRectangle_Feature_value fqf= (pixels in white area) (pixels in shadedarea)
qIf (f) is large it is face ,i.e.qif (f)>threshold, thenq faceqElseq non-face
-
8/3/2019 Face Detection and Recognition Using Image Processing
20/43
Why do we need to find pixel sum ofrectangles?
Answer: We want to get face featuresYou may consider thesefeatures as face features
Two eyes= (Area_A-Area_B)
Nose =(Area_C+Area_E-Area_D)
Mouth =(Area_F+Area_H-Area_G)
They can be differentsizes, polarity and aspectratios
AB
CDE
FGH
-
8/3/2019 Face Detection and Recognition Using Image Processing
21/43
Face feature and example
-1+2IntegralImage
A face
Shaded area
White areaF=Feat_val =pixel sum in shared area - pixel sum in whiteareaExamplePixel sum in white area=216+102+78+129+210+111=846
Pixel sum in shared area=10+20+4+7+45+7=93
Feat_val=F=846-93If F>threshold,feature=+1
Elsefeature=-1 End if;
We can choose threshold =768 , so feature is+1.
10 20 4
7 45 7
216 102 78
129 210 111
Pixel values insideThe areas
-
8/3/2019 Face Detection and Recognition Using Image Processing
22/43
2222
4 basic types of featuresfor white_area-gray_area
Type) Rows xcolumns
Type 1) 1x2
Type 2) 2x1
Type 3) 1x3
Type 4) 3x1
Each basic type canhave differencesizes and aspectratios.
-
8/3/2019 Face Detection and Recognition Using Image Processing
23/43
2323
Feature selection
For a 24x24detection region, thenumber of possiblerectangle features is~160,000!
Some examples and their typesFill in the types for the 2nd, 3rd rows
2 5 1 43
Type
1)
2)
3)
4)
5)
-
8/3/2019 Face Detection and Recognition Using Image Processing
24/43
2424
The detection challenge
Use 24x24 base window
For y=1;y
-
8/3/2019 Face Detection and Recognition Using Image Processing
25/43
2525
Solution to make it efficient
The whole 162,336 feature set is too large
Solution: select good features to make it moreefficient
Use: Boosting
Boosting
Combine many small weak classifiers tobecome a strong classifier.
Training is needed
-
8/3/2019 Face Detection and Recognition Using Image Processing
26/43
2626
Boosting for face detection Define weak learners based on rectangle
features
window
value of rectangle feature
Pt=polarity{+1,-1}
threshold
-
8/3/2019 Face Detection and Recognition Using Image Processing
27/43
2727
AdaBoost training
E.g. Collect 5000 faces, and 9400 non-faces.Different scales.
Use AdaBoost for training to build a strongclassifier.
Pick suitable features of different scales andpositions, pick the best few. (Take months todo , details is in [Viola 2004] paper)
Testing
Scan through the image, pick a window andrescale it to 24x24,
Pass it to the strong classifier for detection.
Report face, if the output is positive
Face detection using Adaboost
-
8/3/2019 Face Detection and Recognition Using Image Processing
28/43
2828
Boosting for face detection [viola2004] In the paper it shows that the following two features
(obtained after training) in cascaded picked by AdaBoost
have 100% detection rate and 50% false positive rate
But 50% false positive rate is not good enough
Approach [viola2004] :Attentional cascade
type2type3
Pick a window in theimage and rescale it to24x24 as image
I.e.H(face)=Sign{1h1(image)+2h2(image)}
H(face)=+1 faceH(face)=-1 non-face
h1(image)h2(image)
-
8/3/2019 Face Detection and Recognition Using Image Processing
29/43
2929
Boosting for face detection A 200-feature classifier can yield 95% detection rate and a false
positive rate of 1 in 14084 (Still not god enough)
Recall: False positive rate
The detector output is positive but it is false (there is actually noface). Definition of False positive: A result that is erroneouslypositive when a situation is normal. An example of a falsepositive: a particular test designed to detect cancer of the toenailis positive but the person does not have toenail cancer.(http://www.medterms.com/script/main/art.asp?articlekey=3377)
Still notgoodenough!
False positive rate
CorrectDetection
rate
X10-3
-
8/3/2019 Face Detection and Recognition Using Image Processing
30/43
3030
To improve false positive rate:Attentional cascade
Cascade of many AdaBoost strongclassifiers
Begin with with simple classifiers to rejectmany negative sub-windows
Many non-faces are rejected at the first fewstages.
Hence the system is efficient enough for
real time processing.AdaboostClassifier1
AdaboostClassifier2
AdaboostClassifier3
True True TrueFacefound
Non-face Non-face
Input image
False False FalseNon-face
-
8/3/2019 Face Detection and Recognition Using Image Processing
31/43
3131
An example
More features for later stages in the cascade
2 features 10 features 25 features 50 features
type3type2
AdaboostClassifier1
AdaboostClassifier2
AdaboostClassifier3
True True TrueFacefound
Non-face Non-face
Input image
False False FalseNon-face
-
8/3/2019 Face Detection and Recognition Using Image Processing
32/43
3232
Attentional cascade Chain classifiers that are
progressively more complex andhave lower false positive rates:
vsfalsenegdetermined by
% False Pos
%
Detection
0 50
0
100
Receiver operating
characteristic
AdaboostClassifier1
AdaboostClassifier2
AdaboostClassifier3
True True TrueFacefound
Non-face Non-face
Input image
False False FalseNon-face
False positive rate
-
8/3/2019 Face Detection and Recognition Using Image Processing
33/43
3333
Attentional cascade
Detection rate for each stage is 0.99 ,
for 10 stages, overall detection rate is 0.9910 0.9
False positive rate at each stage is 0.3,for 10 stages
false positive rate =0.310 610-6)
AdaboostClassifier1
AdaboostClassifier2
AdaboostClassifier3
True True TrueFacefound
Non-face Non-face
Input image
False False FalseNon-face
-
8/3/2019 Face Detection and Recognition Using Image Processing
34/43
3434
Detection process in practice
Use 24x24 sub-window
Scaling
scale the detection (not the input image)
Features evaluated at scales by factors of 1.25at each level
Location : move detector around the image (1pixel increments)
Final detections
A real face may result in multiple nearbydetections (merge them to become the finalresult)
-
8/3/2019 Face Detection and Recognition Using Image Processing
35/43
Face Recognition
-
8/3/2019 Face Detection and Recognition Using Image Processing
36/43
Face Recognition byHumansPerformed routinely and effortlessly by humans
Enormous interest in automatic processing of digitalimages and videos due to wide availability of powerful
and low-cost desktop embedded computing
Applications:
biometric authentication,surveillance,
human-computer interaction
multimedia management
-
8/3/2019 Face Detection and Recognition Using Image Processing
37/43
-
8/3/2019 Face Detection and Recognition Using Image Processing
38/43
Classification
A face recognition system is expected to identify faces present in images
and videos automatically. It can operate in either or both of two
modes:
Face verification (or authentication): involves a one-to-one matchthat compares a query face image against a template face image
whose identity is being claimed.
Face identification (or recognition): involves one-to-manymatches that compares a query face image against all the templateimages in the database to determine the identity of the query face.
First automatic face recognition system was developed by Kanade 1973.
-
8/3/2019 Face Detection and Recognition Using Image Processing
39/43
Face recognition processing
Face recognition is a visual pattern recognitionproblem.
A face is a three-dimensional object subject to varying
illumination, pose, expression is to be identified basedon its two-dimensional image ( or three- dimensionalimages obtained by laser scan).
A face recognition system generally consists of 4
modules - detection, alignment, feature extraction, andmatching.
Localization and normalization (face detection andalignment) are processing steps before face recognition(facial feature extraction and matching) is performed.
-
8/3/2019 Face Detection and Recognition Using Image Processing
40/43
Face recognition processing
Face detection segments the face areas from thebackground.
In the case of video, the detected faces may need tobe tracked using a face tracking component.
Face alignment is aimed at achieving more accurate
localization and at normalizing faces, whereas facedetection provides coarse estimates of the locationand scale of each face.
-
8/3/2019 Face Detection and Recognition Using Image Processing
41/43
Face recognition processing
Facial components and facial outline are located;based on the location points,
The input face image is normalized in respect togeometrical properties, such as size and pose, usinggeometrical transforms or morphing,
The face is further normalized with respect tophotometrical properties such as illumination andgray scale.
-
8/3/2019 Face Detection and Recognition Using Image Processing
42/43
Face recognition processing
After a face is normalized, feature extractionis performed to provide effective informationthat is useful for distinguishing between facesof different persons and stable with respect tothe geometrical and photometrical variations.
For face matching, the extracted featurevector of the input face is matched againstthose of enrolled faces in the database; itoutputs the identity of the face when a matchis found with sufficient confidence or indicatesan unknown face otherwise.
-
8/3/2019 Face Detection and Recognition Using Image Processing
43/43
Face recognition processing
Click to edit Master text stylesSecond level
Third level Fourth level
Fifth level
Face recognition processing flow.