sift flow: dense correspondence across scenes and its ... · sift matching master erasmus mundus of...

FACE DETECTIONLab on Project Based Learning

May 2011

Xin Huang, Mark Ison, Daniel Martínez

Visual Perception

Master Erasmus Mundus of Science in Vision and Robotics (VIBOT) - X.Huang, M.Ison, D.Martínez

Contents

• Introduction

• Exploration into local invariant features

• The final method

• Results

• Conclusions


Introduction• Karlos Arguiñano’s TV show.


Introduction

Which is our goal?


Introduction• Which is our goal? -> Detect presence of a face.


Introduction

Problem definition


Introduction• Problem definition:

• How can we use local invariant feature descriptors to detect Arguinano’s face.

• Use MATLAB programming language.


Introduction• Constraints of the project:

• Use local invariant features descriptors.

• Use MATLAB programming language.


Contents

• Introduction



• Results

• Conclusions


Exploration into local invariant features

First approaches:1. SURF matching.

2. Template database SURF matching.

3. Clustering

4. Filtering best features.

5. SIFT matching


1. SURF matching:

• Speeded up robust features.

• Simplification of SIFT.

• Fast.

• “Combination of novel detection, description, and matching steps.” [ETH Swiss Federal Institute of Technlogy]



1. SURF matching



1. SURF matchingResult with a rigid feature:



1. SURF matchingResult (no more fancy glasses):






3. Clustering


5. SIFT matching.




Test imageClassifier

(NN)Output image

with matchings

FACE/NON FACE templates

(SURF descriptors)


2. Template database SURF matching


+


2. Template database SURF matching Result A



2. Template database SURF matching Result B






3. Clustering


5. SIFT matching.


3. Clustering:




(NN)Output image

with matchings

FACE/NON FACE Clusters of SURF

descriptors

Clustering!

3. Clustering:

Ideal Reality






3. Clustering


5. SIFT matching.


4. Filtering best features:

• Find the extreme ratios.

• Two separate databases.



4. Filtering best features:

Blue BAD, Orange GOOD



SURF FAILS



SURF FAILS



But...WHY?

SURF FAILS



[REF] [A Comparison of SIFT, PCA-SIFT and SURF - Luo Juan, Oubong Gwun- International Journal of Image Processing-2010]




3. Clustering


5. SIFT matching.


5. SIFT Matching:




(NN)Output image

with matchings

FACE/NON FACE Clusters of SIFT

descriptors SIFT!

5. SIFT Matching:

• Bringing in SIFT.



5. SIFT Matching:

• We should do anything else...



Contents

• Introduction



• Results

• Conclusions


1. Skin detection:

• Help SIFT as much as possible.

• Make the ROI of skin color regions.

• Methodology:

• Extract skin segments: find the mean and covariance between CrCb.


The final method

1. Skin detection:

• Methodology:

• Obtain the Gaussian probabilities of skin color pixels for the input image.


The final method

2. Skin detection:

• Methodology:

• Apply an adaptive threshold.


The final method

2. Skin detection:

• Methodology:

• Avoid region without holes and apply morphological operation over the binary image.


The final method

3. Face detection:

• Filtering best features.

• Finding matches between each skin region and each template.


The final method

• D.Lowe

3. Face detection:

• Check if each skin region is a face.

• Threshold = Face matches / No facematches


The final method

Contents

• Introduction



• Results

• Conclusions



Resultsans =

TestImage_num: 1Regioin_num: 1Keypoint_num: 1135

TotalFaceMatch_num: 99TotalNoFaceMatch_num: 14

TemplateMatchNum: [2x37 double]

ans =

TestImage_num: 1Regioin_num: 2Keypoint_num: 74

TotalFaceMatch_num: 3TotalNoFaceMatch_num: 2

TemplateMatchNum: [2x37 double]


Results

Contents

• Introduction



• Results

• Conclusions


Conclusions• We have implement a face-detection method based on feature descriptors with a lot of help.

• SIFT over SURF

• QUALITY OF THE FEATURES over QUANTITY


Conclusions•How can be improved?

• Use shape characteristics.

• A larger template database and a more robust classifier.

• Use morphology advantages.

• Apply other methods instead of feature description (i.e: HAAR).

• Use previous frames information.


sift flow: dense correspondence across scenes and its ... · sift matching master erasmus mundus of...

Documents