phd scholar, image analysis and biometrics …anushs/papers/yearly...phd scholar, image analysis and...

Anush Sankaran PhD Scholar, Image Analysis and Biometrics Indraprastha Institute of Information Technology, Delhi

Advisors:

Dr. Mayank Vatsa Dr. Richa Singh

1 15-02-2012

Biometrics Biometrics are automated methods of

recognizing a person based on a physiological or behavioral characteristic.

An Introduction to Biometric Recognition, Anil K Jain, Arun Ross, Salil Prabhakar - 2004 - IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 2

Fingerprint

Inked Live-scan Latent print

One of the most common biometric modality. Different types based on the mode of capture:

Handbook of Fingerprint Recognition By Davide Maltoni, Dario Maio, Anil K. Jain, Salil Prabhakar 3

Latent print

Inked Live-scan

4

Latent Fingerprint Lifted from surfaces from objects that are

touched or handled by a person.

Impressions are left behind by sweat and natural secretions on the skin surface.

Incidents

5

Commonwealth v Patterson case • Murder of a Boston police detective. • Four latent prints were collected from the detectives

vehicle. • None of the latent impressions in the cluster had

sufficient quality or quantity of similar detail to conclude individualization.

Commonwealth v Patterson, 445 Mass. 626; 840 N.E.2d 12 (2005).

Shirley McKie fingerprint scandal • Police detective fingerprint found in bathroom door of

crime scene. • Error in fingerprint comparison. • Police detective claimed of never visiting the crime

scene. • Released after trial.

Applications

http://www.sciencephoto.com/image/222063/530wm/H2000438-Forensics_officer_lifting_fingerprints-SPL.jpg http://www.drakeinvestigations.com/Portals/56/Fingerprints%20for%20Forensic%20Section.jpg

Personal Identification

• Forensic sciences

• Crime scene evidence

6

http://www.sciencephoto.com/image/222063/530wm/H2000438-Forensics_officer_lifting_fingerprints-SPL.jpg�





http://www.drakeinvestigations.com/Portals/56/Fingerprints for Forensic Section.jpg�

Automated Latent Fingerprint Identification System Latent fingerprint matching is mostly performed

by experts (humans)

Latent Fingerprint

Lifting Enhancement

Segmentation

Quality assessment

Feature Extraction Classification

Figure: Essential components of an automated Latent Fingerprint System

Preprocessing

7

Latent Fingerprint Lifting

Lennard C., “The detection and enhancement of latent fingerprints”. 13th INTERPOL Forensic Science Symposium, Lyon, France. D2-86-D2-98, 2001 Lee H.C. and Gaensslen R.E., “Methods of Latent Fingerprint Development, in Advances in Fingerprint Technology”, second edition, CRC Press, 2001, 105-175.

Lifting

Contact Lifting

Non-contact Lifting

• Dusting • Magna Brush • Fuming • Chemicals • Wax, Rubber, tape lifting

• 3D sensors • Optical methods • Tracers

8

9

Latent Fingerprint Enhancement

1. Feng J. and Jain A. K., “Filtering large fingerprint database for latent matching”. In Proceedings of International Conference on Pattern Recognition, pages 1–4, 2008.2. Yoon S., Feng J., and Jain A. K., “On latent fingerprint enhancement”. In Proceedings of SPIE Biometric Technology for Human Identification, 2010

• Latent fingerprint

• Region of interest

• Singular points

Orientation field regularization

Orientation field estimation

Gabor Filtering

Enhanced fingerprint

10 10 10

Latent Fingerprint Quality Assessment

NBIS – NIST Biometric Imaging Software http://fingerprint.nist.gov/NFIS/

Nill N.B. "IQF (Image Quality of Fingerprint) software application", MTR 070053, MITRE,2007. Available on: http://www.mitre.org/tech/mtf/.

No specialized quality assessment tool for latent fingerprints.

• Some existing algorithms that are used are: • NFIQ (NIST Fingerprint Image Quality): It gives a score range of 1 -5, with 5 being the best quality.

• IQF (Image Quality of Fingerprint): It gives a score range of 0 -100, with 100 being the best quality • Open research problem.

http://fingerprint.nist.gov/NFIS/�

11 11

Level 1 : Level2 : Level 3 :

(a) Arch (b) Tented Arch (c) Left Loop (d) Right Loop (e) Twin Loop (f) Whorl

Pattern Type Minutiae Pores, Dots, Incipients etc..

(a) Ridge bifurcation (b) Ridge ending Pores, Dots, incipients, ridge flow map

Ridge flow information

Fingerprint features

Minimum of 300 ppi images Minimum of 500 ppi images Minimum of 1000 ppi images

12 12

Latent Fingerprint Feature Extratcion

1. Paulino A., Jain A.K., and Feng J., “Latent Fingerprint Matching: Fusion of Manually Marked and Derived Minutiae,” In Proceedings of the 2010 23rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI '10). 2. Jain A.K. and Feng J., “Latent fingerprint matching”. IEEE Transactions on PAMI, 33(1):88–100, 2011.

(a) gray- scale image, (b) minutiae, (c) singular points , (d) ridge quality, (e) ridge flow map, (f) ridge wavelength map, (g) skeletonized image, and (h) dots and incipient ridges.

• Reference points • Minutiae points • Level 3 features • Extended features

• Local matching • Global matching

13 13

Latent Fingerprint Matching

Jain A. K., Nagar A. and Nandakumar K., “Latent fingerprint matching”. Technical Report MSU-CSE-07-203, Michigan State University, 2007 Jain A. K., Feng J., Nagar A. and Nandakumar K., “On matching latent fingerprints”. In Proceedings of Computer Vision and Pattern Recognition Workshop, pages 1–8, 2008.

• Minutiae points • Ridge flow features

• Local matching • Global matching

Weighted summation

Rank of match

Features Feature

matching Match Score computation

Decision

Challenges • Lifting fingerprint needs

experts.

• Presence of partial fingerprints.

• Fingerprints are

smudged.

• Presence of background noise.

• Presence of non-linear

distortion in ridge flow.

Database: NIST Special Database 27a. http://www.nist.gov/itl/iad/ig/sd27a.cfm 14

http://www.nist.gov/itl/iad/ig/sd27a.cfm�

Latent to Latent Fingerprint Matching

IIIT- D Latent Database Multi-Latent Database

15

• Automated matching of Latent to Latent fingerprint is an interesting new problem. • Missing information.

• Crime scene linking. • For grouping criminal activities over several places in the same crime scene or multiple crime scenes.

IIIT- D Latent Database Multi-Latent Database

16

Latent to Latent Fingerprint Matching

A Case Study

• Swiss AFIS system first tried to automate the process.

• An important series of burglaries linked through shoe-marks has first been considered.

• Only 11 persons are identified based on the same finger (1.3 % accuracy).

A. Anthonioz, A. Aguzzi, A. Girod, N. Egli, and O. Ribaux, Potential Use of Fingerprint in Forensic Intelligence: Crime Scene Linking, Z Zagadnien Nauk Sadowych - Problems of Forensic Sciences. 51 (2002) 166-170.

17

“Do they mean that trace-to-trace comparisons are futile, or would a

limitation to criminal phenomena which are known to be serial, burglaries for

instance, be a more relevant approach?”

Latent to Latent Fingerprint matching

Methodology

18

Probe set

Pool of existing

algorithms

Latent database

Result

State-of-art algorithms

• Three state-of-art systems that are used

Algorithm Author Availability Property

NBIS (NIST Biometric Image

Software)

NIST (National Institute of

Standards and Technology)

Open Source Minutiae based algorithm

VeriFinger SDK for Microsoft

Visual C# Neurotechnology Commercial Minutiae based

algorithm

FingerCode Luigi Rosa Open Source Ridge flow based

algorithm

19

20

Information IIIT-D Latent Database Multi-Latent Database

Author IAB Lab, IIIT-Delhi CDEFFS (Committee to

Define an Extended Fingerprint Feature Set )

Number of Subjects 15 4

Number of classes 150 40

Number of instance per class Varying (4 - 18) Varying (3 -5)

Total number of images 1046 166

Property

• Varying background (card / tile) • Quality variation based on dryness of hand.

• Extreme quality variation based on lifting process.

Database formed

Acknowledgement: Tanmay Verma, Final year B.Tech. Student, IIIT-Delhi collected the database while working as RA with IAB Lab.

Experimental Protocol

Information IIIT-D Latent Database Multi-Latent Database

Segmentation Manual Manual

Image Resolution 512 * 512 512 * 512

Split type

30 % gallery (approx.) with condition that at least one image per class must be in

gallery

1 per class in gallery, remaining in probe.

Gallery size 395 images 40 images

SVM train 131 images 26 images

Testing 520 images 100 images

Note: Performed 10 times random cross validation for all results.

21

Results

Experiment Multi Latent Database (%)

Standard deviation

NBIS 29.7 6.2 %

VeriFinger 41.9 6.7 %

FingerCode 38.0 9.5 %

Experiment IIIT-D Latent Database (%)

Standard deviation

NBIS 58.9 2. 4 %

VeriFinger 74.0 1. 8 %

FingerCode 35.4 2. 5 %

Table: Rank 10 Identification Accuracy

22

Observation

Figure: IIIT- D Latent Database Sample Images

Figure: Multi Latent Database Sample Images

NBIS (16%) VeriFinger (13.5 %) FingerCode (7 %)

NBIS (7 %) VeriFinger (11 %) FingerCode (3 %)

23

Fusion

Two different frameworks

• Decision level fusion – Or Fusion

• Match score level fusion – PLR (Product of Likelihood ratio) fusion 24

Results of OR Fusion


Standard Deviation

NBIS 29.7 1. 8 %



Decision Fusion 70.9 2. 4 %


Standard Deviation

NBIS 58.9 1. 8 %



Decision Fusion 77.7 2. 4 %

Table: Rank 10 Identification Accuracy 25

Results of Match Score Fusion

Experiment

Multi Latent Database (%)

Standard deviation

NBIS 29.7 6.2 %



Decision Fusion

70.9 2.0 %

PLR Fusion 42.1 7.8 %

IIIT-D Latent Database (%)

Standard deviation

58.9 2. 4 %

74.0 1. 8 %

35.4 2. 5 %

77.7 3.0 %

55.8 5.3 %


26

• Product of Likelihood Ratio (PLR) based match score fusion.

Observation … 2

Figure: Only Minutiae based algorithm works (26 %)

Figure: Only Ridge flow based algorithm works (9 %) 27

Context Switching

Two different frameworks

• Quality based context switching

• Number of minutiae based context switching 28

Results


Standard deviation

NBIS 29.7 6.2 %



Decision Fusion 70.9 2.0 %


Context Switching (Quality)

29.3 1.8 %

Context Switching (Number of Minutiae)

48.2 2.0 %


29

Sankaran, A., Dhamecha, T. I., Vatsa, M., & Singh, R. (2011). On Matching Latent to Latent Fingerprints. International Joint Conference on Biometrics (accepted).

Results


Standard deviation

NBIS 58.9 2. 4 %


FingerCode 35.4 2. 5 %

Decision Fusion 77.7 3.0 %


Context Switching (Quality) 40.4 2.3 %

Context Switching (Number of Minutiae)

58.7 3.3 %


30

Sankaran, A., Dhamecha, T. I., Vatsa, M., & Singh, R. (2011). On Matching Latent to Latent Fingerprints. International Joint Conference on Biometrics (accepted).

Results … IIIT-D Latent Database

31

Results … Multi Latent Database

32

Conclusion

• Latent to latent fingerprint is an important research problem that requires comprehensive research.

• Though context switching is a good option to achieve better accuracy with lesser time complexity, both image quality and number of minutiae points are not suitable parameters to switch among the classifiers

33

Future directions

• To develop a “Lights-out” latent fingerprint identification system. • Validity and quality assessment of latent fingerprints.

34

Manual Gabor filter enhancement

11 tuple feature

Polynomial function

Academics (2010-2011)

Monsoon Semester ( GPA – 9.67 / 10 ) • Introduction to Biometrics • Probability and Statistics • Image Analysis

• TA – Probability and Statistics

• Winter Semester ( GPA – 10 / 10 ) • Pattern Recognition • Computer Vision • Machine Learning • Technical Communication (Audit)

• TA – Data Structures and Algorithms

• CGPA – 9.83 / 10 35

Other Projects

• On predicting missing minutiae in fingerprint.

• Tom without Jerry.

• Iris recognition without unwrapping, using SURF features.

• On choosing the classifier for Latent fingerprint matching.

A B

36

Accomplishments

• TCS Research Scholar

• Chairman, ACM Student Chapter

Publications

• Sankaran, A., Dhamecha, T. I., Vatsa, M., & Singh, R. (2011). On Matching Latent to Latent Fingerprints. International Joint Conference on Biometrics (accepted).

• Dhamecha, T. I., Sankaran, A., Singh, R., & Vatsa, M. (2011). Is Gender Classification Across Ethnicity Feasible using Discriminant Functions. International Joint Conference on Biometrics (accepted).

37

Conference and Internship

• Attended Indian Conference on Vision, Graphics and Image Processing at IIT, Chennai during December 2010.

PhD Collaborative Program

• Venue: HongKong Polytechnic University

• Collaborator: Dr. Ajay Kumar

• During: June – August 2011

• Project: 10 Print Fingerprint Matching Using Descriptor Level Fusion Technique

38

Back up slides

40

IIIT-D Latent Database

• The database contains 15 subjects. Each subject has all 10 fingerprints captured.

• The impressions are lifted using brush and black powder dusting process.

• The fingerprints are lifted against two different backgrounds – card and tile.

• The fingerprints are captures at semi- controlled environments at varying levels of dryness of the skin. This provides variation in the fiction ridge information of the impression.

41

IIIT-D Latent Database … 2

• The latent fingerprint images are then captured using a Canon EOS 500D camera at a resolution of 4752 × 3168.

• All images are compressed and stored in .jpg format.

42

43 43 43

Latent Fingerprint Segmentation

Karimi-Ashtiani, S.; Kuo, C.-C.J.; , "A robust technique for latent fingerprint image segmentation and enhancement," Image Processing, 2008. ICIP 2008. 15th IEEE International Conference on , vol., no., pp.1492-1495, 12-15 Oct. 2008

• Local window based approach.

Four major steps: Contrast enhancement

• Projection • Orientation frequency estimation • Distance estimation

44

Latent Fingerprint Enhancement

1. Feng J. and Jain A. K., “Filtering large fingerprint database for latent matching”. In Proceedings of International Conference on Pattern Recognition, pages 1–4, 2008.2. Yoon S., Feng J., and Jain A. K., “On latent fingerprint enhancement”. In Proceedings of SPIE Biometric Technology for Human Identification, 2010

No. Algorithm Database Result

1.

Multistage filtering using • Pattern types • Singular Points • Orientation field

258 Latent fingerprints against 10258 Rolled fingerprints

• Rank-1 accuracy of 73.3 % • Matching speed increased three fold.

2.

• Manual ROI and singular point. • Novel orientation field – fits the coarse field obtained by commercial fingerprint SDK.

258 Latent fingerprints

Improved identification accuracy,

• Latent fingerprint

• ROI

• Singular points

45 45 45

Latent Fingerprint Quality

NBIS – NIST Biometric Imaging Software http://fingerprint.nist.gov/NFIS/

Nill N.B. "IQF (Image Quality of Fingerprint) software application", MTR 070053, MITRE,2007. Available on: http://www.mitre.org/tech/mtf/.

Open research problem

• Some existing algorithms that are used are: • NFIQ (NIST Fingerprint Image Quality): It gives a score range of 1 -5, with 5 being the best quality.

• IQF (Image Quality of Fingerprint)

http://fingerprint.nist.gov/NFIS/�

46 46

Latent Fingerprint Feature Extraction Level 1 : Level2 : Level 3 :

(a) Arch (b) Tented Arch (c) Left Loop (d) Right Loop (e) Twin Loop (f) Whorl

Pattern Type Minutiae Pores, Dots, Incipients etc..

(a) Ridge bifurcation (b) Ridge ending Pores, Dots, incipients, ridge flow map

Ridge flow information

47 47

Latent Fingerprint Feature

1. Paulino A., Jain A.K., and Feng J., “Latent Fingerprint Matching: Fusion of Manually Marked and Derived Minutiae,” In Proceedings of the 2010 23rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI '10). 2. Jain A.K. and Feng J., “Latent fingerprint matching”. IEEE Transactions on PAMI, 33(1):88–100, 2011.


1.

• Fusion of manually marked and automated generated minutiae • Different fusion techniques

258 Latent fingerprints

• Enhanced Rank-1 accuracy. • Corrected previous misclassification

2.

• Use Level 3 features. • 1000 ppi images. • Minutiae, singularity, ridge quality map, ridge wavelength map, ridge flow map and skeleton

258 Latent fingerprints against 29,257 rolled fingerprints

Rank-1 accuracy improved from 34.9 % to 74 %.

48 48

Latent Fingerprint Matching

Jain A. K., Nagar A. and Nandakumar K., “Latent fingerprint matching”. Technical Report MSU-CSE-07-203, Michigan State University, 2007 Jain A. K., Feng J., Nagar A. and Nandakumar K., “On matching latent fingerprints”. In Proceedings of Computer Vision and Pattern Recognition Workshop, pages 1–8, 2008.


1.

• Minutiae as well as ridge information. • Latent fingerprint against full fingerprint

258 Latent fingerprints against

• Enhanced Rank-1 accuracy. • Corrected previous misclassification

2.

• Latent fingerprints against rolled fingerprints

258 Latent fingerprints against 29,257 rolled fingerprints

Rank-1 accuracy improved from 34.9 % to 74 %.

Support Vector Machines (SVM)

Class 1

Class 2

Class 1

Class 2

Class 1

Class 2

49

Likelihood: The likelihood of a hypothesis (H) after doing an experiment or gathering data (D) is the probability of the data given the hypothesis.

L(H|D) = P(D|H)

* D is K dimensional corresponding to K different matchers

Likelihood Ratio: “Within the framework of a statistical model, a particular set of data supports one statistical hypothesis better than another if the likelihood of the first hypothesis on the data exceeds the likelihood of the second hypothesis”

Likelihood Ratio = P ( D | H1 ) P ( D | H2 )

Match Score Fusion

• Product of Likelihood Ratio (PLR) based fusion.

Product of Likelihood Ratio: Assume K independent matchers, the product of likelihood ratio provides K 2-dimensional densities for genuine and imposter classes. Product Likelihood Ratio = Пk= 1 to KP(D|H1 or 2 )

50

phd scholar, image analysis and biometrics …anushs/papers/yearly...phd scholar, image analysis and...

Documents