touchless fingerprint and palmprint recognition...

Image Processing forBiometric ApplicationsRuggero Donida Labati

Touchless Fingerprint and Palmprint Recognition Systems

Academic year 2016/2017

• Touchless fingerprint• Touchless palmprint• Publications

Summary

Touchless fingerprint

Traditional biometric systems

• Low usability and user acceptance:o Complex and highly cooperative acquisition

procedureso Can be perceived as privacy invasive

Improving the user acceptance

• Less-constrained biometrics:- touchless- at higher distances- uncontrolled scenarios- natural light conditions- on the move

Fingerprint biometrics• The most used biometric trait:

o high distinctivityo high permanence

• Touch-based sensors:o low usability and user

acceptanceo images with non-linear

distortions and low contrast regions

o latent fingerprint on the sensor platen

o sensibility to dust and dirt

Touchless fingerprint images

TouchlessTouch

• R. Donida Labati, V. Piuri, F. Scotti, TouchlessFingerprint Biometrics, CRC Press, August, 2015.

Possible applications oftouchless fingerprint biometrics

Touchless fingerprint:state of the art• Single view systems:

o enhancement + traditional recognition methods• 2D multiple view systems:

o mosaicing of three different viewso illuminator shaped as a ring-mirror

• 3D reconstruction:o Multiple viewso Structured lighto Photometric stereoo Depth from focuso Acoustic imaging

• Unwrapping methods:o parametric models (e.g. cylinder, sphere, set of rings)o non-parametric models based on minimization functions

Touchless fingerprint:some existing systems (1/2)

Mosaiking

Structured light

Touchless fingerprint:some existing systems (2/2)

Multiple views

Absorbed light

Fingerprint recognition on the move

Researched recognition techniques and their interoperability

The researchedtouchless recognition systems

• Pros:- less-constrainedo absence of distortions in the fingerprint images due

to different pressures of the finger on the sensoro more robust to dust and dirto more user acceptanceo possibility to use the recognition methods in mobile

devices with standard CCD cameras

• Cons:o longer computational timeo interoperability to be further studied

Two-dimensional samples

Single camera acquisition andframe selection

100 mm200 mm

LensStopStart

Featureextraction

NeuralNetwork

• 45 fetures related to:o shape of the ROIo gradient phase and moduleo quality of the focus estimated from the gradient

imageso ridge frequency (FFT and Gabor filters)o entropy

2D touchless fingerprint:touch-equivalent images

Orientationimage

Frequencyimage

Gaborfilters

Enhancement of the ridge pattern

ROIestimation

Enhancement of the ridge visibility

Enhancement of the ridge pattern

Imagebinarization

Three-dimensional minutia points

3D minutiae reconstruction

Features - Minutiae:

xcoordinate

ycoordinate

Angle Quality

- Local Fingercode: 4×2 el.

- HOG features: 3×3×9 el.

Three-dimensional samples and touch-equivalent images

Touchless 3D fingerprint recognition

R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Towardunconstrained fingerprint recognition: a fully-touchless 3-D system basedon two views on the move", in IEEE Transactions on Systems, Man, andCybernetics: Systems, 2015.

Contactless acquisition (1/4)

Contactless acquisition (2/4)

Camera A Camera B

Contactless acquisition (3/4)

Contactless acquisition (4/4)

Preprocessing

Segmentation

Extraction and matching of the reference points (1/2)

Extraction and matching of the reference points (2/2)

Refinement of the pairs of corresponding points

• Based on Thin Plate Spline

3D surface computationand image wrapping

1. Triangulation

2. Linear interpolation

Computation oftouch-equivalent images

• Enhancement– Background subtraction– Non-linear equalization (logarithm)– Butterworth low-pass filter

• Two-dimensional mapping

Two-dimensional mapping (1/2)

• Enrollment:– Compensate for rotations– Computation of 𝑁𝑁𝑅𝑅 rotations

Two-dimensional mapping (2/2)

Template computation

• Neurotechnology VeriFinger– Commercially available– Designed for touch-based

images

Matching

• Database entry– 𝑁𝑁𝑅𝑅 templates 𝑇𝑇𝑒𝑒– One for each rotation

• Live sample– 1 template 𝑇𝑇𝑓𝑓

Experimental results

• Datasets description• Accuracy of 3D reconstruction• Recognition performance• Robustness to finger misplacements• User acceptability• Interoperability• Overview of different technologies

Datasets description• Touchless - one session

– 2368 samples– 10 fingers, 30 volunteers, 8 acquisitions per finger

• Touchless - two sessions– 2368 samples– 10 fingers, 15 volunteers, 16 acquisitions per finger

• 8 acquisitions one year, 8 acquisition subsequent year• Touchless - misplaced fingers

– 1200 samples– 2 fingers (index), 30 volunteers, 20 acquisitions per finger

• Touch-based– One session– Two sessions

Accuracy of 3D reconstruction

• Average error: 0.03m

Recognition performance (1/2)

• Comparable to touch-based systems– One session

– Two-session

Recognition performance (2/2)

Robustness to finger misplacements

• Genuine and impostor match scores remainwell separated

User acceptability

• Survey performed using questionnaires• Results show preference towards contactless

recognition

Interoperability

• Accuracy level obtained by matching imagescaptured by different devices– Matching touchless with touch-based images

• 2 803 712 identity comparisons• EER = 2.00% with 𝑁𝑁𝑅𝑅 = 25• Less than EERs obtained in the literature with similar

experiments

Aspect Touch-based Touchless

Accuracy EER = 0.03% EER = 0.06%

Scalability High To be further investigated

Interoperability High To be improved

Security Latent fingerprints No latent fingerprints

Privacy Data protection techniques Data protection techniques

Cost 10$ to 5000$ 0$ to 5000$

Usability Medium High

User acceptance Medium High

Speed Template extraction + matching

3D reconstruction + template extraction + matching

Overview of different technologies

Three-dimensional samples andthree-dimensional templates

• Template computationo Minutiae extracted from the texture:

each minutiae is described by (i; x; y; z; θ; q)

o Two-dimensional Delaunay triangulation: each triangle is described by (I; L; θMax; C)

3D feature extraction and matching

• Matchingo Iterative algorithm:

searching of the triangle pairs minutiae alignment minutiae matching matching score computation

o Matching score = Nm/(N × M)

Example of 3D template

Computation of synthetictouchless fingerprint samples

R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Accurate 3D fingerprint virtual environment forbiometric technology evaluations and experiment design", Proc. of the 2013 IEEE InternationalConference on Computational Intelligence and Virtual Environments for Measurement Systems andApplications (CIVEMSA 2013), Milan, Italy, July 15-17, 2013, pp. 43-48.

Example of synthetictouchless fingerprint sample

Contactless three-dimensional reconstruction of acient fingerprints

Acquisition

Calibrationof the acquisition system

Image preprocessing and extraction of the

reference points

Point matching and triangulation

Surface estimation and texture mapping

Level 3 features

Conclusions

• Touchless fingerprint recognition:o systems based on two-dimensional samples can be used in

low-cost applications, but the samples present distortionso systems based on three-dimensional samples can obtain

comparable accuracy with respect to traditional systemso touchless systems are characterized by higher usability,

user acceptance, security, and scalabilityo touchless sysemts are partially compatible with AFIS

Touchless palmprint

Contactless and less-constrained palmprint recognition

• Less-constrained fingerprint recognitionhas been studied in previous workso Comparison methods are standard

and publicly available• Results enabled the study of

palmprint recognition methodso Palmprint features are similar to

fingerprint featureso Similar techniques for

acquisition and processing

Palmprint recognition:Comparison with fingerprints• Pros:

o Low resolutions (< 200 dpi) 500 dpi needed for fingerprints

o Can be acquired in more situations Manual workers, elder people

o User acceptabilityo Multibiometric system

Combination with fingerprint, finger shape, hand shape, etc.

• Cons:o High accuracy features not always

usable (e.g., minutiae)

Palmprint recognition:Taxonomy

Palmprint recognition:Contactless vs contact

• Pros:o Less distortiono No dirto Increased user acceptability

• Cons:o Low contrasto Complex backgroundo Sensible to lightingo Sensible to position

Palmprint recognition:3D vs 2D

• Pros:o Robust to lighting, occlusions,

noiseo Robust to spoofing attackso Invariant to position and distanceo Can use also 2D information

• Cons:o Complex equipmento Can be expensive

Palmprint recognition:State of the art (1/3)

• Contact-based 2D systemso CCD-based

scannero Optical deviceo Flatbed scanner

• Contact-based 3D systemso Structured light

illumination

Palmprint recognition:State of the art (2/3)

• Contactless 2D systems:o Cameraso Smartphoneso Webcams

• Contactless 3D systems:o Laser scanners

Palmprint recognition:State of the art (3/3)

• Recognition algorithms:o Ridge basedo Line basedo Subspace basedo Statisticalo Coding based

Researched methods:Palmprint acquisition systems

Contactless palmprint recognition at a fixed

distance

Fully contactless, less-constrained palmprint recognition with uncontrolled

distance

Fully contactless palmprint recognitionwith uncontrolled distance

Acquisition (1/2)

Image A Image B

• Special acquisition systemo Optimization of optics,

illumination, distances• Less-constrained acquisition

o Fully contactlesso Uncontrolled positiono Relaxed hando Palmprint must be visible

Horizontal orientation Small rotations are tolerated

Acquisition (2/2)

• Uniform illumination• Different setups and wavelengths studied

o Three downlights with white ledso Four blue led bars

3D palm reconstruction andmodel normalization• 3D reconstruction

o Point matching and triangulation Homography Cross-correlation

o Point cloud filteringo Surface estimation

• 3D normalization tocompensate rotationso Plane fitting

Palm is almost flato Trigonometry formulas

3D image registration andtexture enhancement• The model is reprojected on the

image planeo Using calibration informationo Normalized position

Invariant to the acquisition positionand distance

• Texture enhancemento Removal of the skin toneo Enhancement of the details of the palmo Removal of ridges

2D feature extraction and matching• SIFT-based alignment for horizontal

rotationso 3D features are not robust to horizontal

rotationso Extraction and matching of pointso Estimation of rotation and translation

RANSAC algorithm

• SIFT-based 2D feature extraction and matchingo Robust to uncontrolled acquisitions o Extraction and matching of SIFT descriptorso Refinement based on collinearity

3D feature extraction and matching

• Delaunay triangulation to refine the matcheso Similar groups of three points are more robusto Computation of 3D coordinateso Delaunay triangulationo Extraction of similar triangles

Match score

Experimental results:Accuracy of different illumination methods

• Palmprints capturedwith uncontrolleddistanceo 64 palms, 640 sampleso White light

Equal Error Rate= 4.13%

o Blue light Equal Error Rate

= 2.53%

Receiver Operating Characteristic

FMR = False Match RateFNMR = False Non-match Rate

Experimental results:Multiple comparisons

• Considered the bestof 3 comparisonso Maximum match score

• Combination of blueand whiteo Mean match score

EER = 0.08%

Fusionscheme EER (%)

FNMR @FMR

= 0.05%

FNMR @FMR

= 0.10%

FNMR @FMR

= 0.25%

FMR@FNMR= 0.10%

FMR@FNMR= 0.25%

Mean 0.08 0.10 0.09 0.07 0.06 0.00

Receiver Operating Characteristic

Experimental results:Robustness to hand orientation

• Hand positioned with differentroll orientationso Good tolerance

Experimental results:Robustness to illumination

• Different environmental illuminationso Laboratory acquisition, morning light,

afternoon light, artificial light• Match scores are not affected

Illumination situation

Match scores

Genuine comparisons

Impostor comparisons

Mean Std Mean Std

Laboratory acquisition 3179.8 942.1 3.3 1.8

Morning light 2677.4 941.6 2 0.8

Afternoon light 2748.9 903.2 2 0.8

Artificial light 2770.9 876.8 2 0.8

Experimental results:Evaluation of usability and social acceptance

• Usabilityo Evaluation of the quality of the sampleso Evaluation of the time needed for the acquisitiono Evaluation of users’ opinion

E.g., Is the acquisition comfortable?

• Social acceptanceo Evaluation of users’ opinion

E.g., Are you worried about hygiene issues? E.g., Do you feel that the system attacks your privacy?

Experimental results:Comparison with the literature (1/2)

• Based on the acquisitiono Fully contactless, less-constrained acquisitiono No pegso No dirt, sweat, or latent impressionso Faster acquisition, simpler setupo Less expensive than the methods based on 3D

models• Based on the accuracy

o Better accuracy than the methods based on 3D models and uncontrolled acquisitions

Comparison of methods in the literature

Reference Type of acquisition Device Size of dataset

(palms)EER(%)

Li et al., 2012Contact

2D

CCD-based with pegs 386 0.02

Cappelli et al., 2012 Optical device 160 < 0.01

Wang et al., 2012 Flatbed scanner 384 0.20

Li et al., 2012 Contact3D

CCD-based andprojector, with pegs 100 0.03

Jia et al. 2012 Contactless 2D

Mobile device 200 0.14

Tiwari et al., 2013 Ad-hoc device 602 0.06

Kanhangad et al. 2011 Contactless

3D

Laser scanner 354 0.22

ProposedMethod

Two-viewCCD-based 64 0.08

Publications

Publications (1/3)

• Research books1. R. Donida Labati, V. Piuri, F. Scotti, Touchless Fingerprint Biometrics, CRC Press, 2015. 2. A. Genovese, V. Piuri, F. Scotti, Touchless Palmprint Recognition Systems, S. Jajodia (ed.),

Springer International Publishing, September, 2014.

• Refereed Papers in International Journals3. R. Donida Labati, A. Genovese, E. Muñoz, V. Piuri, F. Scotti, "A novel pore extraction method for

heterogeneous fingerprint images using Convolutional Neural Networks", in Pattern RecognitionLetters, 2017 (to appear).

4. R. Donida Labati, A. Genovese, V. Piuri, F. Scotti, "Toward Unconstrained Fingerprint Recognition: a Fully Touchless 3D System Based on Two Views on the Move", in IEEE Transactions on Systems, Man, and Cybernetics: Systems, pp. 202-219, February, 2016.

5. R. Donida Labati, A. Genovese, V. Piuri, F. Scotti, "Touchless fingerprint biometrics: a survey on 2D and 3D technologies", in Journal of Internet Technology, pp. 325 - 332, May, 2014. ISSN: 1607-9264.

• Chapters in Books6. R. Donida Labati, F. Scotti, "Fingerprint", in Encyclopedia of Cryptography and Security (2nd

ed.), Springer, pp. 460 - 465, 2011.

Publications (2/3)

• Refereed Papers in Proceedings of International Conferences and Workshops

7. R. Donida Labati, A. Genovese, V. Piuri, F. Scotti, "Contactless Fingerprint Recognition: a Neural Approach for Perspective and Rotation Effects Reduction", in IEEE Workshop on Computational Intelligence in Biometrics and Identity Management, April 16 - 19, 2013.

8. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Two-view Contactless Fingerprint Acquisition Systems: a Case Study for Clay Artworks", in 2012 IEEE Workshop on Biometric Measurements and Systems for Security and Medical Applications, September, 2012.

9. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Virtual Environment for 3-D SyntheticFingerprints", in 2012 IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems, pp. 48 - 53, July, 2012.

10. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Quality Measurement of UnwrappedThree-dimensional Fingerprints: a Neural Networks Approach", in 2012 International Joint Conference on Neural Networks, pp. 1123 - 1130, June, 2012.

10. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Fast 3-D Fingertip Reconstruction Using a Single Two-View Structured Light Acquisition", in IEEE Workshop on Biometric Measurements and Systems for Security and Medical Applications (BioMS 2011), pp. 1 - 8, September 28 , 2011.

Publications (3/3)

• Refereed Papers in Proceedings of International Conferences and Workshops (Cont’d)

11. R. Donida Labati, V. Piuri, and F. Scotti, "A neural-based minutiae pair identification method for touchless fingeprint images", in IEEE Symposium Series in Computational Intelligence 2011 (SSCI 2011), April, 2011.

12. R. Donida Labati, A. Genovese, V. Piuri, F. Scotti, "Measurement of the principal singular point in fingerprint images: a neural approach", in 2010 IEEE International Conference on Computational Intelligence for Measurement Systems and Applications (CIMSA), pp. 18 - 23, September 6-8, 2010.

13. R. Donida Labati, V. Piuri, F. Scotti, "Neural-based Quality Measurement of Fingerprint Images in Contactless Biometric Systems", in The 2010 International Joint Conference on Neural Networks (IJCNN), pp. 1 - 8, July 18-23, 2010.