automatic segmentation of latent...

Anil K. Jain and Kai Cao Michigan State University

Project # 12S-05W-12

Automatic Segmentation of Latent

Fingerprints

Fingerprint Types

Rolled fingerprint Plain fingerprint latent fingerprint

AFIS achieved a rank-1 identification rate of 99.4% (NIST FpVTE 2003)

Latent matcher achieved a rank-1 identification rate of 63.4% (NIST ELFT 2012)

2

Challenges in Latent Matching

Unclear ridges Partial fingerprint

Large distortion Complex background

Reliable

feature

extraction

Robust

feature

matching

3

Goals of Latent Segmentation (Cropping, Region of Interest, Foreground)

• Develop an automatic algorithm to separate friction ridge

pattern from background

• Define a ridge quality measure

• Enhance friction ridge structure in foreground

• Provide a confidence value for segmentation

4 (a) A latent from

NIST SD27

(b) Segmented &

enhanced image of (a) (c) Mate rolled print

Algorithm

Input latent

image

Fine estimation

Segmentation and

enhancement result

Confidence

level (CL)

estimation

CL ≤ TH Reject

Texture

extraction

0 ≤CL ≤1

CL > TH

Dictionary

learning

Dictionary

learning

(0, π/16]

(π/16, 2π/16]

(15π/16, π]

Patch size:

32×32

Patch size:

64×64

Quality map

Frequency field

Orientation field

Quality map

Frequency field

Orientation field

Coarse estimation

Cao, Liu and Jain, Segmentation and Enhancement of Latent Fingerprints: A Coarse to Fine Ridge Structure Dictionary, PAMI, 2014 5

Latent Fingerprint Decomposition

• local total variation (LTV)

• relative reduction rate of LTV

A. Buades, T. Le, J.-M. Morel, and L. Vese, Fast cartoon + texture image filters, IEEE TIP, 19(8):1978 –1986, 2010.

( ) *| | ( ),LTV f L f x

Background patch Fingerprint patch

Separate friction ridge texture from background

= +

Texture part Cartoon part Gray image

• Cartoon part and texture part

Cartoon part

Texture part

Latent Fingerprint Decomposition

Training Set

Rolled prints

from NIST 4

80K 32×32

patches

100K 64×64

patches

Dictionary

learning

1 coarse-level

dictionary

(1,024 elements)

16 fine-level

dictionaries

(64 elements)

Provides coarse-

level quality map

with orientation and

frequency fields

Provides fine-level quality

map with orientation and

frequency fields

Friction Ridge Dictionary Learning

8

(0, π/16]

(π/16, 2π/16]

(15π/16, π]

• Fingerprint selection: NFIQ index <= 2

• Patch selection: average quality >=3.75

(a) A subset of elements on the coarse-

level dictionary (patch size: 64×64). The

total number of dictionary element is 1,024

(b) A subset of elements in the 16 orientation

specific fine-level dictionaries (patch size:

32×32). The total number of elements in

each orientation specific dictionary is 64.

Dictionary Elements

9

Patch Reconstruction: Ridge Quality

• A given latent patch is reconstructed with T

dictionary elements

• Structure similarity (SSIM) measures the “ridge

quality” of a latent patch

(a) (b) (c) (d) (e)

SSIM=0.52 SSIM=0.59 SSIM=0.63 SSIM=0.65



Quality estimation Wang et al., Image Quality Assessment: From Error Visibility to Structural Similarity, IEEE TIP, 13(4): 600-612, 2004.

1 2

2 2 2 2

1 2

2( , )

x y x y

x y x y

C CSSIM x y

C C

T=1 T=2 T=3 T=4

10

Patch Reconstruction: Ridge Flow

• Orientation and frequency fields are estimated

from the reconstructed patch Orientation field estimation

T=1 T=2 T=3 T=4

Instead of extracting level-1 features in the input latent patches,

we extract them in the reconstructed patches 11

Future Work

• Improve ridge quality in dictionary elements

– Use enhanced training set

– Post-process dictionary elements

• Better definition of ridge quality; robust to

– “Dry” fingerprints (broken ridges)

– Linear structures in the background

• Improve confidence value measure

12

Coarse to Fine Strategy

• Why do we need coarse to fine strategy?

32 x 32 patch Most similar

dictionary element 64 x 64 Patch Most similar

dictionary element

Latent Coarse-level quality Fine-level quality

13

Coarse to Fine Strategy

• How does coarse to fine strategy work?

64 x 64

patch

Most similar

dictionary

element

32 x 32

patch

Most similar

dictionary element

in selected fine-

level dictionaries

(0, π/16]

(π/16, 2π/16]

(15π/16, π]

16 fine-level dictionaries

(4π/16, 5π/16]

• Quality map: average of coarse-level and fine-level quality maps

• Ridge flow : fine-level ridge flow

• Frequency field: average of coarse-level and fine-level frequency fields 14

Demo

Texture part

+

Cartoon part

=

Gray image

• Feature: local total variation

• Method: nonlinear filter decomposition

15

Texture part

……

d1 d2 d3 d4 d5 x

Coarse-level dictionary Patch

64 ×64

Coarse-level estimation

16

Texture part

Coarse quality map Coarse orientation and

frequency fields

17

Texture part

Coarse quality map Coarse orientation and

frequency fields

……

d1 d2 d3 d4 d5 x

Specific fine-level dictionary Patch

32 ×32

Fine-level

dictionary selection

Fine-level estimation 18

Coarse orientation and

frequency fields

Texture part

Coarse quality map

Fine quality map Fine orientation and

frequency fields

Segmentation result Enhancement result

19

Results on NIST SD27 Latents

Good

latent

Bad

latent

Ugly

latent

(a) Gray image (b) Texture image (c) Coarse-level quality (d) Fine-level quality 20

Results on NIST SD27 Latents

Good

latent

Bad

latent

Ugly

latent

(a) Gray image (b) Texture image (c) Segmentation result (d) Segmentation and

enhancement result 21

Matching Performance Evaluation • Latent Database: 258 latents in NIST SD27 and 449 latents in WVU DB

• Background Database : ~32K total; rolled prints in NIST SD27 (258), WVU

DB (4,739) and NIST SD14 (27,000)

• Input to COTS : (i) latent image, (ii) segmented & ehnanced latent

(a) NIST SD27 (b) WVU DB 22

Examples

(a) A latent from NIST SD27 (b) Segmented & enhanced

23

(c) Mated rolled print

Mate found at rank 1 Mate found at rank 5

Examples

(a) A latent from WVU DB

(b) Segmented and enhanced

24

(c) Mated rolled print

Mate found at rank 1 Mate found at rank 31,000

• Confidence value:

– mean quality value in the segmented foreground

– Latent is rejected, if confidence in segmentation is low

Confidence Value Evaluation

(a) NIST SD27 (b) WVU DB 25

Contributions

• A fridge dictionary based segmentation

and enhancement algorithm for latents

• Ridge quality definition

• Coarse to fine strategy to balance

accuracy vs. robustness

Cao, Liu and Jain, Segmentation and Enhancement of Latent Fingerprints: A Coarse to Fine Ridge Structure Dictionary, IEEE Trans. PAMI, 2014 (to appear)

26

Thanks!

automatic segmentation of latent...

Documents