facial recognition 2

7/31/2019 Facial Recognition 2

http://slidepdf.com/reader/full/facial-recognition-2 1/24



3

Introduction

For obvious reasons, the human brain and visual organs can be considered the best existing

ace recognition machine – ever – and orever. A specic area o the human brain called usiorm ace

area (FFA) has been proven to be totally dedicated to this task.

Because ace recognition is the most natural thing or any human being, acial recognition would

appear to be the most natural o biometric techniques. Human ace recognition is the most widely

used way o identication or authentication o identity, and accounts or the presence o portraits on

most o the identication documents that we carry in our purses and wallets, be it an ID card, driver

license, credit card, library card or gym club card.This apparent ease o use is the cause or many antasies, inconsistencies and diculties while

implementing automatic ace recognition systems in the eld. Because the technology has an

outstanding competitor : the human brain, trained rom birth to somehow do the exact same thing

There are many applications. Automatic acial recognition is actually used in civil spheres in order to

guarantee the unique nature o identity documents and in military or law enorcement applications,

since the human ace easily leaves “traces” when crimes are recorded by CCTV cameras or the

cameras o witnesses.

The acquisition o portrait images is simple, contactless and does not require any highly specic

equipment; a act that acilitates the implementation o automatic acial recognition. Rapid advances

are being made in acial recognition technology, and it thus has every trait expected o a majorbiometric technique, up to the point where it is on the verge o taking on its greatest challenge: beat

the human brain, best ace recognition machine ever, but not orever. Still, in order to deliver its ull

eciency, a number o caveats have to be taken into account during implementation.

In an eort to answer the increasing number o questions being put orward, Morpho has taken

stock o acial recognition in general, its use, the state o the art o the technology and its technical

and commercial potential.



4

A brie background

The origins

From the very advent o photography, both government

agencies and private organizations have kept collections

o portraits and ID photos have gradually made their

way onto all personal identication documents, rom

the most ocial passports to inormal membership

cards issued by sports clubs.

Beore the use o computers to recognize aces was

even considered a possibility, acial recognition was

already the subject o a great deal o research. Examples

include:

• the development o identication parade or “line-

up”(1) techniques in the United Kingdom, in which a

witness is conronted with a group o physically similar

people, one o whom is a suspect. The witness must

decide whether one o the persons in the group was

present at the scene o the crime.

• the work done by Bertillon on ace classication. In

order to recognize delinquents who are repeatedly

arrested, without having to resort to large collections

o portraits, Bertillon suggested that the portraits be

sorted by common morphological characteristics, i.e.

the specic shapes o the dierent parts o the ace.

This classication is known as the “spoken portrait”.

Facial recognition with goodquality portraits

The rst attempts to automate acial recognition started

in the 1960s in semi-automatic mode. They essentially

consisted in checking the coherence o measurements

between dierent characteristic points o the ace (e.g.

the corners o the eyes, the hairline, etc.). They were not

very successul, because aces are by nature very mobileand measurements between characteristic points are

aected by orientation, to the extent that specially-

developed models quickly proved to be necessary.

At the end o the 1980s, the development o the

eigenaces(2) technique prompted a more intense

research eort. This technique is used to nd a ace in

a photo and to compare images o aces. Researchers

quickly ound that the overall issue o acial recognition

was complex, but could be simplied by only taking

into consideration portraits that are coherent in terms

o orientation, lighting, expression and image quality.

Research ocused on this problem, the ICAO* dened

criteria to obtain controlled portraits and meaningul

test sets were created.

At the start o 2007, the NIST* published the results o

its “FRVT 2006”* test. Its conclusions were quite clear.

Research had reached a point where the operational use

o acial recognition on high-resolution rontal images

taken in a controlled environment was now easible. But

this event obviously did not put an end to work on the

recognition o controlled portraits. More improvements

are expected, but acial recognition has thus become a

biometric technique in its own right.

(1) Alphonse Bertillon, 1853-1914, criminologist who developed

judicial anthropometry in France.

(2) Eigenaces: a acial recognition technique that consists in learning

the distinctive characteristics o aces rom a broad sample o

portraits using each complete image rather than local characteristics

(e.g. the eyes, nose or mouth).

Figure 1: Portrait parlé “class”. Source Library o Congress, USA.



5

General facial recognition

Since 2007, research has been looking into signicantly

more dicult problems, in which aces are not viewed

rontally, resolution is low or the image quality is

sometimes poor. With the MBGC*, the NIST is again

seeking to assess perormance and has provided

researchers with representative data (images and videos

o aces under non-controlled conditions). It is thestart o a new era and we can expect to see signicant

progress over the coming years.

Figure 2: Eigenaces, courtesy o Santiago Serrano Drexel

University, USA.

Figure 3: Facial recognition history.



The Applications

Automatic acial recognition is a orm o biometrics. It is used or authentication

(checking that a person really is who they say they are) and identication (nding

out who someone is rom a group o known persons).

Like most biometric techniques, acial recognition has applications in the policing

and civil elds and or access control. Facial recognition is special due to the portraits

themselves, which are widely available and easy to acquire. Their use is acceptable

to the general public.

Criminal Justice

Identifcation and maintenance o aportrait reerence databaseJust like automatic ingerprint recognition, acial

recognition allows police orces to manage the les o

people o interest by making sure that there are not

several dierent records or a single person. While

this task is already perormed using ngerprints, acial

recognition provides more benets:

• it increases population coverage o the identication

scheme, enabling identication o individuals whose

ngerprints cannot be acquired or various reasons

• by combining the two biometric modalities, superior

identication perormance can be achieved, thereore

reducing the workload involved in the verication

process

The Pierce County Sheri’s Oce in Washington, USA,

demonstrated the high precision o the automatic acial

identication o suspects and that identication is

possible without calling on ngerprint experts.

Identity checks in the feldWith just a camera and suitable means o transmission,

it is possible to check the identity o a person in the

eld using a photograph o their ace. Police ocers

equipped with PDAs can submit search requests to

remote acial recognition systems and quickly determine

whether an individual is already known to the orces o

law and order.

ID checks can be carried out on just the ace or both the

ngerprints and the ace, i the ocer has the equipment

required to take ngerprints. The combination o the

two biometric techniques increases the precision o

searches and allows reliable, automatic decisions to

be sent to the eld, without the ocer requiring any

expertise in ngerprints.

6

Figure 4: mobile

acial recognition.



Criminal investigations and inormationImages are oten made available or inquiries. They

may come rom surveillance videos, a witness’s camera,

Internet sites or copies o identity papers. These images

may show the ace o a suspect.

To begin with, the portraits must be extracted rom the

available evidence. In some investigations, hundreds o

hours o video ootage are analyzed and the “manual”search or excerpts in which aces are visible is a long

and painstaking job. It is the reason why automatic

assistance is necessary. Current automatic ace extraction

techniques work well with almost ull rontal views o

aces and when the quality o the video is good enough.

Research is currently being made into the extraction o

side and three-quarter views o aces.

Even i the quality o the extracted portraits is highly

variable, it is still possible to compare them with portraits

o persons who are known to the police. Morpho’s

experience in this eld shows that these searches can

already solve and correlate crimes. Our French, American

and Australian customers have scored numerous hits

with high-quality images, such as authentic or alse ID

documents or images posted on the Internet. It is also

interesting to note that certain criminal cases have been

solved using low quality images.

Operational examples o the use o surveillance videos

do exist, but they are rarer. By way o example, images

o raudulent use o ATMs or assaults close to an ATM

can be used to solve crimes i the camera obtains well-lit,

acial images. However, they cannot be used to

successully close investigations i the lm only shows

the top o the suspect’s head or i the images are

blurred. The combined advances o video surveillance

systems and acial recognition technology should

enable more crimes to be solved using video data in

the next ew years.

Another source o acial images is the acial composite

picture. I the recollections o the victim or the witnessesare precise enough to make a acial composite picture

resembling the oender, then investigation by acial

recognition may lead to success.

PreventionFacial recognition can also be used or preventive

purposes. It can be used to search or precedents.

By way o example, i a le o pedophiles is available,

then ID photos can be used to check whether people

who work with children are in the le.

In some cases, acial recognition can also be used to

interactively locate persons wanted by the police in

video ootage. This application is subject to controversy,

since it is oten considered to inringe civil liberties. In

any case, it is not currently suited to cases in which a

very small number o persons need to be identied in a

crowd. Even i this technique were to reach the excellent

accuracy level o 90% o persons actually ound with

just 0.1% alse alarms, looking or one person amongst

a crowd o 100,000 passing people per day would

operationally generate 100 alse alarms per day. This

would have a negative impact on the vigilance o

control operators.

Figure 5: acial

recognition

in a crowd.

7



On the other hand, interactive acial recognition is

already possible in controlled passages. By way o

example, when travelers approach the border police

or a travel document check using acial biometry, they

must be cooperative and show their ace. In this case,

it is quite easy to check the travel documents and make

a comparison with the lists o wanted persons. The

operators who check passport control processes can also

check any alerts received in response to these searches.

Civil applications, access controlsand border controls

Issuance o identity documentsFacial recognition is particularly well suited to checks

o the uniqueness o application or identity papers.

In a non-criminal context, it is quite normal to provide

a photo, while ngerprints will always have criminal

connotations and it is more dicult to acquire an image

o an iris than o a ace.

By way o example, driving licenses include a photo o

the holder, but rarely include any other biometric data.

As a consequence, acial recognition applications can be

used to guarantee that a single motorist cannot possess

several driving licenses. Morpho developed a solution

or this very purpose or the state o New South Wales

in Australia.

With regard to travel documents, the ICAO has

recommended that the portrait should be the only

compulsory biometric record.

Control o identity documentsOnce the documents have been issued, acial recognition

can be used to check that they are indeed being used

by their legitimate holders. This check can be made

by simply lming the holders when they present their

documents. By way o example, the SmartGates*

or automatic passport checks deployed by Morpho

have accelerated border ormalities in Australian airports.

Today, document holders are required to stand still in

ront o the camera, but in the near uture the check will

be made as they pass through the checkpoint. Morpho’s

rapid and robust Face on the Fly* technology is capable

o acquiring aces in three dimensions, without requiring

the subject to stand still.

8

Figure 6: acial recognition at airport. Figure 7: biometric passport.



This application is so easy to use that a broad range o

usages is possible. For example, it would be easy or

universities to check the identity o students when they

arrive to take an exam (authentication checks).

Access controlThe purpose o access control is to check that anyone

attempting to access a secure zone is entitled to do so.

Access controls are made in the same way as ID checks.They are very easy or the user i acial recognition is used.

The main advantage o portraits is that checks can still

be made once the person has passed through the access

barrier. I the access control gates are unmanned, then

sta members could easily allow strangers to enter

limited-access zones. But thanks to portraits, which can

be acquired without any special cooperation, it is possible

to check permanently that the people in protected zones

are indeed entitled to be there. Facial recognition can

thereore be used to extend access control by checking

presence in particularly sensitive environments.

Applications for the general public

Access to computerized servicesBiometric acial logins are already possible on certain

computers. But the system has come in or some

criticism, since logins are possible i a photo o the

user is shown instead o the user’s actual ace. Recent

algorithms are capable o detecting whether the ace

is indeed three-dimensional and mobile, and uturegenerations o biometric acial login systems will not be

ooled by photos.

Photo album managementFacial recognition applications are now available to

manage personal collections o photographs by

showing the names o persons in photos, i they already

appear in older pictures in the collection. Products

include iPhoto rom Apple and Picasa rom Google.

While this application may appear trivial, it shows the

ull potential o acial recognition, whose limits are still

ar rom known.

9

Figure 8: acial recognition manages personal collections

o photographs.



10

The Technology

This chapter describes the dierent steps in the acial recognition process and

describes the main technologies that are available today.

The different steps of the automaticfacial recognition process

Step 1: image acquisitionThis step is decisive, because the precision o acial

recognition hinges on the quality o the images

acquired.

In this step, automatic acial recognition systems may

assess the quality o the acquired images. In interactive

acquisition, the portrait can be re-acquired in order to

obtain a better image that meets the criteria o the

image assessment process.

Step 2: ace localization, scaling andalignmentBeore comparing aces, it is rst necessary to nd

them in images that may contain all sorts o other

inormation and adjust them to the same scale, with the

head positioned vertically. This step is quite simple when

working on controlled portraits, because each image onlycontains one ace. But it is much more dicult to extract

a multitude o aces rom a video taken outdoors.

Step 3: enhancement o ace imagesOnce the aces have been ound and calibrated, they

need to be enhanced. By way o example, the eects

o compression can be minimized, inconsistent lighting

can be corrected or unusable zones (masked by a veil,or example) can be detected and excluded. In this step,

models can be applied to correct the orientation o

the ace, the eects o ageing and expressions. While

some enhancements can be made automatically, the

assistance o an operator may prove to be very useul

when working on dicult images.

Step 4: extraction o characteristicsMost acial recognition algorithms use mathematical

transormations in order to compare images. These

transormations can highlight the distinctive specic

eatures o an image: requencies, directions, contours,

etc. Transormed images can not usually be used by the

operator’s naked eye.

Step 5: representation as a template*and comparisonA binary record, or template, is extracted rom the

transormed image. The comparator then compares

this template with those o the images in the reerence

database and scores each image. The higher the score,

the higher the similarity with the image o the wanted

ace.

Figure 10: portrait comparison.

Figure 9: Portrait

acquisition.



Step 6: thresholding and decision-makingI the search is simple, i.e. when the quality o the query

image and the images in the reerence database are

good, the operator should only receive a small number o

images that stand a high chance o matching the wanted

person. The ideal case is when the operator only receives

a candidate list when there’s a “hit” in the database, and

nothing otherwise, thus preserving the operator’s resources

and attention on relevant cases. This operation is calledthresholding. It requires a similarity unction that makes a

clear distinction between ”hits” and alse alarms.

The operator can then make a decision and make

changes to the system’s reerence database* according

to the requirements o the job in hand.

The facial recognition algorithms

This chapter contains an overview o the best known

algorithms. For more details, visit http://www.ace-rec.org/

algorithms/, which is an excellent source o inormation.

There are two prominent categories o algorithms when it

comes to acial recognition: procedural algorithms, which

imitate the analysis made by an operator, and machine

learning algorithms, which apply a mathematical logic

in order to dene and use the criteria that an operator

may not be capable o interpreting. Both categories o

algorithms can be used or dierent types o ace data:

xed images, videos, or 3D acquisitions.

Procedural algorithmsThe main procedural algorithms use the visible acial

landmarks, such as the corner o the eye, the middle othe upper lip, the lowest point o the chin or the details

and color o the skin.

Ater detecting the landmarks o the ace - a process

that may be manually assisted - the procedural

algorithms attempt to measure the coherence between

the parts o the two aces. They do this by using models

designed to demonstrate how a ace is distorted by its

expression, age, orientation and lighting.

The most commonly used algorithms in this category

are Elastic Bunch Graph Matching (EBGM) and the

comparison o acial texture. Facial texture analysis is

used in particular to distinguish twins.

These algorithms are used (at step 5 above) to convert

images into templates in order to compare them.

11

Figure 11: bunch graph matching.



Training algorithmsTraining methods rely on an abstract process in order

to nd independently an optimal organization based on

examples. There is a great number o training algorithms.

Examples include:

• Vector projections. The input used by these methods

is a large vector - the characteristics extracted in

step 4 - which is then projected in a smaller space. Ithe two original images match the same person, they

must have close projected vectors. I they represent

two dierent persons, then the projected vectors are

more distant. It is the denition o the projection that

is complex, and leads to the training process. The most

common methods are:

Principal component analysis (PCA), which extracts

the most distinctive vectors rom a space. Eigenaces

apply this principle.

Linear discriminant analysis (LDA), which separates

dierent objects.

Independent component analysis (ICA), which keeps

the axes as independent rom one another as possible.

Non-linear methods, including Kernels and SVMs*

(Support Vector Machines).

• Neural networks. These networks use a set o cells

that transorm the inormation that they exchange

with one another. They resemble neurons, synapses

and nervous infux. Neural networks are dened by

training. In the comparison phase, the characteristics

extracted rom the portrait are entered or input into

the network. The network output is used to decide

whether the ace resembles the dierent aces in the

reerence database.

• Statistical methods. These methods seek to measure

the probability that a photo matches a statistical

model o the ace. Each ace is represented in the

comparator by its statistical model, e.g. by a number o

states, their respective probability and the probability

o transition rom one state to another. New acial

images are represented as a sequence o successive

Y

X

Y

Detection boundary

0.0496

0.0498

0.0494

0.0492

0.0490

0.0488

0.0486

0.0484

0.0482

1.058 1.060 1.062 1.064 1.066 1;068 1.070 1.072 1.07

= type A image

= type B image

Separation may be easier in higher dimensions

complex in low dimensions simple in higher dimensions

separating

hyperplane

map

Feature

Original image

25 x 25

I1

H1

H2

H50

O1

DB

O2

O40

I2

I3

I625

12

Figure 12: Axes created with principle component analysis.

Figure 13: Boundary detection with linear discriminant analysis.

Figure 14: Space transormation.

Figure 15: Sample neural network.



13

states. For each ace in the reerence database, the

level o probability o the sequence can be determined

in order to decide whether the resemblance between

the two images is high or not.

Training algorithms are obviously used in step 5

(representation as templates and comparison), but they

can also be used to locate aces in images (step 2) and

to extract visible acial landmarks.

Special eatures o video processingA person’s ace appears several times in a video. It can

be viewed rom one video rame* to the next. These

multiple views are useul or acial recognition purposes,

because they can be used to obtain more inormation

about the ace than a single view. A range o tracking

techniques has thereore been developed. The most

robust techniques use movement statistics models. At

each step, they generate probabilized detection and

tracking hypotheses, which are consolidated in order to

make a decision. The actual comparison process uses

a series o images o the same person that are sorted

according to the quality o the views and the dierent

positions that they represent. Using these dierent views

rather than a single view – even i its quality is superior –

will always improve the precision o the search.

Special eatures o 3DFaces are naturally three-dimensional. I a ace is to be

completely represented, then its shape and the color or

texture o every part o the ace must be known. Photos

only contain the color o part o the ace. Thereore

they only contain partial, or 2D, inormation. Traditional

ace comparison techniques rely on these incomplete

data. Using all the inormation o the ace can only

serve to improve the precision o acial recognition. The

European 3D Face project (http://www.3dace.org/) has

demonstrated how the association o 2D data (texture)

and 3D data (shape) improves precision compared with

the use o only the texture or the shape.

The acquisition o three-dimensional images requires

special sensors that are not yet widely available. These

sensors work by projecting structured light onto the

ace or by stereoscopy*. Morpho has developed an

innovative 3D acquisition concept based on stereoscopy

that is capable o acquiring a ace on the fy when a

person passes through a control gate without stopping.

It is called Face on the Fly* technology.

3D technology can be used, even without any 3D

sensors. 3D morphable models can be used to take one

or more images o the same ace and associate a 3Dshape that matches the ace as closely as possible. This

association signicantly improves the robustness o the

comparison o oriented aces.

Light sourcesThe images used or acial recognition are usually taken

in visible light. But inrared images can also be used and

research is currently looking into other data types, such

as terahertz*.

Figure 16:

examples o pose

angles.



14

Perormance… and comparison with otherorms o biometrics

Two gures are required to measure the perormance o a biometric technique:

• the false acceptance rate (FAR*), which measures the percentage of fraudsters

who are mistakenly accepted,

• the false rejection rate (FRR*), which measures the percentage of persons who are

not accepted, whereas they are not raudsters.

Ideally, the alse acceptance and rejection rates should

be zero. But in reality, biometric systems can be

characterized by the graphs on the model below.

The lower the rate o alse acceptances, the more

secure is the application. The lower the rate o alse

rejections, the greater the comort or users and the

more limited the work done by operators.

The tests in NIST’s FRVT 2006 demonstrated that,

with rontal portraits taken in a strictly controlled

environment, with high resolution and only slight

dierences in age, acial recognition is a very precise

biometric technique. False rejections totaled just 1%,

with 0.1% o alse acceptances. This means that, or

example, in the ideal passport control application, out

o 1,000 authentic passport holders, only 10 would

not get through the automatic check and would have

to call on an operator, while out o 1,000 raudsters

carrying passports that do not belong to them, 999

would be detected by the automatic control system.

But we all know that the ideal conditions implemented

by the NIST or FRVT 2006 are dicult to achieve

operationally. In addition to orientation, resolution

and lighting, acial recognition is also conronted with

problems due to physical changes and changes in

appearance: expression, changes o hair, changes in

weight, spectacles, hats, ageing, injuries, illness, etc.

The current acial recognition algorithms can only

tolerate limited variations in the portrait. For example,

i a person allows his beard to grow, then the automatic

portrait recognition system will recognize him with

almost the same reliability, as i his beard had not

changed. I the same person allows his beard to grow

and pulls a ace, the probability that the system will

recognize him drops a little, but stil l remains high. But

i a number o small changes are accumulated (by way

o example, i the person allows his beard to grow,

pulls a ace, does not ace the camera, remains a long

way away rom the camera and conceals a large part

o one o the sides o his ace with his hand), then the

Very low false rejections:comfort for users, weak controls

Very low falseacceptance: security

FALSE ACCEPTANCES

FALSE REJECTIONS

Figure 17: False acceptance – alse rejection graph.

Figure 18: Variations with age and orientation. Figure 19: Variations with accessories.



15

probability that he wil l be recognized is much lower.

Since there are no test samples that can be used to

measure the impact, with the appropriate statistical

validity, o the dierent criteria that minimize the

precision o acial recognition, and since these criteria

are not independent, the drop in the precision o acial

recognition is not predictable in absolute terms.

Consequently, Morpho advises every potential userto proceed with tests and measurements o their own

data, depending on their own operational needs. But

these tests must not simply take account o the purely

algorithmic precision. They must also take the speed o

execution o the algorithms into consideration. A very slow

system will necessarily nd ewer hits than a ast system,

with the same power and precision, and will be more

dicult to adapt to the working procedures o the operators.

I one compares acial recognition with ngerprinting

and iris recognition, then it becomes clear that acial

recognition is intrinsically “trickier” than other biometric

techniques.

Facial recognition already works well and still has plenty

o potential or improvement. Nevertheless, it is quite

improbable that it will achieve the same levels o precision

as iris or ngerprint recognition in the short term.

Iris Fingerprints Face

Uniqueness Every iris is unique Every fngerprint is unique Two persons may

resemble one

another very closely

Number of images Two irises per person 10 fngerprints per person, One ace per person

to be acquired plus other parts o the body

where riction ridges are located

Stability over time Invariable rom birth Invariable rom childhood Changes with age,

state o health, etc.

Representation in Easy to represent

the dimensions in 2D and ew problems Easy to represent in 2D Intrinsically 3D

due to orientation

Distortions Pupil dilation Distortion limited by the elasticity Highly variable,

o the skin according to expressions

Resolution Requires high resolution Usually standard 500 dpi Acquired at any scale

Maturity of the processes Underdeveloped expertise Strong command o the Underdeveloped expertise

associated with identifcation process

the technology by generations o

fngerprinting experts

Comparison of biometric methods



16

Implementing a acial recognition system

The deployment o a acial recognition system must take both technical and human

actors into consideration. Some o these actors, which are specic to acial

recognition, are quite important.

Integrating facial recognition inthe existing technical environment

Photos are a very common orm o data. Most potentialusers o acial recognition thereore already possess a

number o inormation systems that can be interaced

with a acial recognition system:

• databases o individuals (with portraits) or criminal cases

• other systems designed to compare biometric data

• biometric data acquisition systems

There is almost always a business interest to be gained

by integrating a acial recognition system with the

existing environment.

Integration with databases allows or a simplied recovery

o existing data and allows legacy data acquisition

processes to be used. By way o example, in order to

add a acial recognition unction to a civil status register

in order to detect identity raud, it is always preerable

to keep the same civil status register and operate the

acial recognition system in back oce mode, without

impacting the potentially complex processes that are

used with the civil status register.

Integration with other biometric systems provides

or both optimized, redundancy-ree data acquisition

processes and the benets o using a number o

methods that increase the perormance o searches and

require operators to veriy only the most dicult cases.

Even i biometric searches are not natively consolidated,multi-biometrics does increase perormance:

• it allows cases to be processed, when one modality is

absent or is o poor quality,

• it can make more extensive, and consequently more

ecient links. For example, in a criminal police system,

i it is known that a rst oense o bank card raud

and a second oense o shopliting were committed

by the same person, because the images show the

same ace, and i the bank card raud is solved using

ngerprints, then it is highly likely that the shopliting

oense will also be solved. Two isolated systems would

not come to the same conclusion.

In an eort to acilitate the integration o acial

recognition with other inormation systems, Morpho has

developed generic interaces that meet the ANSI/NIST-

ITL 1-2007 standard or the data ormat or ngerprints,

aces and other biometric data.

Managing the expectationsof customers and operators

Facial recognition seems to be so simple and intuitive

that the expectations relative to this technique may be

out o all proportion. The unsuccessul experiment in

Tampa, Florida, USA in 2003 is one notable example.

A acial recognition system was deployed in order

to recognize wanted persons in a crowd. But the

operators only received alse alerts. Preliminary tests in

a controlled environment and an elementary probability

calculation could have concluded that the technology

o the time was not suited to acial recognition

in crowds.

Thereore, it is advisable that every potential user o

acial recognition conducts tests in order to assess

the suitability o the technology to their operational

Figure 20:

identication

verication

screen within

the verication

application.



17

applications beore proceeding with deployment. The

independent tests conducted in 2006 by the BKA (1)

and also the NPIA(2) serve as an example. Research

reports explain perectly the state o the art, the tests

conducted, the conclusions and the outlook.

In order to conduct these tests, Morpho has developeda very simple pilot system containing the most recent

advances in algorithms that can be installed and

programmed in less than one day. In this way, it is

possible to test acial recognition usage scenarios,

check the results that may be obtained or a given

target application and measure the workload required

to obtain these results. Morpho can provide support to

its potential customers in this assessment process.

Developing the expertise o operatorsAcquiring acial images and recognizing people on

photos is not easy and requires the development o

specic expertise. Morpho can provide training in its

products.

Expertise in data acquisitionI acial recognition is to be ecient, then the quality o

the images in the reerence database must be satisactory.

The criteria that measure this quality are dened by the

ISO* standard, and the automatic acquisition systems

are capable o veriying most o these criteria. But the

judgment o an operator remains the best guarantee o

good image acquisition. Automatic checks cannot be

100% reliable and the subject that is lmed may have

particular physical properties that prevent certain criteria

rom being reached.

Similarly, the operator is the person who is best placed

to judge the suitability o an image or acial recognition

searches. When acquiring an image in the eld or an

identity check, lming people behaving violently in a

demonstration or choosing which images rom many

images o the same ace should be used or search

purposes, the operator’s expertise is decisive in making

optimal use o acial recognition.

Expertise in checking searchesVisually recognizing people may appear to be simple.

When working on subjects that one knows well, and

that are seen ace to ace rather than on a photo, even

little children are capable o recognizing a ace. But

even known persons can be misidentied i they appear

on blurred or old photos, or i they have an unusualexpression. And when the person is unknown, the quality

o the photos is variable and the angles and lighting

dier, visual recognition becomes a tricky task, resulting

in many errors detected by academic studies.

A number o methods have been developed to improve

the visual recognition o persons, but they are not

as robust as the methods used to check ngerprints.

The relevance o recognition criteria (e.g. the stability

o wrinkles) remains to be scientically established.

Thereore, in order to avoid making mistakes and wasting

time, it is necessary to develop methods and training or

operators in the use o the acial recognition system. It is

also important to cooperate with academic researchers

and acial recognition technology vendors in order to

make progress in terms o both the practices and the

tools in this eld.

Helping the end usersEnd users react very dierently to biometric systems.

Reactions range rom total hostility (against a society

that some people eel is obsessed with security) to

a certain amusement at being a pioneer in the use o

new technology. Whether they be cooperative or hostile,

they are all novices, and it is essential to give them clear

and concise instructions on the behavior to be adopted.

In applications or the general public, such as passport

controls, it is impossible to support the users one by one.

This is the reason why close attention must be paid to the

ease o use o biometric tools or end users; they must be

as enjoyable to operate as possible.

The SmartGate* passport control system deployed in

Australia meets this need. The end users are happy with

the system and preer automatic passport controls using

acial recognition to conventional control gates.

(1) BKA, Bundeskriminalamt, the German police authorities.

(2) NPIA: National Police Improvement Agency. An organization tasked with making technological recommendations to the British police orce

in order to improve eciency.

Figure 21: on-the-fy portrait acquisition or traveller screening

in airports.



The market associated with automaticacial recognitionThe market for facial recognition

Compared with other biometric techniques

(ngerprinting, iris recognition, etc.), acial recognition

accounts or 16% o the market (source: Frost & Sullivan,

2007, 2008).

Also according to Frost & Sullivan, the value o this

market is growing constantly, with an average annual

growth rate o 54%. Totaling €72.7 million in 2006

and about €250 million in 2009, the acial recognition

market should exceed €622 million by 2011 and

€1 billion in 2013. These estimates correspond to all the

links in the value chain o developments in the eld o

acial recognition.

Driving forces and obstacles

A number o dierent actors drive or hinder the

development o acial recognition, ranging rom

technology to politics, applications and even standards.

The driving orces• The creation o an international standard (ICAO) or

travel documents. This standard species three possible

orms o biometrics: iris recognition, ngerprinting

and acial recognition. It has resulted in the creation

o a reerence database o high quality portraits.

Components /

Imaging sensors

1- 2D/3D cameras

2- Photo-video

Development of high

accuracy face recognition

algorithms,

in 2D and 3D, for static

images and video

Development and

integration of FR

utilization in vertical

markets

Deployement or upgrade

of integrated FR oriented

solutions in public

security market

Face Tech.

DevelopmentSW Development

System

Integration

Facial Recognition: value chain

16% 84%

18

Figure 22: Face recognition share in the biometric market.

Figure 23: Facial recognition: value chain.



• The highly avorable conclusions o FRVT 2006.

Facial recognition o good quality images is a mature

biometric technique and the algorithms continue to

progress, opening the way or the combination o 2D

and 3D acial recognition techniques.

• Existing implementations. Notably, governmental

or inter-governmental identity programs (electronicpassports, ID cards, driving licenses, etc.) and

automatic border passport control programs. These

implementations have been a real success and the

expectations o potential users are becoming more

precise and realistic.

• The proven benets o multi-biometrics. Multi-

biometrics cut stang costs in biometric research,

can solve dicult cases in which the data rom one

o the biometric techniques is o poor quality and can

correlate the connections made by dierent biometric

techniques.

• The availability o video cameras with improved

resolution. These cameras take better pictures, which

allow or more precise acial recognition.

The obstacles• Facial recognition is intrinsically more dicult than

other major biometric techniques.

• Facial recognition is oten quoted as an inringement

o civil liberties.

• Applications are broader than or other biometric

techniques and the potential new users must be

introduced to the eld o biometrics and understand

its benets.

19



The experience o Morphoin the feld o acial recognition

Morpho has been active in the eld o acial recognition since 2002. Some o the

most signicant milestones in our activity include:

• 2002 assessment o the algorithms on the market: we

opted to start with technology rom Cognitec, whichwas ound to be the best on the market at the time

by the NIST’s FRVT 2002. Since then, Morpho has

signicantly improved the technology.

• Morpho deployed a pilot tool designed to assess the

technology with simple and ecient user interaces.

The tool allowed numerous police agencies to test the

possible applications o acial recognition. This pilot

allowed a number o conclusions to be drawn:

The Morpho subsidiary MorphoTrak received the “Best

Biometric Identication Technology” award or its acial

recognition pilot system when it was exhibited at the

Global Border Security Conerence and Expo in Austin,

Texas in May 2008.

The Australian police orce concluded that acial

recognition oered decisive benets or the correlation

o crim inal ca ses and now use s Morph o technology

in operational applications.

The University o Lausanne, in cooperation with

the Romande regional police orce in Switzerland,successully developed a strategy or the use o Morpho’s

acial recognition or demonstrations.

The Pierce County Sheri’s Oce in Washington, USA,

demonstrated how Morpho’s acial recognition reached

a level o precision higher than 94% when identiying

individuals in its collections, even with signicant

dierences in age. It also surmised that the combined

use o Morpho’s acial recognition and ngerprinting

technology could allow subjects to be identied without

calling on ngerprinting experts.

In 2008, the Mexican police conrmed that Morpho’s

technology was the astest and the most accurate on

the market in operational tests.

In 2009, the Paris police authorities helped

investigators by piloting a Morpho acial

recognition system on a reerence database o

470,000 aces.

• Morpho developed the SmartGate border control

system that is now used in all international airports in

Australia and will soon be deployed in New Zealand.

This project uses the photos on biometric passports

to speed up and acilitate passport controls. By the

end o 2008, 150,000 people had passed through a

SmartGate.

20



21

Websites:http://face.nist.gov/mbgc/mbgc_presentations.htm

http://www.hcfdc.org/trophees2009/palmares.php

http://www.3dface.org/

NB : On May 27 2010, Sagem Sécurité changed its name to Morpho. For events, benchmarks having taken place before

this date, we are listed or quoted under the name of Sagem Sécurité.

• Morpho also developed MorphoFace™ Investigate*

on the basis o recommendations made by our users

working in dedicated Focus Groups. This product is

designed to perorm identication tasks, to help the

police to solve criminal cases or simply or identication

purposes in civil applications.

• Morpho’s research laboratories continuously develop

and improve the basic technology. Research ocuses on

acial recognition algorithms, but also the optimization

o portrait acquisition and the extraction o portraits

rom surveillance video ootage. As a result o this

research work, Morpho nished rst in the NIST’s

portal challenge in March 2009. In May 2009, Morpho

received the HCFDC (French High Committee or Civil

Deense) innovation trophy or its Face on the fy ace

acquisition technology.

Morpho takes part in cooperative research projects in the

United States and Europe, including the 3D Face project,

which has concluded that the use o both 2D and 3D

acial data improves the precision o acial recognition

or travel document applications (http://www.3dace.

org/).

Morpho is making signicant investments in innovation

and the development o its acial recognition technology

in order to consolidate its position as leader in biometrics.

Major advances are continuously being made in terms

o both the quality and diversity o the algorithms and

the development o dedicated products adapted to the

needs o the market.



22

Glossary and acronyms

AAlert - An alert is an automatic and

interactive event that is generated when

analyzing video streams. Alerts are generated

when the recognition algorithms conclude

that a person seen in the video is very

probably contained in the reerence

database. Alerts appear in the userinterace by displaying the image o the

person spotted in the video next to the

image in the reerence database.

FFA - False Acceptance. False acceptances

and alse rejections are used to measure

the perormance o a biometric system.

False acceptances correspond to raudsters

that the biometric control ails to detect.

Face on the Fly - Face On The Fly is an

innovative technology developed by

Morpho. The purpose o this

technology is to acquire acial images when

a person passes through a control gate,

without stopping and without having

to look at a particular camera. Several

images are acquired by a series o cameras

to create a three dimensional view o the

ace. A rontal projection o this image can

then be compared and used or

authentication or identifcation purposes.

Face On The Fly technology won the French

High Committee or Civil Deense’s award

or technological innovation in May 2009.

FR - False Rejection. False rejections

correspond to authorized users that the

biometric control ails to recognize.

Frame - An image extracted rom a video

recording or stream.

FRVT - Face Recognition Vendor Test.

I ICAO - International Civil Aviation

Or g a n i za t i o n . An i n t e r n a t i o n a l

organization that is part o the United

Nations. Its mission is to contribute to

the development o standards used to

standardize international air transport

ISO - International Organization orStandardization. An international

organization made up o the national

standardization institutes rom more

than 100 countries.

LLine-up - Line-up techniques are used by

the police to determine whether a witness

has spotted a suspect. The suspect is

presented to the witness amongst a group

o physically similar people. Witnesses

must then decide whether they recognize

one o the members o the group.

MMBGC - Multi Biometric Grand Challenge

(http://ace.nist.gov/mbgc/).

A test organized by the NIST to make

advances in research into the recognition

o persons rom a distance using acial

recognition and iris recognition.

MFI - MorphoFace™ Investigate.

A acial recognition system developed

by Morpho. This system is essentially

designed or use in police investigations.

It can be used to solve cases rom portrait

traces let on the scene o the crime.

Modality - A type o biometrics.

NNIST - National Institute o Standards

a n d T e ch n o l o g y . T h e Am e r i ca n

standardization organization and a

member o ISO.

R

Reference database - A database opersons o interest to be identied in the

images and videos to be processed.

SSmartGate - SmartGate is a project

run by Australian customs. The purpose

o the project is to speed up customs

clearance in Australia’s international

airports using the portraits on electronic

passports and acial recognition. Morpho

deploys a system in Australian airports as

part o this project.

Stereoscopy - Stereoscopy reers to

all the techniques used to reproduce a

perception o a contour rom several fat

images.

SVM - Support Vector Machine. A

prominent technique used to solve

classication problems.

TTemplate - The code extracted rom an

image o a ace by image processing.

Facial comparison is carried out via

extracted templates.

Terahertz - An electromagnetic wave in

the electromagnetic spectrum between

inrared (the optical domain) and

microwaves (the electronic domain).



Bibliography

ForshungsProjekt, Gesichtserkennung als Fahndungshilsmittel,

Fo t o - Fa h n d u n g , Ab s ch l u s sb e r i ch t , B K A , 2 0 0 7 ,

www.bka.de/kriminalwissenschaten/otoahndung/pd/

otoahndung_abschlussbericht.pd - See also: http://www.bka.

de/kriminalwissenschaten/otoahndung/aq.html

Automatic Face Recognition, applications within law

enorcement, Market and technology review, Ambika Suman,October 2006, www.npia.police.uk/en/docs/Face_Recognition_

Report.pd

“CCTV on trials” , Josh P Davis and Tim Valentine, Goldsmiths,

University o London, London, UK, 2007.

“Morphological Classifcation o Facial Features in Adult Caucasian

Males” Vanezis, et al. - Journal o Forensic Sciences, 1996.

“Limitations in Facial Identiication: The Evidence” ,

R. Jenkins, M. Burton, Department o Psychology, University

o Glasgow, Glasgow, UK, Justice O The Peace Volume 172,

January 2008.

“Quality Standards or Forensic Opinions on the Identity o

Living Oenders in Pictures” , 9th Biennial Scientic Meeting

o the International Association or Cranioacial Identication,

F.W. Rosing, Institut ur Humangenetik und Anthropologie

Universitatsklinikum, Ulm, Germany.

“Failure o Anthropometry as a Facial Identifcation Technique

Using High-Quality Photographs” , Kleinber, Pharm, Vanezis,

and Burton - Journal o Forensic Science, July 2007.

“Photo identifcation: acial metrical and morphological eatures

in South Arican males” , Roelose, Steyn, Becker, - Forensic

Science International, 2008.

“Facial comparison ‘area o expertise’: ormulation o the issues,

and international inventory” , Arnout C. Ruirok, Ph.D, Laan van

Ypenburg 6, 2497 GB Den Haag.

ANSI / NIST: Data Format or the Interchange o Fingerprint,

Facial, & Other Biometric Inormation.

ISO/IEC 19794-5, Biometric Data Interchange Format - Part 5:

Face Image Data.

Forensic Art and Illustration Karen T. Taylor, CRC Press, 2000,

ISBN 0849381185, 9780849381188.

“Eigenaces or recognition” , M. Turk, A. Pentland, Journal o

Cognitive Neuroscience, 3(1), 1991.

“Face recognition using eigenaces” , M. Turk and A. Pentland,

Proc. IEEE Conerence on Computer Vision and Pattern

Recognition: 586–591, 1991.

“Low-dimensional procedure or the characterization o human

aces” . L Sirovich and M. Kirby, Journal o the Optical Society o

America A 4: 519–524, 1987.

Préecture de Police de Paris, Identité judiciaire, Etude du

signalement descripti, “Portrait parlé” .

Privacy and technologies o identity: a cross-disciplinary

conversation, Katherine Jo Strandburg, Daniela Stan Raicu,

Springer, 2005, ISBN 0387260501, 9780387260501, page 146.

“Facial Comparisons by Subject Matter Experts: Their Role in

Biometrics and Their Training” , Federal Bureau o Investigation

Forensic Audio, Video and Image Analysis Unit, Nicole A. Spaun,

Advances in Biometrics, June 2009.

“World Biometrics Market” , Frost & Sullivan (2007).



Phone: +33 (0) 58 11 88 76 - Fax: +33 (0) 58 11 87 81

www.morpho.comRegistered Oce: Le Ponant de Paris - 27, rue Leblanc - F-75512 PARIS CEDEX 15 - France

Société anonyme au capital de 159.876.075 e 440 305 282 RCS PARIS

facial recognition 2

Documents