T. Kanade, A. Jain, and N.K. Ratha (Eds.): AVBPA 2005, LNCS 3546, pp. 151�159, 2005. © Springer-Verlag Berlin Heidelberg 2005

A Fingerprint Authentication System Based on Mobile Phone*

Qi Su, Jie Tian**, Xinjian Chen, and Xin Yang

Center for Biometrics and Security Research, Key Laboratory of Complex Systems and Intelligence Science, Institute of Automation, Chinese Academy of Sciences, Graduate School of the Chinese Academy of Sciences,

P.O. Box 2728, Beijing 100080, China tian@doctor.com, jie.tian@mail.ia.ac.cn

http://www.fingerpass.net

Abstract. With the increasing volume of sensitive and private information stored in the mobile phone, the security issue of mobile phone becomes an im-portant field to investigate. This paper proposes a fingerprint authentication sys-tem for mobile phone security application. A prototype of our system is devel-oped from the platform of BIRD E868 mobile phone with external fingerprint capture module. It is composed of two parts. One is the front-end fingerprint capture sub-system, and the other is back-end fingerprint recognition system. A thermal sweep fingerprint sensor is used in the fingerprint capture sub-system to fit the limitations of size, cost, and power consumption. In the fingerprint rec-ognition sub-system, an optimized algorithm is developed from the one partici-pated in the FVC2004. The performance of the proposed system is evaluated on the database built by the thermal sweep fingerprint sensor.

1 Introduction With the rapid evolution of mobile technology, mobile phone is not only a communi-cation tool, but also a MMS center, a scheduler, a recorder, a camera, an mp3 player, and even a mobile web explorer. With the advancement of the hardware, mobile phones can store significant amount of sensitive and private information (e.g. address book, SMS, scheduler and even a bank account). Moreover, with the relative low cost, the number of mobile phone user increases rapidly in recent years. Worldwide mobile phone sales in 2003 was 520 million units, by the end of 2004 the estimated sales was in the range of 580 million units [1]. Nowadays, the mobile phone has become a nec-essary part of our daily life.

Currently, many mobile phones come with a four-digit Personal Identification Number (PIN) and a numerical entry key as a tool for user authentication. Because of the limited length, they may be susceptible to shoulder surfing or systematic trial-and-error attacks [2]. And the PIN may be difficult to remember and prone to input errors when entered via a touch screen. * This paper is supported by the Project of National Science Fund for Distinguished Young

Scholars of China under Grant No. 60225008, the Key Project of National Natural Science Foundation of China under Grant No. 60332010, the Project for Young Scientists� Fund of National Natural Science Foundation of China under Grant No.60303022, and the Project of Natural Science Foundation of Beijing under Grant No.4052026

** Corresponding author: Jie Tian; Telephone: 8610-62532105; Fax: 8610-62527995

152 Qi Su et al.

Because the size of the mobile phone becomes smaller and smaller, it can easily misplaced, unattended or stolen. As the information stored in a mobile device is sensi-tive, the effective protection of the mobile phone against unauthorized access has been increased. Biometrics recognition technology provides replacement or comple-ment passwords to make a higher level of user convenience and security by means of fingerprint, hand geometry, iris, face, and signature etc. Among numerous biometrics technology, fingerprint authentication carries more advantages than others.

Fingerprint authentication has been thoroughly verified through various applica-tions including law enforcement and commercial application for a long time. The fingerprint image may be taken and digitalized by relatively compact and cheap de-vices. Electronic fingerprint capture has been introduced with much success. Combin-ing such methods with powerful microprocessors and pattern matching software has opened a new application in the mobile phone development.

This paper proposes an authentication system based on fingerprint recognition to improve the security protection of the mobile phone. The authentication system is composed of two parts. One is the front-end fingerprint capture sub-system and the other is back-end fingerprint recognition sub-system based on BIRD mobile phone E868. The fingerprint capture sub-system is an external module. It mainly consists of an ARM-Core processor LPC2106 and an Atmel thermal fingerprint sensor AT77C101B. It is responsible for capturing the fingerprint image frames, reconstruct-ing the image and sending it to the recognition sub-system. As a part of the mobile phone operating system, the fingerprint recognition sub-system includes the enroll unit, match unit and system Application Program Interface (API). An optimized fin-gerprint recognition algorithm based on the one participated in the FVC2004 is used in the fingerprint recognition sub-system. The programs of both the fingerprint cap-ture and recognition sub-system are optimized for the embedded environment.

This paper is organized as follows. Section 2 describes the structure of the finger-print authentication system. Section 3 illustrates the software of authentication sys-tem. The fingerprint reconstruction algorithm, fingerprint recognition algorithm and the optimization techniques are described in this section. Section 4 shows the experi-mental results and section 5 concludes our work with future perspectives.

2 The Fingerprint Authentication System The fingerprint authentication system is composed of two parts. One is the front-end fingerprint capture sub-system and the other is back-end fingerprint recognition sub-system. The structure of the whole system is shown in Fig. 1.

Fig. 1. The fingerprint authentication system block diagram

A Fingerprint Authentication System Based on Mobile Phone 153

The fingerprint capture sub-system is an external module, and it is controlled by an ARM-Core processor LPC2106. The external module works in a slave mode. The LPC2106 processor receives the commands from the mobile phone via UART inter-face and controls the thermal fingerprint sensor AT77C101B to capture the image and reconstructs the original image frames to a full fingerprint image, and sends it to the mobile phone.

As a part of the mobile phone operating system, the recognition sub-system oper-ates in the E868 mobile phone. The functions of the fingerprint recognition sub-system include fingerprint enroll and match. Moreover, the recognition sub-system provides a set of APIs. Other applications on the mobile phone operating system can possess the fingerprint authentication functions by calling those APIs.

2.1 E868 Mobile Phone

The hardware platform of the fingerprint authentication system includes the BIRD mobile phone E868 and the external module. Ningbo Bird Mobile Communications Co. Ltd. unveiled the E868 mobile phone in August 2003 [3]. The E868 mainly tar-gets the high-end business market. It is capable of supporting up to 65,000-colors, touch-screen and handwriting recognition. The mobile phone provides PDA func-tions, e-mail, internet, camera, mp3, JAVA� technology and more. The central proc-essing unit of the E868 is a 16-bit embedded processor S1C33. The processor is pro-duced by Epson Company and its working frequency is 13 MHz.

2.2 ARM Core Processor

The fingerprint capture sub-system is based on a LPC2106 ARM-Core embedded processor which is manufactured by PHILIPS [4]. The processor is very powerful. It has a 32-bit ARMTDMI-S core with real-time emulation and embedded trace support, and it can work at 60 MHz clock. Moreover, the LPC2106 processor incorporates 128 KB on-chip Flash and 64 KB on-chip Static RAM. Between the ARMTDMI-S core and the memory block, a 128-bit wide internal memory interface and unique accelera-tor architecture enable 32-bit code execution at the maximum clock rate.

Furthermore, the LPC2106 processor has a high efficient power management unit. The unit can put the whole processor into three statuses: normal, idle and power down. In addition, LPC2106 may turn off individual peripheral when it is not needed in application, resulting for power saving purpose.

Because of the limitation of the E868 hardware platform, the processor of the mo-bile phone can not be connected to the fingerprint sensor. So, as the co-processor, the LPC2106 is in charge of capturing the fingerprint image from the fingerprint sensor. After the process of the fingerprint image reconstruction, the LPC2106 sends the image to the mobile phone by using the UART interface. The LPC2106 processor is suitable for the mobile embedded application with the feature of small package (only 7×7 mm2) and the low power consumption.

2.3 Fingerprint Sensor

The fingerprint authentication system uses the Atmel�s AT77C101B FingerChip IC for taking fingerprint image [5]. It captures the image of a fingerprint as the finger

154 Qi Su et al.

sweeping vertically over the sensor window. The AT77C101B sensor is composed of two main sections: thermal sensor and data conversion. The sensor section comprises a temperature-sensitive pixels array with 8 rows by 280 columns. The data conversion section mainly consists of analog signal amplifier and Analog-to-Digital Converter. The AT77C101B sensor can provide a resolution of 500 dpi fingerprint image. The pixel clock is programmable at up to 2 MHz, giving an output of 1780 frames per second.

The sensor has a very small size. The image zone is only 0.4×14 mm2, it can be embedded into an external mobile module or even into the mobile phone. The AT77C101B�s rectangular sensor window is much smaller than a square window fingerprint sensor with the same image resolution. So it leads to a decreased unit cost and further reduces the cost of the whole fingerprint authentication system. The sweep method of image capture means that the sensor window is self-cleaning with no latent prints left after an image capture. With the technical advance of the smaller size, lower cost and others, AT77C101B sensor is suitable for mobile hand-held devices, such as E868 mobile phone.

3 System Software Descriptions The function of fingerprint authentication system includes fingerprint capture, enroll and match. The core algorithms are fingerprint reconstruction algorithm and verifica-tion algorithm. As the core part of the fingerprint authentication system, the capability of the algorithm influences the system directly. The reconstruction algorithm is based on the linear correlation theory. The verification algorithm is based upon the recogni-tion algorithm participated in the FVC2004. Both of them are optimized prior being ported to E868 mobile phone.

3.1 Fingerprint Capture

The process of fingerprint acquisition is completed in the fingerprint capture sub-system. Because the continuous fingerprint frames are read by the thermal sweep sensor AT77C101B, the reconstruction algorithm is necessary to obtain a full finger-print image. The fingerprint capture process has three main sections: 1) capturing more than 100 continuous fingerprint image frames; 2) reconstructing the fingerprint image; 3) sending the full fingerprint image to the mobile phone. The flow chart of the three sections is shown in Fig. 2.

Fig. 2. The fingerprint capturing block diagram

A Fingerprint Authentication System Based on Mobile Phone 155

When the fingerprint process starts, the thermal fingerprint sensor will read the im-age frames no matter a finger is on the sensor or not. One of the image frames� format is 8×280, 16 gray levels. The fingerprint capture sub-system uses a threshold to fix the start point of the swiping movement. First, the system applies local mean and variance to find the real fingerprint image frames. When the number of continuous image frames is larger than the threshold, the fingerprint capture process goes into the second part, the fingerprint reconstruction. Usually, as long as the threshold is not less than 3 frames, it can match the requirement of fixing the accurate starting point of the fingerprint movement.

The speed of the finger swept over the sensor window is not the same at each time. If the speed is slower than a reasonable rate, there will be an overlapping image in the two successive frames. In image reconstruction section, the program deletes the over-lapping image and output the registration ones. The fingerprint reconstruction algo-rithm is based on the linear correlation theory, meanwhile adopts the virtue of the registration algorithm proposed by Hassan Foroosh etc. [6] Because of the practical movement of finger has been counted in fingerprint reconstruction algorithm, the search range of the translation between a pair of continuous swept fingerprint frames is limited from -45 degrees to +45 degrees. As a result, the computation time of the optimized method is only a quart of the normal method in which the search range is in 360 degrees. Moreover, the quality of fingerprint image after registration is the same as the normal method.

In fact, because of the limitation of the LPC2106�s memory space, the part one and part two do not execute sequentially. The full fingerprint image is 256×280, 256 gray levels after resolution enhancement, and at least 32 frames are needed to reconstruct a full fingerprint image. Therefore the smallest memory space is 71680 bytes and it is larger than the LPC2106�s integrated SRAM memory volume. The real image recon-struction process is shown in Fig. 2. The image capture and reconstruction are exe-cuted alternately. The LPC2106�s SRAM is used as the temporary memory location for image frame data and other temporary data, and the FLASH is performed to save the program and the full fingerprint image.

In the third part, the LPC2106 sends the full fingerprint image to the mobile phone by UART interface at the rate of 230400 bps. Usually the total time of capturing a full fingerprint image is about 1.5 second.

3.2 Fingerprint Recognition Algorithm

The accuracy and the efficiency of fingerprint recognition algorithm directly influ-ence the performance of the authentication system. Maio and Maltoni have presented the direct gray-scale minutiae detection algorithm [7] for fingerprints. However, in this paper, we present the method based on the fingerprint recognition algorithm par-ticipated in the FVC2004 [8]. And our algorithm is modified prior being ported to the E868 mobile phone.

The block diagram of fingerprint recognition sub-system is shown in Fig 3. It basi-cally includes two parts: the enroll part and the match part. Each algorithm is com-posed of 4 stages. The first three processing stages are the same, reading a fingerprint image (256×280 pixels), applying an image filter with the frequency and extracting the minutiae from the fingerprints. The last stage of enroll is to save the fingerprint

156 Qi Su et al.

template to the template file. While the last stage of the match part is to search the template file in order to find the similar template. Different algorithms of the finger-print preprocessing, minutiae extraction and template matching are presented in detail as following.

In fingerprint preprocessing stage, a frequency transformation converts an image from its spatial-domain form of bright intensities into a frequency-domain form of frequency components. In this stage, fingerprint enhancement algorithm based on filtering in frequency domain [8] is used. First, Fourier transforming converts the fingerprints from spatial domain to frequency domain, then the fingerprints are en-hanced by the proposed filter in frequency domain. The frequency domain shows the frequency of brightness variations, the direction of the variation patterns, and the amplitude of the waveforms representing the patterns.

After the enhanced fingerprint image is obtained, the next stage is to generate the thinned ridged fingerprint image and extract the minutiae of the thinned image. We compute the average grey value in every one of the 8 directions to decide the ridge direction of each pixel. To reduce the effect of noise, we use the algorithm proposed by Yuliang He [9] to get the thinned ridge fingerprint map. The thinned ridge map is shown in fig. 4. After the ridge map of filtered image is obtained, the algorithm pro-posed in [9] is performed to extract the minutiae.

(a) (b)

Fig. 4. Example of thinned fingerprint ridge images processed by our algorithm: (a) original image of forefinger and the thinned image; (b) original image of thumb and the thinned image

Fig. 3. Fingerprint verification system

A Fingerprint Authentication System Based on Mobile Phone 157

The fingerprint authentication system employs the thermal sweep fingerprint sen-sor to capture a fingerprint image. As it is well known, the deformations and non-linear distortions of the sweep style sensor are more serious than those of an ordinary press style sensor. Even the fingerprint reconstruction algorithm removes the transla-tion in the fingerprint capture stage, the other deformations are still big problems for fingerprint matching. We use the Maximum-likelihood estimation to calculate the optimal deformation parameters [10].

The main idea of the match algorithm is using an affine transformation model T to relate the fingerprint template and the minutiae set of the live-scanned fingerprint. The variables in the affine transformation model T represent some styles of deforma-tion, including rotation, scale and so on. The deformations are formulated in terms of maximum-likelihood estimation, namely a probability density function. The experi-mental results presented in 4.2 demonstrate the good performance of the fingerprint authentication system by using the proposed algorithm.

3.3 Energy Management

Mobile phone is a kind of hand devices. They are characterized by small size, limited storage and processing power, and battery-powered. The battery-powered embedded computing system needs a set of efficient energy management to prolong the work time. Three approaches toward solving the task scheduling and voltage assignment problem are described in [11]. According to the characters of the LPC2106 processor, we propose a method that is suitable for the external module to prolong the system working time and make it accomplish the balance between system performance and battery duration.

The fingerprint capture sub-system works in the slave mode. It waits for the com-mands come from the application software running into mobile phone, and carries out the relevant operations. The major operations of sub-system are capturing a finger-print image and sending it to the mobile phone. The sub-system does nothing for most situations. So we set the interrupt flag on for the LPC2106 processor�s UART during the sub-system initialization. And afterwards, we make the LPC2106 processor and the sensor, AT77C101B, in sleep mode to save energy. When the mobile phone sends the command to captures a fingerprint image, the communication wakes up the LPC2106 processor to start processing. While the fingerprint capture is finished, the sub-system goes into the sleep mode again. In normal mode, the AT77C101B oper-ates with a power consumption of 20 mW at 3.3V. When the sensor is in the sleep mode, it only consumes less than 10 µA current.

In addition, the LPC2106 processor has another good feature. It possesses a power management unit, which can turn off selected peripheral. When the processor is in the normal mode, the unused peripherals can be turned off automatically to save more energy. For example, when the sub-system is capturing a fingerprint image (described in 3.1), only part 3 needs to use the UART interface to communicate with the mobile phone. The UART interface can be turned off in the period of part 1 and part 2. In the same way, the AT77C101B sensor can be turned off in the period of part 3.

158 Qi Su et al.

4 Experimental Results We have developed a prototype of fingerprint recognition mobile phone based on the E868. The prototype has achieved the application of fingerprint enroll and match. The appearance of the fingerprint recognition mobile phone and the fingerprint image captured by the fingerprint recognition system are shown in Fig. 5.

(a) (b)

Fig. 5. (a) The appearance of the fingerprint recognition mobile phone; (b) Fingerprint image captured by the fingerprint recognition system

4.1 The Data Set

To test the performance of the prototype of fingerprint recognition mobile phone, we have built up a small fingerprint database in the mobile phone. 20 person works as volunteers for providing fingerprints. Thumb, forefinger and middle finger of both hands (six fingers total) of each volunteer were captured by the sweep fingerprint sensor. Four fingerprint images were taken for each of the six fingers from each per-son. The database totally includes 480 fingerprints.

4.2 Results for Fingerprint Match

Four performance tests were measured: genuine match, imposter match, average match time and maximum template size. In genuine match, the number of genuine tests is 720. In imposter match, the total number of false acceptance tests is 7140. The definitions of equal error rate (EER), false non-match rate (FNMR), false match rate (FMR), and receiving operating curve (ROC) are defined in [7].

Fig. 6 presents the experimental results of the proposed algorithm on the database. Figure 6(a) shows the match score distributions. In figure 6(b), the value of EER of the proposed system is 4.13%. The value of FNMR equals to 5.64% for FMR = 1%. Figure 6(c) indicates the ROC curve of the proposed algorithm on the database.

We also measure the average match time of a fingerprint image and the maximum template size. Based on the results of both match methods, the average match time is about 6 seconds and the maximum template size is smaller than 128 bytes.

5 Conclusion Security protection for mobile phone is a desperate issue nowadays. In this paper, we have presented the design and test of a mobile authentication system based on finger-print recognition for security protection of the mobile phone.

A Fingerprint Authentication System Based on Mobile Phone 159

(a) (b) (c)

Fig. 6. Experimental results of the proposed algorithm. (a) Score distributions (b) FMR(t) and FNMR(t) (c) ROC curve

The system consists of front-end fingerprint capture sub-system and back-end fin-gerprint recognition sub-system. The hardware platform is composed of E868 mobile phone and the external fingerprint capture module. The system software includes both fingerprint capture unit and recognition unit. In the recognition sub-system, the opti-mized fingerprint recognition algorithm is used. It is based on the algorithm partici-pated in the FVC2004. The performance of the proposed system was evaluated on the 480 fingerprints database. The EER of the experiment is 4.13%.

The average match time will be decreased in the products of the fingerprint recog-nition mobile phone. Further works will be focused on the system performance opti-mization and the security implementations of the mobile phone based on the authenti-cation system.

References 1. Alexander wolfe, Worldwide Mobile Phone Sales Surge,

http://www.internetnews.com/wireless/article.php/3324061, March 10, 2004 2. Wayne A. Jansen, Authenticating Users on Handheld Devices, Proceedings of the Canadian

Information Technology Security Symposium, May 2003. http://csrc.nist.gov/mobilesecurity/Publications

3. Ningbo Bird Mobile Communications Co. Ltd., BIRD DOEASY E868 Mobile Business Elite Introduce, http://doeasy.net.cn/index_2.htm

4. Philips Semiconductors Co. Ltd., LPC2106/2105/2104 USER MANUAL, http://www.semiconductors.philips.com

5. Atmel Corporation, AT77C101B FingerChip Datasheet, Rev. 2150B�BIOM�09/03, http://www.atmel.com

6. Hassan Foroosh, Josiane B. Zerubia, and Marc Berthod, Extension of Phase Correlation to Sub-pixel Registration, IEEE Trans. Image Processing, vol. 11, No.3,pp.188�200, Mar.2002

7. Dario Maio, Davide Maltoni, Raffaele Cappelli, and etc., FVC2004: Third Fingerprint Verification Competition, Proceedings of ICBA 2004, LNCS 3072, pp.1-7, 2004

8. Xinjian Chen, Jie Tian, Xin Yang, A Matching Algorithm Based on Local Topologic Struc-ture, pp. Proceedings of ICIAR2004, LNCS 3211, pp. 360-367, 2004

9. Yuliang He�Jie Tian, Xiping Luo, and etc., Image Enhancement and Minutia Matching in Fingerprint Verification, Pattern Recognition Letters, Vol.24, pp.1349-1360, 2003

10. Yuliang He, Jie Tian, Qun Ren, and etc.�Maximum-Likehood Deformation Analysis of Dif-ferent-Sized Fingerprints, Proceedings of AVBPA2003,LNCS 2688, pp.421-428, 2003

11. Daler Rakhmatov, Sarma Vrudhula, Energy Management for Battery-Powered Embedded Systems, ACM TECS, Vol. 2, No. 3, 2003