A Prototype Software System for Multi-object Recognition and itsFPGA Implementation

H. Ando, N. Fuchigami, M. Sasaki and A. Iwata

Graduate School of Advanced Sciences of Matter, Hiroshima University1-3-1 Kagamiyama, Higashi-hiroshima, 739-8530, Japan

Phone: +81-824-22-7358, E-mail: ando, futigami, sasaki, iwa@dsl.hiroshima-u.ac.jp

1. IntroductionIn order to recognize various objects such humans, an-

imals, and so on, we have to develop highly intelligentinformation processing systems. Robust image process-ing techniques are also required under variation in size,orientation, lighting condition, etc. Moreover, it is nec-essary to develop VLSI hardware systems with the latestintegrated circuit technologies.

For the aim of achieving highly information processing,we have been proposed a concept of multi-object recog-nition system based on 3D custom stack system (Fig. 1)presented in the 21st century COE of Hiroshima Uni-versity [1]. The preliminary experimental results of facedetection/recognition have already been obtained [2].

In this paper, as a first step toward realizing a fu-ture multi-object recognition system, we propose a realtime human face detection/recognition software systembased on eigenfaces methods. Moreover, we design objectrecognition LSI circuits including eigen-space generationcircuits and DB matching circuits, and implement thesecircuits in FPGA.

Vision

Detection

RecognitionDB

1DB

2DB

N

Normalization

Local Wireless Interconnection

Global Wireless Interconnection

Input Image

Power Supply

Figure 1: A concept of multi-object recognition system based

on 3DCSS.

2. Eigenfaces for face detection and recognitionOne of the well-known face recognition methods is

“Eigenfaces” based on PCA (Principal Component Anal-ysis) [3, 4]. Let us explain the algorithm of this methodbriefly. An i-th face image consists of M pixels is repre-sented as a vector Γi. A preprocessed face Φi is definedby Φi = Γi − Ψ, where Ψ represents the average faceof N images in DB(database), that is Ψ = 1

N

∑Nn=1 Γn.

The “eigenfaces” can be calculated as the eigenvectors ak

of the covariance matrix C of DB, C = 1M

∑Mn=1 ΦnΦt

n,where Φt is a transposed matrix.

A face image is transformed into so-called “eigen-space” ωk by a simple operation: ωk = at

kΦi. The eigen-space ωk forms a vector Ω = [ω1 ω2 . . . ωm] that de-scribes the contribution of each eigenface for face image.

Face detection is performed by generally used thresh-olding methods. A reconstructed image Φr, defined byΦr =

∑mk=1 ωkak, is used as an input of evaluation func-

tion for thresholding. For example, Euclidean distanceε = ∥Φin − Φr∥ is often used as evaluation function,where Φin is a preprocessed unknown input image. Ifthe value ε is lower than a threshold, an unknown inputimage is classified as a human face.

Face recognition is also achieved with the similarmethod. If the face space vector ΩDBi of i-th faceimage in DB leads to the minimum distance εmin =∥Ω − ΩDBi∥, we can know that the input face is thesame as i-th face.

By the eigenfaces methods, only an object of the samekind as DB can be recognized. Therefore, we can recog-nize the various kinds of object by preparing DB corre-sponding with them.

3. Development of prototype software systembased on eigenfaces methodsWe developed the prototype software systems for

multi-object recognition (Fig. 2). Note that the systemis not necessarily limited to the application to humanface. Our proposed algorithm in the case of human facerecognition shown in Fig. 2(b) is as follows: First, aninput image with RGB color spaces is resized to about100×100 pixels in order to improve the processing time.Second, the candidate of face area is determined by skincolor detection. Finally, at the inside area of the candi-date region, face detection and recognition are achievedby using the methods explained in Sec. 2. After face de-tection/recognition was done, the candidate of face areais limited to a small area around the detected face regionfor the speed-up of the movie picture processing.

In the proposed algorithm, we used Euclidean distanceand Correlation coefficient for evaluation functions. Onlyif both of the evaluation values are satisfied, a face can bedetected and the detected face can be recognized. Whenthis condition is not satisfied, the system answers thatthere is not a face or the detected face does not exist inDB.

We achieved the correct face detection and recognitionfrom natural scenes with the developed software systemas shown in Fig. 2(c) and (d). The processing time wasabout 10∼30fps for the captured VGA image by usingthe latest personal computer.

detected skin color area

detected face area

computertarget

USB camera result view

(c) Appearance of software systems

(d) Screen shot of human-face detection/recognition

identified person

face candidate area

Face detection

Skin color detection

Face recognition

Image capture

Image scale-down

(b) Flow chart

Computer

Image capture device

Operating system

Programming language

Xeon3.06GHz x 2CPU

USB Camera

Windows XP

Visual C++7.0 & DirectShow

(a) System configuration

Figure 2: A prototype software system of multi-object recog-

nition.

4. Hardware implementation of object recogni-tion processingWe designed LSI circuits corresponding to object

recognition. As mentioned in Sec. 2, object recognitionneeds calculation of eigen-space, evaluation, and search-ing for minimum value. Therefore, the object recognitioncircuit blocks can be realized by using the eigen-spacegeneration circuits and DB matching circuits as shownin Fig. 3. The eigen-space ωk = at

kΦi are calculatedby using subtracter, multiplier, accumulator and mem-ory storing average face and eigenfaces. If one use theEuclidean distance for an evaluation function, the DBmatching circuits consist of evaluation block includingsubtracter and multiplier, Winner-take-all (WTA) circuitblock for searching for minimum distance, and memorystoring eigen-spaces. In this way, the eigenfaces meth-ods can be easily realized by conventional simple digitalcircuits.

The designed LSI circuits were implemented to theFPGA evaluation board (Fig. 4 (a)). The used FPGAdevice model was “Xilinx Virtex-II Pro(XC2VP7)”. Thespecifications of the implemented circuits are shown inFig. 4 (b), where the evaluation function was the Eu-clidean distance. As a result, the operation time of oneobject recognition was 212µs, where the object image sizeand the clock frequency were 20×20 and 100MHz, respec-tively. Therefore, we can perform 100 objects recognitionper 1ms by employing line-parallel processing.

Subtracter Multiplier AccumulatorΓin

aDBΨDB

Memory

Input image

Average face Eigenfaces

Preprocessed imageEigen-space

ΦinΩin

aDBΨDB

(a) Eigen-space generation circuits

Evaluation block

Eigen-space

Ωin

ΩDB

Memory

Eigen-space

ΩDB

E(Ωin, ΩDB)

Values of evaluation function

WTARecognition

results

(b) DB matching circuits

Figure 3: Object recognition circuits based on PCA.

Face-space generation circuit

Bit width of eigenface

Bit width of input face

Rate of gate use

Processing time

7bit

8bit

1.7%

200µs

DB matching circuit

Bit width of eigen-space

Rate of gate use

Processing time

18bit

16%

12µs

(a) FPGA evaluation board (b) Specifications

FPGA

Figure 4: FPGA board and detail of designed circuits.

5. ConclusionThe prototype software systems for multi-object recog-

nition system were developed and real time human facedetection and recognition were achieved. The process-ing time was about 10∼30fps by using latest personalcomputer. Moreover, we designed the object recognitionLSI circuits and implemented these circuits to FPGA.The operation time of one object recognition was about200µs.

We are scheduled to extend the software so that vari-ous objects can be recognized and to complete the hard-ware system including an image input device and objectdetection circuits.

References

[1] A. Iwata, et al., “Targets and Research Plan of 21stCentury Center of Excellence(COE) on Nanoelectronicsfor Tera-bit Information Processing”, Proc. First Interna-tional Workshop on NTIP, pp. 2–3, 2003.

[2] H. Ando, et al., “Human Face Detection and Recogni-tion using Principal Component Analysis”, Proc. SecondInternational Workshop on NTIP, pp. 78-80, 2004.

[3] W. Zhao, et al., “Face Recognition: A Literature Survey”,ACM CSUR archive, vol. 35, pp. 399-458, 2003.

[4] M. A. Turk, et al., “Eigenfaces for Recognition”,CVPR’91, pp. 586–591, 1991.