I

P

Ta

b

a

ARAA

KSFSL

1

mimoroapctisftrtsteemsM

(

h0

ARTICLE IN PRESSG ModelJLEO-54473; No. of Pages 8

Optik xxx (2014) xxx–xxx

Contents lists available at ScienceDirect

Optik

jo ur nal homepage: www.elsev ier .de / i j leo

roducing virtual face images for single sample face recognition

ao Zhanga,∗, Xianfeng Lia, Rong-Zuo Guob

Engineering Lab on Intelligent Perception for Internet of Things (ELIP), School of Electronic and Computer Engineering, Peking University, Shenzhen, ChinaDepartment of Computer Science, Sichuan Normal University, Chengdu 610068, China

r t i c l e i n f o

rticle history:eceived 3 August 2013ccepted 28 January 2014vailable online xxx

a b s t r a c t

For single sample face recognition, there are limited training samples, so the traditional face recognitionmethods are not applicable to this problem. In this paper we propose to combine two methods to producevirtual face images for single sample face recognition. We firstly use a symmetry transform to produce

eywords:ingle sampleace recognitionymmetrical face

symmetrical face images. We secondly use the linear combination of two samples to generate virtualsamples. As a result, we convert the special single sample problem into a non-single sample problem. Wethen use the 2DPCA method to extract features from the samples and use the nearest neighbor classifierto perform classification. Experimental results show that the proposed method can effectively improvethe recognition rate of single sample face recognition.

inear combination

. Introduction

Face recognition is one of the most challenging branches of bio-etrics recognition [1]. Due to the fact that face recognition are

nfluenced by light, gesture, expression, etc., the existing processingethods of face recognition, such as linear feature extraction meth-

ds [2–6], nonlinear feature extraction methods [7–15] usuallyequire a sufficient number of representative training samples forbtaining good results. But in some special occasions, the avail-ble samples are limited. For example, the id card, student card,assport, diploma, admission ticket, or employee’s card and so onan provide only a single photo, so we can only use this pictureo perform training to achieve the goal of face recognition, whichs known as single sample face recognition. The research of singleample face recognition is very meaningful, because single sampleace recognition can effectively reduce the training samples collec-ion costs, storage costs, and accelerate the processing speed of faceecognition system. But single sample face recognition not only hashe interference factors such as illumination, posture, facial expres-ion, but also has the following problems. First, face recognition is aypical small sample problem, and the single sample problem is thextreme situation. Second, a single training sample does not havenough representative information on the face. Third, the scattering

Please cite this article in press as: T. Zhang, et al., Producing virtual faElectron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.171

atrix of intra-class for single sample does not exist, so some clas-ic methods, such as the LDA method [16] cannot be implemented.oreover, as the intra-class distribution cannot be estimated out,

∗ Corresponding author.E-mail addresses: zhtao123abc@163.com (T. Zhang), lixianfeng@pkusz.edu.cn

X. Li), rongzuoguo@126.com (R.-Z. Guo).

ttp://dx.doi.org/10.1016/j.ijleo.2014.01.171030-4026/© 2014 Elsevier GmbH. All rights reserved.

© 2014 Elsevier GmbH. All rights reserved.

the probability based methods are also not applicable. Fourth, theinter-class variation is overestimated in single sample problem. Theinter-class variations measure the differences between images thathave different class labels. As there is only one image per person insingle sample problem, all the variations are inter-class variations.Though feature extraction methods can try their best to maximizethe inter-class variations, they cannot perform well in single sampleproblem.

Due to the severe challenging of single sample face recogni-tion problem and itself important meanings, it has become oneof the most active branches of face recognition. In recent years,many researchers put forward many important methods [17]. Inliterature [18,19] (PC)2A method is put forward, which uses theoriginal image and integral projection to form a new image, andthen reuses the PCA for identification. In literature [20] sparse PCA(SPCA) method is proposed, which uses the singular value decom-position of original images to reconstruct and to enhance the image,performing special pretreatment of face samples. In literature [21]the authors put forward a method based on the three-layer virtualimage generation, using the singular value disturbance prominentcharacteristics. Literature [22] proposed a method based on sparserepresentation, first exploiting the transformation process for thesingle sample, then using the sparse representation (SR) for classifi-cation. In literature [23] uniform pursuit (UP) approach is proposedto utilize whitening transform and locality dispersing projection touniform the pair wise distance of prototypes in PCA space. Denget al. [24] proposed Extend SRC(ESRC) method, takes advantage of

ce images for single sample face recognition, Optik - Int. J. Light

the pair wise difference images from a generic database to constructthe intra-class variation dictionary, which is further incorporatedinto the framework of SRC to cover the variations between galleryand probe samples. In literature [25] the author proposed a local

ING ModelI

2 tik xx

plltsftflalstcw

2

esge

2

iLisdccHohdsSTvss

2

dauawfff

z

povv

dp

ARTICLEJLEO-54473; No. of Pages 8

T. Zhang et al. / Op

robabilistic approach, where the subspace of each individual isearned and represented by a separate Gaussian distribution. Initerature [26] the authors proposed a method by separating tex-ure and shape information and projecting them into separate PCApaces. After the treatment, they construct separate eigenspacesor texture and the invariant shape features. However, the most ofhe methods have been proposed in the literatures did not makeull use of inter-class information of samples. Due to the particu-arity of the single sample problem, some classic face recognitionlgorithms, such as principal component analysis (PCA) [27] andocal preserve projection (LPP) [28] cannot obtain good effect. Iningle sample face recognition problem, if can find a certain wayo increase the number of training samples and convert the spe-ial single sample problem into a general face recognition issue,e might obtain good results.

. Introduction of related methods

To convert special problems into general issues is a kind of veryffective method to solve the problem. In order to convert singleample face recognition, the extreme small sample problem, into aeneral face recognition issue, literature [29,30] put forward someffective methods of generating the virtual samples as follows.

.1. Generation of symmetry face samples

In this subsection we show how to use every original train-ng sample to generate two symmetrical face training samples.et xi ∈ Rp×q represents the i-th training samples in the form ofmage matrix. Let yi

1 and yi2 respectively stand for the first and

econd symmetrical face training samples generated form xi. Theetails of generating symmetry face are as follows. The left halfolumns of yi

1 is set to the same as that of xi and the right halfolumns of yi

1 is the mirror image of the left half columns of yi1.

owever, the right half columns of yi2 is set to the same as that

f xi and the left half columns of yi2 is the mirror image of right

alf columns of yi2. The mirror image S of an arbitrary image R is

efined as S(i, j) = R(i, V − j + 1), i = 1, ..., U, j = 1, ..., V. U and Vtand for the numbers of the rows and columns of R, respectively.(i,j) denotes the pixel located in the i-th row and j-th columns of S.hus, each single training sample is produced two symmetry face ofirtual sample yi

1 and yi2. As shown in Fig. 1, we randomly selected

ome original samples from the ORL face database to generate theymmetry face of virtual samples.

.2. The linear combination of inter-class samples

Each sample image can be seen as a point in the high-imensional face space. Due to the changes of pose, illuminationnd expression, the same person’s face images are different, so wese different points to denote them respectively. Since these imagesre from the same person, so they have something in common. Weill classify them as the same class. Assume image x and y are taken

rom class 1 and class 2 respectively, regard them as two points inace space. We can perform a linear fitting. The fitting formula is asollow:

= �x + (1 − �)y, 0 ≤ � ≤ 1 (1)

Note that not every fitting out image is real image, the middleart of the linear fitting out images are apparently different fromriginal images, but the images on both ends of linear fitting areery similar to original images, so the selection of parameters is

Please cite this article in press as: T. Zhang, et al., Producing virtual faElectron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.171

ery important.In Fig. 2, we randomly choose two face images from the ORL face

atabase and synthesize nine novel images using (1) by setting thearameter � = 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9. As can be seen

PRESSx (2014) xxx–xxx

from Fig. 2, the synthesized images are quite similar to y whenthe parameter is 0.1 or 0.2, and these are similar to x when theparameter is 0.8 or 0.9. Thus, we can take some of synthesized faceimages as variations of the real face images, and use them to enlargethe training dataset.

By using formula (1) to synthesize different classes x and y toenlarge the number of training samples, we need to set the param-eter value scope. We confines this parameter into the union of twosets s1 = [0, 1/3) and s2 = (2/3, 1]. If � ∈ s1, formula (1) synthesizesa variation for y. If � ∈ s2, formula (1) synthesizes a variation for x.If � = 0 or 1, formula (1) synthesizes a variation for y or x originalimage. In the set consists of the original images and the ones synthe-sized using (1), we can prove that the intra-class variation is smallerthan the inter-class variation in terms of Euclidean distance.

Suppose two images z1 and z2 are synthesized using (1) respec-tively corresponding to parameter �1 and �2, as follows

z1 = �1x + (1 − �1)y (2)

z2 = �2x + (1 − �2)y (3)

The distance between them can be computed.

d2(z1, z2) = d2(�1x + (1 − �1)y, �2x + (1 − �2)y)

= (�1 − �2)2(x − y)T (x − y) = (�1 − �2)2d2(x − y) (4)

Analysis: If z1 and z2 are synthesized images for the same imagey(or x), both �1 and �2 are from the same set s1(or s2). In this sets1(or s2), the difference between these two parameters is smallerthan 1/3. Thus,

d2(z1, z2) = (�1 − �2)2d2(x, y) <19

d2(x, y) (5)

If z1 and z2 are synthesized images for two different images, �1and �2 are from two different sets s1 and s2. Thus, the differencebetween these two parameters is larger than 1/3. Thus,

d2(z1, z2) = (�1 − �2)2d2(x, y) >19

d2(x, y) (6)

Based on the analysis, we know that all the intra-class variationsare smaller than (1/3)d2(x, y) and all the inter-class variations arelarger than (1/3)d2(x, y). Thus, the intra-class variations are smallerthan the inter-class variations. According to this we can conductidentification and classification.

2.3. Two-dimensional principal component analysis (2DPCA)

Principal component analysis (PCA) is a method of data analysiswhich was proposed by K. Pearson in more than a century ago. Itis a mathematical procedure that uses an orthogonal transforma-tion to convert a set of observations of possibly correlated variablesinto a set of values of linearly uncorrelated variables [31]. But thetraditional PCA methods [32–43] need to transform the image intoa one-dimensional vector. However, the dimension of converteddata is very high, the computational complexity is also increased alot. Therefore, Yang et al. [44] proposed two-dimensional principalcomponent analysis (2DPCA) method, which uses original image toconstruct covariance matrix directly, do not need to convert imagematrix into a one-dimensional vector.

According to the method presented in reference [44], let Xdenote an n-dimensional unitary column vector. Regard image Aas a m × n random matrix, project A onto X by the following lineartransformation [45,46],

ce images for single sample face recognition, Optik - Int. J. Light

Y = AX (7)

Thus, we obtain an m-dimensional projected vector Y, which iscalled the projected feature vector of image A. The total scattering

ARTICLE IN PRESSG ModelIJLEO-54473; No. of Pages 8

T. Zhang et al. / Optik xxx (2014) xxx–xxx 3

nd its

mtsof

J

vpptT

S

Fig. 1. Some original samples a

atrix of projection samples can be used to measure the differen-ial ability of projection vector X. The total scatter of the projectedamples can be characterized by the trace of the covariance matrixf the projected feature vectors. From this point of view, adopt theollowing criterion:

(x) = tr(Sx) (8)

Where Sx denotes the covariance matrix of the projected featureectors of the training samples and tr(Sx) denotes the trace of Sx. Thehysical significance of maximizing the criterion in (8) is to find arojection direction X, onto which all samples are projected, so thathe total scatter of the resulting projected samples is maximized.

Please cite this article in press as: T. Zhang, et al., Producing virtual faElectron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.171

he covariance matrix Sx can be denoted by:

x = E(Y − EY)(Y − EY)T = E[AX − E(AX)][AX − E(AX)]T

= E[(A − EA)X][(A − EA)X]T (9)

Fig. 2. Some virtual samples gener

symmetrical faces in the ORL.

So,

tr(Sx) = XT [E(A − EA)T (A − EA)]X (10)

Define the following matrix:

Gt = E(A − EA)T (A − EA) (11)

The matrix Gt is called the image scatter matrix. It is easy to ver-

ce images for single sample face recognition, Optik - Int. J. Light

ify that Gt is a n × n nonnegative definite matrix from its definition.Gt can be calculated directly by using training samples. Supposethat there are M training image samples in total, the j-th train-ing image is denoted by an m × n matrix Aj(j = 1, 2, ..., M), and the

ated by linear combination.

ING ModelI

4 tik xx

ae

G

J

gmtm

iemam

{X

oe

oo

Y

cctot

3

mvsanaiuwhpitvuWevtA

ARTICLEJLEO-54473; No. of Pages 8

T. Zhang et al. / Op

verage image of all training samples is denoted by A, Gt can bevaluated by:

t = 1M

M∑

j=1

(Aj − A)T(Aj − A), (12)

Thus, the criterion in (8) can be expressed by:

(X) = XT GtX (13)

Where X is a unitary column vector. This criterion is called theeneralized total scatter criterion. The unitary vector X that maxi-izes the criterion is called the optimal projection axis. Intuitively,

his means that the total scatter of the projected samples is maxi-ized after the projection of an image matrix onto X.The optimal projection axis Xopt is the unitary vector that max-

mizes J(X), which is the feature vector corresponding to largestigenvalue of Gt . In general, it is not enough to have only one opti-al projection axis. We usually need to select a set of projection

xes X1,X2, · · ·, Xd, subject to the orthogonal constraints and maxi-izing the criterion J(X), that satisfy both (14) with (15):

X1, X2, . . ., Xd} = arg maxJ(X) (14)

iT Xj = 0, i /= j, i = 1, 2, . . ., d; j = 1, 2, . . ., d (15)

In fact, the optimal projection axes X1, X2, . . ., Xd are therthogonal eigenvectors of Gt corresponding to the first d largestigenvalue. d is determined as follows:∑d

i=1�i∑ni=1�i

≥� (16)

where �1≥�2≥· · ·≥�n are n eigenvalues of Gt , � is a preset thresh-ld, usually �≥0.9. The optimal projection vector set X1, X2, . . ., Xd

f 2DPCA are used to extract features. For a given sample A, thus

k = AXk, k = 1, 2, . . ., d (17)

Then can get a set projection feature vectors Y1, Y2, . . ., Yd, weall it the principal component of sample A. After obtain principalomponent, form a m × d size matrix B = [Y1, Y2, . . ., Yd], referredo as the image matrix or the eigenmatrix of the sample. Afterbtaining the eigenmatrix, we use the nearest neighbor classifiero perform classification.

. The proposed method

We noticed that the facial structure and expressions are sym-etrical. Facial symmetry features in face detection have achieved

ery good results [47]. Different classes of human face images areimilar on the outline, the distribution of the eyes, mouth, nosend so on. Moreover, other facial features also have some common-ess. Using the symmetry transform to produce intra-class imagesnd using the linear combination method to produce inter-classmages. The mechanism of generating virtual sample of the twosed methods are totally different, they are complementary some-hat. Virtual samples based on symmetry transform for intra-classave good effect in light conditions change situation. Virtual sam-les based on the linear combination for inter-class have good effect

n expression, posture change cases. Based on the understanding ofhe above, we propose to combine these two methods to produceirtual face images for single sample face recognition. We firstlyse a symmetry transform to produce symmetrical face images.e secondly use the linear combination of two samples to gen-

Please cite this article in press as: T. Zhang, et al., Producing virtual faElectron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.171

rate virtual samples. Using both of the two methods to generateirtual samples, we can make full use of the information both ofhe intra-class and inter-class to obtain virtual training samples.s a result, the single sample face recognition is converted into a

PRESSx (2014) xxx–xxx

general face recognition issue. And then we use 2DPCA method toreduce the dimensionality of the image. Finally, we use the nearestneighbor classifier to perform classification.

3.1. Main steps of the proposed method

The proposed method includes the following main steps.

Step 1. Take out the original training set and testing set.Step 2. Use symmetry transform for intra-class to generate virtual

training samples.Step 3. Use the linear combination method for inter-class to gen-

erate virtual training samples.Step 4. Combine the original and virtual samples as a new training

set.Step 5. Use 2DPCA method for processing.Step 6. Use the nearest neighbor classifier to perform classification.

3.2. Analysis of the proposed method

In this section we will show the rationales of our method,and prove that the proposed method is effective. First, the “sym-metrical face” and “combination of face” in the proposed methodindeed reflect some possible appearance of the face, which are notshown by the original training samples. From Figs. 1 and 2, wecan see that the “symmetry face” and “combination of face” notonly have difference with original sample, but also indeed some-what reflect the possible variation of the face in image scale, poseand illumination. Thus, the “symmetrical face” and “combinationof face” training samples are very useful to overcome the issue ofnon-sufficient training samples. Second, we use two methods ofgenerating virtual samples based on two completely different ways.The proposed method makes full use of both the intra-class andinter-class information. Compared to the virtual samples gener-ated by traditional methods of simply using the stretching, rotationand image enhancement, the virtual samples generated by theproposed method are more representative. First of all, based onthe facial symmetry structure, we use the symmetry transformto obtain symmetrical face images, which well retain the originalimage information, but does not completely equal to the originalsample. The traditional methods to generate the samples, such asrotation, stretching and image enhancement. While in the produc-tion of virtual samples, they did not fully consider how to effectivelyuse and maintain natural class information of original samples. Asa result, the generated samples tend to weaken or even destroy theinner class information of original samples. However, our proposedmethod can effectively avoid this problem. Furthermore, based onthe commonness of inter-class facial contour and distribution aboutfive sense organs, we use formula (1) to produce virtual samples bylinear combination, which make full use of the inter-class informa-tion. The generated virtual samples can effectively enlarge trainingset for single sample face recognition. Third, we use symmetrytransform to produce intra-class images and the linear combinationmethod to produce inter-class images, the mechanism of the twoused methods are totally different, they are complementary some-what. What’s more, our method has more advantages than just useonly one method to enlarge the training set. Fourth, the traditionalmethod use the original sample for training, our proposed methodwill combine the generated virtual samples and original samples fortraining. Thus, the training set not only retain the inherent infor-mation of original samples, but also make full use of the generatedvirtual samples, which can improve the classification accuracy at

ce images for single sample face recognition, Optik - Int. J. Light

the same time. During the register phase in real-world applications,after the face image is obtained by the face detection procedure,the “symmetrical face” and “combination of face” training samplescan be easily and efficiently generated. Since the virtual training

ARTICLE IN PRESSG ModelIJLEO-54473; No. of Pages 8

T. Zhang et al. / Optik xxx (2014) xxx–xxx 5

F

sssos

3

slmoschTiiwoleolivieipcopdtbrtvt0bSeeeslt

ig. 3. Some samples with severely light conditions change from YALE face database.

amples are complementary for the original training samples, theystem can capture only a few original training samples and cantill obtain enough information of the face. To capture only a fewriginal training samples will also allow the system to take only ahort time to complete the register phase.

.3. Analysis of complementarity

In this paper we proposed combine the two methods of usingymmetry transform to produce intra-class images and using theinear combination method to produce inter-class images, the

echanism of the two used methods are totally different. Basedn the analysis, we observed that the virtual samples generated byymmetry transform have very good robustness for light conditionshange, and the virtual samples generated by linear combinationave very good robustness for posture, facial expression change.hey are complementary somewhat. In theory, the former use thenformation of left and right half of face to produce intra-classmages. By mirror transforming to generate the virtual samples,

hich can fully retain the original information. In the conditionf only considering the illumination change, face images are onlyight and shade degree change. In fact, the virtual samples gen-rated by using the symmetry transform are very close to theriginal sample. Intuitively, it is a static change. The latter use theinear combination method to produce inter-class images. Intu-tively, it is a dynamic change. We use formula (1) to produceirtual samples for linear combination. Compared with the orig-nal sample, the generated samples have some stretching, rotationffect, indeed somewhat reflect the possible variation of the facen image scale, pose and expression. In order to prove the com-lementarity of these two methods, we have done the followingonfirmatory experiments. Firstly, we tend to verify the robustnessf symmetry face to illumination change. We take some sam-les with severely light conditions change from the YALE faceatabase as the original samples (as shown in Fig. 3). We take outhe training set and testing set, and then use the nearest neigh-or classifier to perform classification directly. The experimentalesults are as follow. When we only use the original samples ashe training set, the recognition rate is 0.7364. While we use theirtual samples generated by symmetry transform together withhe original samples as the new training set, the recognition rate is.8314. Thus we can conclude that the virtual samples generatedy symmetry transform can well adapt to illumination changes.econdly, in order to verify the virtual samples generated by lin-ar combination have very good robustness for posture, facialxpression changes. We take some samples with significantly facial

Please cite this article in press as: T. Zhang, et al., Producing virtual faElectron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.171

xpression change from the YALE face database as the originalamples (as shown in Fig. 4). The experimental results are as fol-ow. When we only use the original samples as the training set,he recognition rate is 0.7104. While we use linear combination

Fig. 4. Some samples with significantly facial expression change from YALE facedatabase.

method to generate virtual samples, together with the originalsamples as the new training set, the recognition rate is 0.8075.Thus we can conclude that the virtual samples generated by lin-ear combination can well adapt to facial expression and posturechanges.

4. Experimental analysis

In order to test our proposed method, we conducted con-firmatory test on the ORL face database and YALE face databaserespectively. We select one original sample in each class randomlyas training samples, and let the rest as the testing samples. We usesymmetry transform to produce intra-class images and use the lin-ear combination method to produce inter-class images. Then weuse 2DPCA method to extract features from the samples and usethe nearest neighbor classifier to perform classification.

4.1. Experiment on the ORL face database

The ORL face database was created by AT&T LABS of the Univer-sity of Cambridge [48]. The ORL face database includes 40 classes.Each class contains 10 face images. There are 400 face images intotal. Photos are taken in different time. Face images in each classhave different expressions, different gestures and different lightconditions. Fig. 5 shows some samples from the ORL face database.We randomly selected one original sample in each class as trainingsamples. As a result, there are 40 original training samples in total.The rest of 360 images are as test samples. By using the method ofsymmetry transform, each original sample of each class generatestwo symmetry samples, which are based on the left and right halfof original face image respectively. By using the method of linearcombination to generate virtual training samples, we set the valuerange of parameter � ∈ [0, 1], the augment step is 0.1. As a result,each class generates 9 virtual training samples. After the treatment,the total training samples of each class are 12. We take 8 eigen-vectors when use 2DPCA method for processing. The experimentalresults show in Table 1.

Based on analysis of the data in the Table 1, we concluded thefollowing results. When we only use the original samples as thetraining set, the recognition rate is only 0.7167. When only thesymmetric virtual samples are used as the training set, the recog-nition rate is 0.6978. When training samples are only provided bylinear combination virtual samples, the recognition rate is 0.6245.However, when we use the generated virtual samples by the twomethods with original samples for training, the recognition rate

ce images for single sample face recognition, Optik - Int. J. Light

is 0.8739, which greatly improving the accuracy of single sampleface recognition. Thus, the facts that our proposed method achievedfairly good results in the ORL face database. The results prove that

ARTICLE IN PRESSG ModelIJLEO-54473; No. of Pages 8

6 T. Zhang et al. / Optik xxx (2014) xxx–xxx

Fig. 5. Some samples in the ORL face database.

in the

tf

4

topdtrtii

TC

Fig. 6. Some samples

he proposed method can improve the accuracy of the single sampleace recognition.

.2. Experiment on the YALE face database

YALE face database was founded by the Center for Computa-ional Vision and Control of YALE University [49]. The establishmentf this database was supported by 15 volunteers. Each volunteerrovided 11 images. There are 165 images in total. Images in thisatabase contain the changes of illumination, expression and pos-ure. Fig. 6 shows some samples from the YALE face database. We

Please cite this article in press as: T. Zhang, et al., Producing virtual faElectron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.171

andomly selected one original sample in each class. As a result,here are 15 original training samples in total. The rest of 145mages are as the testing samples. We use symmetry transform forntra-class and use the linear combination method for inter-class

able 1omparison of experimental data on the ORL face database.

Source of training samples The number of tra

Only the original samples 40

Only symmetric virtual samples 80

Original samples + symmetric virtual samples 120

Only the linear combination virtual samples 360

Original samples + linear combination virtual samples 400

Original samples + symmetric virtual samples + linearcombination virtual samples

480

YALE face database.

to generate 30 and 135 virtual samples, respectively. There are 165virtual samples in total. We take 8 eigenvectors when use 2DPCAmethod for processing. The experimental results show in Table 2.

Based on analysis of the data in the Table 2, we concluded thefollowing results. When we only use the original samples as thetraining set, the recognition rate is very low, only 0.44. When weuse the proposed method to generate virtual training samples,and let the training set includes both the generated samples andoriginal samples, we could obtain a good result. In this case, thedata information of training set are enriched, meanwhile the num-ber of training set are enlarged. As a result, the recognition ratereaches 0.7986, which was significantly improved than before. The

ce images for single sample face recognition, Optik - Int. J. Light

experimental results on the YALE database prove that the proposedmethod can effectively improve the accuracy of single sample facerecognition.

ining samples The number of testing samples Recognition rate

360 0.7167360 0.6978360 0.8263360 0.6245360 0.8125360 0.8739

ARTICLE IN PRESSG ModelIJLEO-54473; No. of Pages 8

T. Zhang et al. / Optik xxx (2014) xxx–xxx 7

Table 2Comparison of experimental data on the YALE face database.

Source of training samples The number of training samples The number of testing samples Recognition rate

Only the original samples 15 150 0.44Only symmetric virtual samples 30 150 0.5333Original samples + symmetric virtual samples 45 150 0.6421Only the linear combination virtual samples 135 150 0.4467Original samples + linear combination virtual samples 150 150 0.6378Original samples + symmetric virtual samples + linear 180 150 0.7986

5

cspbcptssoh

A

eYgJ

R

[

[[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

combination virtual samples

. Conclusion

For single sample face recognition problem, we put forwardombine two methods to produce virtual face images for singleample face recognition. We firstly use a symmetry transform toroduce symmetrical face images. We secondly use the linear com-ination of two samples to generate virtual samples. As a result, weonvert the special single sample problem into a non-single sampleroblem. We then use the 2DPCA method to extract features fromhe samples and use the nearest neighbor classifier to perform clas-ification. Experimental results in the ORL and YALE face databasehow that the proposed method can largely improve the accuracyf single sample face recognition. In a word, the proposed methodas a positive meaning for single sample face recognition problem.

cknowledgments

This work is supported by the grant of Shenzhen municipal gov-rnment for basic research on Internet technologies (Outstandingoung Scholar, No. JC201005270274A) and Shenzhen municipalovernment for basic research on information technologies (No.CYJ20130331144751105).

eferences

[1] X. Tan, S. Chen, Z.-H. Zhou, F. Zhang, Face recognition from a single image perperson: a survey, Pattern Recognit. 39 (9) (2006) 1725–1745.

[2] M. Kirby, L. Sirovich, Application of the KL phase for the characterization ofhuman faces, IEEE Trans. Pattern Anal. Mach. Intell. 12 (1990) 103–108.

[3] Y. Xu, D. Zhang, J. Yang, J.-Y. Yang, An approach for directly extracting featuresfrom matrix data and its application in face recognition, Neurocomputing 71(10–12) (2008) 1857–1865.

[4] J. Yang, D. Zhang, A.F. Frangi, J.-Y. Yang, Two-dimensional PCA. A new approachto appearance-based face representation and recognition, IEEE Trans. PatternAnal. Mach. Intell. 26 (2004) 131–137.

[5] Y. Xu, D. Zhang, Represent and fuse bimodal biometric images at the featurelevel: complex-matrix-based fusion scheme, Opt. Eng. 49 (3) (2010).

[6] S.W. Park, M. Savvides, A multifactor extension of linear discriminate analysisfor face recognition under varying pose and illumination, EURASIP J. Adv. SignalProcess. 2010 (158395) (2010) 11.

[7] M. Debruyne, T. Verdonck, Robust kernel principal component analysis andclassification, Adv. Data Anal. Classif. 4 (2010) 151–167.

[8] Y. Xu, D. Zhang, F. Song, J.-Y. Yang, Z. Jing, M. Li, A method for speeding upfeature extraction based on KPCA, Neurocomputing 70 (2007) 1056–1061.

[9] M.E. Tipping, Sparse kernel principal component analysis, in: T.K. Leen, T.G.Dietterich, V. Tresp (Eds.), Neural Information Processing Systems, MIT Press,Cambridge, 2000, pp. 633–639.

10] B. Scholkopf, A. Smola, K.-R. Muller, Kernel principal component analysis, in:Proc. ICANN, 1997, pp. 583–588.

11] B. Scholkopf, A. Smola, Learning With Kernels, MIT Press, Cambridge, 2002.12] K.-R. Muller, S. Mika, G. Ratsch, K. Tsuda, B. Scholkopf, An introduction to kernel-

based learning algorithms, IEEE Trans. Neural Netw. 12 (2001) 181–201.13] D. Tao, X. Tang, Kernel full-space biased discriminate analysis, Proc. IEEE ICME,

2004, June, pp. 1287–1290.14] M. Sugiyama, Dimensionality reduction of multimodal labeled data by local

Fisher discriminant analysis, J. Mach. Learn. Res. 8 (2007) 1027–1061.15] V. Vural, G. Fung, B. Krishnapuram, J.G. Dy, B. Rao, Using local dependencies

Please cite this article in press as: T. Zhang, et al., Producing virtual faElectron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.171

within batches to improve large margin classifiers, J. Mach. Learn. Res. 10 (2009)183–206.

16] P.N. Belhumeur, J.P. Hespanha, D.J. Kriegman, Eigenfaces vs. Fisherfaces: recog-nition using class specific linear projection, IEEE Trans. Pattern Anal. Mach.Intell. 19 (9) (1997) 711–720.

[

17] K.-J. Wang, S.-L. Duan, W.-X. Feng, A survey of face recognition using singletraining sample, PR&AI 21 (5) (2008) 77–83, 635–642.

18] X. Wu, Z.H. Zhou, Face recognition with one training image per person, PatternRecognit. Lett. 23 (14) (2002) 1711–1719.

19] J. Wu, Z.-H. Zhou, Face recognition with one training image per person, PatternRecognit. Lett. 23 (14) (2002) 1711–1719.

20] D.-q. Zhang, S.-c. Chen, Z.-h. Zhou, A new face recognition method based onSVD perturbation for single example image per person, Appl. Math. Comput.163 (2) (2005) 895–907.

21] J.-m. Zhang, X. Liu, L.-j. Fan, Face recognition with single sample based on three-layer virtual image generation, Comput. Eng. 36 (9) (2010) 187–189.

22] X.-p. Chang, Z.-l. Zheng, C.-m. Xie, Sparse representation-based face recognitionfor single sample, Comput. Eng. 36 (21) (2010) 175–177.

23] W. Deng, J. Hu, J. Guo, W. Cai, D. Feng, Robust accurate and efficient face recogni-tion from a single training image: a uniform pursuit approach, Pattern Recognit.43 (5) (2010) 1748–1762.

24] W. Deng, J. Hu, J. Guo, Extended src: undersampled face recognition via intra-class variant dictionary, IEEE Trans. Pattern Anal. Mach. Intell. (2012).

25] A. Martinez, Recognizing imprecisely localized, partially occluded, and expres-sion variant faces from a single sample per class, IEEE Trans. Pattern Anal. Mach.Intell. 25 (6) (2002) 748–763.

26] X. Li, G. Mori, H. Zhang, Expression-invariant face recognition with expressionclassification, in: Proc. Third Can. Conf. Comput. Robot Vision, 2006, p. 77.

27] M. Turk, A. Pentland, Eigenfaces for recognition, J. Cogn. Neurosci. 3 (1) (1991)71–86.

28] X. He, S. Yan, Y. Hu, P. Niyogi, H.-J. Zhang, Face recognition using Laplacianfaces,IEEE Trans. Pattern Anal. Mach. Intell. 27 (3) (2005) 328–340.

29] Y. Xu, X. Zhu, Z. Li, G. Liu, Y. Lu, H. Liu, Using the original and ‘symmetrical face’training samples to perform representation based two-step face recognition,Pattern Recognit. 46 (2013) 1151–1158.

30] J.-h. Wang, J. You, Y. Xu, Q. Li, Enlarge the training set based on inter-personalrelationship for face recognition from one image per person, manuscript.

31] X.-g. Zhang, Pattern Recogniton, third ed., Tsinghua University Press, Beijing,2010, pp. 8.

32] Y. Xu, D. Zhang, J. Yang, J.-Y. Yang, An approach for directly extracting featuresfrom matrix data and its application in face recognition, Neurocomputing 71(2008) 1857–1865.

33] Y. Xu, D. Zhang, J.-Y. Yang, A feature extraction method for use with bimodalbiometrics, Pattern Recognit. 43 (3) (2010) 1106–1115.

34] Y. Xu, C. Lin, W. Zhao, Producing computationally efficient KPCA-based featureextraction for classification problems, Electron. Lett. 46 (6) (2010).

35] Y. Xu, D. Zhang, J. Yang, Z. Jin, J. Yang, Evaluate dissimilarity of samples infeature space for improving KPCA, Int. J. Inf. Technol. Decis. Making 10 (3) (2011)479–495.

36] Y. Xu, D. Zhang, J. Yang, J.-Y. Yang, A two-phase test sample sparse representa-tion method for use with face recognition, IEEE Trans. Circ. Syst. Video Technol.21 (9) (2011) 1255–1262.

37] Y. Xu, Q. Zhu, Z. Fan, D. Zhang, J. Mi, Z. Lai, Using the idea of the sparse rep-resentation to perform coarse to fine face recognition, Inf. Sci. 238 (2013)138–148.

38] Y. Xu, W. Zuo, Z. Fan, Supervised sparse representation method with a heuris-tic strategy and face recognition experiments, Neurocomputing 79 (2012)125–131.

39] S. Nedevschi, I.R. Peter, A. Mandrut, PCA type algorithm applied in face recog-nition, in: 2012 IEEE Int. Conf. on Intelligent Computer Communication andProcessing (ICCP), 2012, pp. 167–171.

40] J. Yang, D. Zhang, J.-Y. Yang, Constructing PCA baseline algorithms to reevaluateICA-based face-recognition performance, systems, man, and cybernetics, IEEETrans. B: Cybern. 37 (2007) 1015–1021.

41] Y.-s. Zhao, X.-j. Shen, Georganas, N.D. Petriu, Part-based PCA for Facial FeatureExtraction and Classification, 2009, ISBN 978-1-4244-4218-8, pp. 99–104.

42] G. Heo, P. Gader, H. Frigui, Robust kernel PCA using fuzzy membership, in:International Joint Conference on Neural Networks – IJCNN, 2009, pp. 1213–1220.

43] H.-C. Kim, D. Kim, S.-Y. Bang, A PCA mixture model with an efficient model

ce images for single sample face recognition, Optik - Int. J. Light

selection method, in: International Joint Conference on Neural Networks, 2001,pp. 430–435.

44] J. Yang, D. Zhang, Two-dimensional PCA: a new approach to appearance-basedface representation and recognition, IEEE Trans. Pattern Anal. Mach. Intell. 26(1) (2004) 131–137.

ING ModelI

8 tik xx

[

[

[

[

ARTICLEJLEO-54473; No. of Pages 8

T. Zhang et al. / Op

45] K. Liuetal, Algebraic feature extraction for image recognition based on an opti-

Please cite this article in press as: T. Zhang, et al., Producing virtual faElectron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.171

mal discriminant criterion, Pattern Recognit. 26 (6) (1993) 903–911.46] J. Yang, J.Y. Yang, From image vector to matrix: a straight forward image pro-

jection technique – IMPCA vs. PCA, Pattern Recognit. 35 (9) (2002) 1997–1999.47] P. Ekman, J.C. Hager, W.V. Friesen, The symmetry of emotional and deliberate

facial actions, Psychophysiology 18 (2) (1981) 101–106.

[

PRESSx (2014) xxx–xxx

48] AT&T LAB of the University of Cambridge, UK, ORL face database [EB/OL],

ce images for single sample face recognition, Optik - Int. J. Light

2010, http://www.cl.cam.ac.uk/Research/DTG/attarchive/facedatabase.html(12.05.2010).

49] A.S. Georghiades, P.N. Belhumeur, D.J. Kriegman, From few to many: illumina-tion cone models for face recognition under variable lighting and pose, IEEETrans. Pattern Anal. Mach. Intell. 23 (2001) 643–660.