recognitionoffacialexpressionsundervaryingconditions...

Research ArticleRecognition of Facial Expressions under Varying ConditionsUsing Dual-Feature Fusion

Awais Mahmood1 Shariq Hussain 2 Khalid Iqbal3 and Wail S Elkilani1

1College of Applied Computer Science King Saud University Al Muzahimiyah Campus Riyadh Saudi Arabia2Department of Software Engineering Foundation University Islamabad Islamabad Pakistan3Department of Computer Science COMSATS University Islamabad Attock Campus Attock Pakistan

Correspondence should be addressed to Shariq Hussain shariqfuiedupk

Received 23 July 2019 Accepted 4 August 2019 Published 21 August 2019

Guest Editor Marco Perez-Cisneros

Copyright copy 2019 Awais Mahmood et al is is an open access article distributed under the Creative Commons AttributionLicense which permits unrestricted use distribution and reproduction in any medium provided the original work isproperly cited

Facial expression recognition plays an important role in communicating the emotions and intentions of human beings Facialexpression recognition in uncontrolled environment is more difficult as compared to that in controlled environment due tochange in occlusion illumination and noise In this paper we present a new framework for effective facial expression recognitionfrom real-time facial images Unlike other methods which spendmuch time by dividing the image into blocks or whole face imageour method extracts the discriminative feature from salient face regions and then combine with texture and orientation featuresfor better representation Furthermore we reduce the data dimension by selecting the highly discriminative features eproposed framework is capable of providing high recognition accuracy rate even in the presence of occlusions illumination andnoise To show the robustness of the proposed framework we used three publicly available challenging datasets e experimentalresults show that the performance of the proposed framework is better than existing techniques which indicate the considerablepotential of combining geometric features with appearance-based features

1 Introduction

Facial expression recognition (FER) has emerged as animportant research area over the last two decades Facialexpression is one of the immediate natural and powerfulmeans for humans to communicate their intentions andemotions e FER system can be used in many importantapplications such as driver safety health care video con-ferencing virtual reality and cognitive science etc

Generally facial expression can be classified into neutralanger disgust fear surprise sad and happy Recent re-search shows that the ability of young people to read thefeeling and emotion of other people is getting reduced due tothe extensive use of digital devices [1] erefore it is im-portant to develop a FER system which accurately recognizesfacial expression in real time

An automatic FER system commonly consists of foursteps Preprocessing feature extraction feature selection

and classification of facial expressions In the preprocessingstep face region is first detected and then extracted from theinput image because it is the area that contains expression-related information e most well-known and commonalgorithm used for face detection is the ViolandashJones objectdetection algorithm [2] Subsequently in the feature ex-traction step distinguishable features are extracted from theface image e two popular approaches for feature ex-traction are geometric-based feature extraction and ap-pearance-based feature extraction In the geometric-basedtechniques the facial landmark points are first detected andthen combined into a feature vector which encodes geo-metric information of face from the position distance andangle [3] e appearance-based techniques characterize theappearance information brought by different facial move-ments Next a subset of relevant features is selected in thefeature selection step which contains more discriminatorypower to classify different classes In the last classification

HindawiMathematical Problems in EngineeringVolume 2019 Article ID 9185481 12 pageshttpsdoiorg10115520199185481

step classifiers like K-nearest neighbor (KNN) [4] andsupport vector machine (SVM) [5] are first trained and thenused to classify the input data

Although a lot of work has been done to develop a robustFER system we find that several common problems still existin the real-time environment which hinder the developmentof the FER system (i) e extracted features are sensitive tothe change in illumination occlusion and noiseat meansa slight change in illumination occlusion and noise mayinfluence the recognition accuracy rate (ii) e large datadimension is another problem which deteriorates the per-formance of such systems

e contributions of the proposed work are as follows

(i) A dual-feature fusion technique is proposed in thiswork for effective and efficient classification of facialexpressions in the unconstrained environment

(ii) e proposed framework is based on local andglobal features which make the proposed frame-work robust to change in occlusions illuminationand noise

(iii) Feature selection process is used to obtain thediscriminative features where the redundant fea-tures are discarded e reduction in feature vectorlength also reduces the time complexity whichmakes the proposed framework suitable for real-time applications

e rest of the paper is organized as follows Section 2presents the related work Section 3 provides the descriptionof the materials and methods Experimental results arepresented in Section 4 Finally conclusion is provided inSection 5

2 Related Work

Numerous methods for facial expression recognition havebeen developed due to its increased importance esemethods are mainly categorized into geometric-based andappearance-based methods based on feature extractions

In geometric-based methods information such as shapeof the face and its components are used for feature ex-traction e first important and challenging step in thegeometric-based method is to initialize a set of facial pointsas the facial expression evolves over time e study pre-sented in [6] employed the elastic bunch graph matching(EBGM) algorithm for initialization of facial points ediscriminative features are also selected from triangle andline features with the multiclass AdaBoost algorithm Sunet al [7] proposed an effective method for the selection ofoptimized active face regions ey used convolution neuralnetwork (CNN) to extract features from optimized activeface regions e method used by Hsieh et al [8] wasbased on the active shape model (ASM) ey employedASM to extract different facial expression regionsSimilarly Zangeneh and Moradi [9] first used the activeappearance model (AAM) to reveal the important facialpoints and then differential geometric features are extractedfrom those facial points In the geometric-based features

extraction techniques it is difficult to track and initializefacial feature points in real time If the error occurs duringfacial point initialization process then this error deterioratesthe overall feature extraction process

On the contrary appearance-based features extractionmethods encode the face appearance variations withouttaking muscle motion into account Chen et al [10] in-troduced the multithreading cascade of Speeded Up RobustFeatures (McSURF) which improve the recognition ac-curacy rate Cruz et al [11] explore the temporal derivativeand adjacent frames by using new framework known astemporal patterns of oriented edge magnitudes e casesof out-of-plane head rotations are handled using rotation-reversal invariant HOD presented by Chen et al [12] eyalso developed the cascade learning model to boost theclassification process Alphonse and Dharma [13]employed the maximum response-based directional tex-ture pattern and number pattern for feature extraction eperformance is tested in the constrained and unconstrainedenvironments Recently the work proposed in [14]employed spatiotemporal convolution to jointly extract thetemporal dynamic and multilevel appearance feature offacial expressions Another promising method to enhancethe performance of random forest is proposed in [15] eyreduce the influence of various distortions like occlusionand illumination by extracting the robust features fromsalient facial patches Sajjad et al [16] presented a modelintegrating the histogram-oriented gradient with theuniform-local ternary operator for the extraction of facialfeatures e performance of the proposed method wastested on facial expression images which contains noise andpartial occlusions In another interesting approach theauthors proposed a new framework named local binaryimage cosine transform for computationally efficient fea-ture extractionselection [17] Munir et al [18] proposed amerged binary pattern code (MBPC) to represent the facetexture information ey performed experiments on real-time images In order to normalize the illumination effectsthey preprocessed the images using the fast Fouriertransform and contrast limited adaptive histogramequalization Liu et al [19] made use of deep network tolearn the midlevel representation of face ey tested theeffectiveness of their proposed method both on wild en-vironment images and lab-controlled data

Apart from the appearance-based or geometric-basedfeature extraction fusion of this two-feature extractionmethod is also a promising trend Zhang et al [20] combinedboth texture and geometric-based features to maintainreasonable amount of tolerance against noise and occlusioney used an active shape model and SIFTfor geometric andappearance-based feature respectively To inherit the ad-vantages of geometric and appearance information Yanget al [21] fused deep geometric features and LBP-basedappearance features ey also proposed an improvedrandom forest classifier for effective and efficient recognitionof facial expressions In the method of Tsai and Chang [22]features are extracted via Gabor filter discrete cosinetransform and angular radial transform In the work ofGhimire et al [23] first the face local specific regions were

2 Mathematical Problems in Engineering

selected and then central moments were normalized A localbinary pattern descriptor is used for the extraction ofgeometric and appearance-based features respectively

In this paper different from other methods we select thefacial informative local regions instead of dividing the faceimage into nonoverlapping blocks Such representations canimprove the classification performance compared with theblock-based image representation e appearance-basedfeature is computed from local face regions and also from thewhole face area ese features are then fused which providemore robust features

3 Materials and Methods

e working of the proposed framework based on dual-feature fusion is illustrated in Figure 1 Initially the faceportion is detected and extracted from input images usingthe ViolandashJones algorithm [2] For dual-feature fusion wefirst detect the facial landmark point on the face image andthen the important local regions are locatedeWeber localdescriptor (WLD) excitation and orientation image is alsogenerated from the input images In next step DCT is usedto select the high variance features from local regions alongwith excitation and orientation image of WLD In order toimprove the performance both types of features are thenfussed using the score-level fusion

31 Face Detection and Landmark Position Estimation Inorder to extract the region of interest (ie face portion) weutilized the ViolandashJones algorithm [2] in our study which ismostly cited in literature and also considered as a fast andaccurate object detection algorithm [24]

e spatial misalignment usually occurs due to theexpression and pose variations in the face image Divisionof the face image into nonoverlapped blocks or exploitingholistic features cannot resolve this issue [25] Admit-tedly the intraclass difference is increased due to varia-tion in face appearance because of expressions and facialposes In that case the local features are more robust tothese changes as compared to holistic features ere aresome reliable and stable regions which preserve moreuseful information to deal with these changes at is whyin this study we extract the features from inner faciallandmarks rather than extracting the features from wholeface image

For this purpose we used the method presented byKazemi and Sullivan [26] in which the face landmark po-sition is estimated from subset of pixel intensities usingensemble of regression trees is method is highly effectiveto locate the landmark position not only in the face withneutral expression but also in the face with variation indifferent expressions

After landmark position estimation we use the facialpoint location to divide the face image into 29 local regionse local feature is extracted from all these local regions Inorder to reduce the data dimensions we do not requireexhaustive search technique as performed in [23] to search

for a subset of local regions among 29 local regions becauseour feature selection method is more efficient and effective

32 Construction of WLD Excitation and Orientation Imagee Weber local descriptor is proposed by Chen et al [27]which is inspired from Weberrsquos law WLD consist of twomain components namely differential excitation and gra-dient orientation e differential excitation componentrepresents the intensity differences of the neighbor pixel andthe center pixel where the gradient orientation of the centerpixel is described by the gradient orientation componentBoth the components provide the local texture description ofan image

Formally the differential excitation component can bedefined as

ξm xc( 1113857 arctan α 1113944

pminus 1

i0

xi minus xc

xc

⎛⎝ ⎞⎠ (1)

where the arctangent is used to suppress the noise sideeffect and also to avoid the output of being too large eneighbor pixels are denoted as xi(i 0 1 2 3 p minus 1)while xc represents the center pixel Similarly the differ-ential orientation component of an image can be defined asfollows

ξo xc( 1113857 arctanx1 minus x5

x3 minus x71113888 1113889 (2)

where the intensity difference is indicated by x3 minus x7 andx1 minus x5 in the x and y directions

Figures 2 and 3 illustrate the WLD excitation and ori-entation component images

33 DCT-Based Feature Selection and Fusion We cancompute the DCT of an input scanned image dxy of sizeM times N by using the expression as defined in equation (3)[28] For all values of u 0 1 2 M minus 1 andv 0 1 2 N minus 1 the expression of equation (1) must beevaluated Also given Duv for x 0 1 2 M minus 1 andy 0 1 2 N minus 1 dxy can be obtained by using theinverse DCT transform which is mentioned in equation (4)Note that both equations (3) and (4) consist of a two-di-mensional pair of DCT where x and y are spatial coordinatesand u and v refers to frequency variables

Duv ρ(u)ρ(v) 1113944Mminus 1

x01113944

Nminus 1

y0dxy

cos(2x + 1)uπ

2M1113890 1113891 cos

(2y + 1)vπ2N

1113890 1113891

(3)

duv 1113944Mminus 1

u01113944

Nminus 1

v0ρ(u)ρ(v) Duv

cos(2x + 1)uπ

2M1113890 1113891 cos

(2y + 1)vπ2N

1113890 1113891

(4)

Mathematical Problems in Engineering 3

Input image

Different excitation image Orientation image

The differential excitation component The orientation component

ξm (xc) = arctan (αsum (x1 ndash xcxc)pndash1i=0 ξ0 (xc) = arctan (x1 ndash x5)(x3 ndash x7)

Figure 2 WLD excitation and orientation component

Local regions

WLD images

DCT-based FS

DCT-based FS

Classification using SVM

Figure 1 Proposed framework flow diagram


ρ(u)

1

M

1113970

u 0

2

M

1113970

u 1 2 3 M minus 1

⎧⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎩

ρ(v)

1N

1113970

v 0

2N

1113970

v 1 2 3 N minus 1

⎧⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎩

u 0 1 2 M minus 1

v 0 1 2 N minus 1

x 0 1 2 M minus 1

y 0 1 2 N minus 1

⎧⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎩

(5)

P(u v) is the power spectrum of image dxy and can bedefined as

P(u v) Duv

111386811138681113868111386811138681113868111386811138682 (6)

After selection of appearance-based and geometric-based features we employed a score-level fusion strategy tocombine these features Feature-level fusion and score-levelfusion are the two fusion strategies which are used widely inthe literature In the feature-level fusion different featurevectors are simply concatenated after normalization processIn contrast to the feature-level fusion a distance-basedclassifier is used in the score-level fusion to compute thedistance between the feature vector of training and testingsamples e feature-level fusion mainly produces largedata dimension [29] that is why we prefer score-level fusionin this study In the score-level fusion the extracted

appearance- and geometric-based DCTfeatures are stored inFSap and FSgeo respectively ese features are computed forall training FStr and testing FSte samples Afterward scorevectors namely Sap and Sgeo are produced by computing thedistance between training samples and all the testing samplesof appearance and geometric feature sets In order to per-form normalization the min-max method of normalization[30] is used which is described as

Siprime

Si minus Min(S)

Max(S) minus Min(S) (7)

where the original score i th entry is represented by Si eminimum and the maximum values of the score is denotedby Min(S) and Max(S) Finally the product rule or the sumrule method is used to normalize the score vectors [30]

e procedure of feature extraction and fusion is pre-sented in Algorithm 1

34 Support Vector Machine (SVM) for ExpressionClassification For multi and binary classification problemthe SVM [31] acts as a more powerful toole SVM draws ahyperplane between the two classes by maximizing themargin between the closest points of the class and hyper-plane e decision function for class labels yi ∓1 andtraining data xi(i 1 2 3 N) can be formulated as [23]

f(x) sign wTx + b1113872 1113873 (8)

where the hyperplane separation is denoted by wTx + b 0In order to handle the multiclass problem we have usedSVM with radial basic function kernel implemented aslibsvm [32] and is publicly available for use

Figure 3 First row represents the original image second row is the sample images of excitation component and third row depictsorientations of component images


4 Experimental Results and Discussions

To evaluate the performance of the proposed framework weused 3 publicly available benchmarking databases namelyMMI database extended Cohn-Kanade (CK+) and staticface in the wild (SFEW)

(i) MMI database this image database [33] containsboth video sequences and static images which in-clude head movements and posed expressions Itconsists of images of high resolutions of 88 subjectsand over 2900 videos of male and female For ourexperiment we have selected different video se-quences and extracted a total of 273 images fromthese sequences

(ii) Extended Cohn-Kanade (CK+) this database con-tains 593 video sequence of 123 subjects [34] esubjects are origins from Latino Asian and Afri-can-American and aged from 18 to 30 years Wehave selected different video sequences and ob-tained 540 static images of six basic expressions

(iii) Static face in the wild (SFEW) the SFEW [35]contains real-time movie images which are capturedin unconstrained settings e images are havingdifferent variations like noise pose variation andhigh illumination changes We have taken 291images from the available 1394 images in thedatabase

Sample images of each database is shown in Figure 4 andTable 1 illustrates the number of images taken from MMICK+ and SFEW database

To make maximum use of the available data weemployed 5-fold and 10-fold cross validation for all theexperiments To get the better picture of the facial expressionrecognition accuracy average accuracy rate and confusionmatrices are given across all the three datasets

41 Experiment on MMI CK+ and SFEW Database issection shows the results obtained using MMI CK+ andSFEW datasets MMI dataset contained most of the spon-taneous expressions e proposed framework achieved anaverage recognition accuracy of 96 and 9862 re-spectively for MMI and CK+ database e confusionmatrix of classifying 7 facial expressions for MMI datasetand 6 basic expressions for CK+ is shown in Tables 2 and 3respectively

In Table 2 among the seven facial expressions neutral andsad expressions are the easiest with an average recognitionaccuracy rate of 100 which is followed by happy and sur-prised In contrast angry and fear are the most difficult ex-pressions for classification As shown in the table the fearexpression is mostly confused with neutral and surprisedwhich is expected because of the structural similarities [36]Furthermore the anger facial expression is mostly mis-classifiedwith disgust and neutral expressionsis is probablybecause of the wrinkles of the forehead in anger expressionwhich is also the characteristics of disgust expression

e confusion matrix in Table 3 depicts that disgust sadand happy expressions are classified with 100 recognitionaccuracy rate which is followed by surprised and angerexpressions e recognition accuracy for fear expression isslightly deviated at 95 e results indicate that the fearexpression misclassified either as anger or disgust emotione reason is that the fear disgust and anger expressionsdemonstrated similar muscle activities [37] Moreover it isalso observed that the average recognition accuracy rate ofthe CK+ dataset is slightly higher than theMMI datasetisis because the CK+ dataset contains more expressiveemotions

e confusion matrix for SFEW results is shown inTable 4 e performance on the SFEW database is low ascompared to MMI and CK+ databases is is because theimages of the SFEW database are captured in the un-controlled environment (real-world images) and are morechallenging to classify as compared to other datasets eaverage recognition accuracy rate of 502 is obtainedusing the SFEW database By inspecting the recognitionaccuracy rate of each expression we observed that sad fearand happy expressions are more accurately recognizedHowever the disgust expression obtained the smallestrecognition accuracy of 317

Table 5 illustrates the comparative assessment of theproposed method with the existing state-of-the-art[6 10ndash14] methods In literature the FER system presentedin [11] has achieved the highest recognition accuracy rate of9366 which works on the nonoverlapping patches But intheir method the length of their code is controlled by a newcoding scheme which makes their process more complex forreal-time FER systems e results show that the perfor-mance of our proposed method is superior as compared toexisting techniques in terms of average recognition accuracyFurthermore it is also notable that recognition accuracy rateper expression of our proposed method is also high ascompared to other methods

In Table 6 the results for CK+ database are comparedwith the state-of-the-art methods e average recognitionaccuracy rate of our method is highly competitive with othermethods Although the performance of the method pre-sented in [14] is 111 higher than our method the use of 3Dconvolution neural network makes their method compu-tationally more expensive

Figure 5 illustrates the comparative assessment of theproposed method with other methods on the SFEW data-base It is evident from the results that our proposed methodachieved better results as compared to existing methods inthe literature e average recognition accuracy rate of ourproposed method is 502 For the same dataset present inthe studies [13 19 20 38ndash40] the average accuracy rateswere 261 3014 338 440 4931 and 483 re-spectively e results depict that our strategy of the dual-feature fusion is more appropriate for FER in the un-controlled environment e recognition accuracy rate issignificantly degraded on SFEW as compared to the resultson MMI and CK+ due to its challenging condition egchange in illumination and large pose variations


42 Robustness against Noise and Occlusions In the un-controlled environment noise and occlusions are the mainfactors to degrade the image quality and reduce the facialexpression recognition accuracy rate It is required for anyFER system to perform well in the presence of noise andpartial occlusions In this section we examine the robustnessof our proposed method in the presence of noise and partialocclusions

To check the robustness against noise we randomlyadded salt and pepper noise of different levels to the imagesof MMI and CK+ databaseis type of noise is composed oftwo components

e first component is the salt noise which occurs as abright spot in the image and the second component is thepepper noise which appears as a dark spot As shown inFigure 6 the noise density was increased up to 005 level

Fear Disgust Angry Surprised Sad Happy

MMI

CK+

SFEW

Figure 4 Sample images taken from MMI CK+ and SFEW database

Input Training sample images Itrain with size M times N

Testing sample images ItestOutput FusedfeatProcedure

(1) For each Itrain do(2) Compute WLD images I

aptr and local region images I

geotr

(3) For each Iaptr and I

geotr do

(4) Compute FSaptr and FSgeotr using equations (3) and (4)(5) FSaptr langFSap1 FSap2 FSapsaprang sap size(FSap)

(6) FSgeotr langFSgeo1 FSgeo2 FSgeosaprang sap size(FSgeo)

(7) End For(8) End For(9) For each Itest do(10) ComputeWLD images I

apte and local region images I

geote

(11) For each Iapte and I

geote do

(12) Compute FSapte and FSgeote using equations (3) and (4)

(13) FSapte langFSap1 FSap2 FSapsaprang sap size(FSap)

(14) FSgeote langFSgeo1 FSgeo2 FSgeosaprang sap size(FSgeo)

(15) End For(16) End For(17) For each Itrain do(18) Sap Compute_Distance(FSaptr FSapte )

(19) Sgeo Compute_Distance(FSgeotr FSgeote )

(20) End For(21) For each Itrain do(22) Normalize Sap and Sgeo using equation (7)(23) End For(24) Fusedfeat Score_Level_Fusion(Sap Sgeo)

ALGORITHM 1 e procedure of feature extraction and fusion


because in the real-time system the average noise of thislevel is normally observed [16]

e results illustrated in Figure 7 shows that the rec-ognition accuracy rate of our proposed method does notsignificantly reduce with increase in variance of salt andpepper noise We have also observed that the recognitionaccuracy rate of the CK+ database is more stable in the

presence of noise as compare to the MMI database is isbecause the expression of CK+ is more representative

In order to assess the proposed method performance inthe presence of occlusions we have added a block of randomsize to the test images e range of block size starting from[15times15] to [55times 55] randomly placed to the face images areshown in Figure 8

Table 3 Confusion matrix of recognition accuracy for CK+ database

Fear () Disgust () Angry () Surprised () Sad () Happy ()Fear 950 28 22 0 0 0Disgust 0 1000 0 0 0 0Angry 0 0 978 0 222 0Surprised 0 0 0 989 111 0Sad 0 0 0 0 1000 0Happy 0 0 0 0 1000

Table 4 Confusion matrix of the recognition accuracy for the SFEW database

Fear () Disgust () Angry () Surprised () Sad () Happy ()Fear 640 00 60 140 80 80Disgust 73 317 171 122 195 122Angry 60 100 420 100 140 180Surprised 220 00 160 420 120 80Sad 80 80 80 20 640 100Happy 100 40 140 80 100 540

Table 5 Confusion matrix of recognition accuracy for MMI

Method Fear () Disgust () Angry () Surprised () Sad () Happy () Mean ()Chen et al [10] 6840 6530 6950 8260 6820 8390 7300Cruz et al [11] 9136 9227 8844 9763 9353 9875 9366Ghimire et al [6] 7000 8000 7000 9000 7333 9250 79305Chen et al [12] 7650 6040 7020 8420 6210 8120 7240Alphonse and Dharma [13] 8130 8130 8200 9000 7670 8333 8244Yu et al [14] 8124 8821 8324 8529 8577 9322 8616Proposed method 9270 9490 9110 9740 10000 9740 9558

Table 1 Number of selected images per expression from MMI CK+ and SFEW database

DatasetExpression

Neutral Fear Disgust Angry Surprised Sad Happy TotalMMI 36 41 39 45 39 34 39 273CK+ NA 90 90 90 90 90 90 540SFEW NA 50 41 50 50 50 50 291

Table 2 Confusion matrix of recognition accuracy for MMI database

Neutral () Fear () Disgust () Angry () Surprised () Sad () Happy ()Neutral 100 0 0 0 0 0 0Fear 488 927 0 0 244 0 0Disgust 256 0 949 256 0 0 0Angry 444 0 444 911 0 0 0Surprised 0 256 0 0 974 0 0Sad 0 0 0 0 0 100 0Happy 0 256 0 0 0 0 974


p = 001 p = 002 p = 003 p = 004 p = 005

(a)

p = 001 p = 002 p = 003 p = 004 p = 005

(b)

Figure 6 Sample images of salt and pepper noise from (a) MMI and (b) CK+ where p represents the noise density

60

55

50

45

40

35

30

25

20

15

10

5

Acc

urac

y ra

te (

)

Reference[20]

Reference[19]

Reference[13]

Reference[38]

Reference[39]

Reference[40]

Proposed

Assessment with other methods

Performance () comparison on SFEW database

Figure 5 Comparison between existing method and proposed approach based on recognition accuracy

90

80

70

60

50

40

30

20001 002 003 004 005

Noise density

Acc

urac

y ra

te (

)

MMI databaseCK+ database

Figure 7 Recognition accuracy of MMI and CK+ databases in the presence of noise


e average recognition accuracy rates for both MMIand CK+ are illustrated in Table 7 e results of MMI showthat the accuracy rate decreased up to 36 when the blocksize increased from [15times15] to [45times 45] Howeverthe recognition drops down by 17 when the block size[55times 55] is used is is because most of the important facial

points are hidden due to the large block size In contrast therecognition accuracy on the CK+ database only decreases by75 when [55times 55] block size was used in the experimentsIt is foreseeable that the recognition accuracy reaches to zeroin the presence of total occlusion

To prove the robustness of our proposed method againstnoise and occlusions we also compared the performancewith the existing method [16] as shown in Figures 9 and 10emethods presented in [16] are selected due to their state-of-the-art performance onMMI and CK+ database and theyalso used a similar ratio of noise density and block size Fromthe results we can easily conclude that our dual-featurefusion method is more robust to noise and occlusions ascompared to the methods presented in [16] due to the lessdecline in recognition accuracy

15 times 15 25 times 25 35 times 35 45 times 45 55 times 55

(a)


(b)

Figure 8 Sample images of occlusion from (a) MMI and (b) CK+ databases with varying block size

Table 6 Confusion matrix of recognition accuracy for CK+


Accu

racy

rate

()

100

90

80

70

60

50

40001 002 003 004 003 004005

Noise density001 002

Dual features-MMIHOG-U-LTP-MMI [16]

Dual features-CK+HOG-U-LTP-CK+ [16]

Figure 9 Comparison graph of the proposedmethod accuracy rateassessment with other methods in the presence of noise

Accu

racy

rate

()

100

90

80

70

60

50

40

Dual features (CK+)HOG-U-LTP [16] (CK+)

Dual features (MMI)HOG-U-LTP [16] (MMI)

(25 times 25) (35 times 35) (45 times 45) (55 times 55)(15 times 15)Block size

Figure 10 Competitive assessment with the existing method in thepresence of occlusions

Table 7 Assessment of MMI and CK+ results in the presence ofocclusions

Block size MMI () CK+ ()[15times15] 919 981[25times 25] 908 983[35times 35] 905 906[45times 45] 883 885[55times 55] 751 906


5 Conclusion and Future Work

Facial expression recognition in the real-world case is a long-standing problem e low image quality partial occlusionsand illumination variation in the real-word environmentmake the feature extraction process more challenging In thispaper we exploit both texture and geometric features foreffective facial expression recognition e effective geo-metric features are introduced in this paper from faciallandmark detection which can capture the facial configurechanges Considering that the geometric feature extractionmay fail under various conditions the addition of texturefeature with geometric features is useful for capturing theminor changes in expressions WLD is utilized for the ex-traction of texture feature which is more effective to capturethe facial subtle changes Furthermore we have employedscore-level fusion for fusion of geometric and texture fea-tures which results in decreasing the number of featureseperformance of the proposed approach is evaluated onstandard databases like MMI CK+ and SFEW and theresults are compared with the state-of-the-art approachese effectiveness of our proposed dual-feature fusionstrategy is verified by different experimental results

Although WLD works well on the face images for theextraction of salient features the variation of local intensitycannot effectively be represented by using the standardWLDbecause it neglects different orientations of the neighbor-hood pixel In future work we are planning to address thisissue along with the experimentation with ethnographicdatasets

Data Availability

e authors confirm that the data generated or analyzed andthe information supporting the findings of this study areavailable within the article

Conflicts of Interest

e authors declare no conflicts of interest

Authorsrsquo Contributions

All the co-authors have made significant contribution inconceptualization data analysis experimentations scientificdiscussions preparation of original draft and revision andorganization of the paper

Acknowledgments

is study was supported by the Deanship of ScientificResearch King Saud University Riyadh Saudi Arabiathrough the Research Group under Project RG-1439-039

References

[1] Y T Uhls M Michikyan J Morris et al ldquoFive days atoutdoor education camp without screens improves preteenskills with nonverbal emotion cuesrdquo Computers in HumanBehavior vol 39 pp 387ndash392 2014

[2] P Viola andM Jones ldquoRapid object detection using a boostedcascade of simple featuresrdquo in Proceedings of the 2001 IEEEComputer Society Conference on Computer Vision and PatternRecognition 2001 (CVPR 2001) pp 511ndash518 Kauai HI USADecember 2001

[3] S Jain C Hu and J K Aggarwal ldquoFacial expression rec-ognition with temporal modeling of shapesrdquo in Proceedings ofthe 2011 IEEE International Conference on Computer VisionWorkshops (ICCV Workshops) pp 1642ndash1649 BarcelonaSpain November 2011

[4] N S Altman ldquoAn introduction to kernel and nearest-neighbor nonparametric regressionrdquo =e American Statisti-cian vol 46 no 3 pp 175ndash185 1992

[5] I Kotsia and I Pitas ldquoFacial expression recognition in imagesequences using geometric deformation features and supportvector machinesrdquo IEEE Transactions on Image Processingvol 16 no 1 pp 172ndash187 2007

[6] D Ghimire J Lee Z-N Li and S Jeong ldquoRecognition offacial expressions based on salient geometric features andsupport vector machinesrdquoMultimedia Tools and Applicationsvol 76 no 6 pp 7921ndash7946 2017

[7] A Sun Y Li Y-M Huang Q Li and G Lu ldquoFacial ex-pression recognition using optimized active regionsrdquo Hu-man-Centric Computing and Information Sciences vol 8p 33 2018

[8] C-C Hsieh M-H Hsih M-K Jiang Y-M Cheng andE-H Liang ldquoEffective semantic features for facial expressionsrecognition using SVMrdquo Multimedia Tools and Applicationsvol 75 no 11 pp 6663ndash6682 2016

[9] E Zangeneh and A Moradi ldquoFacial expression recognition byusing differential geometric featuresrdquo =e Imaging ScienceJournal vol 66 no 8 pp 463ndash470 2018

[10] J Chen Z Luo T Takiguchi and Y Ariki ldquoMultithreadingcascade of SURF for facial expression recognitionrdquo EURASIPJournal on Image andVideo Processing vol 2016 no1 p 37 2016

[11] E A S Cruz C R Jung and C H E Franco ldquoFacial ex-pression recognition using temporal POEM featuresrdquo PatternRecognition Letters vol 114 pp 13ndash21 2018

[12] J Chen T Takiguchi and Y Ariki ldquoRotation-reversal invariantHOG cascade for facial expression recognitionrdquo Signal Imageand Video Processing vol 11 no 8 pp 1485ndash1492 2017

[13] A S Alphonse and D Dharma ldquoNovel directional patternsand a generalized supervised dimension reduction system(GSDRS) for facial emotion recognitionrdquo Multimedia Toolsand Applications vol 77 no 8 pp 9455ndash9488 2018

[14] Z Yu G Liu Q Liu and J Deng ldquoSpatio-temporal con-volutional features with nested LSTM for facial expressionrecognitionrdquo Neurocomputing vol 317 pp 50ndash57 2018

[15] Y Liu X Yuan X Gong Z Xie F Fang and Z LuoldquoConditional convolution neural network enhanced randomforest for facial expression recognitionrdquo Pattern Recognitionvol 84 pp 251ndash261 2018

[16] M Sajjad A Shah Z Jan S I Shah S W Baik andI Mehmood ldquoFacial appearance and texture feature-basedrobust facial expression recognition framework for sentimentknowledge discoveryrdquo Cluster Computing vol 21 no 1pp 549ndash567 2018

[17] S A Khan A Hussain and M Usman ldquoReliable facial ex-pression recognition for multi-scale images using weber localbinary image based cosine transform featuresrdquo MultimediaTools and Applications vol 77 no 1 pp 1133ndash1165 2018

[18] A Munir A Hussain S A Khan M Nadeem and S ArshidldquoIllumination invariant facial expression recognition using


selected merged binary patterns for real world imagesrdquo Optikvol 158 pp 1016ndash1025 2018

[19] M Liu S Li S Shan and X Chen ldquoAU-inspired deepnetworks for facial expression feature learningrdquo Neuro-computing vol 159 pp 126ndash136 2015

[20] L Zhang D Tjondronegoro and V Chandran ldquoFacial ex-pression recognition experiments with data from televisionbroadcasts and the World Wide Webrdquo Image and VisionComputing vol 32 no 2 pp 107ndash119 2014

[21] B Yang J-M Cao D-P Jiang and J-D Lv ldquoFacial ex-pression recognition based on dual-feature fusion and im-proved random forest classifierrdquo Multimedia Tools andApplications vol 77 no 16 pp 20477ndash20499 2018

[22] H-H Tsai and Y-C Chang ldquoFacial expression recognition usinga combination of multiple facial features and support vectormachinerdquo Soft Computing vol 22 no 13 pp 4389ndash4405 2018

[23] D Ghimire S Jeong J Lee and S H Park ldquoFacial expressionrecognition based on local region specific features and supportvector machinesrdquoMultimedia Tools and Applications vol 76no 6 pp 7803ndash7821 2017

[24] M Kolsch and M Turk ldquoAnalysis of rotational robustness ofhand detection with a viola-jones detectorrdquo in Proceedings ofthe 17th International Conference on Pattern Recognition2004 (ICPR 2004) pp 107ndash110 Cambridge UK August 2004

[25] Z Zhang L Wang Q Zhu S-K Chen and Y Chen ldquoPose-invariant face recognition using facial landmarks and weberlocal descriptorrdquo Knowledge-Based Systems vol 84 pp 78ndash88 2015

[26] V Kazemi and J Sullivan ldquoOne millisecond face alignmentwith an ensemble of regression treesrdquo in Proceedings of theIEEE Conference on Computer Vision and Pattern Recognitionpp 1867ndash1874 Columbus OH USA June 2014

[27] J Chen S Shan C He et al ldquoWLD a robust local imagedescriptorrdquo IEEE Transactions on Pattern Analysis and Ma-chine Intelligence vol 32 no 9 pp 1705ndash1720 2010

[28] N Ahmed T Natarajan and K R Rao ldquoDiscrete cosinetransformrdquo IEEE Transactions on Computers vol C-23 no 1pp 90ndash93 1974

[29] Z Golrizkhatami and A Acan ldquoECG classification usingthree-level fusion of different feature descriptorsrdquo ExpertSystems with Applications vol 114 pp 54ndash64 2018

[30] M He S-J Horng P Fan et al ldquoPerformance evaluation ofscore level fusion in multimodal biometric systemsrdquo PatternRecognition vol 43 no 5 pp 1789ndash1800 2010

[31] C Cortes and V Vapnik ldquoSupport-vector networksrdquo Ma-chine Learning vol 20 no 3 pp 273ndash297 1995

[32] C-C Chang and C-J Lin ldquoLibsvmrdquo ACM Transactions onIntelligent Systems and Technology vol 2 no 3 pp 1ndash27 2011

[33] M Pantic M Valstar R Rademaker and L Maat ldquoWeb-based database for facial expression analysisrdquo in Proceedingsof the 2005 IEEE International Conference on Multimedia andExpo p 5 Amsterdam Netherlands July 2005

[34] P Lucey J F Cohn T Kanade J Saragih Z Ambadar andI Matthews ldquoe extended Cohn-Kanade dataset (CK+) acomplete dataset for action unit and emotion-specified expres-sionrdquo in Proceedings of the 2010 IEEE Computer Society Con-ference on Computer Vision and Pattern Recognition Workshops(CVPRW) pp 94ndash101 San Francisco CA USA June 2010

[35] A Dhall R Goecke S Lucey and T Gedeon ldquoStatic facialexpression analysis in tough conditions data evaluationprotocol and benchmarkrdquo in Proceedings of the IEEE In-ternational Conference on Computer Vision Workshops (ICCVWorkshops) pp 2106ndash2112 Barcelona Spain November2011

[36] M Yeasin B Bullot and R Sharma ldquoRecognition of facialexpressions and measurement of levels of interest fromvideordquo IEEE Transactions on Multimedia vol 8 no 3pp 500ndash508 2006

[37] U Mlakar and B Potocnik ldquoAutomated facial expressionrecognition based on histograms of oriented gradient featurevector differencesrdquo Signal Image and Video Processing vol 9no S1 pp 245ndash253 2015

[38] W Sun H Zhao and Z Jin ldquoAn efficient unconstrained facialexpression recognition algorithm based on stack binarizedauto-encoders and binarized neural networksrdquo Neuro-computing vol 267 pp 385ndash395 2017

[39] I Gogic M Manhart I S Pandzic and J Ahlberg ldquoFast facialexpression recognition using local binary features and shallowneural networksrdquo =e Visual Computer pp 1ndash16 2018

[40] W Sun H Zhao and Z Jin ldquoA visual attention based ROIdetection method for facial expression recognitionrdquo Neuro-computing vol 296 pp 12ndash22 2018


Hindawiwwwhindawicom Volume 2018

MathematicsJournal of


Mathematical Problems in Engineering

Applied MathematicsJournal of


Probability and StatisticsHindawiwwwhindawicom Volume 2018

Journal of


Mathematical PhysicsAdvances in

Complex AnalysisJournal of


OptimizationJournal of



Engineering Mathematics

International Journal of


Operations ResearchAdvances in

Journal of


Function SpacesAbstract and Applied AnalysisHindawiwwwhindawicom Volume 2018

International Journal of Mathematics and Mathematical Sciences


Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018Volume 2018

Numerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisAdvances inAdvances in Discrete Dynamics in

Nature and SocietyHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Dierential EquationsInternational Journal of

Volume 2018


Decision SciencesAdvances in


AnalysisInternational Journal of


Stochastic AnalysisInternational Journal of

Submit your manuscripts atwwwhindawicom

step classifiers like K-nearest neighbor (KNN) [4] andsupport vector machine (SVM) [5] are first trained and thenused to classify the input data

Although a lot of work has been done to develop a robustFER system we find that several common problems still existin the real-time environment which hinder the developmentof the FER system (i) e extracted features are sensitive tothe change in illumination occlusion and noiseat meansa slight change in illumination occlusion and noise mayinfluence the recognition accuracy rate (ii) e large datadimension is another problem which deteriorates the per-formance of such systems

e contributions of the proposed work are as follows

(i) A dual-feature fusion technique is proposed in thiswork for effective and efficient classification of facialexpressions in the unconstrained environment

(ii) e proposed framework is based on local andglobal features which make the proposed frame-work robust to change in occlusions illuminationand noise

(iii) Feature selection process is used to obtain thediscriminative features where the redundant fea-tures are discarded e reduction in feature vectorlength also reduces the time complexity whichmakes the proposed framework suitable for real-time applications

e rest of the paper is organized as follows Section 2presents the related work Section 3 provides the descriptionof the materials and methods Experimental results arepresented in Section 4 Finally conclusion is provided inSection 5

2 Related Work

Numerous methods for facial expression recognition havebeen developed due to its increased importance esemethods are mainly categorized into geometric-based andappearance-based methods based on feature extractions

In geometric-based methods information such as shapeof the face and its components are used for feature ex-traction e first important and challenging step in thegeometric-based method is to initialize a set of facial pointsas the facial expression evolves over time e study pre-sented in [6] employed the elastic bunch graph matching(EBGM) algorithm for initialization of facial points ediscriminative features are also selected from triangle andline features with the multiclass AdaBoost algorithm Sunet al [7] proposed an effective method for the selection ofoptimized active face regions ey used convolution neuralnetwork (CNN) to extract features from optimized activeface regions e method used by Hsieh et al [8] wasbased on the active shape model (ASM) ey employedASM to extract different facial expression regionsSimilarly Zangeneh and Moradi [9] first used the activeappearance model (AAM) to reveal the important facialpoints and then differential geometric features are extractedfrom those facial points In the geometric-based features

extraction techniques it is difficult to track and initializefacial feature points in real time If the error occurs duringfacial point initialization process then this error deterioratesthe overall feature extraction process

On the contrary appearance-based features extractionmethods encode the face appearance variations withouttaking muscle motion into account Chen et al [10] in-troduced the multithreading cascade of Speeded Up RobustFeatures (McSURF) which improve the recognition ac-curacy rate Cruz et al [11] explore the temporal derivativeand adjacent frames by using new framework known astemporal patterns of oriented edge magnitudes e casesof out-of-plane head rotations are handled using rotation-reversal invariant HOD presented by Chen et al [12] eyalso developed the cascade learning model to boost theclassification process Alphonse and Dharma [13]employed the maximum response-based directional tex-ture pattern and number pattern for feature extraction eperformance is tested in the constrained and unconstrainedenvironments Recently the work proposed in [14]employed spatiotemporal convolution to jointly extract thetemporal dynamic and multilevel appearance feature offacial expressions Another promising method to enhancethe performance of random forest is proposed in [15] eyreduce the influence of various distortions like occlusionand illumination by extracting the robust features fromsalient facial patches Sajjad et al [16] presented a modelintegrating the histogram-oriented gradient with theuniform-local ternary operator for the extraction of facialfeatures e performance of the proposed method wastested on facial expression images which contains noise andpartial occlusions In another interesting approach theauthors proposed a new framework named local binaryimage cosine transform for computationally efficient fea-ture extractionselection [17] Munir et al [18] proposed amerged binary pattern code (MBPC) to represent the facetexture information ey performed experiments on real-time images In order to normalize the illumination effectsthey preprocessed the images using the fast Fouriertransform and contrast limited adaptive histogramequalization Liu et al [19] made use of deep network tolearn the midlevel representation of face ey tested theeffectiveness of their proposed method both on wild en-vironment images and lab-controlled data

Apart from the appearance-based or geometric-basedfeature extraction fusion of this two-feature extractionmethod is also a promising trend Zhang et al [20] combinedboth texture and geometric-based features to maintainreasonable amount of tolerance against noise and occlusioney used an active shape model and SIFTfor geometric andappearance-based feature respectively To inherit the ad-vantages of geometric and appearance information Yanget al [21] fused deep geometric features and LBP-basedappearance features ey also proposed an improvedrandom forest classifier for effective and efficient recognitionof facial expressions In the method of Tsai and Chang [22]features are extracted via Gabor filter discrete cosinetransform and angular radial transform In the work ofGhimire et al [23] first the face local specific regions were













ξm xc( 1113857 arctan α 1113944

pminus 1

i0

xi minus xc

xc

⎛⎝ ⎞⎠ (1)



x3 minus x71113888 1113889 (2)





x01113944

Nminus 1

y0dxy

cos(2x + 1)uπ

2M1113890 1113891 cos

(2y + 1)vπ2N

1113890 1113891

(3)

duv 1113944Mminus 1

u01113944

Nminus 1

v0ρ(u)ρ(v) Duv

cos(2x + 1)uπ

2M1113890 1113891 cos

(2y + 1)vπ2N

1113890 1113891

(4)


Input image





Local regions

WLD images

DCT-based FS

DCT-based FS




ρ(u)

1

M

1113970

u 0

2

M

1113970

u 1 2 3 M minus 1

⎧⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎩

ρ(v)

1N

1113970

v 0

2N

1113970

v 1 2 3 N minus 1

⎧⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎩

u 0 1 2 M minus 1

v 0 1 2 N minus 1

x 0 1 2 M minus 1

y 0 1 2 N minus 1

⎧⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎩

(5)


P(u v) Duv

111386811138681113868111386811138681113868111386811138682 (6)



Siprime

Si minus Min(S)





f(x) sign wTx + b1113872 1113873 (8)























MMI

CK+

SFEW






geotr


geotr do





geote


geote do




















DatasetExpression





p = 001 p = 002 p = 003 p = 004 p = 005

(a)

p = 001 p = 002 p = 003 p = 004 p = 005

(b)


60

55

50

45

40

35

30

25

20

15

10

5

Acc

urac

y ra

te (

)

Reference[20]

Reference[19]

Reference[13]

Reference[38]

Reference[39]

Reference[40]

Proposed




90

80

70

60

50

40

30

20001 002 003 004 005

Noise density

Acc

urac

y ra

te (

)








(a)


(b)




Accu

racy

rate

()

100

90

80

70

60

50

40001 002 003 004 003 004005





Accu

racy

rate

()

100

90

80

70

60

50

40











Data Availability






Acknowledgments


References



















































Journal of












Journal of







Volume 2018






Volume 2018



















ξm xc( 1113857 arctan α 1113944

pminus 1

i0

xi minus xc

xc

⎛⎝ ⎞⎠ (1)



x3 minus x71113888 1113889 (2)





x01113944

Nminus 1

y0dxy

cos(2x + 1)uπ

2M1113890 1113891 cos

(2y + 1)vπ2N

1113890 1113891

(3)

duv 1113944Mminus 1

u01113944

Nminus 1

v0ρ(u)ρ(v) Duv

cos(2x + 1)uπ

2M1113890 1113891 cos

(2y + 1)vπ2N

1113890 1113891

(4)


Input image





Local regions

WLD images

DCT-based FS

DCT-based FS




ρ(u)

1

M

1113970

u 0

2

M

1113970

u 1 2 3 M minus 1

⎧⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎩

ρ(v)

1N

1113970

v 0

2N

1113970

v 1 2 3 N minus 1

⎧⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎩

u 0 1 2 M minus 1

v 0 1 2 N minus 1

x 0 1 2 M minus 1

y 0 1 2 N minus 1

⎧⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎩

(5)


P(u v) Duv

111386811138681113868111386811138681113868111386811138682 (6)



Siprime

Si minus Min(S)





f(x) sign wTx + b1113872 1113873 (8)























MMI

CK+

SFEW






geotr


geotr do





geote


geote do




















DatasetExpression





p = 001 p = 002 p = 003 p = 004 p = 005

(a)

p = 001 p = 002 p = 003 p = 004 p = 005

(b)


60

55

50

45

40

35

30

25

20

15

10

5

Acc

urac

y ra

te (

)

Reference[20]

Reference[19]

Reference[13]

Reference[38]

Reference[39]

Reference[40]

Proposed




90

80

70

60

50

40

30

20001 002 003 004 005

Noise density

Acc

urac

y ra

te (

)








(a)


(b)




Accu

racy

rate

()

100

90

80

70

60

50

40001 002 003 004 003 004005





Accu

racy

rate

()

100

90

80

70

60

50

40











Data Availability






Acknowledgments


References



















































Journal of












Journal of







Volume 2018






Volume 2018








Input image





Local regions

WLD images

DCT-based FS

DCT-based FS




ρ(u)

1

M

1113970

u 0

2

M

1113970

u 1 2 3 M minus 1

⎧⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎩

ρ(v)

1N

1113970

v 0

2N

1113970

v 1 2 3 N minus 1

⎧⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎩

u 0 1 2 M minus 1

v 0 1 2 N minus 1

x 0 1 2 M minus 1

y 0 1 2 N minus 1

⎧⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎩

(5)


P(u v) Duv

111386811138681113868111386811138681113868111386811138682 (6)



Siprime

Si minus Min(S)





f(x) sign wTx + b1113872 1113873 (8)























MMI

CK+

SFEW






geotr


geotr do





geote


geote do




















DatasetExpression





p = 001 p = 002 p = 003 p = 004 p = 005

(a)

p = 001 p = 002 p = 003 p = 004 p = 005

(b)


60

55

50

45

40

35

30

25

20

15

10

5

Acc

urac

y ra

te (

)

Reference[20]

Reference[19]

Reference[13]

Reference[38]

Reference[39]

Reference[40]

Proposed




90

80

70

60

50

40

30

20001 002 003 004 005

Noise density

Acc

urac

y ra

te (

)








(a)


(b)




Accu

racy

rate

()

100

90

80

70

60

50

40001 002 003 004 003 004005





Accu

racy

rate

()

100

90

80

70

60

50

40











Data Availability






Acknowledgments


References



















































Journal of












Journal of







Volume 2018






Volume 2018








ρ(u)

1

M

1113970

u 0

2

M

1113970

u 1 2 3 M minus 1

⎧⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎩

ρ(v)

1N

1113970

v 0

2N

1113970

v 1 2 3 N minus 1

⎧⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎩

u 0 1 2 M minus 1

v 0 1 2 N minus 1

x 0 1 2 M minus 1

y 0 1 2 N minus 1

⎧⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎩

(5)


P(u v) Duv

111386811138681113868111386811138681113868111386811138682 (6)



Siprime

Si minus Min(S)





f(x) sign wTx + b1113872 1113873 (8)























MMI

CK+

SFEW






geotr


geotr do





geote


geote do




















DatasetExpression





p = 001 p = 002 p = 003 p = 004 p = 005

(a)

p = 001 p = 002 p = 003 p = 004 p = 005

(b)


60

55

50

45

40

35

30

25

20

15

10

5

Acc

urac

y ra

te (

)

Reference[20]

Reference[19]

Reference[13]

Reference[38]

Reference[39]

Reference[40]

Proposed




90

80

70

60

50

40

30

20001 002 003 004 005

Noise density

Acc

urac

y ra

te (

)








(a)


(b)




Accu

racy

rate

()

100

90

80

70

60

50

40001 002 003 004 003 004005





Accu

racy

rate

()

100

90

80

70

60

50

40











Data Availability






Acknowledgments


References



















































Journal of












Journal of







Volume 2018






Volume 2018



























MMI

CK+

SFEW






geotr


geotr do





geote


geote do




















DatasetExpression





p = 001 p = 002 p = 003 p = 004 p = 005

(a)

p = 001 p = 002 p = 003 p = 004 p = 005

(b)


60

55

50

45

40

35

30

25

20

15

10

5

Acc

urac

y ra

te (

)

Reference[20]

Reference[19]

Reference[13]

Reference[38]

Reference[39]

Reference[40]

Proposed




90

80

70

60

50

40

30

20001 002 003 004 005

Noise density

Acc

urac

y ra

te (

)








(a)


(b)




Accu

racy

rate

()

100

90

80

70

60

50

40001 002 003 004 003 004005





Accu

racy

rate

()

100

90

80

70

60

50

40











Data Availability






Acknowledgments


References



















































Journal of












Journal of







Volume 2018






Volume 2018



















DatasetExpression





p = 001 p = 002 p = 003 p = 004 p = 005

(a)

p = 001 p = 002 p = 003 p = 004 p = 005

(b)


60

55

50

45

40

35

30

25

20

15

10

5

Acc

urac

y ra

te (

)

Reference[20]

Reference[19]

Reference[13]

Reference[38]

Reference[39]

Reference[40]

Proposed




90

80

70

60

50

40

30

20001 002 003 004 005

Noise density

Acc

urac

y ra

te (

)








(a)


(b)




Accu

racy

rate

()

100

90

80

70

60

50

40001 002 003 004 003 004005





Accu

racy

rate

()

100

90

80

70

60

50

40











Data Availability






Acknowledgments


References



















































Journal of












Journal of







Volume 2018






Volume 2018








p = 001 p = 002 p = 003 p = 004 p = 005

(a)

p = 001 p = 002 p = 003 p = 004 p = 005

(b)


60

55

50

45

40

35

30

25

20

15

10

5

Acc

urac

y ra

te (

)

Reference[20]

Reference[19]

Reference[13]

Reference[38]

Reference[39]

Reference[40]

Proposed




90

80

70

60

50

40

30

20001 002 003 004 005

Noise density

Acc

urac

y ra

te (

)








(a)


(b)




Accu

racy

rate

()

100

90

80

70

60

50

40001 002 003 004 003 004005





Accu

racy

rate

()

100

90

80

70

60

50

40











Data Availability






Acknowledgments


References



















































Journal of












Journal of







Volume 2018






Volume 2018












(a)


(b)




Accu

racy

rate

()

100

90

80

70

60

50

40001 002 003 004 003 004005





Accu

racy

rate

()

100

90

80

70

60

50

40











Data Availability






Acknowledgments


References



















































Journal of












Journal of







Volume 2018






Volume 2018











Data Availability






Acknowledgments


References



















































Journal of












Journal of







Volume 2018






Volume 2018







































Journal of












Journal of







Volume 2018






Volume 2018








recognitionoffacialexpressionsundervaryingconditions...

Documents