machine learning approach

Download Machine Learning Approach

If you can't read please download the document

Upload: arib-elek

Post on 11-Nov-2014

40 views

Category:

Documents


3 download

DESCRIPTION

buku ini gratis

TRANSCRIPT

Machine Learning Approach for Smile Detection in Real Time Images

reference point for any automatic Machine visio Learning n-basedApproach system for Smileconditions. Detection A in new Real database, Time Images GENKI, is presented which attempting to achieve the same functionality. Among the contains pictures, photographed by the subjects themselves, problems, facial expression classification has been studied from thousands of different people in many different realmost, due to its utility in application domains of human world imaging conditions. Results suggest that human-level Harsha Yadappanavar Sin S real-life illumination behavior interpretation and HCI. Most of the surveyed expression recognition Shylaja accuracy systems, Department however, of Computer are based Science on frontal and view Engineering images of faces conditions is achievable with machine learning technology. Department of Information Science and Engineering without facial P.E.S hair and Institute glasses ofwhat Technology is unrealistic to expect in P.E.S Institu te of Technology these application domains. Bangalore 560085, Karnataka, INDIA III. 560085, BangaloreKarnataka, INDIA PROPOSED METHODOLOGY [email protected] [email protected] B. A survey of Facial Expression analysis and synthesis A. Face detection Stelios Krinidis, Ioan Buciu and Ioannis Pitas [2] have surveyed the issues associated with facial expressions Abstract Recognizing facial expressions of human analysis and synthesis. Analysis of human facial expressions beings by a computer is an interesting and challenging consists of three steps: face detection (tracking), facial problem. A system that performs the operation of face feature extraction and facial classification. Automatically detection and facial feature extraction accurately and in classification of human facial expressions is performed real time would form a big stepcoding in achieving a humanaccording to certain facial action schemes, using like interaction between man and machine. In this either spatio-temporal or spatial approaches. An individual paper, propose a method detecting in real specificwe model is obtained and for fitted into the Smile prearranged time Images by Then, machine approach. Machine prototype mesh. the learning constructed individual facial learning method involves training classifier and using model is deformed to produce facial a expression. it in real time images to determine smile. Our implemented approach has been tested on several C. Automatic Recognition of smiling and neutral facial Images from different databases and the achieved expressions results were found to be very satisfactory. For differentiating smile and neutral facial expression, Keywords- Machine Learning, Face Detection, Smile P.Li, S.L.Phung, A. Bouzerdom and F.H.C Tivive [6] have Detection, Haar Classifier, Facial expression analysis described architecture for detecting smile from neutral facial expression. The authors proposed face alignment I. have INTRODUCTION method to address localization error in existing face The detection of faces and the interpretation of facial detection methods. The smiling and neutral facial expression under varying conditions is an everyday expressions are differentiated using a novel neutral task for humans, which we fulfill without effort. the The identity, architecture. The architecture combines fixed and age, gender as well as the emotional state can be seen from adaptive non linear 2D filters. Fixed filter extract primitive someones face. The impression we get from a displayed features, and adaptive filters are trained to extract complex expression will affect our interpretation of the word features. After features are extracted, they are spoken classified by and even our attitude towards the speaker himself. Humor means of linear classifier and classify into smile or neutral . and sympathy are just two examples for essential informations that are primarily communicated via facial D. Facial expression recognition expressions. Hence, they have a high importance for our daily life even though we often are not aware of it. A positive face expression recognition method based on image the based mouthsystems in presented inother [7]. Estimation Foraround computer on the side, it stillof is the direction of face of person is performed hard to open up this very important channel first. of It has been realized by usingThe particle filter and Fast research operator.in Next communication. rapidly expanding facethe image around the mouth ispremise detectedthat based on estimated face processing is based on the information about a position. The feature is calculated by Gabor filter and facial users identity, state, and intent can be extracted from expression isthat recognized by can using SVM. images, and computers then react accordingly, i.e., by observing a persons facial expression. E. Practical Smile Detection Facial expressions are a form of nonverbal communication. They are smile a primary means of conveying A real time and robust detection system is presented social information among humans. The task of automatic in [1] by Jacob Whitehill, Gwen Littlewort, Ian Fasel, facial expression analysis be divided into three main Marian Barlett. The paper can explores whether current machine steps: face detection, facial feature extraction and learning methods can be used to develop an expression classification into expressions. recognition system that operates reliably in more realistic Real time face detection has been performed by using Face process is influenced by several Haar likerecognition Feature Classifiers. Haar-like features are factors such as shape, reflectance, pose, occlusion and illumination rectangular features that can indicate specific characteristics [4]. The face is aThe highly deformable object,features and facial in an image [9]. idea behind Haar-like is to expressions come in a wide variety of possible recognize objects or features based on the value of simple configurations. Time-varying changes include growth and features, instead of pixel values directly. The Haar-like removal have features of facial the advantage hair, wrinkles of very and fast sagging computation, of the skin caused by because it depends aging and only change on the in sum skin of color pixels because within ofa exposure to sunlight. Hence, thevalue. human face is more rectangle instead of every pixel Using anmuch integral difficult tocalculating model and the recognize than most industrial image for sum, one rectangle can be parts. This hard challenge is one of the reasons why computer computed with only four references, independent of the size vision research community has been devoted to face of the feature. recognition for quite some time. The classification error is represented by: This paper describes the machine learning methodology of detecting one of the facial expression, smile. The paper is organized as follows. Section 2 describes the alternate methods for smile detection. The methodology Hinge Loss is given by: used in detecting face region is presented in section 3. Section 4 describes in detail the method for training classifier and method for detecting smile. Finally results obtained are presented in section 5 Total loss shows how good a function (F, ) is : II. LITERATURE SURVEY This section deals with the survey of the recent work carried out by researchers in the domain of Facial Expression Analysis. The papers is to carried out to know Learning issurvey to findof a function minimize the loss:the existing techniques being used for detecting smile in static as well as in real time images. A. Automatic Analysis of Facial Expressions Maja Pantic an d Leon J.M. Rothkrantz [3] have explored and compared a number of approaches to facial expression detection and classification in static images and image sequences. The investigation compared automatic expression information extraction using facial motion analysis, holistic spatial pattern analysis, and analysis of facial features and their spatial arrangement. Analysis of facial expressions is an intriguing problem which humans solve with quite an apparent ease. They have identified three different but related aspects of the problem: face detection, facial expression information extraction, and facial expression classification. Capability of the human visual system in solving these problems has been discussed. It serves as aFig 1: Example of 56 Haar Like Features International Journal of Image Processing and Vision Sciences (IJIPVS) Vo lume-1 Issue-1 ,2012 32 33

Machine Le earning Approach

for Smile Detectio

on in Real Time Im mages

The simples st Haar-like feaature is a two-rrectangle featu ure. T his is calculatedd as the differeence between tthe s The value of th ixels within t the two rectan ngles. This w will sum of the pi in ndicate characcteristics like eedges or bordeers between ligght a ons. A three-r rectangle featuure indicates f for and dark regio in nstance a darkk line or a darkk thin area lyinng between ligght r ding on the si ize of the mid ddle rectangle. A regions, depend feature comp putes the diff fference betweeen F d Four-rectangle of rectangles, and so on. UUsing the integgral diagonal pairs im mage for commputing the su um will allow a two-rectanggle f calculated withh six reference es to the integ gral feature to be c im mage, a three e-rectangle feaature with eig ght references, , a f feature with nin ne references. four-rectangle f B earning Algoritthm B. Adaboost Le The base re esolution [4] oof the detector is 24x24, whi ich r arge amount ( more features than pixels) of p r of these featurres results in a la es available. A small number c possible featureed to form an eeffective classiifier. A variantt of A can be combinesed to select thhis small set oof features andd is la ater also used scade Classifierr. AdaBoost is ust o train the Cas

Am mong the possibble hyper plannes, SVMs seleect the one where the distance off the hyper plaane from the cclosest data points ( margin) is as llarge as possib ble. Giv ven set of traini ing data

For yi = = +1 For r yi = -1

Hyp

per plane is giv ven by

Wid

dth of the marg gin can be clas sified using

Classifi fication functio on which classi fies the data iss given by:

IV. Fig g 2: Features ap pplied to detectt face The first fea ature selected b by AdaBoost s seems to focus on th he property th hat the region oof the eyes is ooften darker thhan th he region of the nose and d cheeks. The e second featu ure s on the propertty that the eye s are darker thhan th he bridge of th he nose. selected relies Mathemat tically final cla assifier is repre sented by:

SMILE DET TECTION

Bef fore training thhe classifier, thhere are two mmain steps required for object classification: a training se et must be constru ucted for whichh the true clas ssifications of tthe objects are kno own, and a set t of object par rameters must be chosen that aree powerful dis scriminators foor classificatioon. Once a possible classifier ha as been identi ified, it is ne ecessary to measur re its accuracy. A. Tra aining Set Cru ucial for train ning the class sifier was col llection of databas se of images consisting of f smile and non smile images. The images collected had to span a widde range of imaging g conditions and also var riability in ag ge, gender, ethnicit ty, facial hair and glasses. HHence meetingg all these requirements, we colllected around 5519 images coonsisting of 258 sm mile and 261 non n smile faces. Afte er collection oof images, we converted all of them to graysca ale and then to standard size o of 64 X 64. B. Fea ature Selection Trai ining classifie ers is an opti imization probblem in a many-d dimen sional sp pace. Increasinng the dimens ionality ofces (IJIPVS) Volu ume-1 Issue-1 ,201 2

C ctor Machine C. Support Vec The goal o of SVM is to s separate the daata with a hyp per p ation of genera al hyper plane iis wx+b=0, w with plane. The equa x nt (vector), w the weights. TThe hyper pla ane s 0 for all the x k of x being a poin e the data, so t that wxk +b > 0 kare should separate o + b < 0 for all xj of other cla ss. If the data a j in n fact separatee one class, wx in this way, thhere is probablly more than oone w way to do it.Internaational Journal of I Image Processing a

and Vision Scienc 34

Machine Le earning Approach

for Smile Detectio

on in Real Time Im mages

th he space by y adding m more paramete ers makes t the o arder (and the difficulty grow ws exponentiaally w optimization haer of parameteers). There e are many fac cial with the nu mb f h change whenn a person smmiles. In order to o we features which lassifier and too lessen the d iffficulty level, w optimize the cl nly mouth as the potential feature becauuse, h m whether person n is have chosen o he major role inn identifying w mouth plays th -smiling. s smiling or non-

machin ne. SVM treatss the feature v values as supp ort vectors and ge enerates a tra ained classifie er model. Th his trained classifier model can be used to cl lassify new daata. In real time, ea ach image fram me is considereed as a static i image. The training g has been per formed by usinng 570 face im mages from the col lection of imaages in databasse. Following High level diagram m describes i in brief the methodology used for detectin ng smile.

Fig 3: Example of fa ace images from m database

We segmen nted the mout th region fromm all images in d converted the segmented im mages to standa ard database and c s 6 making it too 256 feature vvalues or supp ort v size of 16 X 1 vectors.

Fig 55 High Level Desi

ign of the Propose ed Smile Detectio

n system

Fig 6 Dataflo ow diagram for tr

aining a classifier r

Fig 4: Examples of mou uth region from d

database

C Design for D Detecting Smille C. From the d database of faface images, w we extracted t the m and then norm malized it. Theese mouth regi ion im magesregion are then ng Support vec tor mouth an subjected to ttraining by usinInternaational Journal of I Image Processing a and Vision Scienc 35 ces (IJIPVS) Volu ume-1 Issue-1 ,201 2

Machine Le earning Approach

for Smile Detectio

on in Real Time Im mages

Neutral to Smi le0. 4 0. 2 0 1 23 4 56 7 - 0. 2 - 0. 4 - 0. 6 - 0. 8 -1 - 1. 2 8 9 10 11 1 2 13 1 4 1 5 5 1 6 1 7 1 8 1 9 20 2 1 2 2 23 2 4 F a ce 1 F a ce 2 F a ce 3 F a ce 4

F Fig 7: Dataflow di

iagram for classif

fication of curren

t input image fram me

TimeFig 5 : Graph showing g values when fac ce changed neutra al to smile

V. R RESULTS A Accuracy off Face Detectio on A. First and fo oremost, the a ccuracy of de etecting faces in s was determineed. Face detect tion using Ha arstatic imag es w L lassifiers was ttested on total of 336 images of d Like Feature cl e size. I found d out that the overall accuraacy different image face detection a n with negligibble false positivves a g table depicts the accuracy. almost 99% in ives. Following and false negatiTab ble 1: Accuracy of f Face Detection a approach

C. Acc curacy of Smile e Detection For r calculating t the accuracy in smile dete ection, we app lied d the methodol logy for 270 ggrayscale imagges of size 64 X 6 64. We found t that out of 2700 images, theree were 222 correct detections aand 36 false po sitives andd 12 false negativ ves were gener rated. Thus m mak ing it an ac ccuracy of 82.2%.Following tabble depicts the same.Table 2: Acc curacy of smile de etection methodo logy

D. Perf rformance In measuring th e performance e, the total n number of support t vectors used in training andd in real time iis selected . I notice ed that as the ffeature size dec creases, the tim me taken to detect s smile also decrreases. Follow wing chart show ws the time taken to o detect smile when the corre esponding supp port vector is used . B Calculation n of threshold B. The main task in smile e detection ap pproach was to c threshold whenn the person smile would be calculate the t e the classifie er is trained, d we applied t the c detected. Onceal time for arouund 25 faces annd found that tthe th hreshold in was the person actually smile ed. classifier rea -0.4 when F ph shows the calculation o f threshold of f 4 d Following grap different faces

Internaational Journal of I

Image Processing a

and Vision Scienc 36

ces (IJIPVS) Volu

ume-1 Issue-1 ,201

2

Machine Learning Approach for Smile Detection in Real Time Images

The current dataset used in the expression recognition are too small and lack variability in imaging conditions. Current machine learning methods may require on the order of 1000 to 10,000 images per target facial expression. So, more the number of training samples more will be the accuracy. The above method has been implemented in OpenCV platform and takes approximately 2 seconds to detect smile and achieves around 90% accuracy. ACKNOWLEDGMENT The authors would like to extend their sincere gratitude to PES Management and all faculty members and friends in all support provided for carrying out this research work.Fig 6: Chart showing Time taken to detect smile when the corresponding support vectors are used [1]

REFERENCES Jacob Whitehill, Gwen Littlewort, Ian Fasel,Marian Bartlett Toward Practical smile Detection, IEEE Transactions on Pattern Analysis and Machin e Intelligence , pp. 2106 2111, Vol 31, No 11, 2009 Stelios Krinidis, Ioan Buciu and Ioannis Pitas, Facial expression analysis and synthesis: A survey, A white paper by Department of Informatics , Aristotle University of Thessaloniki Maja Pan tic and and Leon J.M. Rothkrantz, Automatic Analysis of Facial Expressions: The State of the Art, IEEE Transactions on Pattern Analysis and Machine Intelligence , vo l 22, no 12, pp 1424 - 1445 2000 Paul Viola and Michael Jones Robust Real Time Object Detection, Second International Workshop On Statistical and Computational Theories Of Vision Modeling, Learning, Computing, and Sampling , 2001 Wataru Hirata, Joo Kooi Tan, Hyongseop Kim, Seiji Ishikawa, Recognizing Facial Expression for Manmachine Interaction, ICROS-SICE International Joint Conference 2009 August 18-21, 2009, Fukuoka International Congress Center, Japan P Li, S.L.Phung, A. Bouzerdom and F.H.C Tivive Automatic Recognition of Smiling and Neutral Facial Expressions, Digital Image Computing: Techniques and Applications , pp. 581 586, 2010 Yu-Hao Huang and Chiou-Shann Fuh Face Detection and Smile Detection, White Paper , 2009 I M. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan, Automatic recognition of facial actions in spontaneous expressions, Journal of Multimedia , 2006 N Phillip Ian Wilson and John Fernandez Facial feature detection using Haar classifiers, Journal of Computing Sciences, Volume 21 Issue 4, 2006 http://www.cs.ucla.edu/~falaki/pub/adaboost.pdf http://opencv.willowgarage.com/wiki/

E. Comparison with existing smile detection methods Finally, I compared the time taken by our implemented smile detection approach with Sony cameras smile shutter function and Samsung Galaxy mini S5570. Both the Sony cameras smile shutter function and Samsung Galaxys smile shutter are only capable of detecting larger smile, i.e. when the person largely smiles, and are not capable of detecting slight smile. The comparison was performed for 10 different faces.

[2]

[3]

[4] Table 3: Comparison of our algorithm with other smile detection methods Sony Smile Shutter Samsung Galaxy mini S5570 Our Implemented Smile Detection algorithm [5]

Camera

Parameters Time Taken(in secs) Slight Smile

[6] 2.5 2 2

No

No

YES

[7] [8]

VI.

SUMMARY AND CONCLUSIONS [9]

Machine learning approach produces the best performance for facial expression analysis. Hence it is possible to produce accurate facial expression analysis by machine learning approaches.

[10] [11]

International Journal of Image Processing and Vision Sciences (IJIPVS) Vo lume-1 Issue-1 ,2012 37