4a026eb425ff87094857e3a0ad4324419468

8/17/2019 4a026eb425ff87094857e3a0ad4324419468

1/15

www.ccsenet.org/cis Computer and Information Science Vol. 5, No. 3; May !"

Sur#ey on $esture %ecognition for &and Image 'ostures%afi(ul )aman *+an" Noor -dnan I ra+eem"

" aculty of Science, 0epartment of Computer Science, -ligar+ Muslim 1ni#ersity, -ligar+, IndiaCorrespondence2 %afi(ul )aman *+an, aculty of Science, 0epartment of Computer Science, -ligar+ Muslim1ni#ersity, -ligar+, ! !! , India. el2 4" 455 6"! 7656. 8 mail2 r9:+an.cs amu.ac.in

%ecei#ed2

8/17/2019 4a026eb425ff87094857e3a0ad4324419468

2/15

www.c senet.org/cis Computer and Information Science Vol. 5, No. 3; May !"

a c

igure ". 8?amples of dat a glo#e ased and #ision ased approac+esa. #i ion ased Afro m image gall ryB; . data glo#e ased A0ipietro, !!FB; c. colored glo#e A>am erti, !""B.

Min A 446B classifi d +and gesture recognitio n system into se#eral categ ories. 'a#lo#ic A"446B presented a

psyc+o logical aspects of gestures. - slig+tly updated #ersion of t+is classification is gi#en in a le ".

a le ". &and gesture recognition system classification

Categor y ype

-pplica ion Sign >anguage, %o ot Control, rac :ing $esture, $amesMotion Static, 0 ynamic

Image a c(uired data CameraA sB, Video, 0ata $lo#e Instrumented 0e#ice,

Colored $lo#e0ata di ensions 0 , 30

num er of +ands used =ne +an , two +andsInput fe atures 30 &and Model, -ppearance Eased, >ow >e#el e atures

$esture modality Commun icati#e, Mani pulati#e

-lt+oug+ ot+er sur# eys +a#e een done wit+ #arious su sets of +and posture and gesture recognition A' a#lo#ic,"446; Moeslund, ! !"; 8rol, !! 6B, t+is wor: is related to t+ e #ision ased approac+es and is up to d ate, andreprese nting a goodstarting point f or in#estigators interested i t+e field of +and postures a nd gestures as well.

+e organi9ation of t+is paper is as follows2 S ection demo nstrates appro ac+es for +an d posture andgesturerecognition. -pplication areas for +and posture and gesture recognition are gi#en in Section 3. Section 7 e?plainsrecognition system m et+odology. Conclusion of t+is paper is e?plained in Section 5.2. Approaches for H and Posture and Gesture Recognition

$esture system can e one of t+e f ollowing t+ree states, glo# ased, #ision ased and low le#el features asedAMurt+ y, !!4B, t+e #ision ase represents t+e most pro ising and effecti#e alternati#e for glo# e asedapproac+es t+at depends on sensors and wires w+ic+ considered costly. +e #ision ased needs cameraAsB attac+ed to a ro ot, and t+e gesture recognition algorit+m responsi le for t anslating t+e +uman gestures into a comm ndto e carried out y t+e mac+ine or ro ot A&asan, !""aB.-. 30 &and Model ased -ppr oac+es2 Many met+ods +a# e already een applied to analysis, model, andreprese nt t+e +and s+ape, w+ic+ gi#es a copio us descriptio n and ma:e a wide range of +uman +a nd to ereprese nted, and a large data ase for storing t+e e?tracted s+ap e c+aracteristics is needed a s well. Since 30+and model +as many 0 = s esides t+ +and is an ar ticulated defo rma le o @ect, features e?traction process ecame more ifficult and formed an o stacle wit+ already e?isting pro lems wit+ 30 model ased approac+

AEilal, !""B.

Published by Canadian Center of Scie nce and Educat on """

8/17/2019 4a026eb425ff87094857e3a0ad4324419468

3/15

8/17/2019 4a026eb425ff87094857e3a0ad4324419468

4/15


3., ele ision Control

>ast application for +and postures and gestures is controlling ele#ision de#ices AC player2 stop operation stop

AHu, !!4B

a 'erforming +and gesture to control t+e #irtualgame control a. ru i:Ds cu game implementation; .ru i:Ds cu game.

8/17/2019 4a026eb425ff87094857e3a0ad4324419468

5/15

Published by Canadian Center of Science and Education ""3

8/17/2019 4a026eb425ff87094857e3a0ad4324419468

6/15

8/17/2019 4a026eb425ff87094857e3a0ad4324419468

7/15

""7 ISSN 1913-8989 E-ISSN 1913-8997

8/17/2019 4a026eb425ff87094857e3a0ad4324419468

8/15


+en t+e input gesture ac(uired form colored camera, instrumented glo#e de#ice or colored glo#e as s+own inigure ". +e first step is segmentation, to e?tract t+e +and region from t+e input image and isolate it from t+e

ac:ground A&asan, !"!B. +ere are two main met+ods for o @ect segmentation, first met+od depends on t+ecolor model t+at can e e?tracted from t+e e?istence %$E color model w+ic+ could e &SV color model A&asan,

!"!; &asan, !""a ; Mo, !""B or C Cr color space AStergiopoulou, !!4B; w+ic+ deals wit+ t+e pigment of t+e s:in of t+e +uman +and A&asan, !"!B, t+e significant property of t+is color space is t+at t+e +uman differentet+ics group can e recogni9ed according to t+eir pigment concentration w+ic+ can e distinguis+ed according tosome s:in color saturation A&asan, !"!B. +en, t+e +and area is isolated from t+e input gesture wit+ somet+res+old #alue. Some normali9ation for t+e segmented image mig+t re(uire for o taining t+e gestures data asew+ic+ s+ould e in#ariant against different pertur ations li:e translation, scaling and rotation A&asan, !"!B. +edata ase created wit+ many samples per single gesture, t+e relation etween t+e num er of samples and t+eaccuracy is directly proportional, and etween num er of samples and t+e speed is in#ersely proportional A&asan,

!"!B.&asan A !"!B used &SV color model to e?tract t+e s:in li:e +and region y estimating t+e parameter #alues for s:in pigment, and used >aplacian filter for detection t+e edges. Stergiopoulou A !!4B used C Cr color model tosegment t+e +and. Mara(a A !!FB used color glo#e for input gestures and &SI color space for t+e segmentation process. $+o adi A !!FB treated t+e segmentation process as clustering met+od y grouping t+e image pi?elsamong image o @ects. >am erti A !""B used &SI color model to segment t+e +and o @ect. a le 3 s+ows some

applications of t+e segmentation met+ods used in t+e discussed met+od.

a le 3. Segmentation process from different +and gesture recognition met+ods

%eference Segmentation 'rocess 0escription

&SV color modelused to e?tract t+e

A&asan, !"!B +and region, and

a c >aplacian filter for detection t+e edges.Segmentation p+ases

a. input image; . segmented image; c. edge detection.

AStergiopoulou, C Cr color model

used to segment t+e!!4B+and.

a

&and segmentation

a. original image; . segmented +and.


8/17/2019 4a026eb425ff87094857e3a0ad4324419468

9/15

8/17/2019 4a026eb425ff87094857e3a0ad4324419468

10/15

""G ISSN 1913-8989 E-ISSN 1913-89 7

8/17/2019 4a026eb425ff87094857e3a0ad4324419468

11/15


)." /eatures 0etection and E traction

+e features are t+e useful information t+at can e e?tracted from t+e segmented +and o @ect y w+ic+ t+emac+ine can understand t+e meaning of t+at posture. +e numerical representation of t+ese features can eo tained from t+e #ision perspecti#e of t+e segmented +and o @ect w+ic+ form t+e feature e?traction p+aseA&asan, !"" B. Many researc+es +a#e een applied to form t+is feature #ector w+ic+ ta:es different si9es aswell as meanings. &asan A !"!B e?tracted t+e features #ector y di#iding t+e segmented +and o @ect into fi?ed loc: si9e 5?5 rig+tness #alue moments; t+is produce G 5 features #ector si9e and only 4F are stored as actualfeatures #ector. Stergiopoulou A !!4B applied Self $rowing and Self =rgani9ed Neural $as AS$=N$B networ: to e?tract t+e e?act s+ape of t+e +and region and determine t+ree c+aracteristics as t+e features; 'alm region,'alm center, and &and slope. Compute angle etween t+e finger root and t+e +and center named %C -ngle, andt+e @oints t+e fingertip and t+e +and center named C, and angle and distance from t+e palm center. >i A !!3Bdefined a grid of fi?ed si9e wit+ " loc:s gray scale features #ector, and eac+ grid cell represents t+e mean#alue of t+e a#erage rig+tness of t+e pi?els in t+e loc:. >am erti A !""B defined t+e distanced from t+e palmto t+e fingersd iAi J ", ..., 5B, and computed t+e angle 2 etween t+e line connecting t+e centroids of t+e palm andt+e fingers, w+ic+ produces four angles 2 iAi J ", ..., 7B, so t+e +and represented y nine numerical features #ector A>am erti, !""B. a le 7 demonstrates features #ector representation of t+ese met+ods.

a le 7. eatures representation from different +and gesture recognition met+ods%eference /eatures representation 0escription

Ey di#iding t+e

segmented +ando @ect into fi?ed

A&asan, !"!B loc: si9e 5?5geometric momentsw+ic+ is rig+tness

a #alue of eac+ loc: separately.

eature Vector representationa. segmented +and; . features rig+tness di#ision.

-fter e?tracted t+e

+and s+ape usingAS$=N$B Neural

AStergiopoulou, Networ: algorit+m.+ree c+aracteristics!!4B are determined;

a c 'alm region, 'almcenter, and &and

eature Vector representation slope.a. %C angle; . 3C angle; c. distance from t+e palm center.

0efined a grid of fi?ed si9e wit+ "

loc:s gray scalefeatures #ector, andeac+ grid cell

A>i, !!3B represents t+emean#alue of t+e a#erage

a rig+tness of t+e pi?els in t+eeature #ector

a. normali9ed +and; . gray scale image partitioned into " loc:s segmented

normali9ed imagefeature #ector.


8/17/2019 4a026eb425ff87094857e3a0ad4324419468

12/15


3+e features #ector formed y fi#e

A>am erti, distances from palmto all fingers and!""B four angles etweent+ose distances.A>am erti, !""B

aeatures representation.

a. segmented +and o @ect; . feature #ector

).3 !ecognition

%ecognition or classification of +and gestures is t+e last p+ase of t+e recognition system. &and gestures can eclassified using two approac+es as mentioned in AMurt+y, !!4B.-. %ule ased -pproac+es2 w+ic+ represents t+e input features as manually encoded rule, and t+e winner gestureis t+e one t+at matc+ed wit+ t+e encoded rules after +is features +as een e?tracted. +e main pro lem of t+istec+ni(ue is t+at t+e +uman a ility to encode t+e rules limits t+e successfulness of t+e recognition processAMurt+y, !!4B.E. Mac+ine >earning ased -pproac+es2 t+e most common approac+es t+at considered t+e gesture as result of some stoc+astic processes AMurt+y, !!4B. Most of t+e pro lems t+at ased on mac+ine learning +a#e eenaddressed ased on t+e statistical modeling A'a#lo#ic, "446B, suc+ as 'C- A*im, !!FB, SM AVerma, !!4B.&idden Mar:o# Models A&MMsB A*es:in, !!3B +a#e een paid attention y many researc+ers AMurt+y, !!4B,*alman filtering AMo, !""B, -rtificial Neural networ:s A-NNsB AMara(a, !!F; Mura:ami, "444; els, "443;

els, "44FB w+ic+ +a#e een utili9ed in gesture recognition as well. Some researc+ers used $aussian distri utionfor gestures classification AStergiopoulou, !!4B, and 8uclidian distance metric A&asan, !"!B.

%. &onc usion&uman mac+ine interaction can e ac+ie#ed y efficient gesture recognition system in w+ic+ its applications#aried from sign language recognition to games and #irtual reality interfaces. In t+is paper a literature re#iew ongesture recognition +as een re#iewed and analy9ed; t+e ma@or tools for classification process include SM,'C-, &MMs, and -NNs are discussed. 0escriptions of recognition system framewor: also presented wit+ ademonstration of t+e main t+ree p+ases of t+e recognition system y detection t+e +and, e?traction t+e features,and recognition t+e gersture. +e ma@or image preprocessing steps necessarily re(uired to features e?traction p+ase are segmentation, edge detection, noise remo#ing, and normali9ation, t+ese steps may not applied toget+er depending on targeted application.

References

Eertsc+, . -., &afner, V. V. A !!4B. %eal time dynamic #isual gesture recognition in +uman ro ot interaction. IEEE-! S 9th International Conference on $u%anoid !obots App. 776 753B. 'aris. +ttp2//d?.doi.org/"!.""!4/IC&%. !!4.536457"

Eilal, S., -:meliawati, %., 8l, S. M.

8/17/2019 4a026eb425ff87094857e3a0ad4324419468

13/15


0ipietro, >., Sa atini, -. M., 0ario, '. A !!FB. Sur#ey of glo#e ased systems and t+eir applications. IEEE ransactions on syste%s# &an and Cybernetics# Part C5 ''lications and re ie6s# 38 A7B, 7G" 7F .

+ttp2//d?.doi.org/"!.""!4/ SMCC. !!F.4 3FG8rol, -., Ee is, $., Nicolescu, M., Eoyle, %. 0., wom ly, H. A !!6B. Vision ased +and pose estimation2 -

re#iew. Else ier Co%'uter (ision and I%age nderstanding# 1 8 A" B, 5 63.+ttp2//d?.doi.org/"!."!"G/@.c#iu. !!G."!.!"

els, S. S., &inton, $. 8. A"443B. $lo#e tal:2 - neural networ: interface etween a data glo#e and a speec+synt+esi9er. IEEE transaction on Neural Net6or s# ) A"B, F. +ttp2//d?.doi.org/"!.""!4/6 ."F G4!

els, S. S., &inton, $. 8. A"44FB. $lo#e tal: IIK- neural networ: interface w+ic+ maps gestures to parallelformant speec+ synt+esi9er controls. IEEE transactions on neural net6or s# 9 A"B, !5 " .+ttp2//d?.doi.org/"!.""!4/6 .G55!7

reeman, . ., eissman, C. 0. A"445B. ele#ision control y +and gestures. IEEE International :or sho'on uto%atic /ace and *esture !ecognition . ;urich . %etrie#ed from www.merl.com/papers/docs/ %47

7.pdf $arg, '., -ggarwal, N., Sofat, S. A !!4B. Vision ased +and gesture recognition.:orld cade%y of Science#

Engineering and echnology . %etrie#ed from www.waset.org/@ournals/waset/#74/#74 "63.pdf

$+o adi, S. 8., >oeppric+, =. 8., -+mado#, ., Eerns+ausen, oLeld, =. A !!FB. %eal time+and ased ro ot control using multimodal images. International 4ournal of Co'%uter Science I4CS.(ol 3+A7B. %etrie#ed from www.iaeng.org/IB.

IEEE transactions on syste%s# %an# and cybernetics-'art B5 Cybernetics# ", A B, 357 354. +ttp2//d?.doi.org/"!.""!4/3766.7F5FFF

>am erti, >., Camastra, . A !""B. %eal time +and gesture recognition using a color glo#e.S'ringer-(erlag Berlin $eidelberg. ICI-' !"", 'art I, >NCS G46F, pp. 3G5 363. %etrie#ed fromwww.springerlin:.com/inde?/! 667&'!3V!175F6.pdf

>i, H. A !!3B. $esture recognition ased on fu99y C Means clustering algorit+m. %etrie#ed fromciteseer?.ist.psu.edu/#iewdoc/downloadPdoiJ"!."."."3G.3435.pdf


8/17/2019 4a026eb425ff87094857e3a0ad4324419468

14/15


>iang, %., =u+young, M. A"44FB. - real time continuous gesture recognition system for sign language. IEEE hird International Conference on uto%atic /ace and *esture !ecognition Procee dings, pp. 55F 5G6.

+ttp2//d?.doi.org/"!.""!4/- $%."44F.G6"!!6Malima, -., Q9gRr, 8., etin, M. A !!GB. - fast algorit+m for #ision ased +and gesture recognition for ro ot

control. IEEE 1) th conference on Signal Processing and Co%%unications ''lications# pp. " 7.+ttp2//d?.doi.org/"!.""!4/SI1. !!G."G54F

Mara(a, M., - u )aiter, %. A !!FB. %ecognition of -ra ic Sign >anguage A-rS>B using recurrent neuralnetwor:s. IEEE /irst International Conference on the ''lications of 0igital Infor%ation and :eb

echnologies# IC-0I !!F, pp. 76F 7F. +ttp2//d?.doi.org/"!.""!4/IC-0I . !!F.7GGA els, "44FB4GMin, E., oon, &., So+, s:incolor model. International Conference on &ulti%edia echnology C media player. International 4ournal of Co%'uter ''lications# 1 A6B. %etrie#ed from [email protected]/#olume"!/num er6/p?c3F6 !" .pdf

%odrigue9, =. '., $on9ale9, =. =., -#i99ano, C. -., %uffaldi, 8., Vercelli, 0., Eergamasco, M. A !!FB.0e#elopment of a 30 real time gesture recognition met+odology for #irtual en#ironment control. IEEE

Proceedings 17th International Sy%'osiu% on !obot and $u%an Interacti e Co%%unication# pp. 64 F7.+ttp2//d?.doi.org/"!.""!4/%=M-N. !!F.7G!!G64

Starner, ., ea#er,

8/17/2019 4a026eb425ff87094857e3a0ad4324419468

15/15


+omas, S., Een@amin, '., Niels, &., Susanne, E. A !!FB.*esture recognition 6ith a :ii controller. -CM,'roceedings of t+e Second International Conference on angi le and 8m edded Interaction A 8IO!FB,$ermany. +ttp2//d?.doi.org/"!.""75/"37634!."376345

Verma, %., 0e#, -. A !!4B. Vision ased +and gesture recognition using finite state mac+ines and fu99y logic. IEEE International Conference on ltra &odern eleco%%unications > :or sho's# IC1M O!4, pp. " G, 'eters urg. +ttp2//d?.doi.org/"!.""!4/IC1M . !!4.53757 5

Vogler, C., Meta?as, 0. A !!"B. - framewor: for recogni9ing t+e simultaneous aspects of -merican Sign>anguage. Ideal Co%'uter (ision and I%age nderstanding 81# pp. 35F 3F7.+ttp2//d?.doi.org/"!."!!G/c#iu. !!!.!F45

ang, &. &. H., Mandal, M. *., Meng, M., >i, 0. A !!3B. 8fficient face and gesture recognition tec+ni(ues for ro ot control. IEEE Canadian Conference on Electrical and Co%'uter Engineering# CCECE " 3# 3 , "656"6G . +ttp2//d?.doi.org/"!.""!4/CC8C8. !!3." G 5!

Hu, )., Hiang, C., >ant9, V., *ong (iao, . A !!4B. &and gesture recognition and #irtual game control asedon 30 accelerometer and 8M$ sensors. C& Proceedings of the 1)th international conference on

Intelligent user interfaces I I 9 . +ttp2//d?.doi.org/doi2"!.""75/"5! G5!."5! 6!Fin, H., Hie, M. A !!6B. &and posture segmentation, !ecognition and ''lication for $u%an-!obot

Interaction . %etrie#ed from www.intec+open.com/download/pdf/pdfs id/5 6G)+u, ., &uang, ., Hu, $., %en, &., cn, ). A"44FB.(ision-based Inter'retation of $and *estures by

&odeling ''earance Changes in I%age Se?uences. - O4F I-'& or:s+op on Mac+ine Vision-pplications.Ma:u+m.

4a026eb425ff87094857e3a0ad4324419468

Documents