an effective dynamic gesture recognition system based on ...bjtf/apresenta%e7%e3o-icann.pdf · an...

An Effective Dynamic Gesture

Recognition System Based on

the Feature Vector Reduction

for SURF and LCS PABLO BARROS, NESTOR JÚNIOR, JUVENAL BISNETO, BRUNO FERNANDES, BYRON BEZERRA, SÉRGIO

FERNANDES.

ESCOLA POLITÉCNICA DE PERNAMBUCO - UNIVERSIDADE DE PERNAMBUCO - BRASIL

RPPDI Dynamic Gesture

Recognition Database

Dynamic Gesture

Frame Sequences

Represent one Gesture

http://rppdi.ecomp.poli.br/gesture/database/

Dynamic Gesture Recognition

System

System Architecture

Feature Extraction Module

Classification Module

Extraction Module

Local Contour Sequence – LCS [1]

Speed Upt Robust Features – SURF [2]

Convexity Approach

CLCS

CSURF

Local Contour Sequence - LCS

Algorithm

Identify Hand Shape

Image Segmentation

Contour Detection

Calculate Feature Vector

LCS – Segmentation

Segmentation

OTSU [3]

Contour Identification

Hand Contour Identification

LCS - Local Contour Sequence

Feature Vector Calculation

Find the top first point of the image

Order the points in clockwise.

Calculate distance of a line formed by two points , ℎ[𝑖−(𝑤−1)⁄2] and ℎ[𝑖−(𝑤+1)⁄2] , and h[i].

Speed Up Robust Features - SURF

Integral Image

Find Interest Points

Describe Interest Points

Intensity

Direction

Descriptors

Convexity Approach

Minimize the hand shape

Douglas-Peucker Algorithm

Apply convex hull

Sklankys Algorithm

Calculate points distances

Convexity Approach

Douglas Peucker Algorithm

Select the two most distant points.

Verify if there is vertex near than a distance T, if there is, remove it.

Recursively do it again with all the points.

Convexity Approach

Sklanky´s Algorithm

Find a convex vertex.

Rename the other vertex in clockwise, starting with p0.

If p0, p1 and p2 turn right:

Put p0 after p2.

Update p0, p1 and p2.

Else:

Put p1 before p0.

Remove p1.

Update p0, p1 and p2.

Repeat until p0 is the initial vertex and p0, p1 and p2 turns right.

For each pair of points draw a line and find the most distant point.

Convexity Local Contour Sequence

Calculate distance

Adaptation of LCS

Use the two external points to draw the line.

Use the inner point to calculate distance.

(a) LCS. (b) SuRF interest points. (c) CLCS. (d) CSURF

Classification Module

Elman Recurrent Neural Network

Hidden Markov Model

Dynamic Time Warping

Results

Convexity Approach

Methodology

Run 30 times

Validation (1/3 for test and 2/3 for training)

Referências

[1] Meena, S. 2011. A Study on Hand Gesture Recognition Technique. Master’s thesis, National Institute Of

Technology, Rourkela,India

[2] Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput. Vis. Image Underst.

110(3), 346–359 (Jun 2008),http://dx.doi.org/10.1016/j.cviu.2007.09.014

[3] Bao, J.; Song, A.; Guo, Y.; and Tang, H. 2011. Dynamic hand gesture recognition based on surf tracking. In

Electric Information and Control Engineering (ICEICE), 2011 International Conference on, 338 –341.

[4] N. Otsu. A threshold selection method from gray-level histograms. Systems, Man and Cybernetics, IEEE

Transactions on, 9(1):62 –66, jan. 1979.