analysis of finger movement by using motion information ... · pdf filecomputer is no longer...

International Journal of Mechanical & Mechatronics Engineering IJMME-IJENS Vol:13 No:03 24

136803-5757-IJMME-IJENS © June 2013 IJENS I J E N S

Abstract – Nowadays, through the advancement of science and

technology, possibility of human finger provide information into

computer is no longer question. Fingers movement and hand

motion continuously being center of research in human computer interaction (HCI) and robotic controls. Using self-develop

DataGlove, an experiment was conducted by using motion

capture System (MOCAP) equipped with five motion capture

cameras to capture human finger movements. The purpose of

this paper is to analyze voltage output from DataGlove and angle obtains from motion capture system while constructing

relationship concerning both outcomes. Polynomial equation is

considered toward the construction of fitting curve line in scatter

data. Through the end of project, differences between finger

graphs slopes will be clarify. Preliminary result of experiment exposed the newly develop DataGlove output might closely relate

into angle of finger bending movement.

Index Term— DataGlove, Finger movements, Human

Computer Interaction, Motion Capture Software (MOCAP),

Polynomial Regression

I. INTRODUCTION

Human Computer Interaction (HCI) is term used to refer an

understanding and designing of differences relationship

between people and computer [1]. According to [2], HCI

involve in various features such as command line, menus,

Papper submitted on 10 May 2013. This work is supported by the

ScienceFund Grant by the Ministry of Science, Technology and Innovation to Universiti Malaysia Perlis (01-01-15-SF0210).

M. Hazwan Ali , Advanced Intelligent Computing and Sustainability Research Group, School of Mechatronic, Universiti Malaysia Perlis

KampusPauh Putra, 02600 Arau, Perlis, MALAYSIA (e-mail: [email protected]).

KhairunizamWAN, Advanced Intelligent Computing and Sustainability Research Group, School of Mechatronic, Universiti Malaysia Perlis

KampusPauh Putra, 02600 Arau, Perlis, MALAYSIA (e-mail:

[email protected]). Nazrul H. ADNAN, Bahagian Sumber Manusia, T ingkat 17 & 18,

IbuPejabat MARA Jalan Raja Laut, 50609 Kuala Lumpur, MALAYSIA & Advanced Intelligent Computing and Sustainability Research Group,

School of Mechatronic, Universiti Malaysia Perlis KampusPauh Putra, 02600 Arau, Perlis, MALAYSIA (e-mail: [email protected]). Y.C Seah, Advanced Intelligent Computing and Sustainability Research

Group, School of Mechatronic, Universiti Malaysia Perlis KampusPauh Putra,

02600 Arau, Perlis, MALAYSIA. Juliana Aida Abu Bakar, School of Multimedia Tech & Communication

College of Arts and Sciences Universiti Utara Malaysia 06010 Sintok, Kedah, MALAYSIA (e-mail: [email protected]).

Zuradzman M Razlan , Advanced Intelligent Computing and Sustainability Research Group, School of Mechatronic, Universiti Malaysia

Perlis Kampus Pauh Putra, 02600 Arau, Perlis, MALAYSIA. (e-mail:

[email protected]).

natural language, direct manipulation, and form fill. Through

direct manipulation, gesture of human body that contains

meaningful information [3] will be interpreted through

pointing device/graphical display [2]. DataGlove is an

example of HCI which provide information about human

finger and motion frequently used for gesture recognitions [4].

DataGlove is input device wears identical to standard glove

capable to capture physical data such as bending of finger [5].

Due to that characteristic, DataGlove often use in Virtual

Reality [6] and hand gesture application [7]-[10].

This research concern about characteristic of finger motion

achieve from preliminary experiment while modeling finger

movement into voltage and angle correlation signal. Effect of

voltage on angle result will be revising using polynomial

regression method. According to [11], polynomial regression

is form of linear regression in which the relationship between

variable x and dependent variable y is modeled as an nth order

polynomial. Consequently, assessing variable voltage and

angle using polynomial regression would estimate the

relationship among variable.

This research paper organized as subsequent; Section 2

encompasses literature review of the related researches,

problem and approach acquiring finger data. Section 3

presents the methodologies of applied procedure. Section 4

divided into 2 sections, first section states about experiment

setup whereas second section demonstrations the result of

experiments. Final section 5 expresses the conclusion over

current research.

II. LITERATURE REVIEW

The modeling of finger motions in this research is based on

GloveMAP and motion capture data. GloveMAP on the

contrary is DataGlove construct using strain gauge sensor to

measure finger flexion [12]. Assessing output voltage from

GloveMAP with proficient signal analysis and programming,

excellent result could be demonstrated. GloveMAP

achievement has been verified by numerous experiments

revolving around GloveMAP such as virtual interaction [13]

whereby the waveform produce by GloveMAP are processes

and display into virtual reality as an alternative by means of

regular Graphical User Interface (GUI) [14]. Furthermore

through GloveMAP, PCA-based finger movement and

grasping classification development [15] has been presented

successful. Angle contradictory to GloveMAP required

dissimilar procedure to obtain the coordinate and magnitude of

Analysis of Finger Movement by Using Motion Information from GloveMAP and Motion

Capture System

M.Hazwan Ali, Khairunizam WAN, Nazrul H. ADNAN, Y.C Seah, Juliana A. Abu Bakar and Zuradzman M. Razlan



finger flexion.

Motion capture system by Qualisys Track Manager

Software (QTM) [16] otherwise present alternative approach

in angle analysis whereby the software already equipped with

calculation to acquire both magnitude and components for

position, angle, velocity and acceleration. Motion Capture

System (MOCAP) is generally system with capability to

record the movements of human, and others motion, and then

using recorded data to animating graphics. MOCAP mostly

use in widespread commercial such as video game and movie

while as well to other area such as biomedics, biomechanic,

education and artistic. Notable usage of MOCAP is in the

study of skeletal parameter by Adam G. Kirk et al. [17]

University of California. Jonathan Maycock, et al. [18] in

2011 also manipulating MOCAP and DataGlove on robust

tracking of human hand postures for robot teaching. While on

International Joint Conference 2006, Young-Il Oh et al. [19]

display a promising research in low cost motion capture

system for PC-based immersive Virtual Environment (PIVE)

system.

III. METHODOLOGY

Fig. 1. Flowchart of GloveMAP and QTM Software

A. Flow Chart of works

Flow Chart of works shown in fig. 1 provides overview of

the proposed system. The works start with the calibration of

MOCAP before measuring finger movements . MOCAP data

are directly transferred to MOCAP’s control computer

whereas GloveMAP data transferal over microcontroller

through serial communication port. Both finger movements

data obtained from MOCAP and GloveMAP are analyzed by

using Polynomial regression method.

B. Kinematic of finger

According to [20], kinematic is the branch of conventional

mechanics that describe the motion points, bodies and systems

of bodies without consideration of the cause’s motion.

Salvador Cobos et al. [21] stated that kinematic model of

human skeleton comprised of 19 links that initiate the

corresponding human bones, and 24 degrees of freedom

(DOF) that represent the joint. That would mean four links and

five DOF for index, middle, ring and little whereas three links

and four DOF for thumb.

Fig. 2 shows detail of kinematic model for human index

finger. In this kinematic finger model, Metacarpophalangeal

joint (MCP) is modeled by two DOF universal joint label as

ϴMCP1 and ϴMCP2 however proximal Interphalangeal (PIP)

and distal Interphalangeal (DIP) similarly have one DOF label

as ϴPIP and ϴDIP individually. Although, both kinematic

modeling and MOCAP provide method to analyze angle, the

nature constrains of human hand have to be taken into account

as it refrain finger flexion in certain angle degree. Finger

motion constrain typically classified into Intrafinger constrains

and Angle range constraints . Intrafinger constrains is a

common constraint occur to same finger joint and can be

calculate by refer to (1), whereas ϴDIP is stated as finger

bending angle of Distal Phalange joint and ϴPIP is structure for

the Proximal Interphalangeal joint bending angle.

( ⁄ ) (1)

Angle range constraints otherwise a types of difficult

constraints rising to the boundary of the range concerning

finger motions of hand anatomy which follow by refer to (2).

( )

And,

(2)

START

GloveMAP start

capturing data

Motion Capture

Start capturing

Data

Computer Process Data

END

Microcontroller

Cubic Polynomial

Marker

Detection

Calibration

Data

Trigonometry

Function

Matlab

Motion Capture Data

Yes

No

No Yes



Fig. 2. Kinematic model human index finger

C. GloveMAP

GloveMAP shown in fig. 3 is adopted flex sensors in the

construction of DataGlove. The resistivity of the sensor

corresponds to the increasing distance between each of carbon

element inside the thin strip of flex sensor. With the change of

the resistance values, the voltage outputs can be calculated

referring to voltage the divider equation, where Vin is the flex

sensor supply voltage, R refer as the resistance of flex sensors,

whereas Vout is voltage output resulting from referring to (3).

By analyzing voltage data into Matlab, waveform as shown in

fig. 4 is obtained.

(3)

Fig. 3. GloveMAP attached with the flex sensor

Fig. 4. Voltage waveform outputted from flex sensor

D. Motion Capture

Qualisys Track Manager (QTM) is used as tracking

software due to fact that QTM is built around set of advanced

motion capture algorithms to ensuring high performance,

accuracy and low latency [16]. QTM measures the finger

movements in 3D space. Fig. 5 shows the magnitude’s

trajectories of finger movements for marker #1, #2 and #3.

Fig. 5. Magnitude of marker trajectories

E. Trigonometry Function

Trigonometric according to [22], is a branch of mathematics

that studies triangles and the relationships between the lengths

of their sides and the angle between those sides. In this

research, trigonometry function is used to model finger

movements of human. Trigonometry used in a calculation is

Pythagorean Theorem and Point-Slope Equation as written in

(4). Whereas ( , ) is a known point, m is a slope of the line

and (x, y) is any point on the line.

( ) (4)

Fig. 6 illustrates the angle ϴ°, which is determined in the

calculation. Thru expending of the equation (4), equation (5) is

produced.

ϴMCP1

ϴMCP2

ϴPIP ϴDIP

Flex sensor #1

(index finger)

LED Power

indicator

Control

Circuit

USB

Flex sensor #2

(middle finger)



( )

( )

( )

( ) And,

( )

( )

( )

( ) (5)

Fig. 6. Angle, Ɵ

Based on the 3 coordinates x, y and z, the correspondence Ɵ

is calculated. Fig. 7(a) shows the magnitude of marker

trajectories for each marker #1, #2 and #3. Fig. 7 (b) shows the

correspondence angles Ɵ calculated by the system.

Fig. 7. (a) Magnitude of marker trajectories for marker #1, #2 and #3; (b)

Angle, Ɵ

F. Polynomial Regression

Polynomial regression method has been used attentively to

nonlinear functions for modeling real-life phenomena and

usually used in mathematical model to expecting dependent

variable y on independent variable x. First degree of regression

analysis (nonconstant linear function) can be used in

constructing best fit straight line in scatter plot data. Second

degree polynomial is a quadratic polynomial, with better data

fit than first degree polynomial. A cubic function is a

polynomial with degree of 3 and has form as refer to (6) [23].

( ) (6)

Cubic polynomial usually provides superior data fit than first

and second order while ensuring high coefficient of

determination on scatter plots. Fig. 8 shows the result of

employing cubic polynomial into the Ɵ signal in fig. 7(b). The

cubic polynomial curve display great comparable with original

signal while expecting the subsequent signal sequence.

Fig. 8. Cubic polynomial plotted graph

IV. EXPERIMENT

Experiments were carried out in MOCAP environment.

Both GloveMAP and QTM Software were simultaneously run

to read finger movements .

A. Experiment Setup

In the experiment, 3 markers were attached to GloveMAP as

shoen in fig. 11. The markers positions were at distal

phalanges, proximal phalanges and metacarpals of the index

finger. The cameras used in the experiment were built on Oqus

100 with average residual of 0.3 to 0.9 mm at 3.7 m at

distance. The experiments were conducted by doing various

movements of index finger. Each experiment was run in 2 s.

ϴ

Marker#1

Marker#2

Marker#3



Fig. 9. MOCAP environment with 5 Oqus 100 cameras

Fig. 10. Anatomy of hand [24]

Fig. 11. GloveMAP and the location of markers placement

B. Experiment Result

For the experimental result, all data obtained was analyzed

by using cubic polynomial. Angle and voltage were plotted

into same graph which voltage performs as dependent variable

y whereas angle as independent variable x. Although both

systems run for 2 s, data sampling rate for GloveMAP and

QTM was difference. The sampling rate of QTM was 100 fps

whereas GloveMAP contain 20 fps. Fig.12 (a) shows the

voltage signal of GloveMAP and fig.12 (b) shows angle signal

of QTM Software. Fig. 12 (c) shows the resampled signal of

angle based on the obtained voltage data.

Fig. 12. Signals (a) Voltage signal (b) Angle signal (c) resampling angle

signal

Finger movements have the directions, which were bending

and straighten movements. Fig. 13(a) shows signal gradient

decline as angle increase indicating the finger was bent

meanwhile fig.13 (b) shows slope rise with angle decreasing

indicate that finger was straighten. Fig.13 (c) and (d) show the

results after employing polynomial regression of finger

movements data. The value of correlation coefficient

indicates linearity relationship between the correlation

Marker#3

Marker#2

Marker#1

Flexible Bend Sensor



coordinate points . Fig.13 (c) and (d) have r2 values of 0.99 and

0.991, respectively. The strong correlation or relationship is

defined that has a value ranging between 0.85 to 1 [25].

Norm of residual value was also observed. Norm of residual

is defined as the difference between observe value with the

estimate function value of the unobservable statically error

[26]. It also may refer as measure of the deviation between

the correlation and data. A lower norm of residual value

implies a better fit of regression to the observe data. Norm of

residual for fig. 13 (a) and fig. 13 (b) is 0.056058 and 0.04691,

respectively. The values show that a small degree of error

when finger bending and straighten. Moreover, the patterns of

dot slopes designate that relationship between GloveMAP

output voltage with angle are may possibly perpendicular to

each other.

Fig. 13. Graph of finger flexion with polynomial (a) finger bend (b) finger

straighten (c) finger bend polynomial (d) finger straighten polynomial

V. DISCUSSIONS

Based on the results, the correlation between voltage signals

outputted from the DataGlove and the angle calculated from

motion data acquired from MOCAP is established. The scope

of research works is to find the correlation between the

voltages outputted from the proposed DataGlove with the

angle of finger’s movement. The correlation could be used in

future experiments for the acquisition of finger movement’s

data of the proposed DataGlove. However, human fingers

have many degrees of freedom, and in the future various

experiments need to be done as a further analysis to evaluate

the performance of the proposed DataGlove.

VI. CONCLUSIONS

The research works proposed the analysis of finger

movements by using polynomial regression approach. The

DataGlove called GloveMAP is used in the experiment.

GloveMAP is a low cost DataGlove developed by our research

group. The output signal from GloveMAP is a voltage signal.

MOCAP system is used to measure the bending angle of the

finger. The analysis is done to correlate the output voltage and

the bending angle of finger by using polynomial regression

approach. The experimental results show that the equation that

represents the correlation between output voltage and angle



could be produced. Furthermore, finger movements include

bending and straighten. The experimental results show that

bending and straighten movements have a similar

characteristic. In the future, the results will be used to acquire

various signals of grasping activities. The classifier will be

employed to train computer knows how to grab various

objects based on shapes and patterns.

ACKNOWLEDGMENT

Special thanks to all members of UNIMAP Advanced

Intelligent Computing and Sustainability Research Group and

School Of Mechatronics Engineering, Universiti Malaysia

Perlis (UNIMAP) for providing the research equipment’s and

internal foundations. This work is supported by the

ScienceFund Grant by the Ministry of Science, Technology

and Innovation to Universiti Malaysia Perlis (01-01-15-

SF0210).

REFERENCES [1] Richard Harper et al., “Being Human: Human-Computer

Interaction in the year 2020,” Microsoft Research Ltd, 2008, pp.

43.

[2] Sean Bechhofer, “Human Computer Interaction,” Lecture note

University of Manchester, 2010, pp. 13-16.

[3] Mark Billinghurst , “Gesture Based Interaction,” Lecture note

Chapter 14, Aug.24, 2011, pp.1.

[4] David L. Quam, “Gesture Recognition with a DataGlove,”

National Conference on Aerospace and Electronics (NAECON 1990), IEEE, 1990, PP. 775-760.

[5] Information on http://en.wikipedia.org/wiki/Wired_glove

[6] S. Sayeed, N. S. Kamel, and R. Besar, “Virtual Reality Based

Dynamic Signature Verification Using Data glove,” International

Conference on Intelligent and Advanced Systems 2007 (ICIAS

2007), Nov.25-28, 2007, pp. 1260-1264.

[7] M. Ishikaws and H. Matsumnra, “Recognition of a Hand-Gesture

Based on Self-organization Using a DataGlove,” 6th International Conference on Neural Information Processing (ICONIP '99), vol.

2, 1999, pp. 739-745.

[8] S. Saengsri, V. Niennattrakul, and C. A. Ratanamahatana, “ TFRS:

Thai Finger-Spelling Sign La Recognition System,” 2012 Second

International Conference on Digital Information and

Communication Technology and it’s Applications (DICTAP),

2012, pp. 457-462.

[9] Wu jiangqin et al., “A Simple Sign Language Recognition System

Based on DataGlove,” 1998 Fourth International Conference on Signal Processing Proceedings (ICSP '98), vol. 2, 1998, pp. 1257-

1260.

[10] T. T . Swee et al., “Wireless Data Gloves Malay Sign Language

Recognition System,” Information, Communications & Signal

Processing (ICICS 2007), Dec. 10-13, 2007, pp. 1-4.

[11] Information on http://en.wikipedia.org/wiki/Polynomial_regression

[12] Nazrul H. ADNAN et al., “The Development of a Low Cost Data

Glove by Using Flexible Bend Sensor for Resistive Interfaces,” The 2nd International Malaysia-Ireland Joint Symposium on

Engineering, Science and Business 2012 (IMiEJS2012) , 2012, pp.

579-587.

[13] Nazrul H. ADNAN et al., “Measurement of the Flexible Bending

Force of the Index and Middle Fingers for Virtual Interaction,”

International Symposium on Robotics and Intelligent Sensors 2012

(IRIS 2012), Procedia Engineering 41 , 2012, pp. 388-394.

[14] Y. A. A. Refaat , and A. A. Ahmed, “Introduction to Graphical User Interface (GUI) MATLAB 6.5,” IEEE UAEU Student Branch

UAE University, pp. 2-35.

[15] Nazrul H. ADNAN et al, “PCA-based Finger Movement and

Grasping Classification using Data Glove “Glove MAP”,”

International Journal of Innovative Technology and Exploring

Engineering (IJITEE), 3rd

ed, vol. 2, Feb. 2013, pp. 66-71.

[16] Information on http://www.qualisys.com/products/ Software/qtm/

[17] A. G. Kirk, J. F. O’Brien, D. A. Forsyth, “Skeletal Parameter

Estimation from Optical Motion Capture Data,” IEEE Computer

Society Conference on Computer Vision and Pattern Recognition

(CVPR 2005), vol. 2, June 20-25, 2005. [18] J. Maycock, J. Steffen, H. Ritter, “Robust Dataglove Mapping for

Recording Human Hand Postures,” 4th International Conference

(ICIRA 201)1, Aachen, Germany, Proceedings, Part II, Dec. 6-8,

2011, pp. 34-45.

[19] Young-Il Oh, Kyoung-Hwan Jo, and Jihong Lee, “Low Cost

Motion Capture System for PC-based Immersive Virtual

Environment (PIVE) System,” International Joint Conference

2006 (SICE-ICASE), Bexco, Busan, Korea, Oct. 18-21, 2006, pp.

3527-3530. [20] Information on http://physics.tutorvista.com/motion/ kinematics-

equations.html

[21] S. Cobos, M. Ferre, R. Aracil, J. Ortego and M. A. Sanchez-Uran,

(2010), “Simplified Human Hand Models for Manipulation

Tasks,” Cutting Edge Robotics 2010 Chapter 10, pp. 156,

Available: http://www.intechopen.com/books/cutting-edge-

robotics-2010/simplified-human-hand-models-for-manipulation-

tasks [22] Information on http://en.wikipedia.org/wiki/ Trigonometry

[23] Information on http://dufu.math.ncu.edu.tw/calculus/

calculus_pre/node7.html

[24] Information on http://www.yalemedicalgroup.org/stw/

Page.asp?Page ID=STW023547

[25] [25]Information on http://www.isixsigma.com/tools-

templates/graphical-analysis-charts/understanding-scatter-

diagrams-and-correlation-analysis.html [26] [26]Information on http://en.wikipedia.org/wiki/Errors_and_

[27] residuals_in_statistics.

M.Hazwan Ali received his Bachelor Engineering

degree in Mechatronic Engineering from University

Malaysia Perlis in 2012. He is currently a MSc

student at University Malaysia Perlis. His research

interest is in Robotics, Human-Computer

Interaction (HCI), Virtual Reality and Artificial Intelligence.

Khairunizam WAN received his B. Eng. degree in

Electrical & Electronic Eng. from Yamaguchi

University and Ph.D. in Mechatronic Eng. from

Kagawa University, in 1999 and 2009 respectively.

He is currently a Senior Lecturer at School Of

Mechatronic Engineering, University Malaysia

Perlis. He is member of Board of Engineer and Institute of Engineer, Malaysia. His research

interest is in Human-Computer Interaction (HCI),

Intelligent Transportation System, Artificial

Intelligence and Robotics.

Nazrul H. ADNAN received his Bachelor

Engineering (Hons) in Power Electrical from

Universiti Teknologi MARA (UiTM) and Master Engineering in Advanced Manufacturing

Technology from Universiti Teknologi Malaysia

(UTM) since 2004 and 2010 respectively. After

graduated in Bachelor Engineering he joined Majlis

Amanah Rakyat (MARA) as Teaching Engineer

where he worked as a lecturer to Mechatronics,

Electronics and Mechanical Diploma students. He

was currently a PhD student in Universiti Malaysia Perlis. His research interest is in Human-Computer Interaction (HCI), Product

Design, Artificial Intelligence, and Machine Design.

http://physics.tutorvista.com/motion/

http://dufu.math.ncu.edu.tw/calculus/

http://www.yalemedicalgroup.org/stw/



Y. C. Seah is currently a Bachelor Degree student in

Mechatronic Engineering at University Malaysia

Perlis. His research interest is in Robotic, Artificial

Intelligence and Mechatronic System,

Juliana A. AbuBakar currently lectures virtual

reality and multimedia technology courses at

University Utara Malaysia (UUM). She received

B.Eng. degree in Electronic Engineering from

University of Leeds, UK in 1999 and MSc. degree in

Information Technology from UUM in 2003. She was awarded Ph.D from International Islamic University

Malaysia in 2012 where her Ph.D thesis covers a

complete cycle of design, development, and user

evaluation of a virtual reality application for

architectural heritage learning. She is passionate in virtual reality research and

development projects since her first involvement in the academic world and

has secured several national grants and published many articles in the area.

Zuradzman M. Razlan received his Bachelor of

Mechanical Engineering from Yamagata University,

Japan from Apr 1993-Mar 1997 and Ph.D. in

Engineering from Mie University, Japan. He is

currently Senior Lecturer at School Of Mechanical

Engineering, University Malaysia Perlis and

experience in his field almost 11 years in R&D

Design Engineering. His research interest in Energy, Thermo-Fluid, Two Phase Flow, Air Flow System,

Heat-Pump and refrigeration system.

analysis of finger movement by using motion information ... · pdf filecomputer is no longer...

Documents