Classification of Hand Gestures using Surface ElectromyographySignals For Upper-Limb Amputees

Gregory LuppescuStanford University

Michael LowneyStanford Univeristy

Raj ShahStanford University

I. INTRODUCTION

For our project we classified hand movements basedon surface electromyography (EMG) signals fromupper-limb amputees. Approximately 38 to 50 percentof patients with upper-limb amputations discontinueuse of their prosthetic because the cost of carryingit outweighs its (limited) usage [1]. Machine learninghas the potential to greatly improve the functionalityof myoelectric prosthetic limbs. After a patient losesa limb, they still contain all the necessary nerves tocontrol their non-existing limb. By using EMG tomeasure the electrical signals sent through these nerves,amputees can potentially control a robotic prosthetic inthe same way that they once controlled their originallimb. Much research still needs to be done in orderfor the latest generation of prosthetics to understandthe electrical signals coming from the user. In ourproject, we utilize Linear Discriminant Analysis, Naı̈veBayes, and Support Vector Machines to classify aset of gestures based on EMG signals provided byseveral upper-limb amputees. We also propose our owncollection of features to use for each classifier.

II. DATASET AND FEATURES

A. Dataset

The data set was provided by [2], and consists of rawEMG signals recorded from nine different transradialamputees (seven traumatic and two congenital) withunilateral amputation, where electrodes were placed onand around the stump where a hand (or prosthetic)would be located. Ten pairs of electrodes were placedon each subject, providing 10 channels of EMG data.Each amputee imagined to perform six different handgestures using their amputated limb. The six gesturesin the data set are the spherical grip, index flexion,hook grip, thumb flexion, fine pinch, and tripod grip.For each of the six gestures, three levels of force -low, medium, and high - were imagined as well. Eachtrial (signal) consists of an 8-12 second holding phase

of a gesture, and 5-8 trials were recorded for eachforce level, giving a total of over 810 distinct gesturesignals. An example of a raw EMG signal is shownin Fig. 1. For our project, we used data from four ofthe nine subjects. These four subjects all had traumaticamputations on their left forearm.

Fig. 1. Raw EMG signal from a single channel

Fig. 2. Experimental setup used in [2] to acquire EMG data

B. Features

A total of 18 features were examined in our project.These features are popular in modern research for EMGpattern recognition [4]. Fifteen of the features wereextracted in the time domain, and three were extractedfrom the frequency domain. Six of the time domainfeatures were time-dependent Power Spectrum densityfeatures suggested in [2]. All the features except thetime-dependent Power Spectrum density features arelisted in Table I. To compute the other features, wefirst calculate the zero, second, and fourth order rootsquared moments by

m0 =

√N

∑i=1

x2i m2 =

√N

∑i=1

∆x2i m4 =

√N

∑i=1

∆2x2i

where N is the total number of samples, and xi isthe sample at index i. The moments are then scaledand normalized by a factor of γ to make them morerobust to noise

mk =mγ

kγ

where a value of γ = 0.1 was used. The six featuresf1, f2, . . . f6 are extracted as follows:

f1 = log(m0)

f2 = log(m0−m2)

f3 = log(m0−m4)

f4 = log(m0√

m0−m2√

m0−m4)

f5 =m2√m0m4

f6 = log(

∑N−1j=0 |∆x|

∑N−1j=0 |∆2x|

)f4 is a measure of sparseness and f5 is a measure

of the ratio of the zero crossings to the number ofpeaks as given in [2]. f6 is simply the ratio of thewaveform length of the first derivative to that of thesecond derivative. In order to negate the effects offorce, the final six features are found by a cosinesimilarity measure between the six features mentionedabove and their logarithmically scaled versions. Thishelps in negating the force effects and only takes intoaccount the orientation of the features [2]. The final6 time-dependent Power Spectrum density features aredefined as

Fi =−2∗ fi ∗ log( f 2

i )

f 2i + log( f 2

i )2

Thus, we have defined all of our features taken intoconsideration for each channel. Each feature describesa particular aspect of the EMG signal. For example,Waveform Length gives a measure of the EMG signal’scomplexity, and both Wilson Amplitude and Log De-tector give a metric for the level of muscle contractionobserved.

Features Definition

Time domain features

Mean absolute value (MAV) 1N ∑

Ni=1 |xi|

Integrated EMG (IEMG) ∑Ni=1 |xi|

Variance (VAR) 1N−1 ∑

Ni=1 x2

i

Root Mean Square (RMS) 1N−1 ∑

Ni=1 x2

i

Waveform Length (WL) ∑N−1i=1 |xi+1− xi|

Log detector (LD) exp( 1N ∑

Ni=1 log(|xi|))

Wilson Amplitude (WA) ∑i 1(|xi− xi+1|> ε

Slope Sign Change (SSC)The number of times thesign of the slope changes

(with a threshold ε)

Zero Crossing (ZC)The number of times thesignal crosses a threshold

ε (ε=0.05)

Frequency domain features

Mean Frequency (MF) ∑Mi=1 f jPj/∑

Mi=1 Pj

Median Frequency (MEF) Median Frequency of thePower Spectrum

Modified-mean Frequency(MMF) ∑

Mi=1 f jA j/∑

Mi=1 A j

Time domain Power Spectrum Descriptors

F1,F2,F3,F4,F5,F6 as defined in II B.

xi - ith sample of the EMG signal, N - Length of theEMG signal, f j - Frequency at the jth sample of thespectrum, Pj - Power spectrum at f j, A j - Amplitude

spectrum at f j, M - Length of the spectrum

TABLE ITHE DEFINITIONS FOR THE FEATURES USED FOR EACH

CHANNEL.

III. METHODOLOGY

A. Preprocessing

Before extracting features, it is important to processthe raw EMG signals to ensure the features of interestare not obfuscated by various sources of noise. To dothis, we apply a cascade of five filters: a high passfifth order Butterworth filter with a cutoff frequency of20 Hz, a low pass third order Butterworth filter with acutoff frequency of 450 Hz, and three notch filters withstop bands centered at 50 Hz, 150 Hz, and 250 Hz [3].The low-pass and high-pass filters smooth the data andrestrict the frequency components of our signal to thefrequency range of normal EMG signals, and the notchfilters suppress interference caused by power lines andelectrical wires (50 Hz in most countries, 60 Hz in theUnited States).

B. Feature Extraction

The preprocessed signals are segmented into win-dows of length 300 ms with 50% overlap. This isdone to leverage the fact that the signals are ”pseudo-stationary” in small regions, and leads to better classi-fication results than if one used non-overlapping win-dows [4]. For each window, the 18 features describedin the previous section are extracted for each of theten channels, giving a grand total of 180 features perwindow. An example window is shown in Fig. 3.

Fig. 3. Windowed EMG signal for which we extracted features

C. Feature Selection

Real time classification is needed for modern myo-electric prosthetics. For this reason, it would be advan-tageous to reduce the size of the feature set from 180to a smaller number to reduce computation time. Foreach classifier, we performed forward feature selectionto determine what the optimal features were, and howfew features could be used while maintaining high testaccuracy.

There are two potential ways to perform featureselection: one in which we consider all 180 featuresacross the ten channels, and one in which we find asubset of the 18 features which is applied to each ofthe ten channels. Due to changes in electrode placementfrom subject to subject, we chose to find which of the18 features per channel would give the best results.Using this method, the optimal feature set can be ap-plied irrespective of the channel placement or numberof channels.

Feature selection was run on each individual foreach classifier. The test errors were calculated using 5-fold cross-validation. The optimal feature set for eachclassifier was decided by using a voting scheme toaverage the results across each of the four subjects.A weighting was applied to each feature depending onits rank in the output of feature selection. We chosethe optimal feature set for each classifier based on thefeatures that cumulatively ranked the highest across allindividuals.

D. Training Procedure

We trained three different classifiers to predict handgestures: Linear Discriminate Analysis (LDA), Naı̈veBayes, and multiclass SVM. For the Naı̈ve Bayesclassifier the feature vectors are discretized into 128different values. The classifier is then trained using amultivariate model. For the multiclass SVM, we tookthe ”one-vs-all” approach, where we trained a singleclassifier per class, with the samples of a specific classas positives and all other samples as negatives. Fortesting, the classifier yielding the highest confidencelevel determined the predicted gesture. The classifierswere trained on each subject individually due to the factthat EMG signals can vary from person to person dueto biological and environmental factors. Two differenttraining and testing procedures were performed, onein which only the gestures were classified (6 classes),and one in which both the gesture and force level wereclassified (18 classes).

Fig. 4. Test accuracy vs. number of features per channel for LDAand NB classification

IV. RESULTS AND DISCUSSIONS

The feature selection algorithm was run for theclassification of hand gestures without force labels.The selected feature sets for each classifier were thenused to classify hand gestures with and without forcelabels. The results of the feature selection algorithmon LDA and Naı̈ve Bayes are shown in Fig 4. TheNaı̈ve Bayes classifier quickly experiences over-fittingwhen there are six or more features. Also, the trainingand testing accuracy is the lowest for Naı̈ve Bayes.This was expected as Naı̈ve Bayes makes some strong

assumptions about the independence between differentfeatures, which is certainly not the case in this appli-cation. The optimal feature set for Naı̈ve Bayes was ofsize five and can be found in Table II. The featuresare listed in the order they were selected from thefeature selection algorithm. The LDA classifier did notexperience any over-fitting, but we still decided to pickthe top eight features in order to reduce the feature set,which would be needed in real time applications. Theoptimal feature set for LDA can also be found in TableII.

Classifier Features used for each channel (byranking)

LDA F1, F6, RMS, F4, WL, F5, F3, IE

Naı̈ve Bayes F6, F5, WA, F1, F3

SVM WL, F3, F1, F4, RMS, MAV, F5

TABLE IITHE FEATURE SETS CHOSEN FOR EACH CLASSIFIER

The confusion matrices for LDA and NB lead toan interesting result. Among both methods, the twogestures that are mistakenly classified for one anotherare the hook grip gesture and the spherical grip gesture.These rates of misclassification (when compared toall other rates of misclassification) are in accordancewith the fact that the two gestures are similar to eachother. In general, the rate of misclassification betweentwo gestures seems to be correlated with the level ofsimilarity between gestures.

Gesture Classification

Classifier TrainingAccuracy

TestingAccuracy

LDA 96.53% 96.18%Naı̈ve Bayes 79.09% 78.65%

SVM 100% 88.76%

TABLE IIITRAINING AND TESTING ACCURACY FOR GESTURE

CLASSIFICATION (6 CLASSES)

For SVM, the optimal feature set can be found inTable II. Due to time and computational constraints,we only ran feature selection for 14 out of the 18features, but noted that features after this point weremost likely either causing over-fitting, or were notcontributing much more to the test accuracy. Thus, wefelt it was valid to halt the process early. We initially

Gesture and Force Classification

Classifier TrainingAccuracy

TestingAccuracy

LDA 93.99% 93.11%Naı̈ve Bayes 77.59% 76.86%

SVM 99.94% 86.53%

TABLE IVTRAINING AND TESTING ACCURACY FOR GESTURE AND FORCE

CLASSIFICATION (18 CLASSES)

implemented the SVMs with linear kernels, which gavepoor testing accuracy. Consequently, we implementedmulticlass SVM with Gaussian kernels which achievedhigh training accuracy, most likely due to the fact thatthe Gaussian kernel can theoretically fit an infinitenumber of points. However, the testing accuracy wasalso considerably high and hence, we decided to gowith a Gaussian kernel.

Fig. 5. Confusion matrix for LDA gesture only classification

Fig. 6. Confusion matrix for NB gesture only classification

The feature selection also presented some importantobservations. Firstly, all the features selected for the

Fig. 7. Confusion matrix for SVM gesture only classificationclassification

classifiers were time domain features. This suggeststhat there is little variance in the frequency domainfeatures to help classify the EMG signals. Also, anumber of time-dependent Power Spectrum densityfeatures from [2] are a part of the optimal feature set.This implies that these features coupled with otherscould prove to be a strong feature set for classifyingEMG signals.

Finally, LDA proved to be best classifier for thisapplication with high training and test accuracy. Also,it was observed that increasing the number of featuresdid not lead to overfitting for the LDA classifier, unlikeNaı̈ve Bayes and SVM. SVM gave high training accu-racies, but testing accuracies lesser than LDA hintingthat regularization may have helped to improve thetesting error.

V. CONCLUSIONS AND FUTURE WORK

In this project, we provided an effective feature setfor real time classification of EMG signals using LDA,Naı̈ve Bayes, and multiclass SVM, and showed thatLDA provided the best overall performance. We werealso able to predict with fairly high accuracy commongestures that would be useful for modern prostheticlimbs. We also demonstrated that the same set offeatures resulted in fairly high performance for theclassification of gesture and force simultaneously. Forfuture work, it is worth exploring ways to make themulticlass SVM classifier more efficient and effectiveby regularization and by using different kernels. Featureselection can be implemented for all 180 featuresacross all channels, which can help to understand thedependence of features on different channels (differentlocations on the arm). Also, it would be useful toimplement a method to identify important channels

out of the 10 channels. This information could beused to make prosthetic arms less cumbersome byincluding only the necessary channels for classficiation.Lastly, it would be worthwhile to explore if the featuresets chosen could generalize to all EMG signals bytesting on data acquired from other sources of the body.For instance, one could acquire data on a lower limbamputee to characterize foot movements and use ourmethodologies for prediction.

ACKNOWLEDGMENT

This project was a part of the CS229 MachineLearning course at Stanford conducted by Prof. JohnDuchi. We would like to thank Prof. Duchi and the TAsfor this insightful course and for guiding us through thisproject.

REFERENCES

[1] Dromerick, Alexander W., et al. ”Feedforward control strate-gies of subjects with transradial amputation in planar reach-ing.” Journal of rehabilitation research and development 47.3(2010): 201.

[2] Al-Timemy, Ali, et al. ”Improving the performance againstforce variation of emg controlled multifunctional upper-limbprostheses for transradial amputees.” (2015).

[3] E. Scheme and K. Englehart, ”A comparison of classifi-cation based confidence metrics for use in the design ofmyoelectric control systems,” 2015 37th Annual InternationalConference of the IEEE Engineering in Medicine and Bi-ology Society (EMBC), Milan, 2015, pp. 7278-7283. doi:10.1109/EMBC.2015.7320072

[4] Hakonen M, Piitulainen H, Visala A. Current state of dig-ital signal processing in myoelectric interfaces and relatedapplications. Biomed Signal Process Control Elsevier Ltd.2015;18:33459.