classification methods for human activity recognition...

1/30

IntroductionPrevious Work

MethodsExperiments

Conclusion

Classification Methods for Human ActivityRecognition (HAR)

Teal Hobson-Lowther

Colorado School of Mines

December 7, 2015

Teal Hobson-Lowther Classification Methods for Human Activity Recognition (HAR)

2/30


MethodsExperiments

Conclusion

Topic

Human Activity Recognition

� Human Activity Recognition (HAR) is a field dedicated tomonitoring the daily activities of humans via computationalmethods.

� Emerging research field since the 1980’s.

� Several classification techniques (HMM, NB, ANNs, etc.)have been applied to the topic, with varying degrees ofsuccess.


3/30


MethodsExperiments

Conclusion

Motivation

Health Insight

� Tracking a person’s activities throughout the day providesinsight into their health.

� Observing different metrics about a human’s activity (mean,std. dev, fourier analysis) gives further information about thequality of their actions

� Proof of concept on simple activities (sitting, standing,walking) can yield insight into more complex activity patterns(pull-up, sit-up, curl)


4/30


MethodsExperiments

Conclusion

Project Goals

Explore Literature

� Explore current literature to extract a decent feature set

� Investigate methodology used in previous papers

Proof of Concept

� Use a simple HAR data set from 1, and compare my results withbenchmarks set by that paper.

� Compare the results of two classification methods: Naive-BayesClassifier and Artificial Neural Network

� Obtain a commercially viable level of classification accuracy (99.9%)

� Use this information as a springboard for an investigation of morecomplex human activity, in particular human exercise classification.


5/30


MethodsExperiments

Conclusion

Data

Groupware@LES HAR Dataset

� Dataset with 5 classes: sitting-down, standing-up, standing,walking, and sitting

� Collected over 8 hours

� 4 healthy subjects

� Four three-axis accelerometers (x,y,z) are placed on the waist, leftthigh, right ankle, and right arm

� Arduino LilyPads used to collect sensor data


6/30


MethodsExperiments

Conclusion

Figure 1: Positioning ofAccelerometers [1]

Figure 2: Table of activity captureinfo [1]


7/30


MethodsExperiments

Conclusion

Figure 3: Table of Previous Work [1]


8/30


MethodsExperiments

Conclusion

Groupware@LES Paper [1]

� Feature extraction techniques mimic those found in [1]

� Determines most relevant features via Mark Hall’s selectionalgorithm

� Uses C4.5 decision tree in conjunction with AdaBoostensemble method for classification

� Gives beautiful summary of previous work


9/30


MethodsExperiments

Conclusion

Naive-Bayes ClassifierNeural NetworksPre-Processing

Introduction to Naive-Bayes Classifier

� Naive-Bayes Classifier (NB) is an extension of maximumlikelihood for classification

� ”Naive” - it is assumed that the elements of the input arestatistically independent

� ”Bayes” - uses Bayes’ Rule for determining the probability ofa class given data, and chooses the maximum

� Highly effective for text classification, using method known as”bag-of-words”


10/30


MethodsExperiments

Conclusion


Simple NB Theory

Bayes’ Rule for Classification

For a particular class Cj :

p(Cj |x) =p(Cj )p(x|Cj )

p(x)∝ p(Cj )p(x|Cj ) (1)

where x is the input vector of features and C is the class

Independent Variables

If we treat the elements xi of x as independent random variables,equation 1 becomes:

p(Cj |x) = p(Cj |x1, x2, ..., xn) ∝ p(Cj )n∏

i=1

p(xi |Cj ) (2)


11/30


MethodsExperiments

Conclusion


Training

� Train on 80% of data

� Find the frequency of all 36 input features: xi , i = 1, 2, ..., 36for a given N input samples of class j: xnj , n = 1, ...,N.

� This frequency gives you p(xi |Cj ) for all training samples.


12/30


MethodsExperiments

Conclusion


Testing

Classifying a new input x requires finding y such that:

y(x) = argmaxy

p(y = Cj )p(x|y = Cj )

= argmaxy

p(y = Cj )n∏

i=1

p(xi |y = Cj )

where

p(xi |y = Cj ) =

(#xi ofinputx)+1

total#xj +1 xi , xj ∈ p(xtrain|Cj )

1total#xj +1 xi /∈ p(xtrain|Cj ), xj ∈ p(xtrain|Cj )


13/30


MethodsExperiments

Conclusion


Neural Networks

Basic Architecture

� An Artificial Neural Network (ANN) consists of many identicalprocessing units called neurons.

� Each connection to a neuron has an adjustable weight factorassociated with it.

� Every neuron in the network has its weighted inputs summed toproduce an activity level, vi , such that:

vi =n∑

j=1

wijxij − wio

where wij is the weight of connection from input j to neuron i , xij isinput signal number j to neuron i , and wio is the thresholdassociated with unit i .


14/30


MethodsExperiments

Conclusion


Neural Networks

Network Activation

� The activity is passed through a non-linear activation function, φ, toget the output of neuron yi :

yi = φ(vi ) = φ( n∑

j=1

wijxij − wio

)� It is this non-linearity that allows the neural network to map an

input to a probability distribution.


15/30


MethodsExperiments

Conclusion


Neuron Activity


16/30


MethodsExperiments

Conclusion


Network Architecture


17/30


MethodsExperiments

Conclusion


Training the Neural Network

Back-Propogation

� In supervised learning, a machine learning algorithm is presented with a set of inputs, xi , and associatedoutputs yi .

� The weights on each neuron connection are first initialized randomly, and then updated via a methodknown as back-propogation

� Back-propogation uses a generalized iterative Least-Mean-Square Error algorithm to alter the neuronweights, such that the the error function:

E =1

2P

∑p

∑k

(dpk − opk )2

is minimized, where p is the index of the P training pair of vectors, k is the index of elements in the outputvector dpk , and opk is the kth element if the output vector when pattern p is presented as input to thenetwork.

� This is a complicated algorithm, and we will skip the details.


18/30


MethodsExperiments

Conclusion


Training vs. Testing

Training

Weights wij are updated until the response of a network to theinput vector xtrain produces output ytrain, such that ytrain ≈ ytrain

Testing

After training, new, unlabeled inputs xtest are presented to thenetwork, and it outputs a predicted response ytest . Thus, newestimations of ytest may be obtained from new inputs.


19/30


MethodsExperiments

Conclusion


Feature Extraction

I) Raw data consists of four three-axis accelerometer readings (12 rawfeatures)

II) Calculate the pitch roll, and L2 norm of the accelerometer data foreach sensor (12 features)

III) Throw out the raw data

IV) Normalize result of II

V) Window the data (take previous time samples of length window size)

VI) Filter the windowed data with a filter size length averaging filter

VII) Calculate the variance of window’s normalized pitch, roll, andacceleration module data. (12 features)

VIII) Calculate the DC Fourier component of window’s normalizedaccelerometer readings, pitch, and roll (12 features)


20/30


MethodsExperiments

Conclusion


Pre-Processing Flowchart


21/30


MethodsExperiments

Conclusion


Naive-Bayes Quality Metrics

� The performance of each trained Naive-Bayes Classifier will becalculated via the following metric:

λj =#(ytest ∈ Cj |ytest ∈ Cj )

#ytest ∈ Cj

for each class j = 1, ..., J of all classes J.

� In other words, the ratio of correctly classified inputs to totalinputs belonging to the class j.


22/30


MethodsExperiments

Conclusion


Neural Network Quality Metrics

The performance of each trained neural network will be calculatedvia the cross-entropy (log-loss) metric:

H(p, q) = −∑

i

pi log(qi ) = −ylog(y)− (1− y)log(1− y)

where the total cross-entropy error is:

L(w) =1

N

N∑n=1

H(pn, qn)


23/30


MethodsExperiments

Conclusion


Programming Implementation

� All code implemented in MATLAB

� Preprocessing involved using built in functions: fspecial(),imfilter(), mean(), var(), fft(), atan(), norm(), randperm()

� Classification involved: Naive-Bayes Classifier usingMATLAB’s Statistics Toolbox, Custom Neural Network builtusing MATLAB’s Neural Network Toolbox


24/30


MethodsExperiments

Conclusion

Naive Bayes Classification

Once the data are pre-processed, the following code is run toimplement NB:

%% Split data into training and testing:

p = .8;

train_length = floor(p*length(target));

data_train = x(1:train_length,:);

data_test = x((train_length):length(target),:);

target_train = target(1:train_length);

target_test = target((train_length):length(target));

%% Naive Bayes Classifier:

NBModel = fitcnb(data_train,target_train);

[label] = predict(NBModel,data_test);

C = confusionmat(target_test,label);

% Calculate the misclassification percentage for activities 1-5:

e = [C(1,1)/sum(C(1,:)), C(2,2)/sum(C(2,:)), C(3,3)/sum(C(3,:)), C(4,4)/sum(C(4,:)), C(5,5)/sum(C(5,:))];

%Returns: e = [0.9846 0.6770 0.9010 0.4868 0.8250]


25/30


MethodsExperiments

Conclusion

Naive-Bayes Results

A sample confusion matrix for one realization of Naive-BayesClassification is:

Predicted AccuracySitting 9907 65 0 90 0 98.46%

Sitting Down 29 1589 452 39 238 67.7%Standing 57 88 8678 211 598 90.1%

Standing Up 67 286 835 1200 77 48.68%Walking 25 33 1367 84 7113 82.5%

The average and standard deviation of the accuracy for each class (10 runs) was:

Activity Mean (%) Std. DevSitting 98.47 .0010

Sitting Down 68.33 .0101Standing 90.27 .0014

Standing Up 48.09 .0106Walking 82.95 .0031


26/30


MethodsExperiments

Conclusion

Neural Network Classification

Once the data are pre-processed, the following code implementsthe NN:

%x - input data. t - target labels for five classes in form of [0 0 0 1 0].

%create a 36 input, two hidden layer (75 and 50 node),

%5 output pattern recognition network

net = patternnet([75,50]);

net = train(net,x,t); %train the network on our data

view(net);

y = net(x); %classify data with trained network

perf = perform(net,t,y); %calculate cross-entropy error

perf

%Returns 4.814e-04


27/30


MethodsExperiments

Conclusion

Neural Network Results

The [75 50 5] configuration of neural network gave the bestexperimental results:

� Average classificationaccuracy was 99.91%, basedon 15 trials.

� Average networkcross-entropy: 5.22948e-04

� Variance of networkcross-entropy: 2.1054e-04

� Training time: ≈ .75sec/epoch, 500 epochs.=⇒ ≈ 6.5 minutes


28/30


MethodsExperiments

Conclusion

Discussion

� Various pre-processing techniques were applied to three-axisaccelerometer data for effective feature extraction

� A Naive-Bayes Classifier was applied with varying degrees ofaccuracy between activity classes

� A Neural Network Structure was found that generatedcommercially viable classification accuracy (99.91%)

� NB trains quickly compared to NN, but has much loweraccuracy


29/30


MethodsExperiments

Conclusion

Discussion Cont’d

� This data was collected with only four subjects doing fiveactivities

� Better feature extraction, including DWT [6], could beimplemented to get better accuracy

� Generalization may be limited by specificity of pre-processing

� Future work will extend NN classification to more complexactivity data (exercises)


30/30


MethodsExperiments

Conclusion

References

[1] Ugulino, W.; Cardador, D.; Vega, K.; Velloso, E.; Milidiu, R.; Fuks, H. Wearable Computing: Accelerometers’Data Classification of Body Postures and Movements. Proceedings of 21st Brazilian Symposium on ArtificialIntelligence. Advances in Artificial Intelligence - SBIA 2012. In: Lecture Notes in Computer Science. , pp.52-61. Curitiba, PR: Springer Berlin / Heidelberg, 2012. ISBN 978-3-642-34458-9. DOI:10.1007/978-3-642-34459-6 6.

[2] M. Mohandes, S. Rehman and T. Halawani, ’A neural networks approach for wind speed prediction’,Renewable Energy, vol. 13, no. 3, pp. 345-354, 1998.

[3] H. Wang, ’Lecture 10: Review of Linear Algebra and Optimization’, Colorado School of Mines, 2015.

[4] MATLAB and Statistics Toolbox Release 2014a, The MathWorks, Inc., Natick, Massachusetts, United States.

[5] MATLAB and Neural Network Toolbox Release 2014a, The MathWorks, Inc., Natick, Massachusetts, UnitedStates.

[6] E. Mitchell, D. Monaghan and N. O’Connor, ’Classification of Sporting Activities Using SmartphoneAccelerometers’, Sensors, vol. 13, no. 4, pp. 5317-5337, 2013.


classification methods for human activity recognition...

Documents