Automatic Electrical Home Appliance Control andSecurity for disabled using electroencephalogram

based brain-computer interfacingS. Paul, T. Sultana, M. Tahmid

Electrical & Electronic Engineering, Electrical & Electronic Engineering, Computer Science & EngineeringSchool of Science, Engineering and Technology, East Delta University

Chittagong, BangladeshEmail: [jishu.astro, tanni.tanin, badhon434]@gmail.com

Abstract—The development of Brain Computer Interface(BCI) system helps to utilize Electroencephalography technologyproviding an effective way of turning human thoughts intoactions as well as communication to other physical deviceswithout any help of the traditional muscular pathways. TheBCI system incorporated with EEG technology has recentlybecome a wonderful solution to provide direct communicationand interaction pathways for old age persons, sick patients,and especially for the severe handicapped person. In this Paperwe propose a novel automatic electrical home appliance controlsystem model using BCI. The proposed system will collect brainsignals through EEG equipment and process using a microcon-troller. The extracted brain thought signals will further be sentvia available wireless communication technology to the input ofthe receiver microcontroller that will directly activate and controlthe electrical appliances. The system will be incorporated witha security alert subsystem along with the control purpose wherethe user can activate an instant alarm by his thoughts in case ofdanger as well. It is expected that the advantage of portabilityand cheapness of this proposed system compared to the otherBCI systems will make it superior and more user-friendly.

Index Terms—BCI, EEG, Home Automation, Microcontrollers,Wireless, Bispectrum.

I. INTRODUCTION

Motor imagery represents the result of conscious accessto the content of the intention of a movement. It can bedefined as a dynamic state during which an individual mentallysimulates a given action. This type of phenomenal experienceimplies that the subject feels him/her self performing theaction. It is usually performed unconsciously during movementpreparation. But a very interesting fact is that, conscious andunconscious motor preparation share common mechanismsand they are functionally equivalent. As a result, a clear imageof an intended action can be present even without the limbbeing involved[1].A brain computer interface (BCI) is a direct communicationpathway between the brain and an external device. It is alsocalled mind-machine interface (MMI), direct neural interface(DNI), or brainmachine interface(BMI). It is a communicationsystem for controlling a device, e.g. computer, wheelchairor a neuro-prosthesis, by human intensions, which does notdepend on the brain’s normal output pathways of peripheral

nerves and muscles but relies on the detectable signals repre-senting responsive or intentional brain activities. It transformsmental intentions into control commands by analyzing thebioelectrical brain activity. BCIs can help patients totallylosing volitional motor ability but having intact cognition andimprove their living standards[2]. A successful BCI systemvery much depends on the following criteria: i. Ability of theextracted features to differentiate the task-oriented brain states,ii. Efficiency of the methods for classifying such features inreal-time[3]. For analyzing BCIs, the brain activity of a patienthas to be recorded.The traditional Electrode system for acquiring brain signal isInternational 10-20 system that works with the help of 21electrodes which are placed on the surface of the human scalp.The usage of common Ag-AgCl small disc metal electrodesfollowed by proper skin preparation, conductive gel etc. cancause discomfort to the human sample for a long term signalacquisition process. Compared to this metal electrode system,user friendly dry electrodes offer more convenient way to EEGtechnology[4].EEG measures voltage fluctuations resulting from ionic currentwithin the neurons of the brain. EEG measures the summationof electrical potentials in the form of electric field generatedfrom millions of neurons having same spatial orientation. Theelectrical field is mainly developed by currents that flow duringsynaptic excitation of the dendrites[4]. Advantages of EEG arethat it is a very low cost technique, non-invasive and recordingprocedures are comparatively easier.The proposed Brain-computer interface (BCI) model willprovide an alternative method of expressing human thoughtsother than the traditional pathways of peripheral nerves ormuscles with the help of the communication based on neuralactivity generated by the brain.So, the objective of this research is to prepare a real time basicBrain Computer Interfaced model that will be able to activateElectric loads along with an alarm especially for the disabledpeople using the classification of EEG signals.

II. GENERAL SYSTEM REVIEW

A. General BCI Diagram

The General BCI system consists of Signal Acquisitionblock, Feature extraction followed by analysis of the signals,classification of the signals and interfacing with the real timemachine through computer with the help of machine learningalgorithm.

Fig. 1. General BCI system

B. EEG Signal Acquisition system

The signal acquisition part is performed with the help ofeither wet electrode system or dry electrode system. Inter-national 10-20 system, which is a standardized system forelectrode placement[5] uses 21 electrodes that are placed uponthe human scalp as Fig 2.

Fig. 2. The 21 Electrode placement of International 10-20 system

The numbers 10 and 20 refer to percentages of relativedistances between different electrode locations on the skull.The general electrode- skin interface diagram is as the Fig 3.

Fig. 3. Skin- Electrode interfacing in the wet electrode system

On the other hand, the newly invented single channelprototype called NeuroSky Mind Wave sensor which is a drytype sensor is able to acquire different brain activities in anon-invasive manner. Compared to the International standardelectrodes, brain wave sensor provides less complexity alongwith increased accuracy. Every NeuroSky product encloses aThink Gear chip which enables the interface between usersbrain and the load activation unit[6]. This TGAM (Think Gear)module consists of an onboard chip that filters the electricalnoise by processing the data sets. Raw brainwaves and theessence (like attention, Excitement and concentration) valuesare determined[7].The sensor (Fig 4) performs extraction of raw EEG signal in anon-invasive manner and transmits wirelessly to the processingunit through RF transmitter. Sensor composed of headset, anear clip and sensor arm[8]. Specifications of such devices are:weight 90g, frequency ranges from 2.42 2.472 GHz and itsmaximum power is 50mw. Sampling rate of EEG signal is512Hz.

Fig. 4. The Neurosky mind wave sensor model

C. EEG rhythms and waveform

The recorded EEG signals from the scalp having amplitudesranging from 20 to 100 microvolts and a frequency contentranging from 0.5 to 30-40 Hz can be conventionally classifiedinto five different frequency bands as the table below:

D. EEG Data Analysis

Signal processing methods can be divided into two generalcategories: methods developed for the analysis of spontaneousbrain activity and brain potentials which are evoked by varioussensory and cognitive stimuli. The alteration of the ongoing

EEG due to the stimuli is called event related potential (ERP),in the case of external stimulation called evoked potential (EP).There are mainly three modalities of stimulation: auditorystimuli are single tones of a determined frequency, or clickswith a broadband frequency distribution. Visual stimuli areproduced by a single light or by the reversal of a pattern as forexample a checkerboard. Somatosensory stimuli are elicited byelectrical stimulation of peripheral nerves.The power spectral analysis provides a quantitative measureof the frequency distribution of the EEG at the expense ofother details in the EEG such as the amplitude distribution andinformation relating to the presence of particular EEG patterns.Hence timefrequency signal-processing algorithms such asdiscrete wavelet transform (DWT) analysis are necessary toaddress different behavior of the EEG in order to describe it inthe time and frequency domain. It should also be emphasizedthat the DWT is suitable for analysis of non-stationary signals,and this represents a major advantage over spectral analysis.

III. METHODOLOGY

The methodology is divided into two sections. The hard-ware configuration and the software processing part. First,we explain the proposed hardware model and then show thealgorithm used in processing of the brain signal with thesimulation results included in the following section.

A. Hardware Model

1) Data Acquisition: Unlike the electrocardiogram (ECG),EEG has a very low amplitude (5-500 uV) and their noisynature make it hard to detect them. Another issue is the DCoffset of the signal due to electrode-tissue interface. This DCoffset is usually 20-50 mV and about 500 times bigger thanthe signal. Thus, a very low noise, high input impedance andhigh CMRR (Common Mode Rejection Ratio) instrumentationamplifier is required to amplify these signals and reject theDC offset[9]. The following block diagram shows the dataacquisition system.

Fig. 5. Block Diagram of the EEG acquisition system

Here, we propose the international 10-20 system of leadsas the electrode system.

2) Classification of data in the Microcontroller: The ac-quired and preprocessed data is considered to be the input tothe microcontroller. Here the software part of the system willbe in use. A prestored matrix containing the feature vectorfor the previously stored training data will be used by themachine learning algorithm. The acquired EEG data will beprocessed according to the algorithm described in the softwarepart to produce the test feature vector. The machine learningalgorithm wil be implemented in the microcontroller using thefollowing block diagram for KNN classifier[10].

START

Acquire (known) training dataset and

Unknown test samples

Define class for training dataset

The test pattern is announced to be of class J if

No. of distances (out of K distances)

corresponding to class J is maximum

Sort out distances and first k distances

And corresponding classes

Calculate distance between training dataset

and unknown data samples. Check it for all

unknown data

Any Unknown

test sample

remains?

Stop Yes No

Fig. 6. Block Diagram of the KNN algorithm

3) Wireless Data Transmission: After the processing of thebrain signal, the acquired binary result is transmitted througha bluetooth module to the desired receiver device. In oursystem, we are currently controlling only one load and thereare two instructions, namely ON or OFF. But the system canbe upgraded easily for multiple loads at the same time.

B. Software Model

We are proposing a classification algorithm which uses thestatistical analysis of the bispectrum of the EEG signal. Theblock diagram in Fig 8 shows the algorithm used in thissystem. For a test run, we have used an available dataset whichfeatures the time delay between the two datasets. It is to benoted that, a real time BCI based system always faces theproblem of time difference between training and testing dataand most of the existing EEG classification algorithms fail

Fig. 7. Overview of the Proposed System

to perform well in this situation. Our proposed algorithm hasperformed well in this circumstances (checked by simulation).

In our system, we will use two imagery motor tasks to detectthe corresponding ON or OFF signals. Corresponding to eachinstruction a 1 or a 0 is generated.

Fig. 8. Proposed Algorithm

1) Details of the Dataset: The dataset used in this researchis the dataset I of the BCI competition III, it contains data ofimagined motor movement of left small finger or tongue. Thetrain set and the test set were recorded from the same subjectin two different days with one week in between. In the BCIexperiment, a subject had to perform imagined movementsof either the left small finger or the tongue. The time seriesof the electrical brain activity was picked up during thesetrials using a 8x8 ECoG platinum electrode grid which wasplaced on the contralateral (right) motor cortex. The grid wasassumed to cover the right motor cortex completely, but dueto its size (approx. 8x8cm) it partly covered also surroundingcortex areas. All recordings were performed with a samplingrate of 1000Hz. Further details about the dataset I in the BCIcompetition III can be found in [11].After amplification the recorded potentials were stored asmicrovolt values. Every trial consisted of either an imaginedtongue or an imagined finger movement and was recordedfor 3 seconds duration. To avoid visually evoked potentialsbeing reflected by the data, the recording intervals started0.5 seconds after the visual cue had ended. The labeledtraining data from the first session was stored in a file calledCompetition train.mat. It consists of two parts: Part 1: thebrain activity during 278 trials. This part is stored in a 3Dmatrix named X using the following format: [trials electrodechannels samples of time series]. Part 2: the labels of the

278 trials. This part is stored as a vector of -1 / 1 valuesnamed Y. The unlabeled test data is also stored in a file calledCompetition test.mat. It contains 100 trials of brain activityin matrix X (3D format is the same as described above) butit contains no labels Y.

2) Feature Extraction: In this method, we have used bis-pectrum of the original ECoG signal and extracted somehigher order statistical features. The feature vector consistsof bispectral higher order statistical features of the wholesignal. Bispectrum is a higher order statistical analysis. Itis an ideal tool to investigate non-linear interactions. TheFourier transform of the third-order cumulant is called thebispectrum. For a non-Gaussian random process x(t), its third-order cumulant is defined as

C3x(m,n) = E[x(k).x(k +m).x(k + n)] (1)

Where, E is the expectation of the of the multiplication of theprocess and its two lagged versions. The bispectrum of thisprocess is defined as the 2D Fourier transform of the cumulant,

Bx(ω1, ω2) =

∞∑m=−∞

∞∑n=−∞

C3x(m,n).e[−j2π(mω1+nω2)]

(2)On the other hand, if the process is Gaussian, then its third-order cumulant is,

C3x(m,n) = 0 (3)

That is, any Gaussian noise in the system is nullified by thebispectral analysis. We can define the brain signal as a sum ofnon-Gaussian random process x(t) and Gaussian noise w(t).

z(t) = x(t) + y(t) (4)

Then, the third-order cumulant of the signal is,C3z(m,n) = C3x(m,n) (5)

In this method, we have extracted 4 features from the cal-culated bispectrum from each channel of a single trial. Allof these are statistical features. The following features wereconsidered,

a) The sum of the logarithmic amplitudes of the bispectrum,

F1 =∑

ω1,ω2∈Flog(|Bx(ω1, ω2)|) (6)

b) The sum of the logarithmic amplitudes of the diagonalelements of the bispectrum,

F2 =∑ω∈F

log(|Bx(ω, ω)|) (7)

c) The 1st order spectral moment of the amplitudes of thediagonal elements of the bispectrum,

F3 =

N∑k=1

k. log(|Bx(ωk, ωk)|) (8)

d) The 2nd order spectral moment of the amplitudes of thediagonal elements of the bispectrum,

F4 =

N∑k=1

(k −H3)2. log(|Bx(ωk, ωk)|) (9)

So, the feature vector is formed as,

F = [F1 F2 F3 F4] (10)

3) Feature Quality: We have verified the qualities of theproposed features. The quality is determined by two param-eters: 1. Inter-class Separability, 2. Intra-class Compactness.These two parameters for each of the features are shown here.Inter-class Separability: The following figures show the inter-class separabilities for the 4 features extracted from thebispectrum of the signal.

0 10 20 30 40 50 60−5

0

5

10

15x 104

Channel No.

Fe

atu

re V

alu

e

Task1

Task2

Fig. 9. Inter-class separability of feature F1

0 10 20 30 40 50 60

0

500

1000

Channel No.

Featu

re V

alu

e

Task1

Task2

Fig. 10. Inter-class separability of feature F2

0 10 20 30 40 50 60−2

0

2

4

6

8x 104

Channel No.

Fe

atu

re V

alu

e

Task1

Task2

Fig. 11. Inter-class separability of feature F3

0 10 20 30 40 50 60

−1

0

1

2

3x 10

12

Channel No.

Fe

atu

re V

alu

e

Task1

Task2

Fig. 12. Inter-class separability of feature F4

So from the above figures, it can be easily seen that, thefeature values of each channel differ by a significant amountfor each task. The red line shows feature values for task-1:movement of left small finger and the green line shows featurevalues for task-2: movement of the tongue.Intra-class Compactness: The following figures show the com-pactness of the feature (taking the second feature as example)for each of the two tasks. The black line in the figure representsthe mean of the feature values for each channel.

0 10 20 30 40 50 60−500

0

500

1000

1500

Channel No.

Feat

ure

Valu

e

Fig. 13. Intra-class compactness for task-1

0 10 20 30 40 50 60−500

0

500

1000

1500

Channel No.

Feat

ure

Valu

e

Fig. 14. Intra-class compactness for task-2

From the above 2 figures, the compactness of the featurescan be seen for individual tasks. With the exception of 2 or3 channels in some of the trials, all the feature values form acompact band for each individual task.

4) Classifier: In our method, we have used KNN classifierto predict the labels of the test set. K-nearest neighbor is a non-parametric method used for classification. Its input consistsof the k closest training examples in the feature space. Theoutput is a class membership. An object is classified by amajority vote of its neighbors, with the object being assignedto the class most common among its k nearest neighbors. Itis among the simplest of all machine learning algorithms. InKNN, k is a positive integer. The value of k depends upon thedata. Generally, larger values of k reduce the effect of noiseon the classification, but make boundaries between classesless distinct. Smaller values of k make boundaries distinct andcomputations easier; but noise will have a higher influence.There are three tie-breaking rules: nearest, random and con-sensus. In our method we have used the first two rules to checkthe accuracy. The distance metrics are of five types: Euclidean,Cityblock, Cosine, Correlation and Hamming.

IV. RESULTS

We have used accuracy as our performance parameter.Accuracy is defined as the number of labels predicted by theproposed method, matching the true labels provided by thedataset providers.

Though the maximum accuracy falls for some cases, we stillget 87% accuracy for the cosine distance metric with nearestrule. Moreover, the optimum value of k is reduced to a verylow value of 6. Thus the proposed feature vector can providea high accuracy with low classification complexity.

V. CONCLUSION

This research paper showed detailed overview of the generalEEG based BCI systems along with a proposed model that willimplement the system with the help of the microcontroller. Arelatively high performing and real-time efficient classifica-tion algorithm is presented here. Moreover our designed andproposed system has the advantage of single processing unitcompared to most of the existing systems. Our system is easily

upgradable to multiple load capacity without the increase ofthe number of processing units. Most of the existing systemsinclude the processing unit with the load, hence increasingnumber of loads becomes tedious and costly. Also, we havedesigned the processing algorithm keeping in mind the time-varying and random nature of EEG signals. Though some fur-ther future expansions are needed, the proposed system surelyhas the qualities to be an effective and efficient automationsystem to be useful for the handicapped.

REFERENCES

[1] Martin Lotze and Ulrike Halsband, Motor imagery, Journal of Physi-ology - Paris 99 (2006) 386-395.

[2] Wolpow JR, Birbaumer N, McFarland DJ, et al. Brain-computer interfacefor communication and control, Clinical Neurophysiology, 2002, 113:767-791.

[3] S.M. Zhou, John Q. Gan, F. Sepulveda, Classifying mental tasks basedon features of higher-order statistics from EEG signals in braincomputerinterface, Information Sciences 178 (2008) 1629-1640.

[4] Lukas Maly, WHEELCHAIR CONTROL USING EEG SIGNAL CLASSI-FICATION, MS Thesis Paper, BRNO UNIVERSITY OF TECHNOL-OGY.

[5] Jasper, H. H., The ten twenty electrode system of the internationalfederation, Electroencephalography and clinical neurophysiology, 10(1958), 371375.

[6] Athanasios Vourvopoulos, Fotis Liarokapis, Evaluation of commercialbrain-computer interfaces in real and virtual world environment: A pilotstudy, Computer and Electrical Engg. 40(2014) 714-729.

[7] L.R. Stephygraph, N.A. Kumar, V. Venkatraman Wireless Mobile RobotControl through Human Machine Interface using Brain Signals, In-ternational Conference on Smart Technologies and Management forComputing, Communication, Controls, Energy and Materials (ICSTM),India. 6 - 8 May 2015. pp.596-603.

[8] NeuroSky related informations available at www.neurosky.com[9] Lin ZHU et al. Design of Portable Multi-Channel EEG Signal Acquisition

System, Biomedical Engineering and Informatics, 2009. BMEI ’09. 2ndInternational Conference on.

[10] Neerja S. D., Rupesh S. M., Methods towards invasive human braincomputer interfaces, International Journal of Scientific and ResearchPublications, Volume 3, Issue 6, June 2013.

[11] Lal TN, Hinterberger T, Widman G Robotic Automation through SpeechRecognition, Advances in Neural Information Processing Systems,Cambridge: MIT Press, 2005, 17: 737-744.