a novel human machine interface using 3d vision and kalman filter optimization

342
da

Upload: alan-cheng

Post on 12-May-2017

250 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

da

Page 2: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Contents1. Abstract........................................................................................................................................5

2. Introduction (Foreword)..............................................................................................................6

3. Reference to Literature................................................................................................................8

3.1 Kinect and 3D Vision.............................................................................................................8

3.1.1 The Microsoft Kinect......................................................................................................8

3.1.2 Depth Calculation with the Microsoft Kinect...............................................................10

3.2 Computer Vision and Image Processing..............................................................................11

3.2.1 OpenCV.........................................................................................................................12

3.2.2 Hue, Saturation, and Value............................................................................................12

3.2.3 Object Recognition........................................................................................................13

3.2.4 Contours........................................................................................................................13

3.2.5 Chain Codes..................................................................................................................14

3.2.6 Contour Area.................................................................................................................14

3.2.7 Bounding Box................................................................................................................15

3.2.8 Circularity......................................................................................................................15

3.2.9 Centroid.........................................................................................................................15

3.3 Kalman Filter.......................................................................................................................16

3.3.1 Kalman Filter Example.................................................................................................17

3.3.2 Model for Kalman Filter................................................................................................18

3.3.3 Prediction Stage.............................................................................................................19

3.3.4 Measurement Stage.......................................................................................................19

3.3.5 Correction Stage............................................................................................................19

4. Project Description....................................................................................................................21

4.1 Purpose and Long Term Goals.............................................................................................21

4.2 Goal......................................................................................................................................21

4.3 Research Question................................................................................................................22

4.4 Constraints...........................................................................................................................22

4.5 Variables..............................................................................................................................22

5. Design Process of Implementation............................................................................................23

Page 3: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

5.1 Drafting................................................................................................................................23

5.2 Current design model – The First Prototype........................................................................24

5.2.1 Microsoft Kinect............................................................................................................24

5.2.2 3M Roku Streaming Projector.......................................................................................25

5.2.3 OpenCV, OpenNI2 and Visual Studios.........................................................................25

6. Methodology..............................................................................................................................28

6.1 Overview of Process............................................................................................................28

6.2 Initialization.........................................................................................................................28

6.3 Calibration............................................................................................................................29

6.4 Finger Detection...................................................................................................................31

6.5 Implementation of Kalman Filter into Current Application................................................35

7. Experimentation.........................................................................................................................38

7.1 Accuracy Testing.................................................................................................................38

7.1.1 Method for Testing........................................................................................................38

7.1.2 Data from Experimentation...........................................................................................39

7.1.3 Observations..................................................................................................................41

7.2 Effects of Use of Kalman Filter...........................................................................................41

7.2.1 Method for Testing........................................................................................................41

7.2.2 Data from Experiment...................................................................................................42

7.2.3 Observations..................................................................................................................45

8. Conclusion.................................................................................................................................46

9. Extensions: Support Vector Machine Classification to Improve Finger Recognition..............48

9.1 Purpose.............................................................................................................................48

9.2 The Support Vector Machine (SVM)...............................................................................48

10. Future Work.............................................................................................................................56

11. References................................................................................................................................58

Appendix A: Experimentation Data from Accuracy Data – Image Files......................................60

Trial 1:........................................................................................................................................61

Trial 2:........................................................................................................................................62

Trial 3:........................................................................................................................................63

Appendix B: Experimentation Data from Kalman Filter Comparison..........................................64

Line 1:........................................................................................................................................65

Page 4: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 1:....................................................................................................................................65

Trial 2:....................................................................................................................................68

Trial 3:....................................................................................................................................72

Trial 4:....................................................................................................................................77

Trial 5:....................................................................................................................................80

Line 2:........................................................................................................................................83

Trial 1:....................................................................................................................................84

Trial 2:....................................................................................................................................87

Trial 3:....................................................................................................................................90

Trial 4:....................................................................................................................................95

Trial 5:..................................................................................................................................100

Circle 1:....................................................................................................................................105

Trial 1:..................................................................................................................................106

Trial 2:..................................................................................................................................111

Trial 3:..................................................................................................................................116

Trial 4:..................................................................................................................................121

Trial 5:..................................................................................................................................126

Star 1:.......................................................................................................................................129

Trial 1:..................................................................................................................................130

Trial 2:..................................................................................................................................137

Trial 3:..................................................................................................................................142

Trial 4:..................................................................................................................................147

Trial 5:..................................................................................................................................152

Triangle 1:................................................................................................................................157

Trial 1:..................................................................................................................................158

Trial 2:..................................................................................................................................163

Trial 3:..................................................................................................................................166

Trial 4:..................................................................................................................................169

Trial 5:..................................................................................................................................172

Figure 8 (1):.............................................................................................................................176

Trial 1:..................................................................................................................................177

Trial 2:..................................................................................................................................182

Page 5: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 3:..................................................................................................................................187

Trial 4:..................................................................................................................................192

Trial 5:..................................................................................................................................197

Heart 1:.....................................................................................................................................202

Trial 1:..................................................................................................................................203

Trial 2:..................................................................................................................................208

Trial 3:..................................................................................................................................213

Trial 4:..................................................................................................................................218

Trial 5:..................................................................................................................................223

Lightning 1:..............................................................................................................................228

Trial 1:..................................................................................................................................229

Trial 2:..................................................................................................................................234

Trial 3:..................................................................................................................................239

Trial 4:..................................................................................................................................244

Trial 5:..................................................................................................................................249

Appendix C: Source Code...........................................................................................................254

Appendix D: Source Code Version 2.0 - SVM...........................................................................270

Page 6: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

1. Abstract

This project presents a novel approach to the creation of an innovative human machine interface (HMI) device using the concepts of 3D vision and Kalman Filter optimization. In the first phase of this project, an HMI device was engineered using the Microsoft Kinect, and was processed using the OpenCV and OpenNI2 libraries. The methodology of the software consists of four main processes: initialization, calibration, finger detection, and optimization. The initialization process scanned and prepared the software for the environment. Calibration helped to align the RGB and Depth maps. Finger detection detected points of finger contact with the environment. Optimization was performed with the Kalman Filter environment to help improve the accuracy of mouse movement. A further improvement was made to drastically improve finger recognition by using the support vector machine classification algorithm.

Two different types of experimentation were performed on the device. The first experiment tested the accuracy of point detection in the software. Results showed that error between the measured and true center has little visible effect on the implementation of the device. The second experiment tested the improvements on the tracing error of shapes when a Kalman Filter is used to optimize mouse movements. Results clearly indicate that the Kalman Filter showed over a 20% improvement on the mean, maximum, number of peaks, and roughness of the tracing error.

The results show the effectiveness of a methodology to create a novel HMI device which has good accuracy and can realistically be used in the real world. Real world applications of the device include: improving teaching techniques in education, collaboration in research, work, and business environments, use for entertainment, and the consumer use for personal entertainment.

Page 7: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

2. Introduction (Foreword)

Foreword:

This idea for this project started in the midst of last year’s Beaverton Hillsboro Science Exposition (BHSE) science fair competition. I would like to thank the Intel manager whom, in response to my overhead camera setup in my project, Improving Precision of Robot Control Using a Hardware Implementation of a Kalman Filter, asked me the question of recognizing fingertip touch on a table surface from an overhead camera. It was after the expo was over which, after failing to place 1st in electrical engineering, I found myself pondering this question over and over. At the moment was when I thought of combining the idea of improving communication and having an interactive table touchscreen, and is a result created this project. Without this enlightening discussion, I would not have started research into this novel idea.

In very recent years, the technology industry has seen the development of consumer based 3D vision products. The most notable being the Microsoft Kinect for the Xbox 360 gaming system. However, the development of these commercially available products opens the path for further research and development on consumer grade products using 3D vision.

On the other hand, my participation at a research group (dedicated to quantum technologies) has shown me the importance of communication. Good communication leads to more efficient and more creative ideas. Over the course of four years, I have repeatedly observed that the most successful ideas come from these discussions. Thus, in order for society as a whole to be more innovative, it is important to increase the efficiency our collaboration.

A lot of discussions occur in conferences rooms. Typically, there is someone using the projector for a presentation. However, besides the use of displaying a PowerPoint and statistical data, there aren’t many other uses. In the situation of a research group, there are normally whiteboards that are used for explanation and planning. By taking the advantage of advancing research with 3D vision, with the suboptimal situation in collaboration, a solution to the problem could be to create an interactive human machine interfacing (HMI) device. When being projected straight down on a table, this allows users to work on the same display. As a result, researchers and businessmen can plan together on big screen, or work in individual partitions that are easily shared with others (since it is on the same big screen). This means that workers can come up with individual ideas, and then easily and directly share their ideas. These interactions introduced by the use of such interfacing technology can help increase collaboration in workspaces.

At the same time, if the HMI device was able to be palm-size, portable, and tangible to the everyday person, it allows for individual use on-the-go. Advantages include: not being restricted to a fixed screen (i.e. enlargement of screen for family entertainment or the use of a small screen to do simple tasks on the go) and the use of electronics in unconventional locations (i.e. locations with water). Thus, with the introduction of such idea, it is possible to create a product that not only is useful to businesses, but also useful to the commonplace person.

Page 8: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

The idea: Think about the above table, but instead it runs any operating system with no designated surface and no restricted size, all enclosed in a small box that can fit in your pocket with a price tag under $500.

Page 9: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

3. Reference to Literature3.1 Kinect and 3D Vision

3.1.1 The Microsoft Kinect

Microsoft’s innovative Kinect was announced at E3 2009 as Project Natal. It was then released on November 4, 2010. The original perception and goal of the Kinect was to introduce a new gesture-based way to play games on the Xbox. However, in the entertainment industry, the Kinect was perceived in negative light.

On the other hand, the Kinect has been in much positive light for use in academic research. Many top universities such as Massachusetts Institute of Technology have used the device for furthering research in various areas of computer science, electrical engineering, and computer engineering. Example uses included 3D videoconferencing (research by University of California, Davis) and simultaneous localization and mapping (SLAM) for autonomous robots (research in many universities, primarily MIT).

Some specs of the Kinect are included below for reference:

640×480 pixels @ 30 Hz (RGB camera) 1280×960 pixels @ 15 Hz (RGB camera) 640×480 pixels @ 30 Hz (IR depth-finding camera) Practical depth range of 1.2–3.5 m (3.9–11.5 ft) Tilt motor for movement in the X, Y, and Z axis Microphone array for voice recognition

Figure 3.1: The Microsoft Kinect for Xbox360 (source: http://upload.wikimedia.org/wikipedia/commons/6/67/Xbox-360-Kinect-Standalone.png)

Page 10: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Figure 3.2: Inside look at the Kinect (source: http://qph.is.quoracdn.net/main-qimg-17601437f077288f37a5d99eaea53726?convert_to_webp=true)

In November 22, 2013, the Xbox One was released. Along with the console was an updated Kinect. The new Kinect allows for high depth sensing and RGB quality, faster processing, and more accurate voice recognition. However, as of now, there are virtually no public available drivers, and the device is not sold separately. Thus, research still uses the original Kinect model.

Page 11: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Figure 3.3: Kinect 2.0 for the Xbox One

3.1.2 Depth Calculation with the Microsoft Kinect

It is common to think that the Kinect uses stereovision to find depth. This statement is actually false. The Kinect uses structured-light in order to detect depth. This is because the use of such technology allows for the use of a cheaper infrared projector and receiver, lowering costs of the device. At the same time, it can still have high-quality depth scanning.

Structured-light works in the following way:

1. A pattern of infrared light is projected on the environment from the infrared projector2. The infrared camera reads the infrared light and calculates distortion from the original

projection to get the depth value.

Page 12: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Figure 3.4: Distortion of multiple different vertical stripe patterns (source: http://3.bp.blogspot.com/_UvGzlBL4wgk/S7FAzysUc4I/AAAAAAAAAII/3_GAI3FMRs4/s1600/Act8-3.jpg)

The Kinect uses a semi-random speckled light pattern for display, has having a large quanitity of points at random intervals can help increase accuracy of the depth calculations.

Figure 3.5: Speckle pattern projection of Kinect on hand to detect depth (source: https://graphics.stanford.edu/~mdfisher/Images/KinectIR.png)

However, some problems do arise with the use of structured-light. The most noticeable is the inability to use the Kinect in plain daylight as the sun emits light. Another problem is the inability to have multiple Kinects look at the same environment, as the projected light would interfere with each other.

3.2 Computer Vision and Image Processing

The general idea of computer vision is to use a camera input as a sensor to do numerical processing. Sample applications of computer vision include autonomous robots, 3d mapping (such as the SLAM algorithm), and the creation of gesture-based interactions. In order for computer vision to be efficient, however, it is necessary to perform image processing, which is the manipulation of an image input for calculations.

Image processing is based-off the idea of using a camera input to perform advanced calculations. It should be noted that the camera input isn’t exactly a picture like what our eyes perceive, but in fact is a grid of number which represents what the camera sees. So, the color of an image is actually just a bunch of numbers encoded as a color.

Page 13: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Although some of the most optimal programs work directly on the raw data of the image input, in most applications it is efficient to use an external image procession library to help with data manipulation and processing. One of the most widely used libraries for this application is OpenCV.

3.2.1 OpenCV

OpenCV is an open source image processing library launched in 1999 by Intel, supported by Willow Garage and Itseez, and is now maintained by the non-profit organization opencv.org. The current stable release is OpenCV 2.4.8, and is written for the c and c++ programming languages. However, adaptations of OpenCV have been made for other languages like Matlab and Python.

Figure 3.6: OpenCV logo (source: http://en.wikipedia.org/wiki/File:OpenCV_Logo_with_text_svg_version.svg)

The primary function of the library is to provide various computer vision related algorithms like thresholding, Haar transform, and Fourier transformation. However, the current builds of OpenCV now contain various machine learning algorithms like support vector machines and the Bayes classifier. Also, with the new emergence of commercial 3D vision sensors, OpenCV has introduced functions to support stereovision, and is compatible with 3D vision sensors like the Microsoft Kinect.

3.2.2 Hue, Saturation, and Value

Hue, saturation, and value, otherwise known as HSV, are color properties. They are used to define the color of a pixel (and ultimately an image). The hue value correlates to the “pure” color (red, green, blue, purple, etc). Saturation is the color’s grayness. Lower saturation values lead to more grayness, with 0% saturation being the gray color itself. Value is the brightness of the color. Thus, we can obtain the grayscale of an image by separating the image into these three color channels.

Page 14: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Figure 3.7: The HSV color cone (source: http://ilab.usc.edu/wiki/index.php/HSV_And_H2SV_Color_Space)

3.2.3 Object Recognition

Object recognition is the idea of classifying an object in an image. Although it may seem easy to perform by the human eye, classifying objects from the pure numerical values is not an easy task to complete. Classification has to be done by finding a shape, and then by analyzing shape properties. Some of the methods will be described below.

3.2.4 Contours

OpenCV’s contour function uses a method presented in 1983 by Satoshi Suzuki and Keiichi Abe in Topological structural analysis of digitized binary images by border following. The general idea of the algorithm is to, by using chain codes, detect the outer and inner edge of both blobs and holes. Inputs are commonly either a thresholded binary image or an image processed by the Canny edge detector algorithm.

Figure 3.8: Contour function on an image processed by the Canny edge detector algorithm (source: http://dasl.mem.drexel.edu/~noahKuntz/opencvtut7-2.png)

Page 15: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

4

1

2

35

6

70

Figure 3.9: Contour function output on a binary thresholded image (source: http://2.bp.blogspot.com/-olgBqVL0C60/UDUDJyRFoUI/AAAAAAAAAQA/7GXI2s145ns/s1600/ObjectDetection-contours.png).

3.2.5 Chain Codes

Chain codes store information of the connected components of an image. Thus, the contour finding algorithm uses chain codes to find the enclosed connect components. One of the most commonly used chain codes is Freeman’s chain code, which contains an encoding for each of the eight cardinal directions. A visual example of Freeman’s chain code is shown below.

Figure 3.10: Example of a “compass” for Freeman’s chain code.

3.2.6 Contour Area

The contour area is a shape feature. It can also be used to filter out small, noise contours. The contour area is the area inside contour, which is the number of pixels enclosed by the contour boundary.

Page 16: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Figure 3.11: The contour area of a contour. The contour area of the contour (light blue) would be two pixels

3.2.7 Bounding Box

A bounding box is a box which bounds or surrounds an object. The box would have the same length of the object’s longest length, and the same width of the object’s longest width.

Figure 3.12: A bounding box of a triangle (green). The bounding box’s area would be four pixels

3.2.8 Circularity

The circularity of an object is the difference in area of an object’s area to a circle’s area with the same perimeter. The equation is P2 / 4πA where P is the perimeter and A is the area. The less circular a shape is, the higher the circularity.

Figure 3.13: The circularity of the triangle is the black circle.

3.2.9 Centroid

Finding the centroid of a shape is very useful in robotics. Using the center coordinates, the robot can then relocate itself to the object and pick it up.

In order to find the centroid of an object, images moments is used. Moments contain the statistical properties of an image, and can be used to find the area, centroid, and object orientation. The centroid can be calculated as following.

M ij=∫−∞

∫−∞

x i y j f ( x , y ) dxdj

Figure 3.14: Moments equation: (top) equation written with integrals; (bottom) equation written with sigma (bottom equation source: http://en.wikipedia.org/wiki/Image_moments.jpg)

Page 17: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

In the equation above, the centroid of a detect object can be found with:

x=M 10

M 00y=

M 01

M 00

3.3 Kalman Filter

The Kalman Filter is utilized in this project to optimize the control and accuracy of the robot arm. It is able to make a model of a problem more noise resistant. Because in the real world, noise arise everywhere (lighting, slight movement of camera, inaccurate servo motors in the robot arm), having a more noise resistant model leads to high accuracy and precision. To simply put it, the Kalman Filter filters out noise, leaving only useful information.

The Kalman Filter has three main parts: prediction, measurement, and correction. The general equations are shown in figure 3.3. The Kalman filter first predicts a state using a model, obtains some sort of measurement from sensors, and then uses both the predicted state and the measurements to create a more accurate state (see figure 3.5). Over time, the model of the problem will become closer to having almost no error. Before explaining each of the states separate, I will present an example outlining the parts of a Kalman Filter.

Figure 3.15: General equations of the Kalman Filter (source: http://bilgin.esme.org/BitsBytes/KalmanFilterforDummies.aspx)

Page 18: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Figure 3.16: Example graph of a Kalman Filter and corresponding measurements in a robot localization problem. (Source: http://2.bp.blogspot.com/_Eogk12oYm-g/S7ZvOnBbTVI/AAAAAAAAAKg/2_IoODo8KoQ/s640/KalmanFusion.png)

3.3.1 Kalman Filter Example

Suppose that there is a river in which you want to monitor the water level. You have a model to predict the water level every hour. However, the model is not 100% accurate. Therefore, you send some to the river to physically check the water level. But the person is only able to check the water level once a day during noon. Also, the measured water level from the person is not 100% accurate. As a result, a Kalman Filter is used to combine the outputs from both sources (model and person) to give a better estimate.

In this example, the three stages are as following:

Prediction – theoretical model of river level Measurement – physical person checking the river level Correction – using both of the outputs from the prediction and measurement stage

to create a better estimate.

Page 19: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Figure 3.17: Sample graph of the river problem.

3.3.2 Model for Kalman Filter

One of the most important steps in the Kalman Filter is to create a model for the problem (general form equation shown in figure 3.7). The purpose of such model is to create a prediction (of the next state) for the current problem. If all the parameters in the project fit into the equations without contradiction, then a Kalman Filter can be used.

In the first equation (which calculates the next state in time), xk−¿¿ is the state being

predicted, A is a transition matrix (similar to slope in y=mx+b; it tells how the model should progress over time), x̂k−1 is the previous state, B is a general form matrix or constant,uk−1 is a control-input, and w k−1 is the process noise value. The second equation calculates the measurement vector zk for the model. It contains the following variables: H is a general matrix (typically used for unit conversion), xk is the current measured state, and vk is a variable denoting measurement noise.

xk−¿=A k−1 x̂k−1+Buk−1+wk−1¿

zk=H xk+vk

Figure 3.18: Equations of the model for a Kalman Filter*

*Note that the general equation will vary depending on function of Kalman Filter and type of Kalman Filter. The equation commonly used is: Xk = Fk−1Xk−1 + Bk−1uk−1 + Wk−1. I have changed the variable name from F to A to match with the equations in the latter portions of this section.

The first equation uses the form of y = mx + b, but also generalizes the equations to matrices and added variables for noise. This is because the common form of a Kalman Filter is for linear functions (although different variants like the Extended Kalman Filter can be used for non-linear models). In the model, a process noise value is added (uk). This value represents the variation in the model. For example, in the river height problem described above, the process noise may be describing the percent error that the model has (since almost all models are not perfectly accurate).

Page 20: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

The second equation is more simplistic than the first. It is used to refine the measurement data. A measurement noise value, which is an estimation of the noise occurring in measurement, is included in the equation. This is because noise has to be accounted for in the model.

The general equation (figure 3.7) can be decomposed into smaller equations. These equations form three major stages of the Kalman Filter: prediction, measurement, and correction. Note that these derived equations will vary depending of the current problem, and therefore are simplified.

3.3.3 Prediction Stage

The first stage in the Kalman Filter is the prediction stage. In this stage, a model is used to predict the future state of a problem (i.e. river level model). In the equations shown below, the variables have the same meaning as the model shown in figure 3.7.

xk−¿=A x̂ k−1+Buk ¿

Pk−¿=A P k−1 A T+Q¿

Figure 3.19: Equations for the prediction stage of the Kalman Filter.

The first equation calculates the value of xk−¿¿, which is the value of next state before

correction. Notice that the only difference from the equation in Figure 3.8 to the model is a noticeable lack of the process noise variable. This is because in many models, process noise isn’t apparent in the model. However, the variable is necessary for problems with significant processing noise.

The next equation is used to calculate the value of the error covariance (Pk−¿ ¿) before

correction. It introduces new values into the equations. Namely these values are: Pk−1 which is the previous error covariance value, and Q is the estimated process error covariance.

3.3.4 Measurement Stage

The next stage of the Kalman Filter is the measurement stage. The purpose of this step is to simply obtain some sort of data from measurement. Therefore, there are no general equations for this stage. This process occurs in the river level problem as the person goes to physically measure the depth of the water.

3.3.5 Correction Stage

The last and most important stage of the Kalman Filter is the correction stage. The purpose of this stage is to give a more accurate predicted state from the data outputted from both the prediction and measurement stages. To do this, there are three equations in the stage, and they are displayed in figure 3.9. New values introduced into the equations are: R which is the measurement error covariance matrix, and K k which is the Kalman Gain.

K k=Pk−¿ H T¿ ¿¿

Page 21: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

x̂k= x̂k−¿+Kk ¿¿

Pk=( I−K k H )Pk−¿ ¿

Figure 3.20: Equations used in the correction stage of the Kalman Filter

The first equation calculates the Kalman Gain (K k). The Kalman Gain is a correction factor. It determines how much correction (measurement – prediction) is needed. If the Kalman Gain is value 0.5, then 50% of correction is added. If the value is closer to 0, then the predicted state has a higher weight than the measurement state. When the value is 0, then the measurement state is completely ignored. The opposite happens as the Kalman Gain approaches 1.

The second equation utilizes the Kalman Gain to calculate the value of the next (corrected) state. The Kalman Gain takes the difference between measurement and prediction values and applies it to the current state. Finally, the third equation calculates the new error covariance value (in a very similar manner to the calculation of the new state).

Page 22: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

4. Project Description4.1 Purpose and Long Term Goals

The long term goal of the project is to create an interactive human machine interfacing (HMI) device. This means that:

1. User(s) would be able to interact on a projection of a screen to control the device.a. Support multiple touch with multiple hands

2. Interaction should be done with real-time speed (little or no lag delay)3. High precision and accuracy4. High portability (users should be able to quickly set up and take down the system;

HMI device should not be locked to a position) 5. Universal support for input devices (should work on devices from smartphones to

computers regardless of operating system)6. Medium-low cost (should in an average consumer’s price range for technological

equipment)

The purpose of the creation of such device is to help technological progression of all of society. This means that the device should be easily tangible to the common man’s life. The main idea in mind is to help productivity in businesses, as a large interactive projection during a conference could spur more creative and collaborative thinking. Other uses include:

Business: Large projection to help collaboration grow in company to foster creative thinking.

Entertainment industry: Interactive videogames Everyday life: can be used in multiple ways in a common household – work, cooking,

map planning, family entertainment, as well as in non-conventional areas like a bathroom.

Research: can influence development of more “futuristic” human-machine interfaces like the holographs of the future.

4.2 Goal

The goal for phase 1 of the project is:

The goal is to develop and engineering an interactive human-machine interface for computers, as well as to analyze the effect of adding the Kalman Filter optimization algorithm.

The goals of the device are to meet the following self-imposed requirements:

10+ frames per second

Page 23: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Universal setup – projection working on various surfaces and orientation Usability – (ideally 80% and above) accuracy of touch

4.3 Research Question

In an interactive human machine interface, will the use of Kalman Filters smoothen and reduce noise in the control of the device? Can the use of Kalman Filters help optimize movement?

4.4 Constraints

Cost (Funding): Little amount of money limits quality and quantity of devices to be used.o Little money led to purchase of 3M Roku Streaming Project for $170 (average

price for medium-high quality project at around $300) Available Resources:

o Kinect for XBOX was available for use before project began, so it was used to save money.

o Medium-low end computers were used for program and testing Current State of Technological Research:

o Since this is a new idea, there isn’t any direct support for the problem. Instead, the project will have to be constructed from pieces of existing research.

4.5 Variables

For Kalman filter testing:

Dependent variable: Presence of Kalman Filter

Independent variables: Accuracy, smoothness and rigidity of movement

Control variables:

Types of algorithms used Type of 3D camera used in HMI device Type of projector used in HMI device Distance of 3D camera from projection Surface of testing area Finger used for testing

Variables to be controlled throughout the whole project:

Image noise Type of camera/projector being used in HMI device Type of computer and software being used

Page 24: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Lightning Surface of projection

5. Design Process of Implementation5.1 Drafting

The first instance of drafting occurred in early March of 2013. The models presented are an initial perspective on the final design. The goal stayed the same: creating a new human machine interface using 3D vision.

Figure 5.1: Different drafts of an interactive HMI device prototype

The idea revolves on having a very thin pico projector and very small 3D vision camera. Unfortunately, given the amount of low-cost technology around us, currently both ideas aren’t too feasible.

One of the most unique devices considered for a prototype is the Samsung Galaxy Beam. The Galaxy Beam is a android powered smartphone with an embedded projector. Unfortunately the projector, with DLP nHD and 15 lumens, displays in fairly low resolution, and needs a very dark

Page 25: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

room. At the same time, the phone itself does not have impressive specifications for a price of $350.

Figure 5.2: The Samsung Galaxy Beam – an android powered phone with an embedded projector (source: http://i-cdn.phonearena.com/images/articles/59957-image/samsung-galaxy-beam-0.jpg).

5.2 Current design model – The First Prototype

The current testing model settled on is utilizing the Microsoft Kinect and the 3M Roku Streaming Projector. There is no stand built (the one displayed on my poster is only for demonstration), though the Kinect normally rests directly above the Projector.

A more in-depth look at the materials used is offered below:

Page 26: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

5.2.1 Microsoft Kinect

The Kinect is used since of my immediate access to it, as well as wide support for the programming of the Kinect. More information can be found in section 3.1.1.

5.2.2 3M Roku Streaming Projector

Figure 5.3: 3M Roku streaming projector (source: http://g-ec2.images-amazon.com/images/G/01/aplus/detail-page/B008X1BV2Y_image2.jpg)

The 3M Roku streaming projector is aimed at being used for entertainment, as it includes a Roku streaming stick that allows for connection to Roku’s streaming service. However, the projector also includes a HDMI input, which allows input from a computer or a smartphone. One of the main reasons for the use was because of the low price in comparison to other pico-projector ($170 on sale compared to the typical $300). The specs are listed below:

Dimensions: 2” x 4” x 4.5” Weight: 1 lb Maximum Resolution: 800 x 480 (16:9 widescreen) DLP technology (Texas Instruments technology allowing for projection using a

lightsource and microscopic mirrors. Allows for creation of pico-projectors) HDMI input WVGA resolution Max image size: 120” Brightness: 60 Lumens

5.2.3 OpenCV, OpenNI2 and Visual Studios

Microsoft’s Visual Studio 2010 was used for the integrated development environment of the project. The programming language of choice is C++ with the OpenCV 2.4.3 library used for image processing and recognition.

Page 27: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Kinect SensorDepth ImageRGB Image

ProjectorProjection of computer

ComputerMain processing unit

Windows 7 OS

Runs finger detection algorithmControls mouse movement

USB

HDMI

Figure 5.4: Screenshot of a code written with OpenCV in the Visual Studio 2010 IDE (from last year)

OpenNI2 is used to handle Kinect processing with OpenCV. It is an open source library created by PrimeSense, the company responsible for creating the original Kinect. The library allows for easy access to the camera channels of the Kinect.

5.2.4 Flowchart of Model

Figure 5.5: Overall connection of current model

5.2.5 Limitations to Current Model for Testing

Only PC supporto Does not recognize double clicks yet

Page 28: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Noise filtering not perfect Software calibration not working perfectly (manual calibration used)

Page 29: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Initialization: Mapping of Background

Calibration:Alignment of Depth and RGB Maps

Detection:Finger Detection

Optimization:Kalman Filter to Smoothen Movement

6. Methodology6.1 Overview of Process

The general process:

Figure 6.1: General outline of software algorithm and methodology

6.2 Initialization

In the initialization process, the main goal is to select the projection as a region of interest (ROI), and then to analyze the properties of the area inside the region of interest. This method allows for the use of the device on irregular surfaces, though flat surfaces are still the most optimal surface type. There are two main steps in the initialization process:

1. User input of ROI using mouse – a rectangle is drawn to outline the projection from the HMI device. This means that the device would calibrate its input to fit the scale of the projection.

2. Mapping of depth of ROI – the initial background in the ROI is mapped and stored. This is so it is possible to determine when touch is applied to the surface.

Page 30: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

The first step defines the region of interest. This implemented by having a user-interface where the user can select the area by drawing a rectangle. This ROI is very important as all the future functions will only process inside the ROI (to have faster processing speed). It is also important for the user to select only the boundaries of the projection, as the program will calibrate the computer screen to the ROI.

The second step is used in the finger detection stage to detect finger touch. This is because there needs to be a reference of how the background looks like. Thus, this step takes all the depth information of aforementioned selected ROI and stores it for later use.

Figure 6.2: Initialization Step

6.3 Calibration

Calibration is performed to align the Kinect’s depth map to its RGB color map. This is because in the physical construct of the device, the depth camera is located slightly away from the RGB camera, and thus they see different images. This means that the (x, y) coordinate of one

Page 31: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

map does not match the other. Thus, the calibration is done to correct the one-to-one mapping of pixels.

Since the calibration is done for aligning the pixels in only two planes (x and y), 3D calibration is not necessary. 3D calibration would only be needed if the distortion of recorded depth values made a noticeable difference in data and in calculations. Therefore, only the shift and the pixel-length ratio need to be calculate. The general idea is described below:

1. Calculate the pixel-length ratio2. Calculate shift between cameras3. Adjust depth map to fit RGB map (described in finger detection process)

The pixel-length ratio is necessary for detecting objects accurately on non-linear surfaces, because this ratio may be different from depth and RGB images. Take the following situation:

Figure 6.3: Environment causing different pixel-length ratios

In this situation, the depth camera (on the left of the RGB camera) will show the object smaller, as it is farther away from the depth camera than the RGB camera. As a result, directly combining the two images do not present a workable combination since the sizing is different. Therefore, it is necessary to find the pixel-length ratio. This is done by taking an object with known size, and then by comparing it to the length in pixels from each camera. In the current implementation, this calibration is done by using a blue block of size 77 by 21 millimeters.

The second step is to find the shift between cameras. This is done by tracking the midpoint of the blue block (same as above) in the depth image, and then subtracting it from the midpoint of the same block in the RGB image. Thus, the x and y shift is obtained.

Page 32: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Figure 6.4: Zoomed in image calibration phase. Two contours are used to calculate shift

6.4 Finger Detection

Finger detection is the main function of the software developed. It is the process of recognizing when a finger touches the projection surface. This is done with the help of the initial 3D Depth map recorded in the initialization step. In order to determine if an object has contact with the projection, an “invisible” layer is formed over the background. If the finger penetrates this layer, then it is detected as a touch. This is done by converting the area in between the “invisible” layer and the background to a binary image, and then by having the computer process the image to determine what is a finger and what isn’t. Thus, there are two main steps to this process:

1. Detection of objects touching the projection2. Recognition of fingers on the objects touching the projection (calculated from step 1)

The first step works similar to the image presented below:

Page 33: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Situation Thresholded binary image

Figure 6.5: Step 1 of the finger detection process – the intersection of an object and the “invisible” layer is described as a binary image

The “invisible” layer is defined as 10 depth units away from the background (which the data was obtained is the initialization process. Thus, the layer is not linear, but instead is based on the background texture, allowing for the processing to work on non-linear surfaces. The binary image is obtained by creating a new binary image based off data from the depth map. If an object is in that aforementioned 10 depth units range, then a white pixel is placed at the (x, y) position. Otherwise, the pixel is deemed black.

The second process is to recognize what shapes in the binary image actually correspond to a finger instead of just noise. The algorithm presented is an optimized version of my detection algorithms from my previous projects. An expanded look at this process is shown figure 6.5.

The first step is to obtain the binary depth image, which is mentioned above. Then, using the binary depth image, a Gaussian blur with kernel size [3, 3] is applied to reduce small noise in the image. This is because the blur uses a statistical normal distribution to operate, which in return is able remove small noise from the image. It is after the blurring in which the contours are found.

Contours are filtered by only two variables, as compared to seven from previous versions of this code, to speed up processing. It is observed through previous years experience that having more image properties used to filter causes more operations to be calculated per contour per image frame. The two properties used are area and circularity. Area is used since it can easily filter out the very large and very small contours that definitely aren’t fingers. Then, circularity is

Page 34: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

used since a finger is relatively circular, and thus can help remove amorphous noise. Before operations can be done while still retaining a frame rate of 15 or more frames in most cases.

The last method of filtering focuses on one frame noises which appear and disappear in the next frame. The general outlook is that it takes are noise with “finger-like” properties and checks if they have existed in the frame before. The drawback is that it does at a 1 frame delay to the detection of a finger, but considering a frame rate of at least 15+ frames a second, there ends up only being a 1/15 second delay. The inclusion of such function drastically improves the filtering and accuracy of the entire algorithm as observed by qualitative data testing.

It is after the entire contour filtering process in which the centroid of each contours is calculated. Then, the centroid is modified by the calibration factors above in the form of:

ne w x=ol dx∗scaling factor+shift factor

ne w y=ol d y∗scaling factor+shift factor

Multiplication operation is very costly in terms of speed for image processing. If the whole 640x480 data was to be operated on by multiplication, speed would drop under 5 frames per second. Thus, in order to help the speed stay the same, the calibration values only affect the centroids (which are the only important points in the first place).

Page 35: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Receive binary image

Gaussian blur on image

Find blobs

Has all blobs been evaluated yet?

Has the blob existed in the frame before?

Calculate centroid using image moments

Draw contour

Modify coordinate by scaling and shift factor

Does blob have right area and circularity?

YesNo

No Yes

YesNo

Go to next blob

Figure 6.6: Finger detection algorithm flowchart

Page 36: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Figure 6.7: Finger is detected by the algorithm. Although it may seem to be off center, the position is actually corrected using the calibration factors. Thus, if the image was zoomed in more, you would see a “cross” projected on the finger, signifying the true position of the touch.

6.5 Implementation of Kalman Filter into Current Application

The use of Kalman Filter is applied after the centroid of the finger is found in the earlier steps. In here, the role of the Kalman Filter is to smooth the motion of cursor the finger controls, as well as to remove noise.

Smoothing occurs as a direct result of Kalman Filter. The prediction given into the model allows for the cursor to fill in the time intervals between data collection. This naturally creates a more fluid movement on the computer. At the same time, the prediction causes the movement to be more realistic; it is based-off velocity. Thus, the effect of any spikes in the centroid position caused by the detection of white noise will be lessened through the use of a Kalman Filter.

Page 37: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

The model of the Kalman Filter uses velocity, as were as position (x, y). This model works because given the previous velocity of the arm; a prediction can be made by adding the previous velocities to the previous state. Thus, the state (xk) is represented by a 4 by 1 vector. The vector is in the format (x, y, velocity_x, velocity_y). The transition matrix (A) performs the operation of adding the velocity to the current state, and is shown in figure 3.12.

`

[1 0 1 00 10 00 0

0 11 00 1]

Figure 3.12: Transition matrix (A)

When implementing the algorithm in coding, there actually exists a function in OpenCV to implement a Kalman Filter. It consists for four separate functions: one to initialize the Kalman Filter, one to predict to next state, one to receive and store the measurements, and one to correct the current state. The program coding is shown in figure 6.7.

Initialization:

KalmanFilter KF(4, 2, 0); Mat_<float> state(4, 1); Mat processNoise(4, 1, CV_32F); Mat_<float> measurement(2,1); measurement.setTo(Scalar(0));

KFs.statePre.at<float>(0) = x;KFs.statePre.at<float>(1) = y;KFs.statePre.at<float>(2) = 0;KFs.statePre.at<float>(3) = 0;

KFs.statePost.at<float>(0) = x;KFs.statePost.at<float>(1) = y;KFs.statePost.at<float>(2) = 0;KFs.statePost.at<float>(3) = 0;

KFs.transitionMatrix = *(Mat_<float>(4, 4) << 1,0,1,0, 0,1,0,1, 0,0,1,0, 0,0,0,1);

setIdentity(KFs.measurementMatrix);setIdentity(KFs.processNoiseCov, Scalar::all(1e-4));setIdentity(KF.measurementNoiseCov, Scalar::all(1e-3));setIdentity(KFs.errorCovPost, Scalar::all(.1));

General Coding to Predict (Step 1)

Point step1()

X Y Vel

ocity

Y

Vel

ocity

X

Velocity YVelocity X

XY

Page 38: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

{ Mat prediction = KFs.predict(); Point predictPt(prediction.at<float>(0),prediction.at<float>(1)); return predictPt;}

General Coding to Input Measurement*

void changeMeasure(float x,float y) { measurements(0) = x; measurements(1) = y; }

General Coding to Correct (Step 2)

Point step2(){

Mat estimated = KFs.correct(measurements);Point statePt(estimated.at<float>(0),estimated.at<float>(1));return statePt;

}Figure 6.7: Example coding for the Kalman Filter

Page 39: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

7. ExperimentationAll current experimentation was done with the following setup:

Figure 7.1: Setup for experimentation of interactive HMI device

7.1 Accuracy Testing

The first test was on the accuracy of the device. The data taken for analysis is the Euclidean distance between the true center (predefined) and the center of touch recognized by the software. A simple procedure is outlined below:

7.1.1 Method for Testing

1. Test points were randomly generated (one pixel points with a larger outer circle to make it visible to the experimenter)

a. 15 points were chosen2. The image was loaded into a drawing software (like Microsoft Paint or Adobe

Photoshop)3. The HMI device software was run

a. The software was calibrated to a projection of the computer (via HDMI cable) on the wall.

4. The experimenter’s finger attempted to touch the center of every single point once at a normal pace.

a. A 1 pixel radius brush was selected.

Page 40: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

5. The distance between the target’s center and the furthest drawn pixel for that target was calculated

a. This is because in the current implementation, a “click” and a “click and hold” is determined by the length of time the finger is touching the surface. However, with that threshold, a small line may be drawn, even though it is still a click.

To following image was used for testing:

Figure 7.2: Image file used for experimentation (omit the number labels).The 1 pixel center points are there; the image has to be compressed in size to fit on this page.

7.1.2 Data from Experimentation

In total, 3 trials were tested, yielding a 45 data point total.

Target Number

Distance of Point from Center (pixels)Trial 1 Trial 2 Trial 3 Average

1 18.02775638 28.0713377 11.40175425 19.166952 22.47220505 8.94427191 16.4924225 15.969633 16.64331698 17.49285568 12.64911064 15.595094 18.38477631 26.01922366 25.29822128 23.234075 18.78829423 19.02629759 30.8058436 22.873486 21.9317122 20.59126028 20.61552813 21.046177 7.280109889 30.08321791 12.04159458 16.468318 25.49509757 25.05992817 22.6715681 24.408869 12.08304597 19.6977156 18.43908891 16.7399510 21.09502311 15.65247584 24.33105012 20.3595211 15.23154621 18.43908891 9.848857802 14.506512 17.2626765 22.47220505 24.51530134 21.4167313 18.43908891 15 18.02775638 17.1556214 25.96150997 26.40075756 17.02938637 23.13055

1 2 3 4

5 67

89

10

11

12

1314 15

Page 41: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

15 29.01723626 38.62641583 22.8035085 30.14905Average: 19.2075597 22.10513678 19.1313995 20.14803

1 2 3 4 5 6 7 8 9 10 11 12 13 14 150

5

10

15

20

25

30

35

Average Distance of Touch from Center

Target Number

Dis

tanc

e fr

om C

ente

r (p

ixel

s)

Figure 7.3: Average distance of point of touch from center point per target over a course of 3 trials

1 2 317.5

1818.5

1919.5

2020.5

2121.5

2222.5

Average Distance of Touch from Center per Trial

Trial Number

Dis

tanc

e fr

om C

ente

r (p

ixel

s)

Figure 7.4: Average distance of point of touch from center point per trial over 15 targets

Page 42: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 390

2

4

6

8

10

12

Distribution of Distance from Center

Pixels from Center

Freq

uenc

y

Figure 7.5: Distribution of distance of point of touch from center point per trial over 15 targets and 3 trials

7.1.3 Observations

Currently, the average mean of the data seems to be around 20 pixels. This means that it is expected to have up to a 20 pixel displacement during a touch with the current beta stage of the algorithm. The standard deviation for the data range is at 6.312967, meaning that 99.7% of the time, the worst possible situation will present a deviation of 39 pixels from the actual area of contact. Since the data was recorded from a computer resolution of 1366 by 768, the error is small, averaging around 1 to 2% error. For reference, the approximate size of my own finger tip is a circle with radius of 30 pixels, thus having an error averaging a 20 pixel displacement will make little to no visual difference in the performance of the software.

7.2 Effects of Use of Kalman Filter

The second test was to compare the smoothness and accuracy of the use of a Kalman Filter to control mouse movement over the movement without the use of a Kalman Filter. The test conducted compared the statistical properties of an array of (x, y) coordinates recorded from a finger tracing various shapes.

8 shapes were used in totalo 5 trials were tested per shape

50-400 data points logged per shape. In total: Over 5000 data points were recorded and analyzed

7.2.1 Method for Testing

1. Several shapes were drawn beforehand as jpeg files.

Page 43: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

2. The image was loaded into a drawing software (like Microsoft Paint or Adobe Photoshop)

3. The HMI device software was run a. The software was calibrated to a projection of the computer (via HDMI cable) on

the wall.4. The experimenter’s finger traced the shape

a. The exact shape and path of the finger does not have to be to pin-point accuracy. What matters most is that the general shape is captured, and both the raw measurement and the Kalman Filter processed data are recorded.

5. The program recorded the movement of both the original raw data and the processed Kalman Filter optimized data.

6. Data analysis was done for all trials on all data points.a. The obtained (x, y) coordinates were compared to a moving average of window 5

7.2.2 Data from Experiment

The entire data can be found in Appendix B.

Data: Vertical Tracing Error from Line 1 (y-coordinate):

The idea here is to see the shift of the vertical movement of the mouse when a horizontal line is drawn. Three statistical properties were analyzed when the respective coordinates were compared with the moving average of a window of 5 elements. Mean is the arithmetic average and maximum is the highest value. RMS stands for Root Mean Square, which is measure of the magnitude of a varying quantity. In this data analysis, root-mean-square roughness is used, and is defined as:

σ rms=√ 1N ∑

i=1

N

( y i− y❑ma )2

Figure 7.6: RMS roughness equation used

Trial Number

Mean (pixel disparity) Maximum (pixel disparity) RMS* (pixel disparity)

Kalman Filter Raw Measurement Kalman Filter Raw

Measurement Kalman Filter Raw Measurement

1 0.226666667 0.368 0.8 1.2 0.295446931 0.5091168822 0.440650407 0.492682927 1.4 1.8 0.536070213 0.6393594363 0.295238095 0.380952381 1.2 1.6 0.43570778 0.5829590644 0.218390805 0.542528736 1 1.4 0.345967465 0.6515528175 0.435714286 0.578571429 1.8 2 0.608276253 0.76997217Average: 0.323332052 0.472547094 1.24 1.6 0.444293728 0.630592074

Page 44: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Mean Max RMS0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

Statistical Comparison for Vertical Tracing Error

Kalman FilterRaw Measurement

Aver

age D

iffer

ence

(pix

els)

Likewise, in the same way, the horizontal tracing error was tested.

Data: Horizontal Component from Line 2 (x-coordinate):

Trial Number

Mean (pixel disparity) Maximum (pixel disparity) RMS (pixel disparity)

Kalman Filter Raw Measurement Kalman Filter Raw

Measurement Kalman Filter Raw Measurement

1 0.412962963 0.703703704 2 2.8 0.608682041 0.9536156022 0.425925926 0.4 1.333333333 1.4 0.544104202 0.5515410513 0.232911392 0.410970464 1 1.6 0.318929292 0.518083674 0.484615385 0.687179487 2.333333333 2.666667 0.664226303 0.8797824135 0.361006289 0.533333333 1 1.4 0.473268664 0.647264847Average: 0.383484391 0.547037398 1.533333333 1.973333333 0.5218421 0.710057517

Page 45: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Mean Max RMS0

0.5

1

1.5

2

2.5

Statistical Comparison for Horizontal Trac-ing Error

Kalman FilterRaw Measurement

Aver

age D

iffer

ence

(pix

els)

Data: Analysis of Shift from Moving Average (Both X and Y coordinates)

Shape Mean (pixel disparity) Maximum (pixel disparity) RMS (pixel disparity)

Kalman Filter Raw Measurement Kalman Filter Raw

Measurement Kalman Filter Raw Measurement

Circle 0.760306277 1.04597043 2.68 5.04 0.950901852 1.392267514Star 1.217517502 1.484209298 5.4 6.92 1.606905262 1.986660438Triangle 0.926119171 1.228783885 4.84 7.12 1.302888577 1.757424069Figure 8 0.922734121 1.17925881 3.56 4.88 1.177789274 1.544603127Heart 0.749682013 0.98949269 3.24 5.04 0.966851601 1.31221052Lightning 1.12444493 1.316371013 4.32 5.64 1.468219327 1.697575176Average: 0.950134003 1.207347688 3.351666667 4.776667 1.245592649 1.615123474Percent Improvement: 21.3040276% - 30.6004619% - 22.8794164% -

Page 46: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Mean Max RMS0

1

2

3

4

5

6

7

Statistical Comparison of Tracing Error

Kalman FilterRaw Measurement

Aver

age D

iffer

ence

(pix

els)

7.2.3 Observations

Since it is impossible to find the true position of the finger movement, a moving average is used as an approximation. One benefit of using moving average is that it checks the “validity” of the mouse movement. That is, in a continuous function, since there are no abrupt motions, the actual recorded data points should be near the moving average. The graphs from the experiment all show an identical pattern: the Kalman Filter has less pixel displacement from the moving average. In fact, the use of a Kalman Filter provided at least a 20% improvement in the accuracy of movement of mouse as compared to without the use of a Kalman Filter. The most improvement was seen the max value, signifying that Kalman Filter is less effected by noise.

Another important observation not visible in the numerical data is the total number of peaks (spikes) present in the movement (refer to Appendix B for the visual graphs). It can be seen that without the use of Kalman Filter, there is a drastic increase in the number of spikes. From observation, it would be reasonable to suggest that the use of a Kalman Filter can reduce the peak count by more than 50%.

Page 47: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

8. ConclusionThis project introduces a completely new idea for a device that can help increase

collaboration in businesses and research, as well as a method to create new human-machine interfaces. The idea proposed is to create a full functional portable interactive HMI Device. In the first phase of the project presented here, a prototype was designed and testing using the Microsoft Kinect and a pico projector. The device used a combination of both depth sensing image analysis and Kalman Filter optimization.

In conclusion of the first phase of the project, it can be seen that the current prototype of the device works well. From the two experiments conducted, it can be seen that the device has high accuracy, and as improved on by the use of a Kalman Filter. In the first experiment, the device was seen, on average, to have only up to a 20 pixel difference from the “true” point. In the second experiment, the use of a Kalman Filter on to the device helped improve rigidity, resistance to noise, and overall the accuracy by over 20%.

The first experiment tested the accuracy of touch in the device. 60 data points were conducted in total over the course of 3 trials of 15 targets, and were tested for the Euclidean distance between the true point and the detecting fingertip point on a laptop screen resolution of 1366 by 768 pixels. It is seen in the analysis of the data that the average mean of this displacement is around 20 pixels, signifying that on average it is expected to have up to a 20 pixel displacement during a finger touch. The largest recorded displacement for these 60 data points is at 39 pixels. However, it is important to consider the fact that when a finger touches any surface, there more than just one pixel making contact with the surface. From observation, it can be approximated that a fingertip touching a surface will have an area of a circle with at least a 30 pixel radius. Thus, it can be concluded that there should not be any noticeable difference in the position of a finger touch with up to 3 standard deviations (99.7%).

The second experiment presented an extensive comparison between the use of an algorithm with Kalman Filter optimization and the use of an algorithm without Kalman Filters for controlling the movement of the mouse. In a data sample of over 5000 data points, spread across 40 trials in 8 unique shapes, a general trend was found. In all shapes, in all trials, and in almost all data points, the Kalman Filter optimization provided over 20% improvement in mean displacement, maximum peak, and root-mean-square roughness over an algorithm without the use of the optimization. In terms of mean displacement between the mouse movement and a calculated moving average, the use of a Kalman Filter boasted 21% improvement. The use of a Kalman Filter saw a 31% improvement in reducing the peak size. Finally, Kalman Filter optimization had a 23% improvement in the root-mean-square roughness of the mouse movement. Another important comparison is the number of peaks present in each shape, as from the graphs it can be expected with high-confidence that the Kalman Filter can provide more than 50% improvement in reducing the number of peaks. All of these points easily show that Kalman Filter optimization is very beneficial to smoothing and improvement accuracy of mouse movement.

Page 48: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

The results of both experiments show a clear indication that the methodology presented in this project works well with high accuracy and low error rates. The initialization stage enables the device the work in almost any environment. The finger detection algorithm allows for accurate measurement of finger touch position, and is supported by experiment 1. Finally, the use of Kalman Filters to optimize mouse movement is clearly justified with the massive improvements in accuracy gained. Thus, the methodology can be used to create a novel HMI device that is portable and can realistically be used.

Page 49: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

9. Extensions: Support Vector Machine Classification to Improve Finger Recognition

9.1 Purpose

In the first variant of the methodology, one of the biggest weaknesses was the accuracy of finger recognition. The old finger recognition technique worked most of the time, though sometimes some small noise was recognized as a finger. However, as the screen projection got larger, the accuracy of the device significantly got worse. One of the main reasons is that the classification was done using fixed values of area, perimeter, and circularity. Therefore, it would make sense that the accuracy would drop for larger screens, as there were less pixels to define a finger touch. Thus, it was decided that a more adaptable and reliable recognition method was needed to improve the overall quality of the device.

9.2 The Support Vector Machine (SVM)

The support vector machine (SVM) is a classification algorithm. This means that the algorithm is able to determine what a data point should be classified depending on the initial training data (data initially used to tell the algorithm what different types of data points are classified). The support vector machine itself was introduced in 1963 by Vapnik as a linear classifier. However, the most well-known paper is, A training algorithm for optimal margin classifiers by Boser, Guyon, and Vapnik (1992), which describes non-linear methods to the SVM. In the last decade, the use of support vector machines in research, such as cancer detection, has increased almost exponentially.

The main benefit from using a support vector machine is that it is able to find the optimal estimator function based on the set of training data. This makes the algorithm highly effective and efficient in classifying new data points (not seen in the training data). Other benefits from using the SVM is that it finds the global optimal, whereas other conventional methods such as neural networks sometimes fail to find the global optimum, finding a non-optimal local maximum instead. Finally, in comparison to other techniques like neural networks and decision trees, the SVM is easier to use. Thus, although highly-optimized data pre-processing used on these methods may yield the same or similar results, the chance of having good results from a SVM is much higher.

Page 50: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Figure 9.1: Example visualization of classification problem

What the SVM finds is the optimal hyperplane that can separate two unique sets of data. The most optimal hyperplane would maximize the distance (margin) between itself and the nearest sets of data. The nearest sets of data points (indicated by yellow in figure 9.1) are called support vectors. Moving support vectors will directly affect the optimum hyperplane.

Figure 9.2: Effects of moving a support vector (source: Berwick 2003 presentation)

Area

ExtentStars Circle

Page 51: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Figure 9.3: In-depth visualization of hyperplanes in SVM problem (source: Berwick 2003 presentation)

The problem can be formalized to the example in figure 9.3. The optimum hyperplane is described as w ∙ x−b=0 where w is the normal vector to the hyperplane, x is the set of points x

satisfying the equation, and the offset from the origin is calculated byb

‖w‖. ‖w‖ is the Euclidian

length of the vector, which is defined as . The dot product in w ∙ x is directly related to the cosine of the angle between both vectors.

Likewise the hyperplane for the support vectors are denoted as w ∙ x−b=1 and w ∙ x−b=−1, which classifies data points above w ∙ x−b=1 as 1 and data points below w ∙ x−b=−1 as -1. The margin can be seen as d+ + d-.

The distance between H (optimum hyperplane) and H1 is |w ∙x+b|

‖w‖ which is1

‖w‖. Thus,

the margin is mathematically defined as2

‖w‖. The goal of the SVM is to minimize‖w‖, which

can more effectively be done by solving the quadratic 12‖w‖2

(modified form) using quadratic

programming techniques.

Page 52: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

The above process shows the general process for support vector machines in linear classifications. For non-linear classification problems, a process called kernelling is used. The general idea of kernelling is to map the problem to a higher dimension, which allows for classification equations to be solved. The structure of the algorithm is still the same, but the dot products are replaced by a kernel function. For example, the kernel function for a polynomial is defined as k ( x i , x j )=( x i ∙ x j )

d. However, one of the more commonly used kernelling function is a Gaussian radial basis function which maps the problem to an infinite dimension, Hilbert space, and is defined as k ( x i , x j )=exp (−γ‖x i−x j‖

2 )

Figure 9.4: Example of SVM classification with different kernels.

Page 53: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Figure 9.5: Visualization of output result of support vector machine with an rbf kernel

9.3 Use of Support Vector Machine in Methodology

Support vector machines were implemented in this newer build of the software in order to improve the accuracy of finger recognition, as the first variant had troubles with this aspect. The machine learning portion of the OpenCV library was used for the implementation. The parameters of the support vector machine used are automatically optimized by a predefined function. Generally speaking, the support vector machine mode is set to a two-class problem, and the kernel used is for non-linear functions (most likely an rbf kernel).

Over 700 contours (shapes) were used in the training data. Of these, around 100 of the contours were fingers. For each shape, the area, perimeter, circularity, and extent was calculated and used as the training data. To obtain the training data, two bitmap image files were created. One file was for fingers, and one was for non-fingers (other shapes, noise, etc).

Page 54: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Figure 9.6: Finger bitmap input for training data

Figure 9.7: Non-finger bitmap input for training data.

Shapes defined as fingers were found by directly taking screenshots on the depth map binary image from the original version of the software. Noise was taken in the same way. At the beginning of the algorithm, the software reads, finds the contours (around 700) and calculates the area, perimeter, circularity, and extent for each contour. The shape feature data was placed into two separate vectors, one vector defining finger properties, and for non-finger properties. These vectors were done used to automatically define SVM parameters and train the SVM. Then, when the software is in the finger recognition step (filtering contours), the SVM is called to classify the current datapoint. If the datapoint is recognized as a finger, then the blob is determined to be a finger.

Page 55: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Read bitmap images for training Data

Find all blobs

Calculate area, perimeter, circularity, and extent for each blob

Find optimal parameters using training data

Train the SVM

Figure 9.8: Initialization flowchart using SVMs

Page 56: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Receive binary image

Gaussian blur on image

Find blobs

Has all blobs been evaluated yet?

Has the blob existed in the frame before?

Calculate centroid using image moments

Draw contour

Modify coordinate by scaling and shift factor

Is blob classified as a finger from SVM?

YesNo

No Yes

YesNo

Go to next blob

Figure 9.8: Shape recognition process using support vector machines

Page 57: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

10. Future WorkFuture work comes into the general sections:

Phase 2:

Adding support for Android phones Fixing automatic calibration Improve noise filtering Improve delay time between computer and HMI device

o Implementing double clicking on computer implementation Further optimization of finger detection

Phase 3:

Condensing HMI device into a more compact model that is truly portable Optimizing support for Android.

o Possibly adding support for iOS platforms Fixing errors in phase 2 Possible addition of support vector machines to improve recognition

Figure 10.1: Sketch of physical model of HMI device for phase 3

Page 58: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Phase 4 and beyond:

Switching 3D vision camera to more accurate and smaller model: Finalization of portable HMI device

Figure 10.2: Occipital’s new micro-sized 3D cameras for iPhones and iPads.

Page 59: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

11. ReferencesAndersen, M.R., T. Jensen, P. Lisouski, A.K. Mortensen, M.K. Hansen, T. Gregersen, and P.

Ahrendt. "Kinect Depth Sensor Evaluation for Computer Vision Applications." Diss.

Aarhus University, Department of Engineering, 2012. Aarhus University, Feb. 2012.

Web. 20 Feb. 2014.

Borenstein, Greg, Andrew Odewahn, and Brian Jepson. Making Things See: 3D Vision with

Kinect, Processing, Arduino, and MakerBot. Sebastopol, CA: O'Reilly, 2012. Print.

Bradski, G., & Kaehler, A. (2008). Learning OpenCV. Sebastopol, CA: O'Reilly Media, Inc.

Bräunl, T. (2006). Embedded Robotics. Springer-Verlag.Chen, G., & Guo, L. (2005).

Catuhe, David. Programming with the Kinect for Windows Software Development Kit. Redmond,

WA: Microsoft, 2012. Print.

Chen, S. Y. "Kalman filter for robot vision: a survey." Industrial Electronics, IEEE Transactions on 59.11 (2012): 4409-4420.

Cheng, Luo. "Fiber Surface Characterization By Laser Back Scattering." Diss. University of

Tennessee, 1989. Print.

Falahati, Soroush. OpenNI Cookbook: Learn How to Write NIUI-based Applications and

Motion-controlled Games. Birmingham: Packt Limited, 2013. Print.

Farrell, J. (2001). Object-Oriented Programming Using C++. Canada: Course Technology.

Kent, J. (2003). C++ Demystified. Emeryville, CA: McGraw-Hill.

Khoshelham, Kourosh. "Accuracy analysis of kinect depth data." ISPRS workshop laser scanning. Vol. 38. No. 5. 2011.

Khoshelham, Kourosh, and Sander Oude Elberink. "Accuracy and resolution of kinect depth data for indoor mapping applications." Sensors 12.2 (2012): 1437-1454.

Malmsten, Paul. "Object Discovery With a Microsoft Kinect." Diss. Worcester Polytechnic

Institute, 2012. 15 Dec. 2012. Web. 20 Feb. 2014.

Mehrotra, Sanjeev, et al. "Low-complexity, near-lossless coding of depth maps from kinect-like depth cameras." Multimedia Signal Processing (MMSP), 2011 IEEE 13th International Workshop on. IEEE, 2011.

Parker, J. R. (1994). Practical Computer Vision Using C. Canada: John Wiley & Sons, Inc..

Prata, S. (2004). C++ Primer Plus (5th ed.). Indianapolis, IN: SAMs Publishing.

Page 60: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raheja, Jagdish L., Ankit Chaudhary, and Kunal Singal. "Tracking of fingertips and centers of palm using kinect." Computational Intelligence, Modelling and Simulation (CIMSiM), 2011 Third International Conference on. IEEE, 2011.

Ren, Zhou, et al. "Robust hand gesture recognition with kinect sensor." Proceedings of the 19th ACM international conference on Multimedia. ACM, 2011.

Zhao, Qi-jie, et al. "Kalman filter based vision predicting and object tracking method and its application [J]." Optics and Precision Engineering 5 (2008): 028.

Page 61: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Appendix A: Experimentation Data from Accuracy Data – Image Files

Page 62: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 1:

Page 63: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 2:

Page 64: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 3:

Page 65: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Appendix B: Experimentation Data from Kalman Filter Comparison

Page 66: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Line 1:

Trial 1:

Kalman Filer Data:

Kalman:Moving Average: Difference:

Difference (abs): Statistics:

X Y Mean:163 56 56 0 0 0.226666667173 55 55.33333333 -0.333333333 0.333333333180 55 55.8 -0.8 0.8 Maximum:188 56 56 0 0 0.8197 57 56.6 0.4 0.4206 57 57.2 -0.2 0.2 Sum:215 58 57.8 0.2 0.2 5.666666667219 58 58.2 -0.2 0.2233 59 58.6 0.4 0.4 RMS:244 59 58.8 0.2 0.2 0.295446931253 59 58.8 0.2 0.2260 59 58.6 0.4 0.4266 58 58.4 -0.4 0.4272 58 58.2 -0.2 0.2277 58 58 0 0280 58 57.8 0.2 0.2284 58 57.6 0.4 0.4286 57 57.4 -0.4 0.4289 57 57.2 -0.2 0.2290 57 57 0 0292 57 57 0 0292 57 57 0 0291 57 56.8 0.2 0.2290 57 56.66666667 0.333333333 0.333333333289 56 56 0 0

Page 67: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average:

Difference:

Difference (abs): Statistics:

X Y Mean:164 56 56 0 0 0.368173 56 56 0 0

180 56 56.4 -0.4 0.4Maximum:

189 57 56.6 0.4 0.4 1.2197 57 57.2 -0.2 0.2208 57 57.8 -0.8 0.8 Sum:216 59 58.2 0.8 0.8 9.2216 59 58.6 0.4 0.4238 59 59 0 0 RMS:246 59 59.2 -0.2 0.2 0.509117252 59 59 0 0259 60 58.8 1.2 1.2264 58 58.6 -0.6 0.6272 58 58.6 -0.6 0.6276 58 58.2 -0.2 0.2278 59 58 1 1283 58 58 0 0285 57 58 -1 1288 58 57.8 0.2 0.2290 58 57.8 0.2 0.2291 58 57.8 0.2 0.2291 58 57.6 0.4 0.4290 57 57.4 -0.4 0.4289 57 57 0 0289 57 57 0 0

Page 68: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

150 170 190 210 230 250 270 29053

54

55

56

57

58

59

60

Line 1 Trial 1 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-c

oord

inat

e

150 170 190 210 230 250 270 29053

54

55

56

57

58

59

60

Line 1 Trial 1 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-c

oord

inat

e

Page 69: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 2:

Kalman Filer Data:

Kalman:

Moving Average: Difference:

Difference (abs): Statistics:

X Y Mean:

11 54 54 0 00.44065040

7

14 56 55.666666670.33333333

3 0.33333333316 57 56 1 1 Maximum:19 56 56.4 -0.4 0.4 1.424 57 56.4 0.6 0.629 56 56.4 -0.4 0.4 Sum:

36 56 56.6 -0.6 0.618.0666666

743 57 56.8 0.2 0.251 57 57.2 -0.2 0.2 RMS:

61 58 57.8 0.2 0.20.53607021

372 58 58 0 077 59 58 1 196 58 57.6 0.4 0.4

111 57 56.8 0.2 0.2126 56 55.6 0.4 0.4139 54 54.8 -0.8 0.8144 53 54.4 -1.4 1.4164 54 54.6 -0.6 0.6182 55 55.4 -0.4 0.4197 57 56.6 0.4 0.4210 58 57.6 0.4 0.4222 59 58.6 0.4 0.4233 59 59 0 0241 60 59.4 0.6 0.6248 59 59.4 -0.4 0.4255 60 59.4 0.6 0.6262 59 58.8 0.2 0.2268 59 58.4 0.6 0.6274 57 57.6 -0.6 0.6280 57 57.2 -0.2 0.2283 56 56.8 -0.8 0.8286 57 56.6 0.4 0.4289 57 56.2 0.8 0.8291 56 56 0 0

Page 70: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

293 55 55.8 -0.8 0.8293 55 55.6 -0.6 0.6293 56 55.6 0.4 0.4292 56 56 0 0291 56 56.4 -0.4 0.4

291 57 56.666666670.33333333

3 0.333333333291 57 57 0 0

0 50 100 150 200 250 30048

50

52

54

56

58

60

62

Line 1 Trial 2 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-c

oord

inat

e

0 50 100 150 200 250 3005152535455565758596061

Line 1 Trial 2 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-c

oord

inat

e

Page 71: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average:

Difference:

Difference (abs): Statistics:

X Y Mean:

12 55 55 0 00.49268292

714 56 56 0 017 57 56.2 0.8 0.8 Maximum:20 56 56.4 -0.4 0.4 1.825 57 56.6 0.4 0.431 56 56.8 -0.8 0.8 Sum:38 57 57.2 -0.2 0.2 20.245 58 57.4 0.6 0.654 58 58 0 0 RMS:

64 58 58.4 -0.4 0.40.63935943

675 59 58.2 0.8 0.875 59 58 1 1

104 57 57.6 -0.6 0.6115 57 56.6 0.4 0.4128 56 55.6 0.4 0.4139 54 55.4 -1.4 1.4139 54 55.4 -1.4 1.4171 56 55.8 0.2 0.2186 57 56.8 0.2 0.2198 58 57.8 0.2 0.2209 59 58.6 0.4 0.4220 59 59.2 -0.2 0.2232 60 59.4 0.6 0.6238 60 59.6 0.4 0.4246 59 59.6 -0.6 0.6254 60 59.4 0.6 0.6261 59 58.6 0.4 0.4267 59 58.4 0.6 0.6274 56 57.8 -1.8 1.8279 58 57.6 0.4 0.4282 57 57.2 -0.2 0.2285 58 57.2 0.8 0.8288 57 56.6 0.4 0.4291 56 56.4 -0.4 0.4292 55 56.2 -1.2 1.2292 56 56.2 -0.2 0.2292 57 56.4 0.6 0.6

Page 72: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

291 57 56.8 0.2 0.2291 57 57 0 0291 57 57 0 0291 57 57 0 0

0 50 100 150 200 250 30051

52

53

54

55

56

57

58

59

60

Line 1 Trial 3 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-c

oord

inat

e

0 50 100 150 200 250 3005152535455565758596061

Line 1 Trial 3 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-c

oord

inat

e

Page 73: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization
Page 74: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 3:

Kalman Filer Data:

Kalman:

Moving Average: Difference:

Difference (abs): Statistics:

X Y Mean:

8 55 55 0 00.29523809

513 56 56 0 020 57 56 1 1 Maximum:22 57 55.8 1.2 1.2 1.230 55 55.6 -0.6 0.633 54 55.2 -1.2 1.2 Sum:

43 55 55 0 016.5333333

355 55 55.6 -0.6 0.668 56 56.4 -0.4 0.4 RMS:80 58 57.2 0.8 0.8 0.4357077895 58 57.8 0.2 0.2

106 59 58 1 1115 58 57.8 0.2 0.2126 57 57.6 -0.6 0.6137 57 57.4 -0.4 0.4142 57 57.2 -0.2 0.2162 58 57.4 0.6 0.6181 57 57.6 -0.6 0.6196 58 58 0 0203 58 58 0 0224 59 58.2 0.8 0.8239 58 58.2 -0.2 0.2244 58 58.2 -0.2 0.2262 58 58.2 -0.2 0.2275 58 58.2 -0.2 0.2281 59 58.2 0.8 0.8285 58 58.2 -0.2 0.2287 58 58.2 -0.2 0.2288 58 58 0 0289 58 57.8 0.2 0.2288 58 57.6 0.4 0.4287 57 57.4 -0.4 0.4287 57 57.2 -0.2 0.2287 57 57 0 0288 57 57 0 0

Page 75: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

289 57 57 0 0290 57 57.2 -0.2 0.2290 57 57.4 -0.4 0.4290 58 57.6 0.4 0.4290 58 57.8 0.2 0.2290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0289 58 57.8 0.2 0.2289 58 57.6 0.4 0.4289 57 57.4 -0.4 0.4289 57 57.2 -0.2 0.2289 57 57 0 0289 57 57 0 0289 57 57 0 0289 57 57 0 0290 57 57.2 -0.2 0.2

290 57 57.33333333

-0.33333333

3 0.333333333290 58 58 0 0

Page 76: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average: Difference:

Difference (abs): Statistics:

X Y Mean:

9 56 56 0 00.38095238

1

13 56 56.33333333

-0.33333333

3 0.33333333321 57 56 1 1 Maximum:21 57 55.6 1.4 1.4 1.632 54 55.6 -1.6 1.632 54 55.4 -1.4 1.4 Sum:

48 56 55.4 0.6 0.621.3333333

359 56 56.4 -0.4 0.471 57 57.4 -0.4 0.4 RMS:

83 59 58 1 10.58295906

498 59 58.4 0.6 0.6

105 59 58.4 0.6 0.6114 58 58.2 -0.2 0.2126 57 58 -1 1138 58 58 0 0138 58 57.8 0.2 0.2171 59 58.2 0.8 0.8185 57 58.4 -1.4 1.4197 59 58.6 0.4 0.4197 59 58.4 0.6 0.6232 59 58.6 0.4 0.4238 58 58.4 -0.4 0.4238 58 58.4 -0.4 0.4267 58 58.6 -0.6 0.6274 59 58.6 0.4 0.4276 60 58.8 1.2 1.2280 58 59 -1 1284 59 58.8 0.2 0.2284 59 58.4 0.6 0.6287 58 58.4 -0.4 0.4286 58 58.2 -0.2 0.2286 58 57.8 0.2 0.2286 58 57.8 0.2 0.2288 57 57.8 -0.8 0.8

Page 77: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

289 58 57.8 0.2 0.2290 58 57.8 0.2 0.2290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0290 58 58 0 0

Page 78: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

0 50 100 150 200 250 30051

52

53

54

55

56

57

58

59

60

Line 1 Trial 3 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-c

oord

inat

e

0 50 100 150 200 250 3005152535455565758596061

Line 1 Trial 3 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-c

oord

inat

e

Page 79: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 4:

Kalman Filer Data:

Kalman:

Moving Average: Difference:

Difference (abs): Statistics:

X Y Mean:

9 53 53 0 00.21839080

510 54 54 0 014 55 55 0 0 Maximum:16 56 55.6 0.4 0.4 122 57 56 1 131 56 56.2 -0.2 0.2 Sum:

35 56 56.4 -0.4 0.46.33333333

359 56 56.4 -0.4 0.482 57 56.4 0.6 0.6 RMS:

102 57 56.2 0.8 0.80.34596746

5122 56 56 0 0131 55 55.6 -0.6 0.6159 55 55.2 -0.2 0.2182 55 55 0 0196 55 55 0 0211 55 55 0 0222 55 55 0 0230 55 55 0 0238 55 55 0 0248 55 55 0 0257 55 55 0 0266 55 55.2 -0.2 0.2275 55 55.4 -0.4 0.4283 56 55.6 0.4 0.4288 56 55.8 0.2 0.2290 56 56 0 0291 56 56.2 -0.2 0.2

291 56 56.33333333

-0.33333333

3 0.333333333289 57 57 0 0

Page 80: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average: Difference:

Difference (abs): Statistics:

X Y Mean:10 54 54 0 0 0.54252873610 54 54.33333333 -0.333333333 0.33333333315 55 55.2 -0.2 0.2 Maximum:17 56 55.6 0.4 0.4 1.424 57 56 1 135 56 56.4 -0.4 0.4 Sum:35 56 56.8 -0.8 0.8 15.7333333373 57 56.8 0.2 0.290 58 56.6 1.4 1.4 RMS:

106 57 56.4 0.6 0.6 0.651552817125 55 56.2 -1.2 1.2125 55 55.8 -0.8 0.8169 56 55.4 0.6 0.6185 56 55.6 0.4 0.4191 55 55.8 -0.8 0.8209 56 55.6 0.4 0.4218 56 55.6 0.4 0.4226 55 55.8 -0.8 0.8235 56 55.8 0.2 0.2247 56 55.6 0.4 0.4257 56 55.6 0.4 0.4266 55 55.8 -0.8 0.8275 55 55.8 -0.8 0.8282 57 55.8 1.2 1.2286 56 56.2 -0.2 0.2287 56 56.6 -0.6 0.6288 57 56.6 0.4 0.4288 57 57 0 0286 57 57 0 0

Page 81: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

0 50 100 150 200 250 30051

52

53

54

55

56

57

58

Line 1 Trial 4 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-c

oord

inat

e

0 50 100 150 200 250 30052

53

54

55

56

57

58

59

Line 1 Trial 4 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-c

oord

inat

e

Page 82: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 5:

Kalman Filer Data:

Kalman:

Moving Average: Difference:

Difference (abs): Statistics:

X Y Mean:

9 56 56 0 00.43571428

66 56 56 0 07 56 56 0 0 Maximum:9 56 55.8 0.2 0.2 1.8

14 56 55.6 0.4 0.419 55 55.4 -0.4 0.4 Sum:25 55 55.2 -0.2 0.2 12.232 55 55.2 -0.2 0.241 55 55.2 -0.2 0.2 RMS:

48 56 55.2 0.8 0.80.60827625

357 55 55.2 -0.2 0.268 55 55.8 -0.8 0.873 55 56.8 -1.8 1.896 58 58.2 -0.2 0.2

117 61 59.8 1.2 1.2136 62 61.2 0.8 0.8155 63 61.8 1.2 1.2174 62 61.6 0.4 0.4191 61 61.2 -0.2 0.2205 60 60.4 -0.4 0.4220 60 59.6 0.4 0.4233 59 58.8 0.2 0.2243 58 58.2 -0.2 0.2258 57 57.8 -0.8 0.8269 57 57.6 -0.6 0.6278 58 57.6 0.4 0.4287 58 58 0 0295 58 58 0 0

Page 83: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average:

Difference:

Difference (abs): Statistics:

X Y Mean:

8 56 56 0 00.57857142

99 56 56 0 09 56 56 0 0 Maximum:

11 56 55.8 0.2 0.2 217 56 55.8 0.2 0.222 55 55.6 -0.6 0.6 Sum:28 56 55.6 0.4 0.4 16.234 55 55.8 -0.8 0.844 56 55.8 0.2 0.2 RMS:50 57 55.8 1.2 1.2 0.7699721759 55 56 -1 171 56 57 -1 171 56 58 -2 2

107 61 59.6 1.4 1.4123 62 60.8 1.2 1.2140 63 62 1 1157 62 61.8 0.2 0.2176 62 61.4 0.6 0.6191 60 60.8 -0.8 0.8203 60 60.2 -0.2 0.2219 60 59.4 0.6 0.6231 59 58.8 0.2 0.2241 58 58.4 -0.4 0.4259 57 58.2 -1.2 1.2268 58 58.2 -0.2 0.2276 59 58.4 0.6 0.6286 59 59 0 0294 59 59 0 0

Page 84: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

0 50 100 150 200 250 30050

52

54

56

58

60

62

64

Line 1 Trial 5 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-c

oord

inat

e

0 50 100 150 200 250 30050

52

54

56

58

60

62

64

Line 1 Trial 5 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-c

oord

inat

e

Page 85: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Line 2:

Page 86: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 1:

Kalman Filer Data:

Kalman:

Moving Average: Difference:

Difference (abs): Statistics:

X Y Mean:

150 22 150 0 00.41296296

3

147 18 148.6666667

-1.66666666

7 1.666666667149 21 149.2 -0.2 0.2 Maximum:150 26 149.2 0.8 0.8 2150 29 149.8 0.2 0.2150 35 149.8 0.2 0.2 Sum:

150 41 149.6 0.4 0.414.8666666

7149 48 149.2 -0.2 0.2149 55 148.8 0.2 0.2 RMS:

148 61 148.2 -0.2 0.20.60868204

1148 67 147.8 0.2 0.2147 74 147.4 -0.4 0.4147 80 147.2 -0.2 0.2147 90 147 0 0147 98 147 0 0147 105 147.4 -0.4 0.4147 114 147.8 -0.8 0.8149 122 148.2 0.8 0.8149 131 148.6 0.4 0.4149 138 148.8 0.2 0.2149 144 148.6 0.4 0.4148 148 148.2 -0.2 0.2148 152 147.8 0.2 0.2147 153 147.6 -0.6 0.6147 155 147.8 -0.8 0.8148 157 147.8 0.2 0.2149 157 148.2 0.8 0.8148 156 149 -1 1149 156 149.2 -0.2 0.2151 159 149 2 2149 163 149 0 0148 165 148.8 -0.8 0.8148 166 148.2 -0.2 0.2

Page 87: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

148 167 148 0 0148 167 148 0 0148 167 148 0 0

Page 88: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average:

Difference:

Difference (abs): Statistics:

X Y Mean:

150 22 150 0 00.70370370

4

148 20 149.3333333-

1.3333333 1.333333333150 23 149.8 0.2 0.2 Maximum:151 28 149.8 1.2 1.2 2.8150 31 150.4 -0.4 0.4150 38 150.2 -0.2 0.2 Sum:

151 43 150 1 125.3333333

3149 51 149.6 -0.6 0.6150 57 149.2 0.8 0.8 RMS:

148 61 148.6 -0.6 0.60.95361560

2148 68 148.2 -0.2 0.2148 75 147.8 0.2 0.2147 80 147.8 -0.8 0.8148 93 147.8 0.2 0.2148 99 147.8 0.2 0.2148 105 148.6 -0.6 0.6148 115 148.6 -0.6 0.6151 123 148.8 2.2 2.2148 132 149 -1 1149 138 149 0 0149 142 148.4 0.6 0.6148 147 148.2 -0.2 0.2148 150 148 0 0147 151 148.2 -1.2 1.2148 154 148.6 -0.6 0.6150 156 148.6 1.4 1.4150 156 149.2 0.8 0.8148 154 150 -2 2150 156 149.6 0.4 0.4152 161 149.2 2.8 2.8148 165 149.2 -1.2 1.2148 166 149 -1 1148 166 148.4 -0.4 0.4149 166 148.6 0.4 0.4149 167 149 0 0

Page 89: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

149 166 149 0 0

146.5 147 147.5 148 148.5 149 149.5 150 150.5 151 151.50

20

40

60

80

100

120

140

160

180

Line 2 Trial 1 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-coo

rdina

te

Page 90: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

146 147 148 149 150 151 152 1530

20

40

60

80

100

120

140

160

180

Line 2 Trial 1 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-coo

rdina

te

Trial 2:

Kalman Filer Data:

Kalman:Moving Average: Difference:

Difference (abs): Statistics:

X Y Mean:151 13 151 0 0 0.425925926154 14 152.6666667 1.333333333 1.333333333153 14 152.6 0.4 0.4 Maximum:152 15 152.8 -0.8 0.8 1.333333333153 24 152.4 0.6 0.6152 34 152.4 -0.4 0.4 Sum:152 49 152.6 -0.6 0.6 7.666666667153 56 152.6 0.4 0.4153 79 152.8 0.2 0.2 RMS:153 99 153.2 -0.2 0.2 0.544104202153 108 153.2 -0.2 0.2154 128 153 1 1153 144 152.8 0.2 0.2152 153 152.6 -0.6 0.6152 160 152.2 -0.2 0.2152 166 152.2 -0.2 0.2

Page 91: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

152 170 152.3333333 -0.333333333 0.333333333153 174 153 0 0

Page 92: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average: Difference:

Difference (abs): Statistics:

X Y Mean:152 14 152 0 0 0.4

153 14 152.33333330.6666666

7 0.666666667152 15 152.4 -0.4 0.4 Maximum:152 15 152.4 -0.4 0.4 1.4153 29 152.4 0.6 0.6152 39 152.6 -0.6 0.6 Sum:153 55 152.8 0.2 0.2 7.2153 55 153 0 0153 90 153.4 -0.4 0.4 RMS:

154 104 153.8 0.2 0.20.55154105

1154 104 153.8 0.2 0.2155 133 153.6 1.4 1.4153 144 153.4 -0.4 0.4152 149 153.2 -1.2 1.2153 156 152.8 0.2 0.2153 163 153 0 0153 167 153.3333333 -0.3333333 0.333333333154 171 154 0 0

Page 93: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

150.5 151 151.5 152 152.5 153 153.5 154 154.50

20406080

100120140160180200

Line 2 Trial 2 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-coo

rdina

te

151.5 152 152.5 153 153.5 154 154.5 155 155.50

20

40

60

80

100

120

140

160

180

Line 2 Trial 2 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-coo

rdina

te

Page 94: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 3:

Kalman Filter Data:

Kalman:

Moving Average: Difference:

Difference (abs): Statistics:

X Y Mean:

167 14 167 0 00.23291139

2

168 14 167.33333330.66666666

7 0.666666667167 18 167 0 0 Maximum:167 18 167 0 0 1166 20 166.8 -0.8 0.8167 19 166.8 0.2 0.2 Sum:167 20 166.6 0.4 0.4 18.4167 22 166.6 0.4 0.4166 24 166.4 -0.4 0.4 RMS:

166 27 166.2 -0.2 0.20.31892929

2166 29 166 0 0166 29 165.8 0.2 0.2166 30 165.6 0.4 0.4165 32 165.4 -0.4 0.4165 33 165.2 -0.2 0.2165 37 165 0 0165 42 164.8 0.2 0.2165 46 164.6 0.4 0.4164 50 164.4 -0.4 0.4164 51 164.2 -0.2 0.2164 52 164 0 0164 55 164.2 -0.2 0.2164 56 164.4 -0.4 0.4165 58 164.6 0.4 0.4165 60 164.8 0.2 0.2165 61 165 0 0165 63 164.8 0.2 0.2165 67 164.6 0.4 0.4164 69 164.4 -0.4 0.4164 72 164.2 -0.2 0.2164 74 164 0 0164 77 164 0 0164 78 164 0 0164 81 164 0 0

Page 95: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

164 83 164 0 0164 85 164 0 0164 88 164 0 0164 91 164 0 0164 93 164 0 0164 95 164 0 0164 96 164 0 0164 99 164 0 0164 101 164 0 0164 104 164 0 0164 105 163.8 0.2 0.2164 105 163.6 0.4 0.4163 106 163.4 -0.4 0.4163 107 163.2 -0.2 0.2163 109 162.8 0.2 0.2163 110 162.4 0.6 0.6162 114 161.8 0.2 0.2161 118 161 0 0160 120 160 0 0159 123 159 0 0158 126 158.2 -0.2 0.2157 127 157.6 -0.6 0.6157 130 157.4 -0.4 0.4157 134 157.2 -0.2 0.2158 137 157.4 0.6 0.6157 140 157.6 -0.6 0.6158 145 158 0 0158 148 158.2 -0.2 0.2159 151 158.6 0.4 0.4159 152 158.6 0.4 0.4159 155 158.6 0.4 0.4158 157 158.2 -0.2 0.2158 159 157.6 0.4 0.4157 163 157.2 -0.2 0.2156 167 157 -1 1157 169 156.8 0.2 0.2157 171 156.8 0.2 0.2157 174 157 0 0157 176 156.8 0.2 0.2157 180 156.6 0.4 0.4156 183 156.4 -0.4 0.4156 187 156.4 -0.4 0.4156 190 156.4 -0.4 0.4

Page 96: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

157 192 156.66666670.33333333

3 0.333333333157 192 157 0 0

Raw Measurement Data:

Raw:Moving Average: Difference:

Difference (abs): Statistics:

X Y Mean:

167 14 167 0 00.41097046

4169 15 168 1 1168 19 167.6 0.4 0.4 Maximum:167 18 168 -1 1 1.6167 21 167.6 -0.6 0.6169 18 167.4 1.6 1.6 Sum:

167 21 167.4 -0.4 0.432.4666666

7167 23 167.4 -0.4 0.4167 26 166.8 0.2 0.2 RMS:167 28 166.6 0.4 0.4 0.51808367166 29 166.6 -0.6 0.6166 29 166.2 -0.2 0.2167 30 166 1 1165 33 166 -1 1166 34 165.8 0.2 0.2166 39 165.4 0.6 0.6165 44 165.4 -0.4 0.4165 48 165 0 0165 50 164.8 0.2 0.2164 50 164.8 -0.8 0.8165 52 164.8 0.2 0.2165 55 165 0 0165 56 165.2 -0.2 0.2166 58 165.2 0.8 0.8165 60 165.2 -0.2 0.2165 61 165.2 -0.2 0.2165 63 164.8 0.2 0.2165 69 164.8 0.2 0.2164 70 164.6 -0.6 0.6165 73 164.4 0.6 0.6164 74 164.4 -0.4 0.4164 77 164.4 -0.4 0.4

Page 97: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

165 78 164.4 0.6 0.6164 82 164.4 -0.4 0.4165 83 164.6 0.4 0.4164 86 164.6 -0.6 0.6165 89 164.6 0.4 0.4165 91 164.4 0.6 0.6164 94 164.4 -0.4 0.4164 95 164.4 -0.4 0.4164 95 164.4 -0.4 0.4165 100 164.4 0.6 0.6165 102 164.4 0.6 0.6164 105 164.4 -0.4 0.4164 104 164.2 -0.2 0.2164 105 164 0 0164 106 164 0 0164 108 163.8 0.2 0.2164 109 163.4 0.6 0.6163 111 162.8 0.2 0.2162 117 162 0 0161 119 161.2 -0.2 0.2160 121 160.2 -0.2 0.2160 124 159.4 0.6 0.6158 126 158.8 -0.8 0.8158 126 158.4 -0.4 0.4158 132 158.2 -0.2 0.2158 135 158 0 0159 138 158.2 0.8 0.8157 141 158.4 -1.4 1.4159 146 158.8 0.2 0.2159 149 158.8 0.2 0.2160 150 159.2 0.8 0.8159 152 159 0 0159 155 158.8 0.2 0.2158 157 158.2 -0.2 0.2158 160 157.8 0.2 0.2157 165 157.6 -0.6 0.6157 168 157.6 -0.6 0.6158 168 157.6 0.4 0.4158 171 157.6 0.4 0.4158 175 157.6 0.4 0.4157 176 157.4 -0.4 0.4157 182 157.2 -0.2 0.2157 184 157 0 0

Page 98: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

157 187 157.2 -0.2 0.2157 191 157.2 -0.2 0.2

158 196 157.33333330.6666666

7 0.666666667157 196 157 0 0

154 156 158 160 162 164 166 168 1700

20406080

100120140160180200

Line 2 Trial 3 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

Page 99: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

156 158 160 162 164 166 168 1700

20406080

100120140160180200

Line 2 Trial 3 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-coo

rdinat

e

Trial 4:

Kalman Filter Data:

Kalman:

Moving Average: Difference:

Difference (abs): Statistics:

X Y Mean:

154 23 154 0 00.48461538

5

159 25 156.66666672.33333333

3 2.333333333157 24 155.8 1.2 1.2 Maximum:

155 25 155.4 -0.4 0.42.33333333

3154 26 153.8 0.2 0.2152 30 152.6 -0.6 0.6 Sum:151 33 152 -1 1 25.2151 34 151.4 -0.4 0.4152 42 151 1 1 RMS:

151 47 150.8 0.2 0.20.66422630

3150 51 150.6 -0.6 0.6

Page 100: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

150 56 150.4 -0.4 0.4150 60 150.6 -0.6 0.6151 66 150.8 0.2 0.2152 68 151 1 1151 72 151.2 -0.2 0.2151 75 151 0 0151 77 150.8 0.2 0.2150 78 150.8 -0.8 0.8151 82 150.8 0.2 0.2151 87 151 0 0151 89 151.4 -0.4 0.4152 94 151.8 0.2 0.2152 96 152.4 -0.4 0.4153 104 153.2 -0.2 0.2154 111 153.8 0.2 0.2155 116 154.4 0.6 0.6155 119 154.8 0.2 0.2155 124 155 0 0155 128 155 0 0155 132 155 0 0155 134 155.2 -0.2 0.2155 136 155.4 -0.4 0.4156 138 155.4 0.6 0.6156 141 155.4 0.6 0.6155 144 155.4 -0.4 0.4155 147 155.6 -0.6 0.6155 150 155.8 -0.8 0.8157 151 156.2 0.8 0.8157 152 157 0 0157 153 158 -1 1159 156 158.6 0.4 0.4160 161 159.2 0.8 0.8160 164 159.6 0.4 0.4160 164 159.8 0.2 0.2159 163 159.8 -0.8 0.8160 163 160 0 0160 163 160.2 -0.2 0.2161 162 160.6 0.4 0.4161 161 159.8 1.2 1.2

161 162 159.33333331.66666666

7 1.666666667156 163 156 0 0

Page 101: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization
Page 102: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average: Difference:

Difference (abs): Statistics:

X Y Mean:

155 24 155 0 00.68717948

7

158 25 156.33333331.6666666

7 1.666666667156 24 155.4 0.6 0.6 Maximum:

154 26 154.8 -0.8 0.82.66666666

7154 26 153.6 0.4 0.4152 33 152.8 -0.8 0.8 Sum:

152 34 152.6 -0.6 0.635.7333333

3152 34 152 0 0153 46 151.6 1.4 1.4 RMS:

151 48 151.4 -0.4 0.40.87978241

3150 51 151 -1 1151 57 151 0 0150 61 151.4 -1.4 1.4153 67 151.6 1.4 1.4153 67 151.6 1.4 1.4151 72 151.8 -0.8 0.8151 75 151.4 -0.4 0.4151 77 151.2 -0.2 0.2151 77 151.4 -0.4 0.4152 84 151.6 0.4 0.4152 88 152 0 0152 88 152.4 -0.4 0.4153 96 152.6 0.4 0.4153 96 153.4 -0.4 0.4153 107 154 -1 1156 113 154.4 1.6 1.6155 116 155 0 0155 118 155.2 -0.2 0.2156 125 155 1 1154 128 155.2 -1.2 1.2155 132 155.4 -0.4 0.4156 133 155.4 0.6 0.6156 135 155.8 0.2 0.2156 138 155.8 0.2 0.2

Page 103: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

156 141 155.8 0.2 0.2155 145 155.8 -0.8 0.8156 148 156.2 -0.2 0.2156 150 156.4 -0.4 0.4158 151 157 1 1157 151 157.8 -0.8 0.8158 153 158.6 -0.6 0.6160 158 159.2 0.8 0.8160 163 159.6 0.4 0.4161 164 159.8 1.2 1.2159 162 160 -1 1159 162 160.2 -1.2 1.2161 163 160.2 0.8 0.8161 163 160.6 0.4 0.4161 162 161 0 0161 161 159.4 1.6 1.6

161 163 158.33333332.6666666

7 2.666666667153 164 153 0 0

Page 104: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

148 150 152 154 156 158 160 1620

20

40

60

80

100

120

140

160

180

Line 2 Trial 4 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-coo

rdina

te

148 150 152 154 156 158 160 1620

20

40

60

80

100

120

140

160

180

Line 2 Trial 4 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-coo

rdina

te

Page 105: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 5:

Kalman:

Moving Average: Difference:

Difference (abs): Statistics:

X Y Mean:

127 12 127 0 00.36100628

9

127 10 127.3333333

-0.33333333

3 0.333333333128 13 129 -1 1 Maximum:131 18 130 1 1 1132 20 131 1 1132 24 131.8 0.2 0.2 Sum:

132 28 131.8 0.2 0.219.1333333

3132 31 131.2 0.8 0.8131 33 130.6 0.4 0.4 RMS:

129 35 129.8 -0.8 0.80.47326866

4129 37 129.2 -0.2 0.2128 40 128.8 -0.8 0.8129 43 128.8 0.2 0.2129 46 128.8 0.2 0.2129 49 129 0 0129 51 129 0 0129 54 128.8 0.2 0.2129 56 128.8 0.2 0.2128 56 129 -1 1129 58 129.2 -0.2 0.2130 59 129.4 0.6 0.6130 59 129.8 0.2 0.2130 63 130.2 -0.2 0.2130 67 130.4 -0.4 0.4131 70 130.6 0.4 0.4131 72 130.8 0.2 0.2131 75 130.8 0.2 0.2131 76 130.4 0.6 0.6130 78 130.2 -0.2 0.2129 80 130 -1 1130 83 129.6 0.4 0.4130 86 129.6 0.4 0.4129 88 129.8 -0.8 0.8130 91 129.8 0.2 0.2

Page 106: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

130 94 129.8 0.2 0.2130 96 129.8 0.2 0.2130 99 129.6 0.4 0.4129 101 129.6 -0.6 0.6129 104 129.6 -0.6 0.6130 108 129.8 0.2 0.2130 113 130 0 0131 116 130.2 0.8 0.8130 120 130.2 -0.2 0.2130 123 130.2 -0.2 0.2130 127 130 0 0130 128 130 0 0130 131 130.2 -0.2 0.2130 135 130.4 -0.4 0.4131 137 130.6 0.4 0.4131 138 130.8 0.2 0.2131 139 131 0 0131 141 131 0 0131 144 131 0 0

Page 107: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average:

Difference:

Difference (abs): Statistics:

X Y Mean:

127 12 127 0 00.53333333

3128 12 128 0 0129 15 129.6 -0.6 0.6 Maximum:132 20 130.6 1.4 1.4 1.4132 20 131.4 0.6 0.6132 26 132 0 0 Sum:

132 29 131.6 0.4 0.428.2666666

7132 32 131 1 1130 33 130.4 -0.4 0.4 RMS:

129 34 129.8 -0.8 0.80.64726484

7129 38 129.4 -0.4 0.4129 41 129.4 -0.4 0.4130 43 129.6 0.4 0.4130 47 129.8 0.2 0.2130 50 129.8 0.2 0.2130 51 129.6 0.4 0.4129 54 129.4 -0.4 0.4129 56 129.4 -0.4 0.4129 55 129.6 -0.6 0.6130 59 130 0 0131 59 130.2 0.8 0.8131 59 130.6 0.4 0.4130 65 131 -1 1131 69 131 0 0132 71 131.2 0.8 0.8131 72 131.4 -0.4 0.4132 75 131.2 0.8 0.8131 76 130.6 0.4 0.4130 78 130.6 -0.6 0.6129 80 130.4 -1.4 1.4131 85 130 1 1131 86 130.2 0.8 0.8129 89 130.4 -1.4 1.4131 91 130.4 0.6 0.6130 95 130.2 -0.2 0.2131 96 130.2 0.8 0.8

Page 108: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

130 100 130 0 0129 101 130.2 -1.2 1.2130 105 130.2 -0.2 0.2131 110 130.4 0.6 0.6131 115 130.6 0.4 0.4131 115 130.6 0.4 0.4130 122 130.6 -0.6 0.6130 123 130.6 -0.6 0.6131 127 130.4 0.6 0.6131 127 130.6 0.4 0.4130 132 131 -1 1131 136 131 0 0132 137 131.2 0.8 0.8131 137 131.4 -0.4 0.4132 139 131.6 0.4 0.4

131 142 131.6666667-

0.6666667 0.666666667132 145 132 0 0

Page 109: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

126 127 128 129 130 131 132 1330

20

40

60

80

100

120

140

160

Line 2 Trial 5 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-coo

rdina

te

126 127 128 129 130 131 132 1330

20

40

60

80

100

120

140

160

Line 2 Trial 5 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-coo

rdina

te

Page 110: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Circle 1:

Page 111: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 1:

Kalman Filter Data:

Kalman:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:140 47 140 47 0 0 0 0 0.639906103134 46 136 46.33333333 -2 -0.333333333 2 0.333333333134 46 132 45.6 2 0.4 2 0.4 Maximum:128 45 128 45 0 0 0 0 2.4124 44 124.8 44.8 -0.8 -0.8 0.8 0.8120 44 120.4 45.2 -0.4 -1.2 0.4 1.2 Sum:118 45 116.4 46.4 1.6 -1.4 1.6 1.4 90.86666667112 48 112.8 48.2 -0.8 -0.2 0.8 0.2108 51 109.2 50.6 -1.2 0.4 1.2 0.4 RMS:106 53 105.4 53.6 0.6 -0.6 0.6 0.6 0.82060705102 56 102.4 57 -0.4 -1 0.4 1

99 60 100 61 -1 -1 1 1 Mean X:97 65 97.8 65.8 -0.8 -0.8 0.8 0.8 0.56244131596 71 96.2 71 -0.2 0 0.2 095 77 95.2 76.2 -0.2 0.8 0.2 0.8 Mean Y:94 82 94.6 81 -0.6 1 0.6 1 0.71737089294 86 94.4 85.8 -0.4 0.2 0.4 0.294 89 94.6 90.6 -0.6 -1.6 0.6 1.6 Maximum X:95 95 95.6 95.4 -0.6 -0.4 0.6 0.4 296 101 97.2 100.4 -1.2 0.6 1.2 0.699 106 99 105.2 0 0.8 0 0.8 Maximum Y:

102 111 101.6 109.8 0.4 1.2 0.4 1.2 2.4103 113 105 114 -2 -1 2 1108 118 108.6 118 -0.6 0 0.6 0 Sum X:113 122 112.6 121.6 0.4 0.4 0.4 0.4 39.93333333117 126 117.2 125 -0.2 1 0.2 1122 129 121.8 127.8 0.2 1.2 0.2 1.2 Sum Y:126 130 126.4 129.8 -0.4 0.2 0.4 0.2 50.93333333131 132 131 131 0 1 0 1136 132 135.4 131.2 0.6 0.8 0.6 0.8 RMS X:140 132 140 130.8 0 1.2 0 1.2 0.762049785144 130 144.6 129.4 -0.6 0.6 0.6 0.6149 128 149.2 127.6 -0.2 0.4 0.2 0.4 RMS Y:154 125 153.6 125 0.4 0 0.4 0 0.875255384159 123 157.8 121.6 1.2 1.4 1.2 1.4162 119 161.6 117.8 0.4 1.2 0.4 1.2165 113 164.8 113.6 0.2 -0.6 0.2 0.6168 109 167.4 108.6 0.6 0.4 0.6 0.4170 104 169.8 103.4 0.2 0.6 0.2 0.6172 98 171.8 98.2 0.2 -0.2 0.2 0.2174 93 173.2 92.2 0.8 0.8 0.8 0.8175 87 174.2 86 0.8 1 0.8 1175 79 174.6 80.2 0.4 -1.2 0.4 1.2175 73 174.4 74 0.6 -1 0.6 1174 69 173.6 68.2 0.4 0.8 0.4 0.8173 62 172 63.4 1 -1.4 1 1.4171 58 169.8 59.4 1.2 -1.4 1.2 1.4167 55 167 55.6 0 -0.6 0 0.6164 53 163.4 53 0.6 0 0.6 0160 50 159.2 51 0.8 -1 0.8 1155 49 154.8 49.4 0.2 -0.4 0.2 0.4150 48 150 48.2 0 -0.2 0 0.2145 47 145 47.4 0 -0.4 0 0.4140 47 140 47 0 0 0 0135 46 135 47 0 -1 0 1130 47 129.8 47.4 0.2 -0.4 0.2 0.4125 48 124.4 48.2 0.6 -0.2 0.6 0.2119 49 118.8 50 0.2 -1 0.2 1

Page 112: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

113 51 113.4 52.2 -0.4 -1.2 0.4 1.2107 55 108.6 54.4 -1.6 0.6 1.6 0.6103 58 104.4 57.4 -1.4 0.6 1.4 0.6101 59 100.8 61.4 0.2 -2.4 0.2 2.4

98 64 97.8 66 0.2 -2 0.2 295 71 95.6 71.4 -0.6 -0.4 0.6 0.492 78 93.8 77.6 -1.8 0.4 1.8 0.492 85 92.4 83.8 -0.4 1.2 0.4 1.292 90 91.4 89.2 0.6 0.8 0.6 0.891 95 90.8 93.8 0.2 1.2 0.2 1.290 98 90.2 97.6 -0.2 0.4 0.2 0.489 101 89.33333333 101 -0.333333333 0 0.333333333 089 104 89 104 0 0 0 0

Page 113: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:140 47 140 47 0 0 0 0 0.983568075

135 47 136.6666667 47-

1.666666667 0 1.666666667 0135 47 132.4 46 2.6 1 2.6 1 Maximum:128 45 128.4 45.8 -0.4 -0.8 0.4 0.8 4.4124 44 125.4 45.6 -1.4 -1.6 1.4 1.6120 46 120.6 46.4 -0.6 -0.4 0.6 0.4 Sum:120 46 116.6 48 3.4 -2 3.4 2 139.6666667111 51 113.4 49.8 -2.4 1.2 2.4 1.2108 53 109.8 52.2 -1.8 0.8 1.8 0.8 RMS:108 53 105.8 55.4 2.2 -2.4 2.2 2.4 1.281760819102 58 103.2 58.4 -1.2 -0.4 1.2 0.4100 62 101.2 62.4 -1.2 -0.4 1.2 0.4 Mean X:

98 66 99 67.6 -1 -1.6 1 1.6 0.95492957798 73 97.6 72.4 0.4 0.6 0.4 0.697 79 96.8 77.2 0.2 1.8 0.2 1.8 Mean Y:95 82 96.2 81.8 -1.2 0.2 1.2 0.2 1.01220657396 86 96 86.6 0 -0.6 0 0.695 89 96 91.2 -1 -2.2 1 2.2 Maximum X:97 97 97.4 96.2 -0.4 0.8 0.4 0.8 497 102 98.8 101.2 -1.8 0.8 1.8 0.8

102 107 100.4 105.6 1.6 1.4 1.6 1.4 Maximum Y:103 111 103.2 110 -0.2 1 0.2 1 4.4103 111 107 114.2 -4 -3.2 4 3.2111 119 110 118 1 1 1 1 Sum X:116 123 114.2 121.4 1.8 1.6 1.8 1.6 67.8117 126 119 125 -2 1 2 1124 128 123.2 127.4 0.8 0.6 0.8 0.6 Sum Y:127 129 127.4 129 -0.4 0 0.4 0 71.86666667132 131 132 130 0 1 0 1137 131 136.2 130.2 0.8 0.8 0.8 0.8 RMS X:140 131 140.8 129.6 -0.8 1.4 0.8 1.4 1.274500117145 129 145.4 128.4 -0.4 0.6 0.4 0.6150 126 149.8 126.6 0.2 -0.6 0.2 0.6 RMS Y:155 125 154.2 124 0.8 1 0.8 1 1.288980624159 122 158.2 120.4 0.8 1.6 0.8 1.6162 118 161.6 117 0.4 1 0.4 1165 111 164.4 112.6 0.6 -1.6 0.6 1.6167 109 167.2 107.6 -0.2 1.4 0.2 1.4169 103 169.4 102.8 -0.4 0.2 0.4 0.2173 97 171.2 98 1.8 -1 1.8 1173 94 172.6 91.8 0.4 2.2 0.4 2.2174 87 173.6 86 0.4 1 0.4 1174 78 173.8 80.6 0.2 -2.6 0.2 2.6174 74 173.6 74.2 0.4 -0.2 0.4 0.2174 70 172.8 68.6 1.2 1.4 1.2 1.4172 62 171.2 64.4 0.8 -2.4 0.8 2.4170 59 169 60.6 1 -1.6 1 1.6166 57 166 57 0 0 0 0163 55 162.4 54.8 0.6 0.2 0.6 0.2

Page 114: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

159 52 158.2 52.8 0.8 -0.8 0.8 0.8154 51 154.2 51.2 -0.2 -0.2 0.2 0.2149 49 149.6 50 -0.6 -1 0.6 1146 49 144.8 49 1.2 0 1.2 0140 49 140 48.4 0 0.6 0 0.6135 47 135.4 48.4 -0.4 -1.4 0.4 1.4130 48 130 48.8 0 -0.8 0 0.8126 49 124.6 49.6 1.4 -0.6 1.4 0.6119 51 119 51.6 0 -0.6 0 0.6113 53 113.8 53.8 -0.8 -0.8 0.8 0.8107 57 109.4 55.8 -2.4 1.2 2.4 1.2104 59 105.6 59 -1.6 0 1.6 0104 59 102 63.4 2 -4.4 2 4.4100 67 99.4 68 0.6 -1 0.6 1

95 75 97.6 73.4 -2.6 1.6 2.6 1.694 80 95.4 79.6 -1.4 0.4 1.4 0.495 86 93.8 85.2 1.2 0.8 1.2 0.893 90 92.8 89.6 0.2 0.4 0.2 0.492 95 92 93.8 0 1.2 0 1.290 97 91.2 97.2 -1.2 -0.2 1.2 0.2

90 101 90.33333333 100.3333333-

0.333333333 0.666666667 0.333333333 0.66666666791 103 91 103 0 0 0 0

Page 115: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

80 90 100 110 120 130 140 150 160 170 180405060708090

100110120130140

Circle 1 Trial 1 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

80 90 100 110 120 130 140 150 160 170 180405060708090

100110120130140

Circle 1 Trial 1 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 116: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 2:

Kalman Filter Data:

Kalman:

Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:

10812

9 108 129 0 0 0 00.5371069

18

11213

0110.66666

67129.66666

671.3333333

330.3333333

331.3333333

330.3333333

33

11213

0 112.4 132 -0.4 -2 0.4 2 Maximum:

11413

4 114.2 133.8 -0.2 0.2 0.2 0.2 2.2

11613

7 115.4 135.6 0.6 1.4 0.6 1.4

11713

8 116.8 137.2 0.2 0.8 0.2 0.8 Sum:

11813

9 118.2 138.2 -0.2 0.8 0.2 0.8113.86666

67

11913

8 119.8 138.6 -0.8 -0.6 0.8 0.6

12113

9 121.8 138.8 -0.8 0.2 0.8 0.2 RMS:

12413

9 124.2 138.8 -0.2 0.2 0.2 0.20.6808860

92

12713

9 126.8 139.2 0.2 -0.2 0.2 0.2

13013

9 129.6 139.6 0.4 -0.6 0.4 0.6 Mean X:

13214

0 132.2 140 -0.2 0 0.2 00.5188679

25

13514

1 134.6 140.2 0.4 0.8 0.4 0.8

13714

1 136.8 140.2 0.2 0.8 0.2 0.8 Mean Y:

13914

0 139.4 140 -0.4 0 0.4 00.5553459

12

14113

9 141.8 139.4 -0.8 -0.4 0.8 0.4

14513

9 144.2 138.6 0.8 0.4 0.8 0.4Maximum X:

14713

8 146.6 137.6 0.4 0.4 0.4 0.4 2

14913

7 149.2 136.4 -0.2 0.6 0.2 0.6

15113

5 151.2 134.8 -0.2 0.2 0.2 0.2Maximum Y:

15413

3 153.2 133 0.8 0 0.8 0 2.2

15513

1 155.4 130.8 -0.4 0.2 0.4 0.2

15712

9 157.6 128.6 -0.6 0.4 0.6 0.4 Sum X:

16012

6 159.4 126.2 0.6 -0.2 0.6 0.2 55

16212

4 161.4 123.6 0.6 0.4 0.6 0.4

16312

1 163 120.8 0 0.2 0 0.2 Sum Y:

16511

8 164.4 117.8 0.6 0.2 0.6 0.258.866666

67

16511

5 165.8 114.6 -0.8 0.4 0.8 0.4

Page 117: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

16711

1 167.2 111.4 -0.2 -0.4 0.2 0.4 RMS X:

16910

8 168.2 107.8 0.8 0.2 0.8 0.20.6260253

64

17010

5 169.2 104.2 0.8 0.8 0.8 0.8

17010

0 170.2 100.6 -0.2 -0.6 0.2 0.6 RMS Y:

170 97 170.8 96.8 -0.8 0.2 0.8 0.20.7316447

11172 93 171.2 92.8 0.8 0.2 0.8 0.2172 89 171.4 88.8 0.6 0.2 0.6 0.2172 85 171.2 84.4 0.8 0.6 0.8 0.6171 80 170.2 80.2 0.8 -0.2 0.8 0.2169 75 168.8 76.2 0.2 -1.2 0.2 1.2167 72 167.2 72.8 -0.2 -0.8 0.2 0.8165 69 165.6 70.2 -0.6 -1.2 0.6 1.2164 68 164.2 68 -0.2 0 0.2 0163 67 162.8 65.8 0.2 1.2 0.2 1.2162 64 161.6 64 0.4 0 0.4 0160 61 160.2 61.6 -0.2 -0.6 0.2 0.6159 60 158.4 58.6 0.6 1.4 0.6 1.4157 56 156 55.2 1 0.8 1 0.8154 52 153.6 52.2 0.4 -0.2 0.4 0.2150 47 151 49.2 -1 -2.2 1 2.2148 46 148.4 46.8 -0.4 -0.8 0.4 0.8146 45 145.6 45 0.4 0 0.4 0144 44 142.8 44 1.2 0 1.2 0140 43 139.8 43.2 0.2 -0.2 0.2 0.2136 42 136.6 42.4 -0.6 -0.4 0.6 0.4133 42 133.4 41.8 -0.4 0.2 0.4 0.2130 41 130.2 41.2 -0.2 -0.2 0.2 0.2128 41 127.2 41 0.8 0 0.8 0124 40 124.6 41 -0.6 -1 0.6 1121 41 121.6 41.2 -0.6 -0.2 0.6 0.2120 42 118 41.8 2 0.2 2 0.2115 42 114.2 42.8 0.8 -0.8 0.8 0.8110 44 110.2 44.4 -0.2 -0.4 0.2 0.4105 45 105.6 46.4 -0.6 -1.4 0.6 1.4101 49 101.6 49.2 -0.6 -0.2 0.6 0.2

97 52 98 52.4 -1 -0.4 1 0.495 56 95.2 56 -0.2 0 0.2 092 60 92.8 59.4 -0.8 0.6 0.8 0.691 63 91.2 62.6 -0.2 0.4 0.2 0.489 66 89.6 65.4 -0.6 0.6 0.6 0.689 68 88.8 68.4 0.2 -0.4 0.2 0.487 70 88.4 72 -1.4 -2 1.4 288 75 88.4 75.6 -0.4 -0.6 0.4 0.689 81 88.4 79.2 0.6 1.8 0.6 1.889 84 88.8 82.4 0.2 1.6 0.2 1.689 86 89.2 85.2 -0.2 0.8 0.2 0.889 86 89.4 87.4 -0.4 -1.4 0.4 1.490 89 89.8 90 0.2 -1 0.2 190 92 90.6 93.2 -0.6 -1.2 0.6 1.291 97 91.6 97.2 -0.6 -0.2 0.6 0.2

9310

2 92.6 101.2 0.4 0.8 0.4 0.8

9410

6 94 105 0 1 0 1

9510

9 95.4 108 -0.4 1 0.4 1

9711

1 96.4 110.6 0.6 0.4 0.6 0.4

9811

2 97.4 113.2 0.6 -1.2 0.6 1.298 11 98.4 115.8 -0.4 -0.8 0.4 0.8

Page 118: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

5

9911

9 99 118.4 0 0.6 0 0.6

10012

2 99.8 121.2 0.2 0.8 0.2 0.8

10012

4 101 123.6 -1 0.4 1 0.4

10212

6 102.2 125.4 -0.2 0.6 0.2 0.6

10412

7 103.8 126.8 0.2 0.2 0.2 0.2

10512

8 106 128 -1 0 1 0

10812

9 108.2 129.4 -0.2 -0.4 0.2 0.4

11113

0 110.6 131 0.4 -1 0.4 1

11313

3 113.8 132.6 -0.8 0.4 0.8 0.4

11613

5 117.2 134.2 -1.2 0.8 1.2 0.8

12113

6 120.4 135.6 0.6 0.4 0.6 0.4

12513

7 123.8 136.4 1.2 0.6 1.2 0.6

12713

7 127.6 136.8 -0.6 0.2 0.6 0.2

13013

7 130.8 136.8 -0.8 0.2 0.8 0.2

13513

7 133.8 136.4 1.2 0.6 1.2 0.6

13713

6 136.8 136 0.2 0 0.2 0

14013

5 139.8 135.4 0.2 -0.4 0.2 0.4

14213

5 142.4 134.6 -0.4 0.4 0.4 0.4

14513

4 144.8 133.6 0.2 0.4 0.2 0.4

14813

3147.33333

33132.66666

670.6666666

670.3333333

330.6666666

670.3333333

33

14913

1 149 131 0 0 0 0

Page 119: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:

Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs):

Statistics:

X YMean:

109

130 109 130 0 0 0 0

0.7452830

19

112

130 111

130.33333

33 1

-0.3333333

33 1

0.3333333

33112

131 112.8 132.8 -0.8 -1.8 0.8 1.8

Maximum:

115

135 114.4 134.4 0.6 0.6 0.6 0.6 3.8

116

138 115.6 136.2 0.4 1.8 0.4 1.8

117

138 117 137.4 0 0.6 0 0.6 Sum:

118

139 118.4 138.2 -0.4 0.8 0.4 0.8 158

119

137 120.2 138.4 -1.2 -1.4 1.2 1.4

122

139 122.4 138.6 -0.4 0.4 0.4 0.4 RMS:

125

139 125 138.8 0 0.2 0 0.2

0.9885508

1128

139 127.6 139.6 0.4 -0.6 0.4 0.6

131

140 130.4 140.2 0.6 -0.2 0.6 0.2

Mean X:

132

141 132.8 140.6 -0.8 0.4 0.8 0.4

0.6937106

92136

142 135 140.8 1 1.2 1 1.2

137

141 137.2 140.6 -0.2 0.4 0.2 0.4

Mean Y:

139

140 140 140.4 -1 -0.4 1 0.4

0.7968553

46142

139 142.2 139.6 -0.2 -0.6 0.2 0.6

146

140 144.6 138.8 1.4 1.2 1.4 1.2

Maximum X:

147

138 147 137.6 0 0.4 0 0.4 3.8

Page 120: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

149

137 149.6 136.4 -0.6 0.6 0.6 0.6

151

134 151.4 134.6 -0.4 -0.6 0.4 0.6

Maximum Y:

155

133 153.6 132.8 1.4 0.2 1.4 0.2 3

155

131 156 130.6 -1 0.4 1 0.4

158

129 158.2 128.6 -0.2 0.4 0.2 0.4

Sum X:

161

126 159.8 126.2 1.2 -0.2 1.2 0.2

73.533333

33162

124 161.8 123.8 0.2 0.2 0.2 0.2

163

121 163.2 120.8 -0.2 0.2 0.2 0.2

Sum Y:

165

119 164.6 117.8 0.4 1.2 0.4 1.2

84.466666

67165

114 166 114.8 -1 -0.8 1 0.8

168

111 167.4 111.6 0.6 -0.6 0.6 0.6

RMS X:

169

109 168.4 107.6 0.6 1.4 0.6 1.4

0.9301608

26170

105 169.4 104.4 0.6 0.6 0.6 0.6

170

99 170.4 100.8 -0.4 -1.8 0.4 1.8

RMS Y:

170

98 171 97 -1 1 1 1

1.0436791

88173

93 171.4 93 1.6 0 1.6 0

172

90 171.4 89.2 0.6 0.8 0.6 0.8

172

85 171 84.6 1 0.4 1 0.4

170

80 169.8 80.8 0.2 -0.8 0.2 0.8

168

75 168.4 76.8 -0.4 -1.8 0.4 1.8

167

74 166.8 73.8 0.2 0.2 0.2 0.2

165

70 165.6 71.6 -0.6 -1.6 0.6 1.6

Page 121: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

164

70 164.6 69.4 -0.6 0.6 0.6 0.6

164

69 163.2 66.8 0.8 2.2 0.8 2.2

163

64 162.2 65 0.8 -1 0.8 1

160

61 160.6 62 -0.6 -1 0.6 1

160

61 158.6 58.4 1.4 2.6 1.4 2.6

156

55 155.8 55 0.2 0 0.2 0

154

51 153.6 52.6 0.4 -1.6 0.4 1.6

149

47 151 49.6 -2 -2.6 2 2.6

149

49 148.6 47.6 0.4 1.4 0.4 1.4

147

46 145.8 46.4 1.2 -0.4 1.2 0.4

144

45 143.2 45.6 0.8 -0.6 0.8 0.6

140

45 140 44.4 0 0.6 0 0.6

136

43 136.6 43.4 -0.6 -0.4 0.6 0.4

133

43 133.6 42.8 -0.6 0.2 0.6 0.2

130

41 130.4 42 -0.4 -1 0.4 1

129

42 127.6 42 1.4 0 1.4 0

124

41 125.4 42 -1.4 -1 1.4 1

122

43 122 42.4 0 0.6 0 0.6

122

43 118.2 43 3.8 0 3.8 0

113

43 114.4 44 -1.4 -1 1.4 1

110

45 110.4 45.6 -0.4 -0.6 0.4 0.6

105

46 105.6 47.8 -0.6 -1.8 0.6 1.8

Page 122: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

102

51 102.4 50.6 -0.4 0.4 0.4 0.4

98

54 98.8 53.8 -0.8 0.2 0.8 0.2

97

57 96.4 57.2 0.6 -0.2 0.6 0.2

92

61 94.2 60.2 -2.2 0.8 2.2 0.8

93

63 92.6 63 0.4 0 0.4 0

91

66 90.8 65.6 0.2 0.4 0.2 0.4

90

68 90.4 69 -0.4 -1 0.4 1

88

70 89.8 73 -1.8 -3 1.8 3

90

78 89.6 76.6 0.4 1.4 0.4 1.4

90

83 89.6 80 0.4 3 0.4 3

90

84 90 83 0 1 0 1

90

85 90 85.2 0 -0.2 0 0.2

90

85 90 87.2 0 -2.2 0 2.2

90

89 90.4 90.4 -0.4 -1.4 0.4 1.4

90

93 91.2 94 -1.2 -1 1.2 1

92

100 92.2 98.4 -0.2 1.6 0.2 1.6

94

103 93.4 102.2 0.6 0.8 0.6 0.8

95

107 94.8 105.8 0.2 1.2 0.2 1.2

96

108 96 108 0 0 0 0

97

111 96.8 110.6 0.2 0.4 0.2 0.4

98

111 97.8 113.2 0.2 -2.2 0.2 2.2

98

116 98.6 116.2 -0.6 -0.2 0.6 0.2

100

120 99.2 118.6 0.8 1.4 0.8 1.4

100

123 100.2 121.6 -0.2 1.4 0.2 1.4

100

123 101.6 123.8 -1.6 -0.8 1.6 0.8

103

126 102.8 125.2 0.2 0.8 0.2 0.8

10

12

104.8 126.4 0.2 0.6 0.2 0.6

Page 123: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

5 7106

127 107.2 128 -1.2 -1 1.2 1

110

129 109.4 129.6 0.6 -0.6 0.6 0.6

112

131 111.8 131.2 0.2 -0.2 0.2 0.2

114

134 115.2 133.2 -1.2 0.8 1.2 0.8

117

135 118.4 134.8 -1.4 0.2 1.4 0.2

123

137 121.2 136 1.8 1 1.8 1

126

137 124.6 136.6 1.4 0.4 1.4 0.4

126

137 128.4 137 -2.4 0 2.4 0

131

137 131.2 136.8 -0.2 0.2 0.2 0.2

136

137 134 136.4 2 0.6 2 0.6

137

136 137.2 136 -0.2 0 0.2 0

140

135 140.2 135.4 -0.2 -0.4 0.2 0.4

142

135 142.6 134.6 -0.6 0.4 0.6 0.4

146

134 145 133.6 1 0.4 1 0.4

148

133

147.66666

67

132.66666

67

0.3333333

33

0.3333333

33

0.3333333

33

0.3333333

33149

131 149 131 0 0 0 0

Page 124: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

80 90 100 110 120 130 140 150 160 170 18040

60

80

100

120

140

160

Circle 1 Trial 2 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

80 90 100 110 120 130 140 150 160 170 18040

60

80

100

120

140

160

Circle 1 Trial 2 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 125: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 3:

Kalman:

Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs):

Statistics:

X Y

Mean:

148 1

2

148

123 0 0 0 0

0.86512820

5

154 1

2

153

122.

666666

7 1

0.33333333

3 1

0.33333333

3

157 1

2

156.2

122

0.8 0

0.8 0

Maximum:

160 1

2

159.6

121.4

0.4

-0.4

0.4

0.4

3.6

162 1

2

162.6

120.4

-0.6

0.6

0.6

0.6

165 1

2

165.6

118.4

-0.6

1.6

0.6

1.6

Sum:

169 1

1

168.2

115

0.8 3

0.8 3

112.

466666

7172 1

1

170.8

109.8

1.2

2.2

1.2

2.2

173 1

0

172.6

102.6

0.4

1.4

0.4

1.4

RMS:

175 9

5

173.4

94.4

1.6

0.6

1.6

0.6

1.06157438

3174 8

4

173.2

86.2

0.8

-2.2

0.8

2.2

173 7

7

172.2

78.2

0.8

-1.2

0.8

1.2

Mean X:

Page 126: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

171 7

1

170 71 1 0 1 0

0.78051282

1168 6

4

167

64.8 1

-0.8 1

0.8

164 5

9

163.2

59

0.8

0

0.8

0 Mean Y:

159 5

3

158.8

53.8

0.2

-0.8

0.2

0.8

0.94974359

154 4

8

154.6 50

-0.6 -2

0.6 2

149

45

149.4

46.6

-0.4

-1.6

0.4

1.6

Maximum X:

147 4

5

143.4

44.4

3.6

0.6

3.6

0.6

3.6

138 4

2

137

43.6 1

-1.6 1

1.6

129

42

130 44 -1 -2 1 2

Maximum Y:

122 4

4

122.2 45

-0.2 -1

0.2 1 3

114 4

7

115

47.4 -1

-0.4 1

0.4

108 5

0

109 51 -1 -1 1 1

Sum X:

102

54

104 55 -2 -1 2 1

50.7

333333

3

99 6

0

100.6

58.8

-1.6

1.2

1.6

1.2

97 6

498 63 -1 1 1 1

Sum Y:

97 6

6

96.4

67.4

0.6

-1.4

0.6

1.4

61.7

333333

3

Page 127: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

95 7

1

95.4

71.6

-0.4

-0.6

0.4

0.6

94 7

6

94.6 76

-0.6 0

0.6 0

RMS X:

94 8

1

93.8

81.4

0.2

-0.4

0.2

0.4

0.94617538

7

93 8

6

93.4

87.2

-0.4

-1.2

0.4

1.2

93 9

3

93.4

93.2

-0.4

-0.2

0.4

0.2

RMS Y:

93 1

094

99.2 -1

0.8 1

0.8

1.16560391

2

94 1

0

95.4

105.4

-1.4

0.6

1.4

0.6

97 1

1

97.6

111.2

-0.6

-0.2

0.6

0.2

100 1

1

100.4

116.4

-0.4

0.6

0.4

0.6

104 1

2

103.4

120.8

0.6

1.2

0.6

1.2

107 1

2

106.6

124.6

0.4

1.4

0.4

1.4

109 1

2

110.2

127.6

-1.2

0.4

1.2

0.4

113 1

3

114

130 -1 0 1 0

118 1

3

118.4

131.8

-0.4

0.2

0.4

0.2

123 1

3

123.2

133.2

-0.2

0.8

0.2

0.8

129 1

3

128.4

134

0.6 1

0.6 1

133 1

3

133.8

134

-0.8 1

0.8 1

139 1

3

139.2

133.2

-0.2

0.8

0.2

0.8

145 1

3

144.6

131.2

0.4

0.8

0.4

0.8

1 13

15 12 - 2 0. 2

Page 128: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

50

0.2 8

0.2 2

156 1

2

155.2

124.2

0.8

0.8

0.8

0.8

161 1

1

159.6

119.4

1.4

-0.4

1.4

0.4

164 1

1

163.4

113.8

0.6

1.2

0.6

1.2

167 1

0

166.2

108.4

0.8

-0.4

0.8

0.4

169 1

0

168.2

102.8

0.8

-0.8

0.8

0.8

170 9

8

169.6

96.2

0.4

1.8

0.4

1.8

171 9

1

170.2

89.4

0.8

1.6

0.8

1.6

171 8

2

170

82.6 1

-0.6 1

0.6

170 7

4

168.8

74.6

1.2

-0.6

1.2

0.6

168 6

8

166.6

66.8

1.4

1.2

1.4

1.2

164 5

8

163.2

59.6

0.8

-1.6

0.8

1.6

160 5

2

159

53.2 1

-1.2 1

1.2

154 4

6

154.4

47.8

-0.4

-1.8

0.4

1.8

149 4

2

150

44.4 -1

-2.4 1

2.4

145 4

1

145.6

42.2

-0.6

-1.2

0.6

1.2

142 4

1

141.

666666

7 41

0.33333333

3 0

0.33333333

3 0138 4

1

138 41 0 0 0 0

Page 129: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y: Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:149 124 149 124 0 0 0 0 1.061538462154 123 153.3333333 123 0.666666667 0 0.666666667 0157 122 156.4 122.4 0.6 -0.4 0.6 0.4 Maximum:160 122 159.8 121.8 0.2 0.2 0.2 0.2 6.4162 121 162.8 120.6 -0.8 0.4 0.8 0.4166 121 165.8 118 0.2 3 0.2 3 Sum:169 117 168.4 113.6 0.6 3.4 0.6 3.4 138172 109 171 108 1 1 1 1173 100 172.2 99.8 0.8 0.2 0.8 0.2 RMS:175 93 172.8 92 2.2 1 2.2 1 1.488095209172 80 172.4 84.8 -0.4 -4.8 0.4 4.8172 78 171.2 77.8 0.8 0.2 0.8 0.2 Mean X:170 73 168.8 71.2 1.2 1.8 1.2 1.8 0.916923077167 65 166 66 1 -1 1 1163 60 162.2 60.4 0.8 -0.4 0.8 0.4 Mean Y:158 54 158 55.4 0 -1.4 0 1.4 1.206153846153 50 154.4 52 -1.4 -2 1.4 2149 48 148.6 48.6 0.4 -0.6 0.4 0.6 Maximum X:149 48 142.6 46.6 6.4 1.4 6.4 1.4 6.4134 43 136.2 46.2 -2.2 -3.2 2.2 3.2128 44 129.2 46.6 -1.2 -2.6 1.2 2.6 Maximum Y:121 48 121.4 47.6 -0.4 0.4 0.4 0.4 4.8114 50 115 50 -1 0 1 0110 53 109.8 53.8 0.2 -0.8 0.2 0.8 Sum X:102 55 105.6 57.2 -3.6 -2.2 3.6 2.2 59.6102 63 102.8 60.2 -0.8 2.8 0.8 2.8100 65 100.2 64.2 -0.2 0.8 0.2 0.8 Sum Y:100 65 98.8 68.6 1.2 -3.6 1.2 3.6 78.4

97 73 97.6 72.6 -0.6 0.4 0.6 0.495 77 96.2 77 -1.2 0 1.2 0 RMS X:96 83 95 83 1 0 1 0 1.31333723893 87 94.4 88.6 -1.4 -1.6 1.4 1.694 95 94.6 94.4 -0.6 0.6 0.6 0.6 RMS Y:94 101 95.4 100 -1.4 1 1.4 1 1.64438438396 106 97.2 106 -1.2 0 1.2 0

100 111 99.6 111.4 0.4 -0.4 0.4 0.4102 117 102.2 116.2 -0.2 0.8 0.2 0.8106 122 104.8 120.2 1.2 1.8 1.2 1.8107 125 107.8 123.8 -0.8 1.2 0.8 1.2109 126 111.2 126.8 -2.2 -0.8 2.2 0.8115 129 115 129 0 0 0 0119 132 119.6 130.8 -0.6 1.2 0.6 1.2125 133 124.6 132.4 0.4 0.6 0.4 0.6130 134 129.6 133.2 0.4 0.8 0.4 0.8134 134 135 133 -1 1 1 1140 133 140 132.2 0 0.8 0 0.8146 131 145.4 129.8 0.6 1.2 0.6 1.2150 129 150.8 126.6 -0.8 2.4 0.8 2.4157 122 155.4 122.8 1.6 -0.8 1.6 0.8161 118 159.4 118 1.6 0 1.6 0163 114 163 112.4 0 1.6 0 1.6166 107 165.4 107.8 0.6 -0.8 0.6 0.8168 101 167.2 102.2 0.8 -1.2 0.8 1.2169 99 168.6 95.4 0.4 3.6 0.4 3.6170 90 169.2 88.8 0.8 1.2 0.8 1.2170 80 169 82.4 1 -2.4 1 2.4169 74 167.6 74 1.4 0 1.4 0167 69 165.2 66.8 1.8 2.2 1.8 2.2162 57 161.6 60.2 0.4 -3.2 0.4 3.2

Page 130: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

158 54 157.6 54.4 0.4 -0.4 0.4 0.4152 47 153.4 49.6 -1.4 -2.6 1.4 2.6149 45 149.6 47 -0.6 -2 0.6 2146 45 145.8 44.8 0.2 0.2 0.2 0.2143 44 142.6666667 44 0.333333333 0 0.333333333 0139 43 139 43 0 0 0 0

Page 131: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

80 90 100 110 120 130 140 150 160 170 180405060708090

100110120130140

Circle 1 Trial 3 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

80 90 100 110 120 130 140 150 160 170 180405060708090

100110120130140

Circle 1 Trial 3 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 132: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 4:

Kalman Filter Data:

Kalman:Moving Average X:

Moving Average Y:

Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:100 62 100 62 0 0 0 0 0.724774775

99 63 99 65.33333333 0 -2.333333333 0 2.33333333398 71 98 69.8 0 1.2 0 1.2 Maximum:97 76 97 73.6 0 2.4 0 2.4 2.496 77 96 77.8 0 -0.8 0 0.895 81 95.2 80.8 -0.2 0.2 0.2 0.2 Sum:94 84 94.6 82.8 -0.6 1.2 0.6 1.2 107.266666794 86 94 85.6 0 0.4 0 0.494 86 93.6 88.2 0.4 -2.2 0.4 2.2 RMS:93 91 93.8 91.2 -0.8 -0.2 0.8 0.2 0.92413420293 94 94.2 94.4 -1.2 -0.4 1.2 0.495 99 95 98.4 0 0.6 0 0.6 Mean X:96 102 96.4 102.4 -0.4 -0.4 0.4 0.4 0.60810810898 106 98.4 106.8 -0.4 -0.8 0.4 0.8

100 111 100.2 111 -0.2 0 0.2 0 Mean Y:103 116 102.2 115.4 0.8 0.6 0.8 0.6 0.841441441104 120 104.2 119.4 -0.2 0.6 0.2 0.6106 124 106.8 123 -0.8 1 0.8 1 Maximum X:108 126 109.6 126.4 -1.6 -0.4 1.6 0.4 1.8113 129 113.2 129.6 -0.2 -0.6 0.2 0.6117 133 117.4 132.4 -0.4 0.6 0.4 0.6 Maximum Y:122 136 122 135 0 1 0 1 2.4127 138 126.2 137 0.8 1 0.8 1131 139 130.6 138.2 0.4 0.8 0.4 0.8 Sum X:134 139 135 138.8 -1 0.2 1 0.2 45139 139 139.8 138.6 -0.8 0.4 0.8 0.4144 139 145.2 137.8 -1.2 1.2 1.2 1.2 Sum Y:151 137 151 136.4 0 0.6 0 0.6 62.26666667158 135 156.6 134.4 1.4 0.6 1.4 0.6163 132 161.8 131.8 1.2 0.2 1.2 0.2 RMS X:167 129 166 128.6 1 0.4 1 0.4 0.779119392170 126 169.2 125 0.8 1 0.8 1172 121 171.6 120.8 0.4 0.2 0.4 0.2 RMS Y:174 117 173.6 116 0.4 1 0.4 1 1.049295488175 111 175.2 110.6 -0.2 0.4 0.2 0.4177 105 176.4 104.8 0.6 0.2 0.6 0.2178 99 177.2 98.8 0.8 0.2 0.8 0.2178 92 177.4 92.2 0.6 -0.2 0.6 0.2178 87 176.6 85.4 1.4 1.6 1.4 1.6176 78 175.2 78.2 0.8 -0.2 0.8 0.2173 71 173 70.8 0 0.2 0 0.2171 63 169.8 62.8 1.2 0.2 1.2 0.2167 55 166.2 55.6 0.8 -0.6 0.8 0.6162 47 162.2 49 -0.2 -2 0.2 2158 42 157.2 43.4 0.8 -1.4 0.8 1.4153 38 151.6 39.4 1.4 -1.4 1.4 1.4146 35 145.8 36.8 0.2 -1.8 0.2 1.8139 35 139.8 35.4 -0.8 -0.4 0.8 0.4133 34 134 35.4 -1 -1.4 1 1.4128 35 128.6 36.4 -0.6 -1.4 0.6 1.4124 38 123.8 38.2 0.2 -0.2 0.2 0.2119 40 119 41 0 -1 0 1115 44 114.2 44.6 0.8 -0.6 0.8 0.6109 48 109.4 48.8 -0.4 -0.8 0.4 0.8104 53 105.4 54.2 -1.4 -1.2 1.4 1.2100 59 101.8 60.6 -1.8 -1.6 1.8 1.6

99 67 99 67.8 0 -0.8 0 0.897 76 97.2 75.2 -0.2 0.8 0.2 0.8

Page 133: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

95 84 96.2 82.6 -1.2 1.4 1.2 1.495 90 95.8 89.6 -0.8 0.4 0.8 0.495 96 96.4 96 -1.4 0 1.4 097 102 98 102 -1 0 1 0

100 108 100 107.2 0 0.8 0 0.8103 114 102.8 112.8 0.2 1.2 0.2 1.2105 116 106 118 -1 -2 1 2109 124 109.4 122.6 -0.4 1.4 0.4 1.4113 128 113.6 126.6 -0.6 1.4 0.6 1.4117 131 118.8 130.6 -1.8 0.4 1.8 0.4124 134 124.6 132.8 -0.6 1.2 0.6 1.2131 136 131.2 134 -0.2 2 0.2 2138 135 138.6 133.6 -0.6 1.4 0.6 1.4146 134 145.6 131.8 0.4 2.2 0.4 2.2154 129 153 129.3333333 1 -0.333333333 1 0.333333333159 125 159 125 0 0 0 0

Page 134: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:100 63 100 63 0 0 0 0 1.035616438100 63 99.33333333 66.33333333 0.666666667 -3.333333333 0.666666667 3.333333333

98 73 98.4 70.2 -0.4 2.8 0.4 2.8 Maximum:97 76 97.4 73.8 -0.4 2.2 0.4 2.2 3.897 76 96.2 78 0.8 -2 0.8 295 81 95.6 80.4 -0.6 0.6 0.6 0.6 Sum:94 84 95.2 82.2 -1.2 1.8 1.2 1.8 151.295 85 94.4 85.8 0.6 -0.8 0.6 0.895 85 94.2 88.6 0.8 -3.6 0.8 3.6 RMS:93 94 94.8 91.8 -1.8 2.2 1.8 2.2 1.31674613994 95 95.2 95.4 -1.2 -0.4 1.2 0.497 100 96 99.8 1 0.2 1 0.2 Mean X:97 103 97.8 103.4 -0.8 -0.4 0.8 0.4 0.88767123399 107 99.8 108 -0.8 -1 0.8 1

102 112 101.2 112 0.8 0 0.8 0 Mean Y:104 118 103.2 116.2 0.8 1.8 0.8 1.8 1.183561644104 120 105.2 119.8 -1.2 0.2 1.2 0.2107 124 107.8 123.2 -0.8 0.8 0.8 0.8 Maximum X:109 125 110.8 126.4 -1.8 -1.4 1.8 1.4 2.6115 129 114.6 129.8 0.4 -0.8 0.4 0.8119 134 118.8 132.4 0.2 1.6 0.2 1.6 Maximum Y:123 137 123.2 135 -0.2 2 0.2 2 3.8128 137 127 136.8 1 0.2 1 0.2131 138 131.2 137.8 -0.2 0.2 0.2 0.2 Sum X:134 138 135.8 138 -1.8 0 1.8 0 64.8140 139 140.8 137.8 -0.8 1.2 0.8 1.2146 138 146.4 137.2 -0.4 0.8 0.4 0.8 Sum Y:153 136 152.4 135.8 0.6 0.2 0.6 0.2 86.4159 135 157.4 133.8 1.6 1.2 1.6 1.2164 131 162 131.2 2 -0.2 2 0.2 RMS X:165 129 165.6 128 -0.6 1 0.6 1 1.100491104169 125 168.4 124.4 0.6 0.6 0.6 0.6171 120 170.4 120.2 0.6 -0.2 0.6 0.2 RMS Y:173 117 172.8 115.4 0.2 1.6 0.2 1.6 1.502185116174 110 174.4 110.2 -0.4 -0.2 0.4 0.2177 105 175.8 104.4 1.2 0.6 1.2 0.6177 99 176.6 98.6 0.4 0.4 0.4 0.4178 91 176.6 92 1.4 -1 1.4 1177 88 175.6 85.2 1.4 2.8 1.4 2.8174 77 174.2 77.8 -0.2 -0.8 0.2 0.8172 71 171.8 70.6 0.2 0.4 0.2 0.4170 62 168.6 62.6 1.4 -0.6 1.4 0.6166 55 165.2 56 0.8 -1 0.8 1161 48 161.2 50 -0.2 -2 0.2 2157 44 156.2 45.2 0.8 -1.2 0.8 1.2152 41 150.6 41.8 1.4 -0.8 1.4 0.8145 38 145.2 39.6 -0.2 -1.6 0.2 1.6138 38 139.6 38.4 -1.6 -0.4 1.6 0.4134 37 134.2 38.4 -0.2 -1.4 0.2 1.4129 38 129 39.2 0 -1.2 0 1.2125 41 124.6 41 0.4 0 0.4 0119 42 119.6 43.4 -0.6 -1.4 0.6 1.4116 47 114.6 46.8 1.4 0.2 1.4 0.2109 49 110 50.8 -1 -1.8 1 1.8104 55 106.6 56.4 -2.6 -1.4 2.6 1.4102 61 103 62.8 -1 -1.8 1 1.8102 70 100.4 70 1.6 0 1.6 0

98 79 99.2 77 -1.2 2 1.2 296 85 98 83.8 -2 1.2 2 1.298 90 97.6 90.4 0.4 -0.4 0.4 0.496 95 98.4 96.2 -2.4 -1.2 2.4 1.2

Page 135: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

100 103 100.2 102 -0.2 1 0.2 1102 108 101.6 106.8 0.4 1.2 0.4 1.2105 114 104.8 113 0.2 1 0.2 1105 114 107.6 117.8 -2.6 -3.8 2.6 3.8112 126 111 122.2 1 3.8 1 3.8114 127 115.2 126 -1.2 1 1.2 1119 130 120.8 130.2 -1.8 -0.2 1.8 0.2126 133 126.4 131.6 -0.4 1.4 0.4 1.4133 135 133 132.6 0 2.4 0 2.4140 133 140.4 131.6 -0.4 1.4 0.4 1.4147 132 146.8 129.6 0.2 2.4 0.2 2.4156 125 153.6666667 126.6666667 2.333333333 -1.666666667 2.333333333 1.666666667

Page 136: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

80 90 100 110 120 130 140 150 160 170 18020

40

60

80

100

120

140

Circle 1 Trial 4 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

80 90 100 110 120 130 140 150 160 170 18020

40

60

80

100

120

140

Circle 1 Trial 4 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 137: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 5:

Kalman Filter Data:

Kalman:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:97 110 97 110 0 0 0 0 1.03461538599 116 99.33333333 115.6666667 -0.333333333 0.333333333 0.333333333 0.333333333

102 121 101.8 119.6 0.2 1.4 0.2 1.4 Maximum:105 124 104.4 123 0.6 1 0.6 1 2.8106 127 107.2 125.4 -1.2 1.6 1.2 1.6110 127 110.2 127 -0.2 0 0.2 0 Sum:113 128 114.2 128.4 -1.2 -0.4 1.2 0.4 107.6117 129 119.4 129.8 -2.4 -0.8 2.4 0.8125 131 125.2 131.4 -0.2 -0.4 0.2 0.4 RMS:132 134 131.6 132.8 0.4 1.2 0.4 1.2 1.26730753139 135 138.2 133.6 0.8 1.4 0.8 1.4145 135 144.6 133 0.4 2 0.4 2 Mean X:150 133 150.8 130.8 -0.8 2.2 0.8 2.2 0.896153846157 128 156.8 127.6 0.2 0.4 0.2 0.4163 123 162.4 122.8 0.6 0.2 0.6 0.2 Mean Y:169 119 167.4 116.8 1.6 2.2 1.6 2.2 1.173076923173 111 171.4 110 1.6 1 1.6 1175 103 174 102 1 1 1 1 Maximum X:177 94 175.2 92.8 1.8 1.2 1.8 1.2 2.8176 83 175.4 84.4 0.6 -1.4 0.6 1.4175 73 174 75.4 1 -2.4 1 2.4 Maximum Y:174 69 171.2 66.4 2.8 2.6 2.8 2.6 2.8168 58 167.4 58.4 0.6 -0.4 0.6 0.4163 49 162.4 51.4 0.6 -2.4 0.6 2.4 Sum X:157 43 156 44.4 1 -1.4 1 1.4 46.6150 38 149.4 39.4 0.6 -1.4 0.6 1.4142 34 142.4 36.2 -0.4 -2.2 0.4 2.2 Sum Y:135 33 134.6 34.8 0.4 -1.8 0.4 1.8 61128 33 127.2 34.8 0.8 -1.8 0.8 1.8118 36 119.6 37 -1.6 -1 1.6 1 RMS X:113 38 111.8 40.8 1.2 -2.8 1.2 2.8 1.110747804104 45 104.4 46 -0.4 -1 0.4 1

96 52 98.2 52.2 -2.2 -0.2 2.2 0.2 RMS Y:91 59 92.6 58.8 -1.6 0.2 1.6 0.2 1.40654757187 67 88.8 65.6 -1.8 1.4 1.8 1.485 71 86.6 72.2 -1.6 -1.2 1.6 1.285 79 85.4 78.8 -0.4 0.2 0.4 0.285 85 85.6 85.8 -0.6 -0.8 0.6 0.885 92 87 93.4 -2 -1.4 2 1.488 102 89.2 100.8 -1.2 1.2 1.2 1.292 109 92.4 108.6 -0.4 0.4 0.4 0.496 116 96.8 116 -0.8 0 0.8 0

101 124 102.4 122.2 -1.4 1.8 1.4 1.8107 129 108.8 127.6 -1.8 1.4 1.8 1.4116 133 116 131.6 0 1.4 0 1.4124 136 123.8 134 0.2 2 0.2 2132 136 131.6 135 0.4 1 0.4 1140 136 139.2 134 0.8 2 0.8 2146 134 146.6 131.6 -0.6 2.4 0.6 2.4154 128 153.4 128 0.6 0 0.6 0161 124 160.3333333 123.3333333 0.666666667 0.666666667 0.666666667 0.666666667166 118 166 118 0 0 0 0

Page 138: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:98 111 98 111 0 0 0 0 1.403846154

100 116 100.3333333 116 -0.333333333 0 0.333333333 0103 121 102.4 119.6 0.6 1.4 0.6 1.4 Maximum:105 124 105 122.4 0 1.6 0 1.6 6.8106 126 107.8 124.6 -1.8 1.4 1.8 1.4111 125 111 126.4 0 -1.4 0 1.4 Sum:114 127 115.6 128 -1.6 -1 1.6 1 146119 130 121.4 129.8 -2.4 0.2 2.4 0.2128 132 127.2 131.6 0.8 0.4 0.8 0.4 RMS:135 135 133.6 133 1.4 2 1.4 2 1.88618459140 134 139.8 133.2 0.2 0.8 0.2 0.8146 134 145.8 131.8 0.2 2.2 0.2 2.2 Mean X:150 131 151.6 129 -1.6 2 1.6 2 1.205128205158 125 157.4 125.8 0.6 -0.8 0.6 0.8164 121 162.6 120.8 1.4 0.2 1.4 0.2 Mean Y:169 118 167.2 114.8 1.8 3.2 1.8 3.2 1.602564103172 109 170.8 108.2 1.2 0.8 1.2 0.8173 101 172.6 100.2 0.4 0.8 0.4 0.8 Maximum X:176 92 173.4 91.2 2.6 0.8 2.6 0.8 4.4173 81 173.6 84 -0.6 -3 0.6 3173 73 171.8 74.8 1.2 -1.8 1.2 1.8 Maximum Y:173 73 168.8 66.2 4.2 6.8 4.2 6.8 6.8164 55 165.2 59 -1.2 -4 1.2 4161 49 160.2 52.6 0.8 -3.6 0.8 3.6 Sum X:155 45 153.8 45.4 1.2 -0.4 1.2 0.4 62.66666667148 41 148 41.8 0 -0.8 0 0.8141 37 141.4 39.4 -0.4 -2.4 0.4 2.4 Sum Y:135 37 133.6 38.4 1.4 -1.4 1.4 1.4 83.33333333128 37 127.2 38.2 0.8 -1.2 0.8 1.2116 40 119.4 41 -3.4 -1 3.4 1 RMS X:116 40 111.6 44.8 4.4 -4.8 4.4 4.8 1.627540749102 51 104.8 49.8 -2.8 1.2 2.8 1.2

96 56 99.4 55.8 -3.4 0.2 3.4 0.2 RMS Y:94 62 94 61.8 0 0.2 0 0.2 2.11340855689 70 91.2 67.8 -2.2 2.2 2.2 2.289 70 89.6 73.8 -0.6 -3.8 0.6 3.888 81 88.2 80 -0.2 1 0.2 188 86 88.6 87 -0.6 -1 0.6 187 93 89.8 94.8 -2.8 -1.8 2.8 1.891 105 92 101.8 -1 3.2 1 3.295 109 95 109.6 0 -0.6 0 0.699 116 99.6 116.4 -0.6 -0.4 0.6 0.4

103 125 105.2 121.8 -2.2 3.2 2.2 3.2110 127 111.4 126.8 -1.4 0.2 1.4 0.2119 132 118.4 130.2 0.6 1.8 0.6 1.8126 134 126 132.2 0 1.8 0 1.8134 133 133 133 1 0 1 0141 135 140.2 131.4 0.8 3.6 0.8 3.6145 131 147.2 129 -2.2 2 2.2 2155 124 153.6 125.6 1.4 -1.6 1.4 1.6161 122 160.6666667 120.6666667 0.333333333 1.333333333 0.333333333 1.333333333166 116 166 116 0 0 0 0

Page 139: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

80 90 100 110 120 130 140 150 160 170 18020

40

60

80

100

120

140

Circle 1 Trial 5 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

80 90 100 110 120 130 140 150 160 170 18020

40

60

80

100

120

140

Circle 1 Trial 5 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 140: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Star 1:

Page 141: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 1:

Kalman Filter Data:

Kalman:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:142 28 142 28 0 0 0 0 0.687313433142 29 141.3333333 28.66666667 0.666666667 0.333333333 0.666666667 0.333333333140 29 140.6 29.2 -0.6 -0.2 0.6 0.2 Maximum:139 30 140.4 29.8 -1.4 0.2 1.4 0.2 3.4140 30 140.4 30.4 -0.4 -0.4 0.4 0.4141 31 140.8 31.8 0.2 -0.8 0.2 0.8 Sum:142 32 141.8 33.6 0.2 -1.6 0.2 1.6 184.2142 36 142.8 36 -0.8 0 0.8 0144 39 143.8 38.8 0.2 0.2 0.2 0.2 RMS:145 42 144.8 42 0.2 0 0.2 0 0.897157849146 45 146 45 0 0 0 0147 48 147 48 0 0 0 0 Mean X:148 51 148.2 51.2 -0.2 -0.2 0.2 0.2 0.647263682149 54 149.4 54.8 -0.4 -0.8 0.4 0.8151 58 150.4 58.4 0.6 -0.4 0.6 0.4 Mean Y:152 63 151.6 62.2 0.4 0.8 0.4 0.8 0.727363184152 66 152.8 66.4 -0.8 -0.4 0.8 0.4154 70 154 70.6 0 -0.6 0 0.6 Maximum X:155 75 155.6 74 -0.6 1 0.6 1 3.4157 79 157.6 76.6 -0.6 2.4 0.6 2.4160 80 159.6 78.4 0.4 1.6 0.4 1.6 Maximum Y:162 79 162.2 79.2 -0.2 -0.2 0.2 0.2 3164 79 165.2 79.2 -1.2 -0.2 1.2 0.2168 79 168 78.6 0 0.4 0 0.4 Sum X:172 79 171.4 78.2 0.6 0.8 0.6 0.8 86.73333333174 77 175 77.6 -1 -0.6 1 0.6179 77 178.4 76.8 0.6 0.2 0.6 0.2 Sum Y:182 76 181.6 75.8 0.4 0.2 0.4 0.2 97.46666667185 75 184.6 75.4 0.4 -0.4 0.4 0.4188 74 186.6 75 1.4 -1 1.4 1 RMS X:189 75 188.2 75 0.8 0 0.8 0 0.852603077189 75 188.8 75.6 0.2 -0.6 0.2 0.6190 76 188.6 77 1.4 -1 1.4 1 RMS Y:188 78 188 78.6 0 -0.6 0 0.6 0.939602259187 81 187.4 80.4 -0.4 0.6 0.4 0.6186 83 186.4 82 -0.4 1 0.4 1186 84 185.8 83.2 0.2 0.8 0.2 0.8185 84 185.4 83.8 -0.4 0.2 0.4 0.2185 84 185 84 0 0 0 0185 84 184.2 84 0.8 0 0.8 0184 84 183 84 1 0 1 0182 84 181.4 84.4 0.6 -0.4 0.6 0.4179 84 179.4 85.2 -0.4 -1.2 0.4 1.2177 86 177 86.4 0 -0.4 0 0.4175 88 174.6 87.8 0.4 0.2 0.4 0.2172 90 172.6 89.2 -0.6 0.8 0.6 0.8170 91 171 90 -1 1 1 1169 91 169.8 90.6 -0.8 0.4 0.8 0.4169 90 169.2 91.2 -0.2 -1.2 0.2 1.2169 91 168.6 92.2 0.4 -1.2 0.4 1.2169 93 167.8 93.6 1.2 -0.6 1.2 0.6167 96 166.8 95.4 0.2 0.6 0.2 0.6165 98 165.6 97 -0.6 1 0.6 1164 99 164.4 98.4 -0.4 0.6 0.4 0.6163 99 163.8 100 -0.8 -1 0.8 1163 100 163.8 102 -0.8 -2 0.8 2164 104 164.4 105 -0.4 -1 0.4 1165 108 165.2 109 -0.2 -1 0.2 1

Page 142: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

167 114 166.4 113.4 0.6 0.6 0.6 0.6167 119 167.6 117.6 -0.6 1.4 0.6 1.4169 122 168.8 122 0.2 0 0.2 0170 125 170.2 126.2 -0.2 -1.2 0.2 1.2171 130 172 130.4 -1 -0.4 1 0.4174 135 173.6 134.8 0.4 0.2 0.4 0.2176 140 174.8 138.6 1.2 1.4 1.2 1.4177 144 175.2 141 1.8 3 1.8 3176 144 174 141.8 2 2.2 2 2.2173 142 171.6 140.8 1.4 1.2 1.4 1.2168 139 167.8 137.8 0.2 1.2 0.2 1.2164 135 163 133.8 1 1.2 1 1.2158 129 158.2 129.6 -0.2 -0.6 0.2 0.6152 124 153.8 125.4 -1.8 -1.4 1.8 1.4149 121 149.8 121.8 -0.8 -0.8 0.8 0.8146 118 146.6 119.4 -0.6 -1.4 0.6 1.4144 117 144.2 118.6 -0.2 -1.6 0.2 1.6142 117 142 118.8 0 -1.8 0 1.8140 120 140 120 0 0 0 0138 122 138 122 0 0 0 0136 124 135.4 124.6 0.6 -0.6 0.6 0.6134 127 132.2 127.2 1.8 -0.2 1.8 0.2129 130 128.8 130 0.2 0 0.2 0124 133 125 133 -1 0 1 0121 136 120.6 136 0.4 0 0.4 0117 139 116.4 138.8 0.6 0.2 0.6 0.2112 142 113 141 -1 1 1 1108 144 110.2 142 -2.2 2 2.2 2107 144 108.6 141.6 -1.6 2.4 1.6 2.4107 141 108.2 139.4 -1.2 1.6 1.2 1.6109 137 108.6 136.2 0.4 0.8 0.4 0.8110 131 110 131.4 0 -0.4 0 0.4110 128 112 126.2 -2 1.8 2 1.8114 120 113.8 120.8 0.2 -0.8 0.2 0.8117 115 115.8 115.8 1.2 -0.8 1.2 0.8118 110 118.2 110.8 -0.2 -0.8 0.2 0.8120 106 120 106.8 0 -0.8 0 0.8122 103 121.4 103.6 0.6 -0.6 0.6 0.6123 100 122.6 101.4 0.4 -1.4 0.4 1.4124 99 123.2 99.8 0.8 -0.8 0.8 0.8124 99 123.2 98.6 0.8 0.4 0.8 0.4123 98 121.6 97.4 1.4 0.6 1.4 0.6122 97 118.6 95.6 3.4 1.4 3.4 1.4115 94 114.4 92.8 0.6 1.2 0.6 1.2109 90 109.4 89.6 -0.4 0.4 0.4 0.4103 85 103.6 85.8 -0.6 -0.8 0.6 0.8

98 82 98.4 82.2 -0.4 -0.2 0.4 0.293 78 93.8 78.8 -0.8 -0.8 0.8 0.889 76 90.2 76.2 -1.2 -0.2 1.2 0.286 73 87.8 74.2 -1.8 -1.2 1.8 1.285 72 86.6 73.2 -1.6 -1.2 1.6 1.286 72 87 72.6 -1 -0.6 1 0.687 73 89 72.6 -2 0.4 2 0.491 73 91.8 72.8 -0.8 0.2 0.8 0.296 73 95.4 73 0.6 0 0.6 099 73 99.6 73.2 -0.6 -0.2 0.6 0.2

104 73 103.8 73.4 0.2 -0.4 0.2 0.4108 74 107.8 73.4 0.2 0.6 0.2 0.6112 74 112 73.4 0 0.6 0 0.6116 73 115.8 73.4 0.2 -0.4 0.2 0.4120 73 119.4 73.6 0.6 -0.6 0.6 0.6123 73 122.6 73.8 0.4 -0.8 0.4 0.8126 75 125.2 74 0.8 1 0.8 1128 75 127.2 73.8 0.8 1.2 0.8 1.2129 74 128.6 73.2 0.4 0.8 0.4 0.8130 72 129.6 71 0.4 1 0.4 1

Page 143: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

130 70 130.6 67.8 -0.6 2.2 0.6 2.2131 64 131.4 63.6 -0.4 0.4 0.4 0.4133 59 132 58.6 1 0.4 1 0.4133 53 132.8 53 0.2 0 0.2 0133 47 133.6 48 -0.6 -1 0.6 1134 42 134.4 43.4 -0.4 -1.4 0.4 1.4135 39 135.8 39.4 -0.8 -0.4 0.8 0.4137 36 137.4 36.2 -0.4 -0.2 0.4 0.2

140 33 139.3333333 33.33333333 0.666666667-

0.333333333 0.666666667 0.333333333141 31 141 31 0 0 0 0

Page 144: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:143 29 143 29 0 0 0 0 0.918905473

142 29 141.6666667 29.33333333 0.333333333-

0.333333333 0.333333333 0.333333333140 30 141.2 29.8 -1.2 0.2 1.2 0.2 Maximum:140 30 141 30.4 -1 -0.4 1 0.4 4141 31 141.2 31.2 -0.2 -0.2 0.2 0.2142 32 141.8 32.8 0.2 -0.8 0.2 0.8 Sum:143 33 142.8 35 0.2 -2 0.2 2 246.2666667143 38 143.8 37.4 -0.8 0.6 0.8 0.6145 41 144.6 40.2 0.4 0.8 0.4 0.8 RMS:146 43 145.4 43.4 0.6 -0.4 0.6 0.4 1.177427277146 46 146.6 46.2 -0.6 -0.2 0.6 0.2147 49 147.4 49 -0.4 0 0.4 0 Mean X:149 52 148.8 52.2 0.2 -0.2 0.2 0.2 0.910945274149 55 150 55.8 -1 -0.8 1 0.8153 59 151 59.4 2 -0.4 2 0.4 Mean Y:152 64 152.2 63.2 -0.2 0.8 0.2 0.8 0.926865672152 67 153.6 67.4 -1.6 -0.4 1.6 0.4155 71 154.6 71.4 0.4 -0.4 0.4 0.4 Maximum X:156 76 156.4 74.4 -0.4 1.6 0.4 1.6 4158 79 158.6 76.4 -0.6 2.6 0.6 2.6161 79 160.6 77.8 0.4 1.2 0.4 1.2 Maximum Y:163 77 163.2 78.2 -0.2 -1.2 0.2 1.2 3.4165 78 166.4 78.2 -1.4 -0.2 1.4 0.2169 78 168.8 77.6 0.2 0.4 0.2 0.4 Sum X:174 79 172.4 77.6 1.6 1.4 1.6 1.4 122.0666667173 76 176 77.4 -3 -1.4 3 1.4181 77 179 76.8 2 0.2 2 0.2 Sum Y:183 77 181.8 76 1.2 1 1.2 1 124.2184 75 185 76 -1 -1 1 1188 75 186.4 75.8 1.6 -0.8 1.6 0.8 RMS X:189 76 187.6 76 1.4 0 1.4 0 1.16754054188 76 188.2 77 -0.2 -1 0.2 1189 78 187.8 78.6 1.2 -0.6 1.2 0.6 RMS Y:187 80 187.4 80 -0.4 0 0.4 0 1.187231684186 83 187 81.6 -1 1.4 1 1.4187 83 186.4 82.8 0.6 0.2 0.6 0.2186 84 186.2 83.6 -0.2 0.4 0.2 0.4186 84 186 83.8 0 0.2 0 0.2186 84 185.4 84 0.6 0 0.6 0185 84 184.4 84 0.6 0 0.6 0184 84 183 84.2 1 -0.2 1 0.2181 84 181.2 84.8 -0.2 -0.8 0.2 0.8179 85 179.2 86 -0.2 -1 0.2 1177 87 176.8 87.4 0.2 -0.4 0.2 0.4175 90 174.8 88.8 0.2 1.2 0.2 1.2172 91 173.2 89.8 -1.2 1.2 1.2 1.2171 91 171.8 90.4 -0.8 0.6 0.8 0.6171 90 170.8 90.8 0.2 -0.8 0.2 0.8170 90 170.4 91.4 -0.4 -1.4 0.4 1.4170 92 169.4 92.8 0.6 -0.8 0.6 0.8170 94 168.2 94.4 1.8 -0.4 1.8 0.4166 98 167 96.2 -1 1.8 1 1.8165 98 165.8 97.6 -0.8 0.4 0.8 0.4164 99 164.8 98.8 -0.8 0.2 0.8 0.2164 99 164.6 100.4 -0.6 -1.4 0.6 1.4165 100 165 102.8 0 -2.8 0 2.8165 106 165.8 106.4 -0.8 -0.4 0.8 0.4167 110 166.6 110.6 0.4 -0.6 0.4 0.6168 117 167.6 114.8 0.4 2.2 0.4 2.2168 120 168.6 118.6 -0.6 1.4 0.6 1.4

Page 145: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

170 121 169.4 122.8 0.6 -1.8 0.6 1.8170 125 171 126.8 -1 -1.8 1 1.8171 131 172.6 131 -1.6 0 1.6 0176 137 174.2 135.4 1.8 1.6 1.8 1.6176 141 175 138.6 1 2.4 1 2.4178 143 175 140.2 3 2.8 3 2.8174 141 173 140 1 1 1 1171 139 170.2 138.4 0.8 0.6 0.8 0.6166 136 165.8 135 0.2 1 0.2 1162 133 161.4 131.4 0.6 1.6 0.6 1.6156 126 157.4 128.2 -1.4 -2.2 1.4 2.2152 123 153.6 124.8 -1.6 -1.8 1.6 1.8151 123 150.4 122 0.6 1 0.6 1147 119 148 120.6 -1 -1.6 1 1.6146 119 145.8 120.8 0.2 -1.8 0.2 1.8144 119 143.2 121 0.8 -2 0.8 2141 124 141.2 122.4 -0.2 1.6 0.2 1.6138 124 138.8 124.2 -0.8 -0.2 0.8 0.2137 126 135.4 126.6 1.6 -0.6 1.6 0.6134 128 132 128.8 2 -0.8 2 0.8127 131 128.6 131.4 -1.6 -0.4 1.6 0.4124 135 124.6 134 -0.6 1 0.6 1121 137 120.2 137 0.8 0 0.8 0117 139 116.6 139.4 0.4 -0.4 0.4 0.4112 143 113.6 141 -1.6 2 1.6 2109 143 111.4 141.2 -2.4 1.8 2.4 1.8109 143 110.6 140.2 -1.6 2.8 1.6 2.8110 138 110.4 137.2 -0.4 0.8 0.4 0.8113 134 110.8 134.2 2.2 -0.2 2.2 0.2111 128 112.6 129 -1.6 -1 1.6 1111 128 114.2 124.6 -3.2 3.4 3.2 3.4118 117 115.2 119.8 2.8 -2.8 2.8 2.8118 116 117.2 115.8 0.8 0.2 0.8 0.2118 110 119.6 111 -1.6 -1 1.6 1121 108 120.4 108 0.6 0 0.6 0123 104 121.6 105 1.4 -1 1.4 1122 102 122.8 103.2 -0.8 -1.2 0.8 1.2124 101 123 101.6 1 -0.6 1 0.6124 101 122.6 100.4 1.4 0.6 1.4 0.6122 100 120.4 98.6 1.6 1.4 1.6 1.4121 98 117 96.2 4 1.8 4 1.8111 93 112.4 92.8 -1.4 0.2 1.4 0.2107 89 107.6 89.4 -0.6 -0.4 0.6 0.4101 84 102.2 85.4 -1.2 -1.4 1.2 1.4

98 83 98 82.4 0 0.6 0 0.694 78 94.2 79.4 -0.2 -1.4 0.2 1.490 78 91.4 77.4 -1.4 0.6 1.4 0.688 74 89.6 75.6 -1.6 -1.6 1.6 1.687 74 88.8 75 -1.8 -1 1.8 189 74 89.6 74.4 -0.6 -0.4 0.6 0.490 75 91.8 74.4 -1.8 0.6 1.8 0.694 75 94.6 74.4 -0.6 0.6 0.6 0.699 74 97.8 74.4 1.2 -0.4 1.2 0.4

101 74 101.6 74.4 -0.6 -0.4 0.6 0.4105 74 105.4 74.4 -0.4 -0.4 0.4 0.4109 75 109 74.2 0 0.8 0 0.8113 75 113 74.2 0 0.8 0 0.8117 73 116.6 74.2 0.4 -1.2 0.4 1.2121 74 119.8 74.4 1.2 -0.4 1.2 0.4123 74 122.8 74.6 0.2 -0.6 0.2 0.6125 76 125 74.6 0 1.4 0 1.4128 76 126.6 74 1.4 2 1.4 2128 73 127.8 73 0.2 0 0.2 0129 71 129.2 70.2 -0.2 0.8 0.2 0.8129 69 130.4 66.4 -1.4 2.6 1.4 2.6132 62 131.4 62.2 0.6 -0.2 0.6 0.2

Page 146: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

134 57 132.2 57.4 1.8 -0.4 1.8 0.4133 52 133.2 52 -0.2 0 0.2 0133 47 134 47.8 -1 -0.8 1 0.8134 42 135 44 -1 -2 1 2136 41 136.6 40.4 -0.6 0.6 0.6 0.6139 38 138.4 37.4 0.6 0.6 0.6 0.6

141 34 140.6666667 34.66666667 0.333333333-

0.666666667 0.333333333 0.666666667142 32 142 32 0 0 0 0

Page 147: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

80 100 120 140 160 180 20020

40

60

80

100

120

140

160

Star 1 Trial 1 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

80 100 120 140 160 180 20020

40

60

80

100

120

140

160

Star 1 Trial 1 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 148: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 2:

Kalman Filter Data:

Kalman:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:139 27 139 27 0 0 0 0 0.955902778138 31 138.3333333 30 -0.333333333 1 0.333333333 1138 32 138.6 32.8 -0.6 -0.8 0.6 0.8 Maximum:138 34 139.2 36.6 -1.2 -2.6 1.2 2.6 3.4140 40 140.2 40.8 -0.2 -0.8 0.2 0.8142 46 141.6 45.6 0.4 0.4 0.4 0.4 Sum:143 52 143.4 51.2 -0.4 0.8 0.4 0.8 183.5333333145 56 145.2 57.2 -0.2 -1.2 0.2 1.2147 62 147 62.8 0 -0.8 0 0.8 RMS:149 70 148.8 67.6 0.2 2.4 0.2 2.4 1.230825152151 74 150.2 71.8 0.8 2.2 0.8 2.2152 76 151.6 74.8 0.4 1.2 0.4 1.2 Mean X:152 77 153.4 76.2 -1.4 0.8 1.4 0.8 0.941666667154 77 156 76.8 -2 0.2 2 0.2158 77 159.2 76.8 -1.2 0.2 1.2 0.2 Mean Y:164 77 163 76.4 1 0.6 1 0.6 0.970138889168 76 167 76 1 0 1 0171 75 170.8 75.4 0.2 -0.4 0.2 0.4 Maximum X:174 75 173.8 75 0.2 0 0.2 0 3.4177 74 176.2 74.8 0.8 -0.8 0.8 0.8179 75 178.4 75.2 0.6 -0.2 0.6 0.2 Maximum Y:180 75 180.2 76 -0.2 -1 0.2 1 3.2182 77 180.6 78 1.4 -1 1.4 1183 79 180 81 3 -2 3 2 Sum X:179 84 178.2 84.8 0.8 -0.8 0.8 0.8 90.4176 90 175.2 88.4 0.8 1.6 0.8 1.6171 94 171.4 92 -0.4 2 0.4 2 Sum Y:167 95 167.6 94.8 -0.6 0.2 0.6 0.2 93.13333333164 97 163.8 97.2 0.2 -0.2 0.2 0.2160 98 160.8 99.6 -0.8 -1.6 0.8 1.6 RMS X:157 102 158.6 102.8 -1.6 -0.8 1.6 0.8 1.216704718156 106 157.6 107 -1.6 -1 1.6 1156 111 157.8 112.2 -1.8 -1.2 1.8 1.2 RMS Y:159 118 159.2 118 -0.2 0 0.2 0 1.24478542161 124 161.6 124.4 -0.6 -0.4 0.6 0.4164 131 164.8 130.6 -0.8 0.4 0.8 0.4168 138 167.6 136.2 0.4 1.8 0.4 1.8172 142 170.4 141.2 1.6 0.8 1.6 0.8173 146 172.6 144.6 0.4 1.4 0.4 1.4175 149 173.6 146.2 1.4 2.8 1.4 2.8175 148 172.6 146 2.4 2 2.4 2173 146 170 144 3 2 3 2167 141 165.4 140.4 1.6 0.6 1.6 0.6160 136 159.2 136 0.8 0 0.8 0152 131 152.4 131.2 -0.4 -0.2 0.4 0.2144 126 146.2 127.2 -2.2 -1.2 2.2 1.2139 122 141.2 124.2 -2.2 -2.2 2.2 2.2136 121 137.6 122.6 -1.6 -1.6 1.6 1.6135 121 135 122.6 0 -1.6 0 1.6134 123 132.4 124.6 1.6 -1.6 1.6 1.6131 126 129.6 127.8 1.4 -1.8 1.4 1.8126 132 125.8 132 0.2 0 0.2 0122 137 121 136.4 1 0.6 1 0.6116 142 116.2 140.6 -0.2 1.4 0.2 1.4110 145 112 144 -2 1 2 1107 147 108.6 145.8 -1.6 1.2 1.6 1.2105 149 106.8 145.8 -1.8 3.2 1.8 3.2105 146 106.8 143.6 -1.8 2.4 1.8 2.4

Page 149: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

107 142 108 139.8 -1 2.2 1 2.2110 134 110.2 134.6 -0.2 -0.6 0.2 0.6113 128 112.8 129.6 0.2 -1.6 0.2 1.6116 123 115.4 124 0.6 -1 0.6 1118 121 117.6 119.2 0.4 1.8 0.4 1.8120 114 119.2 114.8 0.8 -0.8 0.8 0.8121 110 119.2 110.6 1.8 -0.6 1.8 0.6121 106 117.8 106 3.2 0 3.2 0116 102 115 102 1 0 1 0111 98 111 98.2 0 -0.2 0 0.2106 94 106 94.4 0 -0.4 0 0.4101 91 101.4 91 -0.4 0 0.4 0

96 87 96.8 87.6 -0.8 -0.6 0.8 0.693 85 92.6 84.4 0.4 0.6 0.4 0.688 81 89.2 81.4 -1.2 -0.4 1.2 0.485 78 86.8 79.2 -1.8 -1.2 1.8 1.284 76 86 77.6 -2 -1.6 2 1.684 76 87.4 77 -3.4 -1 3.4 189 77 90.8 77 -1.8 0 1.8 095 78 96 77.4 -1 0.6 1 0.6

102 78 102.4 77.8 -0.4 0.2 0.4 0.2110 78 109 78 1 0 1 0116 78 115.2 77.8 0.8 0.2 0.8 0.2122 78 121 76.8 1 1.2 1 1.2126 77 125.6 74.4 0.4 2.6 0.4 2.6131 73 129.4 70.8 1.6 2.2 1.6 2.2133 66 132.2 65.6 0.8 0.4 0.8 0.4135 60 134.2 59.6 0.8 0.4 0.8 0.4136 52 135.6 53.6 0.4 -1.6 0.4 1.6136 47 136.6 48.8 -0.6 -1.8 0.6 1.8138 43 137.6 44.8 0.4 -1.8 0.4 1.8138 42 138.2 42.2 -0.2 -0.2 0.2 0.2140 40 138.8 40.4 1.2 -0.4 1.2 0.4139 39 139 39.6 0 -0.6 0 0.6139 38 139.2 39 -0.2 -1 0.2 1139 39 138.6 39 0.4 0 0.4 0139 39 138.3333333 39.33333333 0.666666667 -0.333333333 0.666666667 0.333333333137 40 137 40 0 0 0 0

Page 150: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw: Moving Moving Difference X: Difference Y: Difference X Difference Y Statistics:X Y Mean:139 28 139 28 0 0 0 0 1.128819444139 32 139 30.66666667 0 1.333333333 0 1.333333333139 32 139.6 34 -0.6 -2 0.6 2 Maximum:139 35 140.4 38 -1.4 -3 1.4 3 4.2142 43 141.2 42.2 0.8 0.8 0.8 0.8143 48 142.8 47.2 0.2 0.8 0.2 0.8 Sum:143 53 144.4 52.8 -1.4 0.2 1.4 0.2 216.7333333147 57 146 58.6 1 -1.6 1 1.6147 63 147.6 63.6 -0.6 -0.6 0.6 0.6 RMS:150 72 149.4 67.8 0.6 4.2 0.6 4.2 1.466879719151 73 150.4 71.4 0.6 1.6 0.6 1.6152 74 152 74 0 0 0 0 Mean X:152 75 154.2 74.8 -2.2 0.2 2.2 0.2 1.146527778155 76 157.2 75.4 -2.2 0.6 2.2 0.6161 76 160.6 75.6 0.4 0.4 0.4 0.4 Mean Y:166 76 164.4 75.6 1.6 0.4 1.6 0.4 1.111111111169 75 168.4 75.4 0.6 -0.4 0.6 0.4171 75 171.6 75.2 -0.6 -0.2 0.6 0.2 Maximum X:175 75 174.2 75.2 0.8 -0.2 0.8 0.2 4.2177 75 176.2 75.4 0.8 -0.4 0.8 0.4179 76 178.4 76 0.6 0 0.6 0 Maximum Y:179 76 180 77.2 -1 -1.2 1 1.2 4.2182 78 179.8 79.8 2.2 -1.8 2.2 1.8183 81 178.8 83.2 4.2 -2.2 4.2 2.2 Sum X:176 88 176.8 86.8 -0.8 1.2 0.8 1.2 110.0666667174 93 173.8 90 0.2 3 0.2 3169 94 170 93 -1 1 1 1 Sum Y:167 94 166.8 95 0.2 -1 0.2 1 106.6666667164 96 163.6 97 0.4 -1 0.4 1160 98 161.4 99.8 -1.4 -1.8 1.4 1.8 RMS X:158 103 159.8 103.4 -1.8 -0.4 1.8 0.4 1.457467997158 108 159.4 108.4 -1.4 -0.4 1.4 0.4159 112 160 114 -1 -2 1 2 RMS Y:162 121 161.6 119.8 0.4 1.2 0.4 1.2 1.476231438163 126 164 126 -1 0 1 0166 132 167 131.8 -1 0.2 1 0.2170 139 169 136.6 1 2.4 1 2.4174 141 171.6 141 2.4 0 2.4 0172 145 173 143.6 -1 1.4 1 1.4176 148 173.2 144.4 2.8 3.6 2.8 3.6173 145 171.2 143.8 1.8 1.2 1.8 1.2171 143 168.2 141.6 2.8 1.4 2.8 1.4164 138 162.8 137.8 1.2 0.2 1.2 0.2157 134 156.8 134 0.2 0 0.2 0149 129 150.6 130 -1.6 -1 1.6 1143 126 145.4 127 -2.4 -1 2.4 1140 123 141.6 125 -1.6 -2 1.6 2138 123 139 124.6 -1 -1.6 1 1.6138 124 136.8 125.2 1.2 -1.2 1.2 1.2136 127 133.8 127.8 2.2 -0.8 2.2 0.8132 129 130.4 131.2 1.6 -2.2 1.6 2.2125 136 125.8 135 -0.8 1 0.8 1121 140 120.6 138.6 0.4 1.4 0.4 1.4115 143 115.8 142.2 -0.8 0.8 0.8 0.8110 145 112.2 144.6 -2.2 0.4 2.2 0.4108 147 109.6 145.2 -1.6 1.8 1.6 1.8107 148 108.8 144.4 -1.8 3.6 1.8 3.6108 143 109.4 141.4 -1.4 1.6 1.4 1.6111 139 111 137.2 0 1.8 0 1.8113 130 113.2 132.2 -0.2 -2.2 0.2 2.2116 126 115.2 128.2 0.8 -2.2 0.8 2.2118 123 117 123 1 0 1 0118 123 118.8 119 -0.8 4 0.8 4

Page 151: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

120 113 119.6 115.2 0.4 -2.2 0.4 2.2122 110 118.4 111.2 3.6 -1.2 3.6 1.2120 107 116.6 106.4 3.4 0.6 3.4 0.6112 103 113.6 102.8 -1.6 0.2 1.6 0.2109 99 109.2 99.2 -0.2 -0.2 0.2 0.2105 95 104.4 95.4 0.6 -0.4 0.6 0.4100 92 101 92 -1 0 1 0

96 88 96.6 88.4 -0.6 -0.4 0.6 0.495 86 92.8 85.2 2.2 0.8 2.2 0.887 81 90.2 82.4 -3.2 -1.4 3.2 1.486 79 88.4 80.6 -2.4 -1.6 2.4 1.687 78 88.2 79.4 -1.2 -1.4 1.2 1.487 79 90.6 79 -3.6 0 3.6 094 80 94.6 79 -0.6 1 0.6 199 79 99.8 79.2 -0.8 -0.2 0.8 0.2

106 79 105.8 79.2 0.2 -0.2 0.2 0.2113 79 111.4 78.8 1.6 0.2 1.6 0.2117 79 116.6 78.4 0.4 0.6 0.4 0.6122 78 121.6 76.8 0.4 1.2 0.4 1.2125 77 125.4 73.6 -0.4 3.4 0.4 3.4131 71 128.6 69.4 2.4 1.6 2.4 1.6132 63 131.4 63.8 0.6 -0.8 0.6 0.8133 58 133.4 58 -0.4 0 0.4 0136 50 134.8 52.8 1.2 -2.8 1.2 2.8135 48 136 49.2 -1 -1.2 1 1.2138 45 137.4 45.8 0.6 -0.8 0.6 0.8138 45 138 44 0 1 0 1140 41 138.6 42.4 1.4 -1.4 1.4 1.4139 41 138.8 41.6 0.2 -0.6 0.2 0.6138 40 139 40.8 -1 -0.8 1 0.8139 41 138.4 41 0.6 0 0.6 0139 41 138.3333333 41.33333333 0.666666667 -0.333333333 0.666666667 0.333333333137 42 137 42 0 0 0 0

Page 152: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

80 100 120 140 160 180 20020

40

60

80

100

120

140

160

Star 1 Trial 2 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

80 100 120 140 160 180 20020

40

60

80

100

120

140

160

Star 1 Trial 2 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 153: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 3:

Kalman Filter Data:

Kalman:

Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:

139 26 139 26 0 0 0 01.31418439

7

142 28 140.6666667 271.33333333

3 1 1.333333333 1141 27 140.8 27.6 0.2 -0.6 0.2 0.6 Maximum:141 28 141.2 28.2 -0.2 -0.2 0.2 0.2 7141 29 141 28.4 0 0.6 0 0.6141 29 141 28.8 0 0.2 0 0.2 Sum:

141 29 141 29 0 0 0 0247.066666

7141 29 141 29.4 0 -0.4 0 0.4141 29 141.2 30.4 -0.2 -1.4 0.2 1.4 RMS:

141 31 141.6 33 -0.6 -2 0.6 21.76178235

6142 34 142.2 36.8 -0.2 -2.8 0.2 2.8143 42 143.2 42 -0.2 0 0.2 0 Mean X:

144 48 144.6 48.2 -0.6 -0.2 0.6 0.21.21773049

6146 55 146.4 55.2 -0.4 -0.2 0.4 0.2148 62 148.2 61.6 -0.2 0.4 0.2 0.4 Mean Y:

151 69 150 67.6 1 1.4 1 1.41.41063829

8152 74 151.8 72.4 0.2 1.6 0.2 1.6153 78 153.8 76 -0.8 2 0.8 2 Maximum X:155 79 156.2 78.2 -1.2 0.8 1.2 0.8 5.2158 80 159.6 79.4 -1.6 0.6 1.6 0.6163 80 163.8 79.6 -0.8 0.4 0.8 0.4 Maximum Y:169 80 168.6 79.6 0.4 0.4 0.4 0.4 7174 79 173.4 79.4 0.6 -0.4 0.6 0.4179 79 177.8 79.2 1.2 -0.2 1.2 0.2 Sum X:

182 79 181.2 79 0.8 0 0.8 0114.466666

7185 79 183.6 79.2 1.4 -0.2 1.4 0.2186 79 185.2 79.6 0.8 -0.6 0.8 0.6 Sum Y:186 80 185.6 80.2 0.4 -0.2 0.4 0.2 132.6187 81 184.2 82 2.8 -1 2.8 1184 82 181.4 85.4 2.6 -3.4 2.6 3.4 RMS X:

178 88 177.4 89.4 0.6 -1.4 0.6 1.41.62915226

9172 96 172.6 93.8 -0.6 2.2 0.6 2.2166 100 168.4 98.4 -2.4 1.6 2.4 1.6 RMS Y:

163 103 165.4 102.6 -2.4 0.4 2.4 0.41.88510398

2163 105 164.4 107.4 -1.4 -2.4 1.4 2.4163 109 165 113.6 -2 -4.6 2 4.6167 120 167 120.8 0 -0.8 0 0.8169 131 169.6 128.6 -0.6 2.4 0.6 2.4173 139 172.4 136.2 0.6 2.8 0.6 2.8176 144 174.4 141.6 1.6 2.4 1.6 2.4177 147 175.2 144.4 1.8 2.6 1.8 2.6177 147 174 144.8 3 2.2 3 2.2173 145 170.4 143.4 2.6 1.6 2.6 1.6167 141 165.2 140.2 1.8 0.8 1.8 0.8158 137 158.6 136.2 -0.6 0.8 0.6 0.8151 131 152 132.2 -1 -1.2 1 1.2144 127 146.4 129 -2.4 -2 2.4 2140 125 142.4 127.2 -2.4 -2.2 2.4 2.2139 125 138.8 127.6 0.2 -2.6 0.2 2.6

Page 154: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

138 128 135.2 130.6 2.8 -2.6 2.8 2.6133 133 130.6 135.4 2.4 -2.4 2.4 2.4126 142 124.8 140.8 1.2 1.2 1.2 1.2117 149 118.6 145.8 -1.6 3.2 1.6 3.2110 152 113.6 149.6 -3.6 2.4 3.6 2.4107 153 110.6 150.6 -3.6 2.4 3.6 2.4108 152 110 147.8 -2 4.2 2 4.2111 147 111.4 142.8 -0.4 4.2 0.4 4.2114 135 113.6 136.6 0.4 -1.6 0.4 1.6117 127 116.2 129 0.8 -2 0.8 2118 122 118.6 121.4 -0.6 0.6 0.6 0.6121 114 120.4 115.8 0.6 -1.8 0.6 1.8123 109 121.4 111.4 1.6 -2.4 1.6 2.4123 107 122 108 1 -1 1 1122 105 120.6 105 1.4 0 1.4 0121 105 116.8 101.4 4.2 3.6 4.2 3.6114 99 112 97.6 2 1.4 2 1.4104 91 105.4 92.6 -1.4 -1.6 1.4 1.6

99 88 98 87 1 1 1 189 80 91.4 82.4 -2.4 -2.4 2.4 2.484 77 87.6 79.4 -3.6 -2.4 3.6 2.481 76 86.2 77.4 -5.2 -1.4 5.2 1.485 76 88.2 77.2 -3.2 -1.2 3.2 1.292 78 93 77.6 -1 0.4 1 0.499 79 100 78.2 -1 0.8 1 0.8

108 79 107.4 78.8 0.6 0.2 0.6 0.2116 79 114.2 79 1.8 0 1.8 0122 79 119.8 78.8 2.2 0.2 2.2 0.2126 79 123.6 78.4 2.4 0.6 2.4 0.6127 78 125.6 78 1.4 0 1.4 0127 77 127.2 75.2 -0.2 1.8 0.2 1.8126 77 128.6 70 -2.6 7 2.6 7130 65 130.4 63.2 -0.4 1.8 0.4 1.8133 53 132.4 55.8 0.6 -2.8 0.6 2.8136 44 135 47 1 -3 1 3137 40 136.8 40.2 0.2 -0.2 0.2 0.2139 33 138.2 35.6 0.8 -2.6 0.8 2.6139 31 138.8 32.8 0.2 -1.8 0.2 1.8140 30 139.2 30.8 0.8 -0.8 0.8 0.8139 30 139.2 30.4 -0.2 -0.4 0.2 0.4139 30 139.4 30.4 -0.4 -0.4 0.4 0.4139 31 139.4 30.8 -0.4 0.2 0.4 0.2140 31 139.8 31.4 0.2 -0.4 0.2 0.4

140 32 140.3333333 32

-0.33333333

3 0 0.333333333 0141 33 141 33 0 0 0 0

Page 155: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:140 27 140 27 0 0 0 0 1.609929078141 28 140.3333333 27.33333333 0.666666667 0.666666667 0.666666667 0.666666667140 27 140.6 28 -0.6 -1 0.6 1 Maximum:141 29 140.8 28.4 0.2 0.6 0.2 0.6 9.2141 29 140.8 28.6 0.2 0.4 0.2 0.4141 29 141.2 29 -0.2 0 0.2 0 Sum:141 29 141.2 29.2 -0.2 -0.2 0.2 0.2 302.6666667142 29 141.4 29.8 0.6 -0.8 0.6 0.8141 30 141.8 31.4 -0.8 -1.4 0.8 1.4 RMS:142 32 142.4 34.8 -0.4 -2.8 0.4 2.8 2.231686701143 37 143 39 0 -2 0 2144 46 144.4 44.4 -0.4 1.6 0.4 1.6 Mean X:145 50 145.6 50.6 -0.6 -0.6 0.6 0.6 1.44893617148 57 147.6 57.2 0.4 -0.2 0.4 0.2148 63 149 62.8 -1 0.2 1 0.2 Mean Y:153 70 150.6 68.2 2.4 1.8 2.4 1.8 1.770921986151 74 152.2 72.2 -1.2 1.8 1.2 1.8153 77 154.6 75.2 -1.6 1.8 1.6 1.8 Maximum X:156 77 157.2 76.8 -1.2 0.2 1.2 0.2 5.6160 78 161.4 77.8 -1.4 0.2 1.4 0.2166 78 165.8 78.2 0.2 -0.2 0.2 0.2 Maximum Y:172 79 170.4 78.6 1.6 0.4 1.6 0.4 9.2175 79 174.8 78.8 0.2 0.2 0.2 0.2179 79 178.6 79.2 0.4 -0.2 0.4 0.2 Sum X:182 79 181.2 79.4 0.8 -0.4 0.8 0.4 136.2185 80 183.2 79.8 1.8 0.2 1.8 0.2185 80 184.6 80.4 0.4 -0.4 0.4 0.4 Sum Y:185 81 184.6 81.2 0.4 -0.2 0.4 0.2 166.4666667186 82 182.4 83.8 3.6 -1.8 3.6 1.8182 83 179.4 87.8 2.6 -4.8 2.6 4.8 RMS X:174 93 175.4 91.6 -1.4 1.4 1.4 1.4 1.955231813170 100 171 95.6 -1 4.4 1 4.4165 100 167.8 99.8 -2.8 0.2 2.8 0.2 RMS Y:164 102 166.2 103.2 -2.2 -1.2 2.2 1.2 2.477482517166 104 166.4 108.4 -0.4 -4.4 0.4 4.4166 110 167.6 115.4 -1.6 -5.4 1.6 5.4171 126 170 123 1 3 1 3171 135 172.2 130.6 -1.2 4.4 1.2 4.4176 140 174.2 137.4 1.8 2.6 1.8 2.6177 142 175.4 141 1.6 1 1.6 1176 144 175.2 142.2 0.8 1.8 0.8 1.8177 144 172.8 141.8 4.2 2.2 4.2 2.2170 141 168 140.2 2 0.8 2 0.8164 138 162.8 137.2 1.2 0.8 1.2 0.8153 134 156.2 133.6 -3.2 0.4 3.2 0.4150 129 150.6 130.8 -0.6 -1.8 0.6 1.8144 126 146.4 128.6 -2.4 -2.6 2.4 2.6142 127 143.8 128.2 -1.8 -1.2 1.8 1.2143 127 140.4 130 2.6 -3 2.6 3140 132 136.4 134.2 3.6 -2.2 3.6 2.2133 138 130.8 139 2.2 -1 2.2 1124 147 124.2 143.8 -0.2 3.2 0.2 3.2114 151 118.4 147.6 -4.4 3.4 4.4 3.4110 151 114.4 150 -4.4 1 4.4 1111 151 112.6 149 -1.6 2 1.6 2113 150 113.4 144.4 -0.4 5.6 0.4 5.6115 142 115.2 139 -0.2 3 0.2 3118 128 116.8 133.6 1.2 -5.6 1.2 5.6119 124 118.6 126 0.4 -2 0.4 2119 124 120.2 119.6 -1.2 4.4 1.2 4.4122 112 121.2 116 0.8 -4 0.8 4

Page 156: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

123 110 121.4 112.8 1.6 -2.8 1.6 2.8123 110 121.6 109.6 1.4 0.4 1.4 0.4120 108 119 106.6 1 1.4 1 1.4120 108 114.4 102.4 5.6 5.6 5.6 5.6109 97 109.8 98.2 -0.8 -1.2 0.8 1.2100 89 103 92.4 -3 -3.4 3 3.4100 89 96 86.6 4 2.4 4 2.4

86 79 91.2 83 -5.2 -4 5.2 485 79 89.6 81.2 -4.6 -2.2 4.6 2.285 79 89.2 79.6 -4.2 -0.6 4.2 0.692 80 93 79.8 -1 0.2 1 0.298 81 98.4 80 -0.4 1 0.4 1

105 80 105 80.2 0 -0.2 0 0.2112 80 111 80 1 0 1 0118 80 116.4 79.6 1.6 0.4 1.6 0.4122 79 120.4 79.4 1.6 -0.4 1.6 0.4125 79 122.8 78.8 2.2 0.2 2.2 0.2125 79 124 78.2 1 0.8 1 0.8124 77 126.2 73.8 -2.2 3.2 2.2 3.2124 77 128 67.8 -4 9.2 4 9.2133 57 130.2 60.6 2.8 -3.6 2.8 3.6134 49 132.6 53.8 1.4 -4.8 1.4 4.8136 43 135.6 45.2 0.4 -2.2 0.4 2.2136 43 136.8 40.6 -0.8 2.4 0.8 2.4139 34 137.8 37.4 1.2 -3.4 1.2 3.4139 34 138.4 35.4 0.6 -1.4 0.6 1.4139 33 139 33.4 0 -0.4 0 0.4139 33 139 33.2 0 -0.2 0 0.2139 33 139.4 32.8 -0.4 0.2 0.4 0.2139 33 139.8 32.8 -0.8 0.2 0.8 0.2141 32 140.4 33 0.6 -1 0.6 1141 33 141.3333333 33 -0.333333333 0 0.333333333 0142 34 142 34 0 0 0 0

Page 157: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

80 100 120 140 160 180 20020

40

60

80

100

120

140

160

Star 1 Trial 3 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

80 100 120 140 160 180 20020

40

60

80

100

120

140

160

Star 1 Trial 3 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 158: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 4:

Kalman Filter Data:

Kalman:

Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:87 75 87 75 0 0 0 0 1.55721393

90 76 90.33333333 76.33333333

-0.33333333

3

-0.33333333

3 0.333333333 0.33333333394 78 95.6 77.4 -1.6 0.6 1.6 0.6 Maximum:

100 79 101.2 78.2 -1.2 0.8 1.2 0.8 6107 79 107.8 78.4 -0.8 0.6 0.8 0.6115 79 114.6 78 0.4 1 0.4 1 Sum:

123 77 120.6 77.2 2.4 -0.2 2.4 0.2208.666666

7128 76 125.6 75.6 2.4 0.4 2.4 0.4130 75 129.4 72.2 0.6 2.8 0.6 2.8 RMS:

132 71 132 67.4 0 3.6 0 3.62.04425336

2134 62 134.2 61.4 -0.2 0.6 0.2 0.6136 53 136.6 53.8 -0.6 -0.8 0.6 0.8 Mean X:

139 46 138.8 46 0.2 0 0.2 01.38407960

2142 37 140.8 39.6 1.2 -2.6 1.2 2.6143 32 142.4 34.6 0.6 -2.6 0.6 2.6 Mean Y:

144 30 143.2 31 0.8 -1 0.8 11.73034825

9144 28 143.4 29.6 0.6 -1.6 0.6 1.6

143 28 143.8 30.4 -0.8 -2.4 0.8 2.4Maximum X:

143 30 144.2 33.2 -1.2 -3.2 1.2 3.2 4.6145 36 144.8 38.2 0.2 -2.2 0.2 2.2

146 44 146 45 0 -1 0 1Maximum Y:

147 53 147.6 53 -0.6 0 0.6 0 6149 62 149 60.8 0 1.2 0 1.2151 70 150.4 67.2 0.6 2.8 0.6 2.8 Sum X:

152 75 152.8 71.8 -0.8 3.2 0.8 3.292.7333333

3153 76 156 74.6 -3 1.4 3 1.4159 76 160.4 75.6 -1.4 0.4 1.4 0.4 Sum Y:

165 76 166.2 75.6 -1.2 0.4 1.2 0.4115.933333

3173 75 173.2 75.2 -0.2 -0.2 0.2 0.2181 75 180.2 74.8 0.8 0.2 0.8 0.2 RMS X:

188 74 186.6 74.4 1.4 -0.4 1.4 0.41.79165191

5194 74 191.6 74 2.4 0 2.4 0197 74 195 73.8 2 0.2 2 0.2 RMS Y:

198 73 196 74.4 2 -1.4 2 1.42.26890436

8198 74 194.4 76.4 3.6 -2.4 3.6 2.4193 77 190.6 79.6 2.4 -2.6 2.4 2.6186 84 185.2 84 0.8 0 0.8 0178 90 178.8 89.2 -0.8 0.8 0.8 0.8171 95 172.8 95.2 -1.8 -0.2 1.8 0.2166 100 168.2 101.4 -2.2 -1.4 2.2 1.4163 107 166 109 -3 -2 3 2163 115 166 117.6 -3 -2.6 3 2.6167 128 167.4 126.2 -0.4 1.8 0.4 1.8171 138 169.4 133.6 1.6 4.4 1.6 4.4173 143 170.8 138.6 2.2 4.4 2.2 4.4173 144 169.8 140 3.2 4 3.2 4

Page 159: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

170 140 166.4 138.2 3.6 1.8 3.6 1.8162 135 160.8 134 1.2 1 1.2 1154 129 154 129 0 0 0 0145 122 147.2 124.6 -2.2 -2.6 2.2 2.6139 119 141.4 122 -2.4 -3 2.4 3136 118 136 122.2 0 -4.2 0 4.2133 122 131 125.4 2 -3.4 2 3.4127 130 125.6 130.4 1.4 -0.4 1.4 0.4120 138 120 136.2 0 1.8 0 1.8112 144 115.4 140.4 -3.4 3.6 3.4 3.6108 147 112.6 141 -4.6 6 4.6 6110 143 112 137.2 -2 5.8 2 5.8113 133 113.8 129.8 -0.8 3.2 0.8 3.2117 119 117 120.2 0 -1.2 0 1.2121 107 119.4 110.4 1.6 -3.4 1.6 3.4124 99 120.4 102 3.6 -3 3.6 3122 94 119.2 95.6 2.8 -1.6 2.8 1.6118 91 115.6 90.2 2.4 0.8 2.4 0.8111 87 109.8 85.6 1.2 1.4 1.2 1.4103 80 103 81 0 -1 0 1

95 76 95 76 0 0 0 0

Page 160: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:88 76 88 76 0 0 0 0 1.80348258790 76 90.66666667 77 -0.666666667 -1 0.666666667 194 79 96.6 77.8 -2.6 1.2 2.6 1.2 Maximum:

102 79 102.4 78.2 -0.4 0.8 0.4 0.8 6.6109 79 109.2 78.2 -0.2 0.8 0.2 0.8117 78 116 77.6 1 0.4 1 0.4 Sum:124 76 121 76.8 3 -0.8 3 0.8 241.6666667128 76 125.2 75 2.8 1 2.8 1127 75 128.6 70.6 -1.6 4.4 1.6 4.4 RMS:130 70 131 65.6 -1 4.4 1 4.4 2.34006846134 56 133.2 59.6 0.8 -3.6 0.8 3.6136 51 136.4 51.6 -0.4 -0.6 0.4 0.6 Mean X:139 46 138.8 44.4 0.2 1.6 0.2 1.6 1.617910448143 35 140.8 40 2.2 -5 2.2 5142 34 142.2 36 -0.2 -2 0.2 2 Mean Y:144 34 142.8 33 1.2 1 1.2 1 1.989054726143 31 143 32.6 0 -1.6 0 1.6142 31 143.8 34.2 -1.8 -3.2 1.8 3.2 Maximum X:144 33 144.2 37.4 -0.2 -4.4 0.2 4.4 5.6146 42 145.2 42.6 0.8 -0.6 0.8 0.6146 50 147 49.2 -1 0.8 1 0.8 Maximum Y:148 57 148.6 56.8 -0.6 0.2 0.6 0.2 6.6151 64 149.6 63.2 1.4 0.8 1.4 0.8152 71 151.2 67.8 0.8 3.2 0.8 3.2 Sum X:151 74 154 71 -3 3 3 3 108.4154 73 157.4 73 -3.4 0 3.4 0162 73 162.2 73.6 -0.2 -0.6 0.2 0.6 Sum Y:168 74 168.6 73.6 -0.6 0.4 0.6 0.4 133.2666667176 74 175.6 73.8 0.4 0.2 0.4 0.2183 74 182 74 1 0 1 0 RMS X:189 74 187.4 74 1.6 0 1.6 0 2.034266974194 74 191.4 74 2.6 0 2.6 0195 74 193.8 74.2 1.2 -0.2 1.2 0.2 RMS Y:196 74 193.8 75.4 2.2 -1.4 2.2 1.4 2.610287086195 75 191.2 78.4 3.8 -3.4 3.8 3.4189 80 187 82.2 2 -2.2 2 2.2181 89 181.8 86.6 -0.8 2.4 0.8 2.4174 93 176 91.8 -2 1.2 2 1.2170 96 171.4 97.6 -1.4 -1.6 1.4 1.6166 101 168.4 103.4 -2.4 -2.4 2.4 2.4166 109 168 111.4 -2 -2.4 2 2.4166 118 169 120.2 -3 -2.2 3 2.2172 133 170.6 128 1.4 5 1.4 5175 140 172 134 3 6 3 6174 140 172.2 137.2 1.8 2.8 1.8 2.8173 139 169.4 136.6 3.6 2.4 3.6 2.4167 134 164.4 133.6 2.6 0.4 2.6 0.4158 130 158.2 129.6 -0.2 0.4 0.2 0.4150 125 151.4 125.8 -1.4 -0.8 1.4 0.8143 120 145.8 123 -2.8 -3 2.8 3139 120 141.2 122.6 -2.2 -2.6 2.2 2.6139 120 136.6 125 2.4 -5 2.4 5135 128 131.8 129.4 3.2 -1.4 3.2 1.4127 137 126.2 134.6 0.8 2.4 0.8 2.4119 142 120.4 139.8 -1.4 2.2 1.4 2.2111 146 116.6 141.8 -5.6 4.2 5.6 4.2110 146 114.8 139.4 -4.8 6.6 4.8 6.6116 138 115.2 133.2 0.8 4.8 0.8 4.8118 125 117.6 124.6 0.4 0.4 0.4 0.4121 111 120.6 115.6 0.4 -4.6 0.4 4.6123 103 121.4 107.4 1.6 -4.4 1.6 4.4

Page 161: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

125 101 120.6 101.2 4.4 -0.2 4.4 0.2120 97 118 96.8 2 0.2 2 0.2114 94 113.2 92 0.8 2 0.8 2108 89 107 87.2 1 1.8 1 1.8

99 79 100.3333333 81.66666667 -1.333333333 -2.666666667 1.333333333 2.66666666794 77 94 77 0 0 0 0

Page 162: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

80 100 120 140 160 180 200 22020

40

60

80

100

120

140

160

Star 1 Trial 4 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

80 100 120 140 160 180 200 22020

40

60

80

100

120

140

160

Star 1 Trial 4 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 163: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 5:

Kalman Filter Data:

Kalman:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:136 119 136 119 0 0 0 0 1.572972973139 122 139 121 0 1 0 1142 122 142 122.8 0 -0.8 0 0.8 Maximum:145 124 145.4 125 -0.4 -1 0.4 1 7.2148 127 149.6 127.6 -1.6 -0.6 1.6 0.6153 130 154.6 131 -1.6 -1 1.6 1 Sum:160 135 160 134.4 0 0.6 0 0.6 232.8167 139 165.6 137.2 1.4 1.8 1.4 1.8172 141 170 138.2 2 2.8 2 2.8 RMS:176 141 172.8 137.6 3.2 3.4 3.2 3.4 2.100507589175 135 173.2 133.2 1.8 1.8 1.8 1.8174 132 171.6 126.2 2.4 5.8 2.4 5.8 Mean X:169 117 168.8 118.2 0.2 -1.2 0.2 1.2 1.436936937164 106 166.2 110.8 -2.2 -4.8 2.2 4.8162 101 164.6 103.2 -2.6 -2.2 2.6 2.2 Mean Y:162 98 165 98 -3 0 3 0 1.709009009166 94 167.8 94.4 -1.8 -0.4 1.8 0.4171 91 172 90.8 -1 0.2 1 0.2 Maximum X:178 88 176.8 87.2 1.2 0.8 1.2 0.8 4183 83 181.8 83.6 1.2 -0.6 1.2 0.6186 80 186 80.2 0 -0.2 0 0.2 Maximum Y:191 76 188.2 77.4 2.8 -1.4 2.8 1.4 7.2192 74 188.2 75.4 3.8 -1.4 3.8 1.4189 74 185.4 73.8 3.6 0.2 3.6 0.2 Sum X:183 73 180 73 3 0 3 0 106.3333333172 72 173.2 72.6 -1.2 -0.6 1.2 0.6164 72 166 72 -2 0 2 0 Sum Y:158 72 159.4 71.6 -1.4 0.4 1.4 0.4 126.4666667153 71 155 70.2 -2 0.8 2 0.8150 71 152 66.6 -2 4.4 2 4.4 RMS X:150 65 150 60.4 0 4.6 0 4.6 1.765246752149 54 148.4 51.8 0.6 2.2 0.6 2.2148 41 147 42 1 -1 1 1 RMS Y:145 28 145.2 33 -0.2 -5 0.2 5 2.389177299143 22 143.6 26.4 -0.6 -4.4 0.6 4.4141 20 142 22.6 -1 -2.6 1 2.6141 21 140.4 23.8 0.6 -2.8 0.6 2.8140 22 138 29 2 -7 2 7137 34 135.4 36.2 1.6 -2.2 1.6 2.2131 48 132 45.8 -1 2.2 1 2.2128 56 128.4 56.4 -0.4 -0.4 0.4 0.4124 69 125 65 -1 4 1 4122 75 122 70.8 0 4.2 0 4.2120 77 118.2 75 1.8 2 1.8 2116 77 113.8 76.4 2.2 0.6 2.2 0.6109 77 108.2 76.4 0.8 0.6 0.8 0.6102 76 101.6 75.8 0.4 0.2 0.4 0.2

94 75 95.6 75.6 -1.6 -0.6 1.6 0.687 74 91 75.6 -4 -1.6 4 1.686 76 88.6 76.4 -2.6 -0.4 2.6 0.486 77 88.8 77.8 -2.8 -0.8 2.8 0.890 80 91.6 80.2 -1.6 -0.2 1.6 0.295 82 96 83.2 -1 -1.2 1 1.2

101 86 101.6 86.6 -0.6 -0.6 0.6 0.6108 91 107.4 90.2 0.6 0.8 0.6 0.8114 94 112.6 93.6 1.4 0.4 1.4 0.4119 98 116.6 96.8 2.4 1.2 2.4 1.2121 99 119.2 99.2 1.8 -0.2 1.8 0.2

Page 164: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

121 102 119.6 104 1.4 -2 1.4 2121 103 118.2 110.2 2.8 -7.2 2.8 7.2116 118 115.8 118.2 0.2 -0.2 0.2 0.2112 129 113 126.8 -1 2.2 1 2.2109 139 110 136.2 -1 2.8 1 2.8107 145 108.2 143.2 -1.2 1.8 1.2 1.8106 150 107.4 147.6 -1.4 2.4 1.4 2.4107 153 108 149.2 -1 3.8 1 3.8108 151 110.8 148.2 -2.8 2.8 2.8 2.8112 147 115 145.2 -3 1.8 3 1.8121 140 120.8 140.2 0.2 -0.2 0.2 0.2127 135 127.6 134.6 -0.6 0.4 0.6 0.4136 128 134.2 129.4 1.8 -1.4 1.8 1.4142 123 139.4 125.6 2.6 -2.6 2.6 2.6145 121 144.6666667 121.6666667 0.333333333 -0.666666667 0.333333333 0.666666667147 121 147 121 0 0 0 0

Page 165: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:137 120 137 120 0 0 0 0 1.95990991138 121 139 121 -1 0 1 0142 122 142.2 123 -0.2 -1 0.2 1 Maximum:145 125 145.6 125.4 -0.6 -0.4 0.6 0.4 10.6149 127 150.6 128.6 -1.6 -1.6 1.6 1.6154 132 155.8 132 -1.8 0 1.8 0 Sum:163 137 161.4 135 1.6 2 1.6 2 290.0666667168 139 166.6 137.6 1.4 1.4 1.4 1.4173 140 170.2 137.2 2.8 2.8 2.8 2.8 RMS:175 140 172 135.8 3 4.2 3 4.2 2.717240032172 130 171.2 129.6 0.8 0.4 0.8 0.4172 130 169 122.4 3 7.6 3 7.6 Mean X:164 108 166.6 115 -2.6 -7 2.6 7 1.699099099162 104 165 109.4 -3 -5.4 3 5.4163 103 164.8 102.6 -1.8 0.4 1.8 0.4 Mean Y:164 102 167 99.6 -3 2.4 3 2.4 2.220720721171 96 171 96.6 0 -0.6 0 0.6175 93 175.4 92.4 -0.4 0.6 0.4 0.6 Maximum X:182 89 179.6 88.4 2.4 0.6 2.4 0.6 5185 82 183.8 84.4 1.2 -2.4 1.2 2.4185 82 186.8 81 -1.8 1 1.8 1 Maximum Y:192 76 187.4 78.4 4.6 -2.4 4.6 2.4 10.6190 76 186 77 4 -1 4 1185 76 182.2 75.2 2.8 0.8 2.8 0.8 Sum X:178 75 176.2 74.6 1.8 0.4 1.8 0.4 125.7333333166 73 169.8 74 -3.8 -1 3.8 1162 73 163.6 73.2 -1.6 -0.2 1.6 0.2 Sum Y:158 73 158.4 72.4 -0.4 0.6 0.4 0.6 164.3333333154 72 156 70.2 -2 1.8 2 1.8152 71 153.8 65.4 -1.8 5.6 1.8 5.6 RMS X:154 62 152.2 57.8 1.8 4.2 1.8 4.2 2.047249675151 49 150.2 48.4 0.8 0.6 0.8 0.6150 35 148.6 39.2 1.4 -4.2 1.4 4.2 RMS Y:144 25 146.2 32 -2.2 -7 2.2 7 3.252007927144 25 144.4 27.4 -0.4 -2.4 0.4 2.4142 26 142.8 25.6 -0.8 0.4 0.8 0.4142 26 141 29.6 1 -3.6 1 3.6142 26 138 36 4 -10 4 10135 45 135.4 42.2 -0.4 2.8 0.4 2.8129 57 131.8 51.6 -2.8 5.4 2.8 5.4129 57 128 61 1 -4 1 4124 73 125.2 66.6 -1.2 6.4 1.2 6.4123 73 122.6 69.8 0.4 3.2 0.4 3.2121 73 118.2 73.4 2.8 -0.4 2.8 0.4116 73 113.4 73.6 2.6 -0.6 2.6 0.6107 75 107.2 73.8 -0.2 1.2 0.2 1.2100 74 100.4 74 -0.4 0 0.4 0

92 74 95.2 74.8 -3.2 -0.8 3.2 0.887 74 92 75.6 -5 -1.6 5 1.690 77 91 77 -1 0 1 091 79 92.4 79 -1.4 0 1.4 095 81 96 81.8 -1 -0.8 1 0.899 84 100.2 85 -1.2 -1 1.2 1

105 88 105.2 88.2 -0.2 -0.2 0.2 0.2111 93 110 91.6 1 1.4 1 1.4116 95 114.2 94.4 1.8 0.6 1.8 0.6119 98 117 97.2 2 0.8 2 0.8120 98 118.6 99 1.4 -1 1.4 1119 102 117.8 105.6 1.2 -3.6 1.2 3.6119 102 116 112.6 3 -10.6 3 10.6112 128 113.6 121.2 -1.6 6.8 1.6 6.8

Page 166: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

110 133 111.4 129.4 -1.4 3.6 1.4 3.6108 141 109 138.8 -1 2.2 1 2.2108 143 108.4 143.2 -0.4 -0.2 0.4 0.2107 149 108.4 146 -1.4 3 1.4 3109 150 110 146.2 -1 3.8 1 3.8110 147 113.6 144.8 -3.6 2.2 3.6 2.2116 142 118 141.6 -2 0.4 2 0.4126 136 124 136.6 2 -0.6 2 0.6129 133 130.4 131.8 -1.4 1.2 1.4 1.2139 125 136 128 3 -3 3 3142 123 139.8 125.6 2.2 -2.6 2.2 2.6144 123 143.6666667 123.3333333 0.333333333 -0.333333333 0.333333333 0.333333333145 124 145 124 0 0 0 0

Page 167: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

80 100 120 140 160 180 20020

40

60

80

100

120

140

160

Star 1 Trial 5 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

80 100 120 140 160 180 20020

40

60

80

100

120

140

160

Star 1 Trial 5 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 168: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Triangle 1:

Page 169: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 1:

Kalman Filter Data:

Kalman:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:106 122 106 122 0 0 0 0 0.731343284104 129 104.3333333 127 -0.333333333 2 0.333333333 2103 130 103.8 127.8 -0.8 2.2 0.8 2.2 Maximum:103 130 103.4 129.4 -0.4 0.6 0.4 0.6 4.4103 128 103.6 129.4 -0.6 -1.4 0.6 1.4104 130 104.2 129.4 -0.2 0.6 0.2 0.6 Sum:105 129 104.8 129.4 0.2 -0.4 0.2 0.4 98106 130 105.8 130.2 0.2 -0.2 0.2 0.2106 130 106.8 131 -0.8 -1 0.8 1 RMS:108 132 108.2 132.2 -0.2 -0.2 0.2 0.2 1.044006824109 134 109.8 133.6 -0.8 0.4 0.8 0.4112 135 112.2 135.2 -0.2 -0.2 0.2 0.2 Mean X:114 137 115.2 136.6 -1.2 0.4 1.2 0.4 0.580099502118 138 118.8 137.4 -0.8 0.6 0.8 0.6123 139 123 138 0 1 0 1 Mean Y:127 138 127.8 138.2 -0.8 -0.2 0.8 0.2 0.882587065133 138 133 138 0 0 0 0138 138 138.2 137.4 -0.2 0.6 0.2 0.6 Maximum X:144 137 143.6 136.8 0.4 0.2 0.4 0.2 2.4149 136 149 136 0 0 0 0154 135 154.4 135 -0.4 0 0.4 0 Maximum Y:160 134 159.8 134 0.2 0 0.2 0 4.4165 133 165.4 133.2 -0.4 -0.2 0.4 0.2171 132 171.2 132.4 -0.2 -0.4 0.2 0.4 Sum X:177 132 177 131.6 0 0.4 0 0.4 38.86666667183 131 182.4 131 0.6 0 0.6 0189 130 187 130.4 2 -0.4 2 0.4 Sum Y:192 130 190 129.8 2 0.2 2 0.2 59.13333333194 129 191.6 129 2.4 0 2.4 0192 129 191 127.6 1 1.4 1 1.4 RMS X:191 127 189.2 125.8 1.8 1.2 1.8 1.2 0.783621732186 123 185.8 122.6 0.2 0.4 0.2 0.4183 121 181.6 117.8 1.4 3.2 1.4 3.2 RMS Y:177 113 176.8 111.8 0.2 1.2 0.2 1.2 1.25133428171 105 172.2 105.2 -1.2 -0.2 1.2 0.2167 97 167.2 97.6 -0.2 -0.6 0.2 0.6163 90 162.6 90 0.4 0 0.4 0158 83 158.4 82.4 -0.4 0.6 0.4 0.6154 75 154.4 75 -0.4 0 0.4 0150 67 150.8 67.4 -0.8 -0.4 0.8 0.4147 60 147.4 59.8 -0.4 0.2 0.4 0.2145 52 144.6 53.2 0.4 -1.2 0.4 1.2141 45 142.2 48 -1.2 -3 1.2 3140 42 140.2 44.6 -0.2 -2.6 0.2 2.6138 41 138.6 43 -0.6 -2 0.6 2137 43 137.8 43.2 -0.8 -0.2 0.8 0.2137 44 137.2 45 -0.2 -1 0.2 1137 46 136.8 47.4 0.2 -1.4 0.2 1.4137 51 136.2 51.4 0.8 -0.4 0.8 0.4136 53 134.8 57.4 1.2 -4.4 1.2 4.4134 63 132.6 64.8 1.4 -1.8 1.4 1.8130 74 129.8 72.4 0.2 1.6 0.2 1.6126 83 126.4 81 -0.4 2 0.4 2123 89 123 88.2 0 0.8 0 0.8119 96 119.6 94.6 -0.6 1.4 0.6 1.4117 99 116.2 100.2 0.8 -1.2 0.8 1.2113 106 113 105 0 1 0 1109 111 109.8 109.8 -0.8 1.2 0.8 1.2

Page 170: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

107 113 106.6 115.2 0.4 -2.2 0.4 2.2103 120 103.6 120.2 -0.6 -0.2 0.6 0.2101 126 101.2 125 -0.2 1 0.2 1

98 131 99 129.6 -1 1.4 1 1.497 135 97.8 133 -0.8 2 0.8 296 136 97 135.2 -1 0.8 1 0.897 137 97 136.2 0 0.8 0 0.897 137 97.33333333 136.6666667 -0.333333333 0.333333333 0.333333333 0.33333333398 136 98 136 0 0 0 0

Page 171: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:

106 123 106 123 0 0 0 01.08557213

9

105 129 104.6666667 1270.33333333

3 2 0.333333333 2103 129 104.4 127.4 -1.4 1.6 1.4 1.6 Maximum:104 129 104.4 129 -0.4 0 0.4 0 7104 127 104.4 129 -0.4 -2 0.4 2106 131 105.2 129.4 0.8 1.6 0.8 1.6 Sum:

105 129 105.6 129.6 -0.6 -0.6 0.6 0.6145.466666

7107 131 106.8 131 0.2 0 0.2 0106 130 107.6 131.8 -1.6 -1.8 1.6 1.8 RMS:

110 134 109.4 133.2 0.6 0.8 0.6 0.81.51170118

2110 135 111 134.4 -1 0.6 1 0.6114 136 113.6 136 0.4 0 0.4 0 Mean X:

115 137 116.6 137 -1.6 0 1.6 00.95124378

1119 138 120.4 137.6 -1.4 0.4 1.4 0.4125 139 124.4 138 0.6 1 0.6 1 Mean Y:

129 138 129.2 138.2 -0.2 -0.2 0.2 0.21.21990049

8134 138 134.6 137.8 -0.6 0.2 0.6 0.2139 138 139.4 137.2 -0.4 0.8 0.4 0.8 Maximum X:146 136 144.4 136.6 1.6 -0.6 1.6 0.6 2.6149 136 149.8 135.8 -0.8 0.2 0.8 0.2154 135 155.2 135 -1.2 0 1.2 0 Maximum Y:161 134 160.2 134.2 0.8 -0.2 0.8 0.2 7166 134 166.2 133.6 -0.2 0.4 0.2 0.4171 132 172.2 132.8 -1.2 -0.8 1.2 0.8 Sum X:

179 133 177.8 132.2 1.2 0.8 1.2 0.863.7333333

3184 131 182.8 131.4 1.2 -0.4 1.2 0.4189 131 187 131 2 0 2 0 Sum Y:

191 130 189 130.2 2 -0.2 2 0.281.7333333

3192 130 190 129.4 2 0.6 2 0.6189 129 188.6 127.4 0.4 1.6 0.4 1.6 RMS X:

189 127 186.8 125.6 2.2 1.4 2.2 1.41.17327905

8182 121 183.4 121.6 -1.4 -0.6 1.4 0.6182 121 179.4 116.2 2.6 4.8 2.6 4.8 RMS Y:

175 110 175.2 110 -0.2 0 0.2 01.78714777

8169 102 171.6 103.8 -2.6 -1.8 2.6 1.8168 96 166.8 96.2 1.2 -0.2 1.2 0.2164 90 162.8 89 1.2 1 1.2 1158 83 159 82.2 -1 0.8 1 0.8155 74 155.2 75.2 -0.2 -1.2 0.2 1.2150 68 151.6 67.4 -1.6 0.6 1.6 0.6149 61 148.4 60.2 0.6 0.8 0.6 0.8146 51 145.6 54.4 0.4 -3.4 0.4 3.4142 47 143.4 50 -1.4 -3 1.4 3141 45 141.4 47.2 -0.4 -2.2 0.4 2.2139 46 140 46.4 -1 -0.4 1 0.4139 47 139.4 46.8 -0.4 0.2 0.4 0.2139 47 138.6 48.6 0.4 -1.6 0.4 1.6139 49 138.2 50.2 0.8 -1.2 0.8 1.2137 54 137 54.8 0 -0.8 0 0.8137 54 135 61 2 -7 2 7133 70 132.2 68.2 0.8 1.8 0.8 1.8

Page 172: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

129 78 129.4 75.2 -0.4 2.8 0.4 2.8125 85 125.8 83.6 -0.8 1.4 0.8 1.4123 89 123 88.8 0 0.2 0 0.2119 96 119.6 94.6 -0.6 1.4 0.6 1.4119 96 116.4 99.8 2.6 -3.8 2.6 3.8112 107 113.6 104.2 -1.6 2.8 1.6 2.8109 111 110.4 109.6 -1.4 1.4 1.4 1.4109 111 107 115.6 2 -4.6 2 4.6103 123 104.4 120.6 -1.4 2.4 1.4 2.4102 126 102.2 125.4 -0.2 0.6 0.2 0.6

99 132 100 130 -1 2 1 298 135 99.2 132.6 -1.2 2.4 1.2 2.498 134 98.6 134.6 -0.6 -0.6 0.6 0.699 136 98.6 135.2 0.4 0.8 0.4 0.8

99 136 99 135.6666667 00.33333333

3 0 0.33333333399 135 99 135 0 0 0 0

Page 173: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

80 100 120 140 160 180 20020

40

60

80

100

120

140

Triangle 1 Trial 1 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

80 100 120 140 160 180 20020

40

60

80

100

120

140

Triangle 1 Trial 1 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 174: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 2:

Kalman Filter Data:

Kalman:

Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:

143 40 143 40 0 0 0 01.070748

299

146 42144.6666

66741.33333

3331.333333

3330.666666

6671.333333

3330.666666

667

145 42 143.6 42.6 1.4 -0.6 1.4 0.6Maximum:

144 43 142.2 44.4 1.8 -1.4 1.8 1.4 5.4140 46 139.8 46.8 0.2 -0.8 0.2 0.8136 49 136.8 50.6 -0.8 -1.6 0.8 1.6 Sum:

134 54 133.4 55.6 0.6 -1.6 0.6 1.6104.9333

333130 61 130.2 61.4 -0.2 -0.4 0.2 0.4127 68 127.6 67.8 -0.6 0.2 0.6 0.2 RMS:

124 75 125 74 -1 1 1 11.483147

967123 81 123.2 79.6 -0.2 1.4 0.2 1.4121 85 121.4 84.4 -0.4 0.6 0.4 0.6 Mean X:

121 89 119.8 88.2 1.2 0.8 1.2 0.81.017687

075118 92 117.8 92.2 0.2 -0.2 0.2 0.2116 94 116 96.6 0 -2.6 0 2.6 Mean Y:

11310

1 113.8 101.6 -0.8 -0.6 0.8 0.61.123809

524

11210

7 111.4 107.6 0.6 -0.6 0.6 0.6

11011

4 108.8 114.2 1.2 -0.2 1.2 0.2Maximum X:

10612

2 106.6 120 -0.6 2 0.6 2 3.8

10312

7 104.2 125.4 -1.2 1.6 1.2 1.6

10213

0 102.2 129.6 -0.2 0.4 0.2 0.4Maximum Y:

10013

4 101.2 132.4 -1.2 1.6 1.2 1.6 5.4

10013

5 101 134.4 -1 0.6 1 0.6

10113

6 101.8 135.8 -0.8 0.2 0.8 0.2 Sum X:

10213

7 104.4 136.4 -2.4 0.6 2.4 0.649.86666

667

10613

7 108.6 137 -2.6 0 2.6 0

11313

7 114 137.4 -1 -0.4 1 0.4 Sum Y:

12113

8 121.2 137.8 -0.2 0.2 0.2 0.255.06666

667

12813

8 129.8 138.2 -1.8 -0.2 1.8 0.2

13813

9 139 138.4 -1 0.6 1 0.6 RMS X:

14913

9 148 138.2 1 0.8 1 0.81.328562

895

15913

8 157.2 137.6 1.8 0.4 1.8 0.4166 13 165.8 136.6 0.2 0.4 0.2 0.4 RMS Y:

Page 175: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

7

17413

5 173.2 135.2 0.8 -0.2 0.8 0.21.623076

22

18113

4 179 133.8 2 0.2 2 0.2

18613

2 183.4 131.6 2.6 0.4 2.6 0.4

18813

1 185.2 127.4 2.8 3.6 2.8 3.6

18812

6 184.2 120.6 3.8 5.4 3.8 5.4

18311

4 181 112.4 2 1.6 2 1.6

17610

0 176.4 102.8 -0.4 -2.8 0.4 2.8170 91 170.6 92 -0.6 -1 0.6 1165 83 164.8 81.2 0.2 1.8 0.2 1.8159 72 159.4 71 -0.4 1 0.4 1154 60 154.2 61.2 -0.2 -1.2 0.2 1.2149 49 149.4 52.4 -0.4 -3.4 0.4 3.4144 42 145.8 46 -1.8 -4 1.8 4141 39 143 42.2 -2 -3.2 2 3.2

141 40140.6666

667 400.333333

333 00.333333

333 0140 41 140 41 0 0 0 0

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:144 41 144 41 0 0 0 0 1.33537415

145 42 144.3333333 41.666666670.66666666

70.33333333

3 0.666666667 0.333333333144 42 143 43.2 1 -1.2 1 1.2 Maximum:143 44 141.2 45.2 1.8 -1.2 1.8 1.2 6139 47 139 48.2 0 -1.2 0 1.2135 51 136.2 52.6 -1.2 -1.6 1.2 1.6 Sum:

134 57 133 58 1 -1 1 1130.866666

7130 64 130.2 63.8 -0.2 0.2 0.2 0.2127 71 128.2 69.8 -1.2 1.2 1.2 1.2 RMS:

125 76 125.8 75.2 -0.8 0.8 0.8 0.81.87736245

7125 81 124.2 80 0.8 1 0.8 1122 84 122.2 84.2 -0.2 -0.2 0.2 0.2 Mean X:122 88 120.6 87.4 1.4 0.6 1.4 0.6 1.22585034117 92 118.2 92.2 -1.2 -0.2 1.2 0.2117 92 116.6 97 0.4 -5 0.4 5 Mean Y:

113 105 114.2 102.6 -1.2 2.4 1.2 2.41.44489795

9114 108 111.8 109 2.2 -1 2.2 1

110 116 109.2 115.8 0.8 0.2 0.8 0.2Maximum X:

105 124 107.2 120.6 -2.2 3.4 2.2 3.4 3.8104 126 104.6 125.6 -0.6 0.4 0.6 0.4

103 129 103 129.2 0 -0.2 0 0.2Maximum Y:

101 133 102.6 131.2 -1.6 1.8 1.6 1.8 6102 134 102.6 133.2 -0.6 0.8 0.6 0.8

Page 176: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

103 134 103.8 134.6 -0.8 -0.6 0.8 0.6 Sum X:

104 136 107.2 135.4 -3.2 0.6 3.2 0.660.0666666

7109 136 111.6 136.2 -2.6 -0.2 2.6 0.2118 137 117 137 1 0 1 0 Sum Y:124 138 124.6 137.8 -0.6 0.2 0.6 0.2 70.8130 138 133.2 138.2 -3.2 -0.2 3.2 0.2142 140 141.6 138.4 0.4 1.6 0.4 1.6 RMS X:

152 138 149.8 138 2.2 0 2.2 01.54235735

6160 138 158.8 137.2 1.2 0.8 1.2 0.8165 136 166.4 136 -1.4 0 1.4 0 RMS Y:

175 134 172.8 135 2.2 -1 2.2 12.16104451

1180 134 178 133.6 2 0.4 2 0.4184 133 181.8 131 2.2 2 2.2 2186 131 182.2 125.8 3.8 5.2 3.8 5.2184 123 180.6 117.8 3.4 5.2 3.4 5.2177 108 177.2 109.4 -0.2 -1.4 0.2 1.4172 94 172.8 100 -0.8 -6 0.8 6167 91 167.8 89.6 -0.8 1.4 0.8 1.4164 84 163.2 79.6 0.8 4.4 0.8 4.4159 71 158.6 70.6 0.4 0.4 0.4 0.4154 58 154.2 61.4 -0.2 -3.4 0.2 3.4149 49 150 53.6 -1 -4.6 1 4.6145 45 147 48.6 -2 -3.6 2 3.6143 45 144.6 46 -1.6 -1 1.6 1

144 46 143 45.33333333 10.66666666

7 1 0.666666667142 45 142 45 0 0 0 0

Page 177: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

80 100 120 140 160 180 20020

40

60

80

100

120

140

Triangle 1 Trial 2 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

80 100 120 140 160 180 20020

40

60

80

100

120

140

Triangle 1 Trial 2 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 178: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 3:

Kalman Filter Data:

Kalman:

Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:

134 26 134 26 0 0 0 01.17777777

8

138 30 136.6666667 29.333333331.33333333

30.66666666

7 1.333333333 0.666666667138 32 137 32 1 0 1 0 Maximum:139 33 137 35.6 2 -2.6 2 2.6 6.2136 39 135.6 40 0.4 -1 0.4 1134 44 133.6 45.6 0.4 -1.6 0.4 1.6 Sum:

131 52 130.6 53.2 0.4 -1.2 0.4 1.298.9333333

3128 60 127.6 61.6 0.4 -1.6 0.4 1.6124 71 124.2 70.6 -0.2 0.4 0.2 0.4 RMS:

121 81 120.8 79.4 0.2 1.6 0.2 1.61.67527933

4117 89 117 88 0 1 0 1114 96 113.6 95 0.4 1 0.4 1 Mean X:

109 103 109.8 102.6 -0.8 0.4 0.8 0.41.05238095

2107 106 106.2 110.8 0.8 -4.8 0.8 4.8102 119 103 119.2 -1 -0.2 1 0.2 Mean Y:

99 130 100.6 126.6 -1.6 3.4 1.6 3.41.30317460

398 138 98.8 133.4 -0.8 4.6 0.8 4.697 140 98.8 137.4 -1.8 2.6 1.8 2.6 Maximum X:98 140 101.2 139 -3.2 1 3.2 1 3.4

102 139 105.4 138.8 -3.4 0.2 3.4 0.2111 138 111.4 138 -0.4 0 0.4 0 Maximum Y:119 137 118.4 137 0.6 0 0.6 0 6.2127 136 126.2 136.4 0.8 -0.4 0.8 0.4133 135 133.6 136 -0.6 -1 0.6 1 Sum X:141 136 141.2 135.6 -0.2 0.4 0.2 0.4 44.2148 136 148.8 135.4 -0.8 0.6 0.8 0.6157 135 156.6 135.2 0.4 -0.2 0.4 0.2 Sum Y:

165 135 164 134.6 1 0.4 1 0.454.7333333

3172 134 170.6 134 1.4 0 1.4 0178 133 175.6 132.8 2.4 0.2 2.4 0.2 RMS X:

181 133 178.8 130 2.2 3 2.2 31.38101258

8182 129 180.2 126.6 1.8 2.4 1.8 2.4181 121 179 120 2 1 2 1 RMS Y:

179 117 175.6 110.8 3.4 6.2 3.4 6.21.92507816

1172 100 170.6 100 1.4 0 1.4 0164 87 165 89 -1 -2 1 2157 75 158.8 77 -1.8 -2 1.8 2153 66 153.2 66.8 -0.2 -0.8 0.2 0.8148 57 148.4 58.2 -0.4 -1.2 0.4 1.2144 49 144.6 51.4 -0.6 -2.4 0.6 2.4140 44 140.6666667 44.66666667 -

0.66666666-

0.666666660.666666667 0.666666667

Page 179: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

7 7138 41 138 41 0 0 0 0

Page 180: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:

135 27 135 27 0 0 0 0 1.556349206

137 30 136.6666667 29.66666667 0.333333333 0.333333333 0.333333333 0.333333333

138 32 136.6 32.8 1.4 -0.8 1.4 0.8 Maximum:

138 33 136 36.6 2 -3.6 2 3.6 10

135 42 134.8 41.4 0.2 0.6 0.2 0.6

132 46 132.6 47.6 -0.6 -1.6 0.6 1.6 Sum:

131 54 129.8 55.8 1.2 -1.8 1.2 1.8 130.7333333

127 63 126.8 64 0.2 -1 0.2 1

124 74 123.8 72.6 0.2 1.4 0.2 1.4 RMS:

120 83 120.4 80.8 -0.4 2.2 0.4 2.2 2.411447566

117 89 116.8 88.8 0.2 0.2 0.2 0.2

114 95 113.8 94.6 0.2 0.4 0.2 0.4 Mean X:

109 103 110 103 -1 0 1 0 1.336507937

109 103 106.6 111.6 2.4 -8.6 2.4 8.6

101 125 103.8 120 -2.8 5 2.8 5 Mean Y:

100 132 101.8 126.6 -1.8 5.4 1.8 5.4 1.776190476

100 137 100 133 0 4 0 4

99 136 101 135.2 -2 0.8 2 0.8 Maximum X:

100 135 104.6 136 -4.6 -1 4.6 1 5

106 136 109 135.8 -3 0.2 3 0.2

118 136 115 135.8 3 0.2 3 0.2 Maximum Y:

122 136 121.8 135.8 0.2 0.2 0.2 0.2 10

129 136 129.2 136 -0.2 0 0.2 0

134 135 135.2 136 -1.2 -1 1.2 1 Sum X:

143 137 142.6 135.8 0.4 1.2 0.4 1.2 56.13333333

148 136 150 135.6 -2 0.4 2 0.4

159 135 157.6 135.4 1.4 -0.4 1.4 0.4 Sum Y:

166 135 164.4 134.8 1.6 0.2 1.6 0.2 74.6

172 134 170.4 134.4 1.6 -0.4 1.6 0.4

177 134 174.6 132.6 2.4 1.4 2.4 1.4 RMS X:

178 134 176.8 129 1.2 5 1.2 5 1.781325825

180 126 177.8 125.6 2.2 0.4 2.2 0.4

177 117 175.6 117 1.4 0 1.4 0 RMS Y:

177 117 172 107 5 10 5 10 2.908098526

166 91 167.2 96.6 -1.2 -5.6 1.2 5.6

160 84 162.6 86.8 -2.6 -2.8 2.6 2.8

156 74 157 75.2 -1 -1.2 1 1.2

154 68 152.8 67.2 1.2 0.8 1.2 0.8

149 59 149 59.8 0 -0.8 0 0.8

145 51 145.8 54 -0.8 -3 0.8 3

141 47 142 47.66666667 -1 -0.666666667 1 0.666666667

140 45 140 45 0 0 0 0

Page 181: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

80 100 120 140 160 180 20020

40

60

80

100

120

140

Triangle 1 Trial 3 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

80 100 120 140 160 180 20020

40

60

80

100

120

140

Triangle 1 Trial 3 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 182: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 4:

Kalman Filter Data:

Kalman:Moving Average X:

Moving Average Y: Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:104 136 104 136 0 0 0 0 0.934615385102 141 102.3333333 140 -0.333333333 1 0.333333333 1101 143 102.2 140.2 -1.2 2.8 1.2 2.8 Maximum:102 141 101.8 141 0.2 0 0.2 0 5.4102 140 101.8 140.6 0.2 -0.6 0.2 0.6102 140 102 139.8 0 0.2 0 0.2 Sum:102 139 102 139.4 0 -0.4 0 0.4 97.2102 139 101.8 139.2 0.2 -0.2 0.2 0.2102 139 101.4 138.6 0.6 0.4 0.6 0.4 RMS:101 139 101.2 138 -0.2 1 0.2 1 1.344059421100 137 101.6 137.2 -1.6 -0.2 1.6 0.2101 136 103.2 136.2 -2.2 -0.2 2.2 0.2 Mean X:104 135 107 135.2 -3 -0.2 3 0.2 0.815384615110 134 112.8 134.6 -2.8 -0.6 2.8 0.6120 134 120.6 134.2 -0.6 -0.2 0.6 0.2 Mean Y:129 134 129.6 133.8 -0.6 0.2 0.6 0.2 1.053846154140 134 139 133.4 1 0.6 1 0.6149 133 148 133 1 0 1 0 Maximum X:157 132 156.4 132.4 0.6 -0.4 0.6 0.4 3165 132 163.6 131.8 1.4 0.2 1.4 0.2171 131 169.6 131.2 1.4 -0.2 1.4 0.2 Maximum Y:176 131 174.6 130.4 1.4 0.6 1.4 0.6 5.4179 130 178.4 129.4 0.6 0.6 0.6 0.6182 128 181 128.2 1 -0.2 1 0.2 Sum X:184 127 182 126.4 2 0.6 2 0.6 42.4184 125 182 123.8 2 1.2 2 1.2181 122 181 121.2 0 0.8 0 0.8 Sum Y:179 117 178.4 116.6 0.6 0.4 0.6 0.4 54.8177 115 174.8 109.6 2.2 5.4 2.2 5.4171 104 170.4 100.8 0.6 3.2 0.6 3.2 RMS X:166 90 165.8 91.4 0.2 -1.4 0.2 1.4 1.099961149159 78 160.8 80.8 -1.8 -2.8 1.8 2.8156 70 156.2 71 -0.2 -1 0.2 1 RMS Y:152 62 152.2 62.8 -0.2 -0.8 0.2 0.8 1.550186093148 55 149.2 55.8 -1.2 -0.8 1.2 0.8146 49 146.8 49.8 -0.8 -0.8 0.8 0.8144 43 144.6 45 -0.6 -2 0.6 2144 40 143 41.8 1 -1.8 1 1.8141 38 141.4 40.4 -0.4 -2.4 0.4 2.4140 39 139.4 41.6 0.6 -2.6 0.6 2.6138 42 136.4 45.8 1.6 -3.8 1.6 3.8134 49 133.4 52.2 0.6 -3.2 0.6 3.2129 61 129.8 59.8 -0.8 1.2 0.8 1.2126 70 126 68.4 0 1.6 0 1.6122 77 122.6 77 -0.6 0 0.6 0119 85 119.8 85.2 -0.8 -0.2 0.8 0.2117 92 117 93.8 0 -1.8 0 1.8115 102 114.4 103 0.6 -1 0.6 1112 113 112 112 0 1 0 1109 123 109.2 121 -0.2 2 0.2 2107 130 106.3333333 130 0.666666667 0 0.666666667 0103 137 103 137 0 0 0 0

Page 183: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:

104 137 104 137 0 0 0 01.24551282

1103 141 103 140 0 1 0 1102 142 103 139.6 -1 2.4 1 2.4 Maximum:103 139 102.8 140.2 0.2 -1.2 0.2 1.2 9.2103 139 102.8 139.8 0.2 -0.8 0.2 0.8103 140 102.8 139.4 0.2 0.6 0.2 0.6 Sum:

103 139 102.6 139.6 0.4 -0.6 0.4 0.6129.533333

3102 140 102.4 139.8 -0.4 0.2 0.4 0.2102 140 101.8 138.8 0.2 1.2 0.2 1.2 RMS:

102 140 101.6 138.2 0.4 1.8 0.4 1.81.80624510

2100 135 102.6 137.4 -2.6 -2.4 2.6 2.4102 136 105 136.4 -3 -0.4 3 0.4 Mean X:

107 136 109.6 135.4 -2.6 0.6 2.6 0.61.03205128

2114 135 116.2 135.4 -2.2 -0.4 2.2 0.4125 135 124.4 135 0.6 0 0.6 0 Mean Y:

133 135 133.2 134.4 -0.2 0.6 0.2 0.61.45897435

9143 134 141.6 133.8 1.4 0.2 1.4 0.2151 133 149.6 133.4 1.4 -0.4 1.4 0.4 Maximum X:156 132 157 132.8 -1 -0.8 1 0.8 4.2165 133 163.2 132.2 1.8 0.8 1.8 0.8170 132 168.6 131.8 1.4 0.2 1.4 0.2 Maximum Y:174 131 173.6 130.8 0.4 0.2 0.4 0.2 9.2178 131 177 129.6 1 1.4 1 1.4181 127 179.4 128.4 1.6 -1.4 1.6 1.4 Sum X:

182 127 180.2 126.2 1.8 0.8 1.8 0.853.6666666

7182 126 180 123.2 2 2.8 2 2.8178 120 179.2 121 -1.2 -1 1.2 1 Sum Y:

177 116 176.4 115.2 0.6 0.8 0.6 0.875.8666666

7177 116 172.8 106.8 4.2 9.2 4.2 9.2168 98 168.8 98.2 -0.8 -0.2 0.8 0.2 RMS X:

164 84 164.8 89.2 -0.8 -5.2 0.8 5.21.36196815

9158 77 160 78.8 -2 -1.8 2 1.8157 71 156.2 70.6 0.8 0.4 0.8 0.4 RMS Y:

153 64 153 64 0 0 0 02.16103805

4149 57 150.6 57.4 -1.6 -0.4 1.6 0.4148 51 148.4 52 -0.4 -1 0.4 1146 44 146 47.2 0 -3.2 0 3.2146 44 144.4 44.6 1.6 -0.6 1.6 0.6141 40 142.4 43.8 -1.4 -3.8 1.4 3.8141 44 140 46 1 -2 1 2138 47 136.4 50.8 1.6 -3.8 1.6 3.8134 55 133.4 57.6 0.6 -2.6 0.6 2.6128 68 129.8 64.4 -1.8 3.6 1.8 3.6126 74 126 72.2 0 1.8 0 1.8123 78 122.8 79.6 0.2 -1.6 0.2 1.6119 86 120.4 87 -1.4 -1 1.4 1118 92 117.8 95.2 0.2 -3.2 0.2 3.2116 105 115 104.4 1 0.6 1 0.6

Page 184: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

113 115 112.8 113 0.2 2 0.2 2109 124 109.6 121.8 -0.6 2.2 0.6 2.2

108 129 106.3333333 129.66666671.66666666

7 -0.666666667 1.666666667 0.666666667102 136 102 136 0 0 0 0

80 100 120 140 160 180 20023

43

63

83

103

123

143

Triangle 1 Trial 4 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

Page 185: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

80 100 120 140 160 180 20023

43

63

83

103

123

143

Triangle 1 Trial 4 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Trial 5:

Trial 5:Kalman:

Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:169 39 169 39 0 0 0 0 0.716111

169 38 169.3333333 39.33333333

-0.33333333

3

-1.33333333

3 0.333333333 1.333333333

170 41 171.4 43.4 -1.4 -2.4 1.4 2.4Maximum:

173 46 173.2 47.4 -0.2 -1.4 0.2 1.4 2.8176 53 175.4 53.2 0.6 -0.2 0.6 0.2178 59 178 59.4 0 -0.4 0 0.4 Sum:180 67 180.6 65.6 -0.6 1.4 0.6 1.4 85.93333183 72 183.4 71.4 -0.4 0.6 0.4 0.6186 77 186.6 77.2 -0.6 -0.2 0.6 0.2 RMS:190 82 190 82.6 0 -0.6 0 0.6 0.967949194 88 193.8 88.6 0.2 -0.6 0.2 0.6197 94 197.8 95 -0.8 -1 0.8 1 Mean X:202 102 201.4 101.8 0.6 0.2 0.6 0.2 0.714444206 109 204.8 109 1.2 0 1.2 0208 116 208.2 116.4 -0.2 -0.4 0.2 0.4 Mean Y:211 124 211 123.2 0 0.8 0 0.8 0.717778214 131 213.2 129 0.8 2 0.8 2

216 136 214.6 133.6 1.4 2.4 1.4 2.4Maximum X:

217 138 214.4 136.8 2.6 1.2 2.6 1.2 2.8215 139 212.2 138.6 2.8 0.4 2.8 0.4

Page 186: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

210 140 207.6 139.2 2.4 0.8 2.4 0.8Maximum Y:

203 140 200.8 139 2.2 1 2.2 1 2.4193 139 192.8 138.4 0.2 0.6 0.2 0.6183 137 184.6 137.6 -1.6 -0.6 1.6 0.6 Sum X:175 136 176.6 137 -1.6 -1 1.6 1 42.86667169 136 169.6 136.6 -0.6 -0.6 0.6 0.6163 137 163.4 136.8 -0.4 0.2 0.4 0.2 Sum Y:158 137 157.4 137.4 0.6 -0.4 0.6 0.4 43.06667152 138 151.2 137.8 0.8 0.2 0.8 0.2145 139 145.4 138 -0.4 1 0.4 1 RMS X:138 138 140.2 138.2 -2.2 -0.2 2.2 0.2 0.98744134 138 135.8 138.2 -1.8 -0.2 1.8 0.2132 138 132.4 137.8 -0.4 0.2 0.4 0.2 RMS Y:130 138 130.4 137.6 -0.4 0.4 0.4 0.4 0.948058128 137 129.4 137.2 -1.4 -0.2 1.4 0.2128 137 129.2 136.2 -1.2 0.8 1.2 0.8129 136 129.8 134.4 -0.8 1.6 0.8 1.6131 133 131.4 131.6 -0.4 1.4 0.4 1.4133 129 133.4 128 -0.4 1 0.4 1136 123 135.6 123.8 0.4 -0.8 0.4 0.8138 119 137.8 119 0.2 0 0.2 0140 115 140 113.8 0 1.2 0 1.2142 109 142.2 108.6 -0.2 0.4 0.2 0.4144 103 144.6 102.8 -0.6 0.2 0.6 0.2147 97 146.8 96.2 0.2 0.8 0.2 0.8150 90 149.4 89.6 0.6 0.4 0.6 0.4151 82 152.2 82.8 -1.2 -0.8 1.2 0.8155 76 155 76 0 0 0 0158 69 157.6 69.4 0.4 -0.4 0.4 0.4161 63 160.2 63 0.8 0 0.8 0163 57 162.4 56.6 0.6 0.4 0.6 0.4164 50 164.2 50.8 -0.2 -0.8 0.2 0.8166 44 165.6 46 0.4 -2 0.4 2167 40 166.6 42.4 0.4 -2.4 0.4 2.4168 39 167.2 40.2 0.8 -1.2 0.8 1.2168 39 167.4 39.4 0.6 -0.4 0.6 0.4167 39 167.4 39.6 -0.4 -0.6 0.4 0.6167 40 167 40 0 0 0 0

167 41 166.6666667 40.666666670.33333333

30.33333333

3 0.333333333 0.333333333166 41 166 41 0 0 0 0

Page 187: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw:Moving Average X:

Moving Average Y:

Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:170 39 170 39 0 0 0 0 0.921111111169 39 170 40 -1 -1 1 1171 42 172.2 44.8 -1.2 -2.8 1.2 2.8 Maximum:174 48 173.8 49 0.2 -1 0.2 1 3.4177 56 176 55 1 1 1 1178 60 178.6 61 -0.6 -1 0.6 1 Sum:180 69 181.4 66.8 -1.4 2.2 1.4 2.2 110.5333333184 72 184.2 72 -0.2 0 0.2 0188 77 187.6 77.8 0.4 -0.8 0.4 0.8 RMS:191 82 191.2 83 -0.2 -1 0.2 1 1.180364037195 89 195 89.4 0 -0.4 0 0.4198 95 198.6 96 -0.6 -1 0.6 1 Mean X:203 104 202 103 1 1 1 1 0.883333333206 110 205.2 110.2 0.8 -0.2 0.8 0.2208 117 208.6 117.4 -0.6 -0.4 0.6 0.4 Mean Y:211 125 211.2 123.6 -0.2 1.4 0.2 1.4 0.958888889215 131 213.2 128.8 1.8 2.2 1.8 2.2216 135 214.2 132.8 1.8 2.2 1.8 2.2 Maximum X:216 136 213.2 135.4 2.8 0.6 2.8 0.6 3.4213 137 210 137 3 0 3 0206 138 204.6 137.4 1.4 0.6 1.4 0.6 Maximum Y:199 139 197.6 137.4 1.4 1.6 1.4 1.6 2.8189 137 190 137.2 -1 -0.2 1 0.2181 136 183 137 -2 -1 2 1 Sum X:175 136 176 136.8 -1 -0.8 1 0.8 53171 137 170 136.8 1 0.2 1 0.2164 138 164.4 137.6 -0.4 0.4 0.4 0.4 Sum Y:159 137 158.4 138.2 0.6 -1.2 0.6 1.2 57.53333333153 140 151.8 138.4 1.2 1.6 1.2 1.6145 139 146.2 138.4 -1.2 0.6 1.2 0.6 RMS X:138 138 141.4 138.8 -3.4 -0.8 3.4 0.8 1.14629839136 138 137.2 138.4 -1.2 -0.4 1.2 0.4135 139 134.2 138 0.8 1 0.8 1 RMS Y:132 138 132.8 137.8 -0.8 0.2 0.8 0.2 1.21347374130 137 132 137.6 -2 -0.6 2 0.6131 137 131.8 136.2 -0.8 0.8 0.8 0.8132 137 132.4 134.2 -0.4 2.8 0.4 2.8134 132 134 131 0 1 0 1135 128 135.4 127.4 -0.4 0.6 0.4 0.6138 121 137.4 123.2 0.6 -2.2 0.6 2.2138 119 139.2 118.2 -1.2 0.8 1.2 0.8142 116 141.2 113.2 0.8 2.8 0.8 2.8143 107 143.2 108.4 -0.2 -1.4 0.2 1.4145 103 145.8 102.4 -0.8 0.6 0.8 0.6148 97 147.6 95.6 0.4 1.4 0.4 1.4151 89 150.4 89.4 0.6 -0.4 0.6 0.4151 82 153.2 82.6 -2.2 -0.6 2.2 0.6157 76 155.8 76.2 1.2 -0.2 1.2 0.2159 69 158.2 69.8 0.8 -0.8 0.8 0.8161 65 160.8 63.6 0.2 1.4 0.2 1.4163 57 162.6 57.4 0.4 -0.4 0.4 0.4164 51 164.2 52 -0.2 -1 0.2 1166 45 165.6 47.4 0.4 -2.4 0.4 2.4167 42 166.4 44.4 0.6 -2.4 0.6 2.4168 42 167 42.6 1 -0.6 1 0.6167 42 167.2 42.2 -0.2 -0.2 0.2 0.2167 42 167.4 42.4 -0.4 -0.4 0.4 0.4167 43 167 42.4 0 0.6 0 0.6168 43 167 42.66666667 1 0.333333333 1 0.333333333166 42 166 42 0 0 0 0

Page 188: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

80 100 120 140 160 180 200 220 24020

40

60

80

100

120

140

Triangle 1 Trial 5 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

80 100 120 140 160 180 200 220 24020

40

60

80

100

120

140

Triangle 1 Trial 5 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 189: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Figure 8 (1):

Page 190: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 1:

Kalman Filter Data:

Kalman:

Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:76 37 76 37 0 0 0 0 1.05959596 0

74 41 73.33333333 420.66666666

7 -1 0.666666667 1 170 48 70.6 48.4 -0.6 -0.4 0.6 0.4 Maximum: 267 55 68.6 54.4 -1.6 0.6 1.6 0.6 4.8 266 61 67.2 60.2 -1.2 0.8 1.2 0.8 266 67 66.8 65.2 -0.8 1.8 0.8 1.8 Sum: 2

67 70 67.2 69.2 -0.2 0.8 0.2 0.8139.866666

7 268 73 68.6 73.2 -0.6 -0.2 0.6 0.2 269 75 71 76.6 -2 -1.6 2 1.6 RMS: 2

73 81 74 80 -1 1 1 11.36283028

9 278 84 77.4 83 0.6 1 0.6 1 282 87 81.6 85.8 0.4 1.2 0.4 1.2 Mean X: 2

85 88 86 87.2 -1 0.8 1 0.80.86767676

8 290 89 89.8 87.6 0.2 1.4 0.2 1.4 295 88 93.8 86.6 1.2 1.4 1.2 1.4 Mean Y: 2

97 86 98.2 84.8 -1.2 1.2 1.2 1.21.25151515

2 2102 82 102 82.4 0 -0.4 0 0.4 2

107 79 106 79 1 0 1 0Maximum X: 2

109 77 110.8 74.8 -1.8 2.2 1.8 2.2 4 2115 71 115.6 70.8 -0.6 0.2 0.6 0.2 2

121 65 120.6 66.8 0.4 -1.8 0.4 1.8Maximum Y: 2

126 62 126.2 63.2 -0.2 -1.2 0.2 1.2 4.8 2132 59 131.6 61.2 0.4 -2.2 0.4 2.2 2137 59 136.8 60.8 0.2 -1.8 0.2 1.8 Sum X: 2

142 61 141.6 61.6 0.4 -0.6 0.4 0.657.2666666

7 2147 63 145.8 63.6 1.2 -0.6 1.2 0.6 2150 66 149.6 66.4 0.4 -0.4 0.4 0.4 Sum Y: 2153 69 152.8 69.4 0.2 -0.4 0.2 0.4 82.6 2156 73 155.2 72.6 0.8 0.4 0.8 0.4 2158 76 157 76.4 1 -0.4 1 0.4 RMS X: 2159 79 158 81 1 -2 1 2 1.10703292 2159 85 157.8 86.6 1.2 -1.6 1.2 1.6 2158 92 156.4 92.8 1.6 -0.8 1.6 0.8 RMS Y: 2

155 101 154.4 99.4 0.6 1.6 0.6 1.61.57768530

1 2151 107 152.2 105.2 -1.2 1.8 1.2 1.8 2149 112 149 110.6 0 1.4 0 1.4 2148 114 145 114.4 3 -0.4 3 0.4 2142 119 140.8 116.8 1.2 2.2 1.2 2.2 2135 120 136 117.6 -1 2.4 1 2.4 2130 119 130.8 117 -0.8 2 0.8 2 2125 116 126.2 114 -1.2 2 1.2 2 2122 111 122.4 109.6 -0.4 1.4 0.4 1.4 2

Page 191: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

119 104 119.4 104 -0.4 0 0.4 0 2116 98 117.4 98 -1.4 0 1.4 0 2115 91 116 91.8 -1 -0.8 1 0.8 2115 86 115.2 86.4 -0.2 -0.4 0.2 0.4 2115 80 115.2 81.4 -0.2 -1.4 0.2 1.4 2115 77 115.8 77.2 -0.8 -0.2 0.8 0.2 2116 73 116.4 72.6 -0.4 0.4 0.4 0.4 2118 70 117 67.6 1 2.4 1 2.4 2118 63 117.8 61.8 0.2 1.2 0.2 1.2 2118 55 118.2 55.8 -0.2 -0.8 0.2 0.8 2119 48 117.8 49.2 1.2 -1.2 1.2 1.2 2118 43 116.8 44.6 1.2 -1.6 1.2 1.6 2116 37 115.2 41.8 0.8 -4.8 0.8 4.8 2113 40 111.2 39 1.8 1 1.8 1 2110 41 106 36.6 4 4.4 4 4.4 2

99 34 99.6 35.4 -0.6 -1.4 0.6 1.4 292 31 92.8 34 -0.8 -3 0.8 3 284 31 85.8 33.2 -1.8 -2.2 1.8 2.2 279 33 80.4 35.4 -1.4 -2.4 1.4 2.4 275 37 76 39.4 -1 -2.4 1 2.4 272 45 73 44 -1 1 1 1 270 51 70.8 48.8 -0.8 2.2 0.8 2.2 269 54 69 54 0 0 0 0 168 57 68 57 0 0 0 0 0

Page 192: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:

76 37 76 37 0 0 0 01.47727272

7

74 42 73.33333333 42.666666670.66666666

7

-0.66666666

7 0.666666667 0.66666666770 49 71.2 49.2 -1.2 -0.2 1.2 0.2 Maximum:68 56 69.6 55.2 -1.6 0.8 1.6 0.8 6.668 62 68.6 60.4 -0.6 1.6 0.6 1.668 67 68.6 65.2 -0.6 1.8 0.6 1.8 Sum:69 68 69 68.6 0 -0.6 0 0.6 19570 73 70.8 72.8 -0.8 0.2 0.8 0.270 73 73.2 76 -3.2 -3 3.2 3 RMS:

77 83 76 80 1 3 1 31.99496841

880 83 79.2 82.8 0.8 0.2 0.8 0.283 88 83.4 85.8 -0.4 2.2 0.4 2.2 Mean X:

86 87 87.4 86.6 -1.4 0.4 1.4 0.41.24949494

991 88 90.6 86.8 0.4 1.2 0.4 1.297 87 94.6 85.2 2.4 1.8 2.4 1.8 Mean Y:

96 84 99 83.4 -3 0.6 3 0.61.70505050

5103 80 102.4 81.4 0.6 -1.4 0.6 1.4

108 78 106.6 77.6 1.4 0.4 1.4 0.4Maximum X:

108 78 111.8 73.6 -3.8 4.4 3.8 4.4 5.2118 68 116.4 70.2 1.6 -2.2 1.6 2.2

122 64 121.4 66.8 0.6 -2.8 0.6 2.8Maximum Y:

126 63 127.4 63.4 -1.4 -0.4 1.4 0.4 6.6133 61 132.2 62.8 0.8 -1.8 0.8 1.8138 61 137.2 63.2 0.8 -2.2 0.8 2.2 Sum X:

142 65 141.8 64.2 0.2 0.8 0.2 0.882.4666666

7147 66 145.6 66.2 1.4 -0.2 1.4 0.2149 68 149.2 69 -0.2 -1 0.2 1 Sum Y:

152 71 152.4 71.4 -0.4 -0.4 0.4 0.4112.533333

3156 75 154.4 74.2 1.6 0.8 1.6 0.8158 77 156.2 78 1.8 -1 1.8 1 RMS X:

157 80 157 82.8 0 -2.8 0 2.81.66680807

5158 87 156.6 88.6 1.4 -1.6 1.4 1.6156 95 154.6 94.8 1.4 0.2 1.4 0.2 RMS Y:

154 104 153 101.2 1 2.8 1 2.82.27630156

6148 108 151.2 106.2 -3.2 1.8 3.2 1.8149 112 148 111 1 1 1 1149 112 143.8 113.8 5.2 -1.8 5.2 1.8140 119 140.2 115.6 -0.2 3.4 0.2 3.4133 118 135.6 115.8 -2.6 2.2 2.6 2.2130 117 130.6 114.8 -0.6 2.2 0.6 2.2126 113 126.6 111.2 -0.6 1.8 0.6 1.8124 107 123.6 106.8 0.4 0.2 0.4 0.2120 101 121 101.4 -1 -0.4 1 0.4118 96 119.4 96 -1.4 0 1.4 0117 90 117.8 90.8 -0.8 -0.8 0.8 0.8118 86 117.2 86.4 0.8 -0.4 0.8 0.4116 81 117.2 82 -1.2 -1 1.2 1117 79 117.6 78.2 -0.6 0.8 0.6 0.8118 74 117.6 73.2 0.4 0.8 0.4 0.8

Page 193: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

119 71 118.2 67.8 0.8 3.2 0.8 3.2118 61 118.6 61.6 -0.6 -0.6 0.6 0.6119 54 118.4 55.6 0.6 -1.6 0.6 1.6119 48 117.6 49 1.4 -1 1.4 1117 44 116.4 46 0.6 -2 0.6 2115 38 114.4 44.2 0.6 -6.2 0.6 6.2112 46 109.4 40.8 2.6 5.2 2.6 5.2109 45 104 38.4 5 6.6 5 6.6

94 31 97.8 37.4 -3.8 -6.4 3.8 6.490 32 91.4 35.6 -1.4 -3.6 1.4 3.684 33 85.2 34.8 -1.2 -1.8 1.2 1.880 37 81 38.6 -1 -1.6 1 1.678 41 77.4 42.8 0.6 -1.8 0.6 1.873 50 75 46.8 -2 3.2 2 3.272 53 72.8 51 -0.8 2 0.8 2

72 53 71 54.66666667 1

-1.66666666

7 1 1.66666666769 58 69 58 0 0 0 0

Page 194: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

50 70 90 110 130 150 17020

40

60

80

100

120

140

Figure 8 Trial 1 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

50 70 90 110 130 150 17020

40

60

80

100

120

140

Figure 8 Trial 1 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 195: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 2:

Kalman Filter:

Kalman:

Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:133 66 133 66 0 0 0 0 0.782564103138 64 137.6666667 65 0.333333333 -1 0.333333333 1142 65 141.2 65.2 0.8 -0.2 0.8 0.2 Maximum:145 65 144.6 65.2 0.4 -0.2 0.4 0.2 3.4148 66 147.2 65.8 0.8 0.2 0.8 0.2150 66 149.6 66.6 0.4 -0.6 0.4 0.6 Sum:151 67 151.6 67.8 -0.6 -0.8 0.6 0.8 101.7333333154 69 153.2 69.2 0.8 -0.2 0.8 0.2155 71 154.8 71.4 0.2 -0.4 0.2 0.4 RMS:156 73 156.6 74 -0.6 -1 0.6 1 1.020507667158 77 158.2 77.2 -0.2 -0.2 0.2 0.2160 80 159.8 80.8 0.2 -0.8 0.2 0.8 Mean X:162 85 161 85 1 0 1 0 0.662564103163 89 161.4 89.6 1.6 -0.6 1.6 0.6162 94 160.4 94.6 1.6 -0.6 1.6 0.6 Mean Y:160 100 158.2 99.6 1.8 0.4 1.8 0.4 0.902564103155 105 155 104.8 0 0.2 0 0.2151 110 151.2 109.8 -0.2 0.2 0.2 0.2 Maximum X:147 115 146.8 113.6 0.2 1.4 0.2 1.4 2.2143 119 142.6 116 0.4 3 0.4 3138 119 138.6 116.8 -0.6 2.2 0.6 2.2 Maximum Y:134 117 135 116.4 -1 0.6 1 0.6 3.4131 114 131.6 114.2 -0.6 -0.2 0.6 0.2129 113 128.6 110.6 0.4 2.4 0.4 2.4 Sum X:126 108 125.8 106.4 0.2 1.6 0.2 1.6 43.06666667123 101 122.8 101.6 0.2 -0.6 0.2 0.6120 96 120 95.4 0 0.6 0 0.6 Sum Y:116 90 118.2 88.8 -2.2 1.2 2.2 1.2 58.66666667115 82 117 82.4 -2 -0.4 2 0.4117 75 116.4 75.2 0.6 -0.2 0.6 0.2 RMS X:117 69 116.8 68.2 0.2 0.8 0.2 0.8 0.85415025117 60 117.4 61.4 -0.4 -1.4 0.4 1.4118 55 117.4 54.8 0.6 0.2 0.6 0.2 RMS Y:118 48 116.8 48 1.2 0 1.2 0 1.163313864117 42 115.8 42.2 1.2 -0.2 1.2 0.2114 35 113.4 36.8 0.6 -1.8 0.6 1.8112 31 110 32.6 2 -1.6 2 1.6106 28 105.6 29.4 0.4 -1.4 0.4 1.4101 27 100.4 27.8 0.6 -0.8 0.6 0.8

95 26 94.6 27.4 0.4 -1.4 0.4 1.488 27 89.4 28.4 -1.4 -1.4 1.4 1.483 29 84.6 31.4 -1.6 -2.4 1.6 2.480 33 80.8 36.4 -0.8 -3.4 0.8 3.477 42 78.4 42.8 -1.4 -0.8 1.4 0.876 51 77.2 49.8 -1.2 1.2 1.2 1.276 59 76.8 56.8 -0.8 2.2 0.8 2.277 64 77.6 63 -0.6 1 0.6 178 68 79.2 68.2 -1.2 -0.2 1.2 0.281 73 81.6 72.4 -0.6 0.6 0.6 0.684 77 84.6 75.8 -0.6 1.2 0.6 1.288 80 88.2 79 -0.2 1 0.2 192 81 92 81.4 0 -0.4 0 0.496 84 95.8 82.8 0.2 1.2 0.2 1.2

100 85 99.6 83.4 0.4 1.6 0.4 1.6103 84 103.2 83.4 -0.2 0.6 0.2 0.6107 83 106.6 82.2 0.4 0.8 0.4 0.8110 81 109.8 80 0.2 1 0.2 1113 78 113 77.6 0 0.4 0 0.4

Page 196: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

116 74 116.4 74.4 -0.4 -0.4 0.4 0.4119 72 120 70.8 -1 1.2 1 1.2124 67 123.6 67.4 0.4 -0.4 0.4 0.4128 63 127 64.6 1 -1.6 1 1.6131 61 130.4 62.4 0.6 -1.4 0.6 1.4133 60 133.3333333 60.66666667 -0.333333333 -0.666666667 0.333333333 0.666666667136 61 136 61 0 0 0 0

Page 197: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y:

DifferenceX:

DifferenceY:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:134 66 134 66 0 0 0 0 0.968205128138 65 138 65.66666667 0 -0.666666667 0 0.666666667142 66 141.4 65.8 0.6 0.2 0.6 0.2 Maximum:145 65 144.4 66 0.6 -1 0.6 1 3.6148 67 147 66.6 1 0.4 1 0.4149 67 149.4 67.4 -0.4 -0.4 0.4 0.4 Sum:151 68 151.2 68.8 -0.2 -0.8 0.2 0.8 125.8666667154 70 153 70.4 1 -0.4 1 0.4154 72 154.8 72.6 -0.8 -0.6 0.8 0.6 RMS:157 75 156.8 75.4 0.2 -0.4 0.2 0.4 1.276185556158 78 158.6 78.8 -0.6 -0.8 0.6 0.8161 82 160.4 82.2 0.6 -0.2 0.6 0.2 Mean X:163 87 161.2 86.4 1.8 0.6 1.8 0.6 0.806153846163 89 161.2 91.2 1.8 -2.2 1.8 2.2161 96 159.4 96 1.6 0 1.6 0 Mean Y:158 102 157 100.8 1 1.2 1 1.2 1.13025641152 106 153.6 106.2 -1.6 -0.2 1.6 0.2151 111 150 110.6 1 0.4 1 0.4 Maximum X:146 116 146 113.6 0 2.4 0 2.4 3143 118 142.6 115.2 0.4 2.8 0.4 2.8138 117 138.8 115.4 -0.8 1.6 0.8 1.6 Maximum Y:135 114 135.6 114.6 -0.6 -0.6 0.6 0.6 3.6132 112 132.4 112.2 -0.4 -0.2 0.4 0.2130 112 129.4 108.4 0.6 3.6 0.6 3.6 Sum X:127 106 126.4 104.6 0.6 1.4 0.6 1.4 52.4123 98 123.4 100 -0.4 -2 0.4 2120 95 120.8 93.8 -0.8 1.2 0.8 1.2 Sum Y:117 89 119.4 87.6 -2.4 1.4 2.4 1.4 73.46666667117 81 118.4 82 -1.4 -1 1.4 1120 75 118.2 74.8 1.8 0.2 1.8 0.2 RMS X:118 70 118.6 68.2 -0.6 1.8 0.6 1.8 1.027019586119 59 119 61.6 0 -2.6 0 2.6119 56 118.2 55.2 0.8 0.8 0.8 0.8 RMS Y:119 48 117.2 48.2 1.8 -0.2 1.8 0.2 1.484092286116 43 115.8 43.2 0.2 -0.2 0.2 0.2113 35 112.8 38.2 0.2 -3.2 0.2 3.2112 34 109 34.4 3 -0.4 3 0.4104 31 104.4 31.6 -0.4 -0.6 0.4 0.6100 29 99.4 30.6 0.6 -1.6 0.6 1.6

93 29 93.8 30.2 -0.8 -1.2 0.8 1.288 30 89.4 31.4 -1.4 -1.4 1.4 1.484 32 85.2 35 -1.2 -3 1.2 382 37 82.2 40.4 -0.2 -3.4 0.2 3.479 47 80.4 46.6 -1.4 0.4 1.4 0.478 56 79.4 52.8 -1.4 3.2 1.4 3.279 61 79 59 0 2 0 279 63 80 64.2 -1 -1.2 1 1.280 68 81.6 68.2 -1.6 -0.2 1.6 0.284 73 84 71.8 0 1.2 0 1.286 76 86.8 75.2 -0.8 0.8 0.8 0.891 79 90.2 78.6 0.8 0.4 0.8 0.493 80 93.6 80.6 -0.6 -0.6 0.6 0.6

Page 198: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

97 85 97 81.8 0 3.2 0 3.2101 83 100.4 82.4 0.6 0.6 0.6 0.6103 82 103.8 82.4 -0.8 -0.4 0.8 0.4108 82 107 80.8 1 1.2 1 1.2110 80 110.2 78.8 -0.2 1.2 0.2 1.2113 77 113.4 76.8 -0.4 0.2 0.4 0.2117 73 117 73.6 0 -0.6 0 0.6119 72 120.6 70.2 -1.6 1.8 1.6 1.8126 66 124.2 67.2 1.8 -1.2 1.8 1.2128 63 127.2 65.2 0.8 -2.2 0.8 2.2131 62 130.6 63.6 0.4 -1.6 0.4 1.6132 63 133 63 -1 0 1 0136 64 136 64 0 0 0 0

Page 199: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

50 70 90 110 130 150 17020

40

60

80

100

120

140

Figure 8 Trial 2 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

50 70 90 110 130 150 17020

40

60

80

100

120

140

Figure 8 Trial 2 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 200: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 3:

Kalman Filter Data:

Kalman:

Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:

85 76 85 76 0 0 0 00.89761904

8

90 82 89.33333333 80.666666670.66666666

7 1.333333333 0.666666667 1.33333333393 84 92.6 82.8 0.4 1.2 0.4 1.2 Maximum:96 86 96.4 85 -0.4 1 0.4 1 2.899 86 100.2 85.8 -1.2 0.2 1.2 0.2

104 87 104 85.8 0 1.2 0 1.2 Sum:

109 86 107.8 84.8 1.2 1.2 1.2 1.2125.666666

7112 84 111.8 83 0.2 1 0.2 1115 81 115.6 80 -0.6 1 0.6 1 RMS:

119 77 118.8 76.6 0.2 0.4 0.2 0.41.10475164

2123 72 122.4 72.6 0.6 -0.6 0.6 0.6125 69 126 68.6 -1 0.4 1 0.4 Mean X:

130 64 129 65.2 1 -1.2 1 1.20.78666666

7133 61 132.4 62.6 0.6 -1.6 0.6 1.6134 60 136.4 60.6 -2.4 -0.6 2.4 0.6 Mean Y:

140 59 140 60 0 -1 0 11.00857142

9145 59 143.6 60.4 1.4 -1.4 1.4 1.4148 61 147.6 61.6 0.4 -0.6 0.4 0.6 Maximum X:151 63 150.8 63.6 0.2 -0.6 0.2 0.6 2.4154 66 153.6 66.4 0.4 -0.4 0.4 0.4156 69 156.4 70 -0.4 -1 0.4 1 Maximum Y:159 73 158.8 74.2 0.2 -1.2 0.2 1.2 2.8162 79 160.6 78.8 1.4 0.2 1.4 0.2163 84 161.6 83.8 1.4 0.2 1.4 0.2 Sum X:

163 89 161.4 89.2 1.6 -0.2 1.6 0.255.0666666

7161 94 159.6 94.4 1.4 -0.4 1.4 0.4158 100 157.2 99.2 0.8 0.8 0.8 0.8 Sum Y:153 105 154 103.6 -1 1.4 1 1.4 70.6151 108 150.4 107.6 0.6 0.4 0.6 0.4147 111 146.6 110.6 0.4 0.4 0.4 0.4 RMS X:

143 114 143.2 112.8 -0.2 1.2 0.2 1.20.99802980

5139 115 139 114 0 1 0 1136 116 134.8 114 1.2 2 1.2 2 RMS Y:

130 114 130.8 113.2 -0.8 0.8 0.8 0.81.20203531

1126 111 127.4 111.4 -1.4 -0.4 1.4 0.4123 110 124.2 108.6 -1.2 1.4 1.2 1.4122 106 122 105.4 0 0.6 0 0.6120 102 120.2 102 -0.2 0 0.2 0119 98 118.4 98 0.6 0 0.6 0117 94 116.6 93.2 0.4 0.8 0.4 0.8114 90 115.4 87.6 -1.4 2.4 1.4 2.4113 82 114.6 80.4 -1.6 1.6 1.6 1.6114 74 114.4 72.6 -0.4 1.4 0.4 1.4115 62 114.6 64.2 0.4 -2.2 0.4 2.2116 55 114.8 56.2 1.2 -1.2 1.2 1.2115 48 114.2 48.2 0.8 -0.2 0.8 0.2114 42 112.4 41.6 1.6 0.4 1.6 0.4111 34 109 35.6 2 -1.6 2 1.6106 29 104.4 31 1.6 -2 1.6 2

Page 201: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

99 25 99 27.6 0 -2.6 0 2.692 25 93.4 26.2 -1.4 -1.2 1.4 1.287 25 87.8 27 -0.8 -2 0.8 283 27 82.6 29.8 0.4 -2.8 0.4 2.878 33 78.4 33.8 -0.4 -0.8 0.4 0.873 39 75.2 39.6 -2.2 -0.6 2.2 0.671 45 73.2 46.4 -2.2 -1.4 2.2 1.471 54 72.6 52.8 -1.6 1.2 1.6 1.273 61 74 59.4 -1 1.6 1 1.675 65 76.6 65.8 -1.6 -0.8 1.6 0.880 72 80.6 71 -0.6 1 0.6 184 77 85.2 75.2 -1.2 1.8 1.2 1.891 80 90.6 78.4 0.4 1.6 0.4 1.696 82 96.2 79.8 -0.2 2.2 0.2 2.2

102 81 102 79.8 0 1.2 0 1.2108 79 107.8 78.2 0.2 0.8 0.2 0.8113 77 113.6 75.6 -0.6 1.4 0.6 1.4120 72 119.2 72.2 0.8 -0.2 0.8 0.2125 69 124.6 68.6 0.4 0.4 0.4 0.4130 64 130 64.66666667 0 -0.666666667 0 0.666666667135 61 135 61 0 0 0 0

Page 202: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y:

DifferenceX:

DifferenceY:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:86 77 86 77 0 0 0 0 1.13619047691 82 90 81 1 1 1 193 84 93.2 82.8 -0.2 1.2 0.2 1.2 Maximum:96 85 97.2 84.8 -1.2 0.2 1.2 0.2 4.2

100 86 101 85.4 -1 0.6 1 0.6106 87 104.8 85.2 1.2 1.8 1.2 1.8 Sum:110 85 108.6 84.2 1.4 0.8 1.4 0.8 159.0666667112 83 112.6 82 -0.6 1 0.6 1115 80 116 78.8 -1 1.2 1 1.2 RMS:120 75 119 75.6 1 -0.6 1 0.6 1.422695854123 71 122.8 71.6 0.2 -0.6 0.2 0.6125 69 126.4 68 -1.4 1 1.4 1 Mean X:131 63 129 65.4 2 -2.4 2 2.4 1.079047619133 62 132.8 63.4 0.2 -1.4 0.2 1.4133 62 137 61.8 -4 0.2 4 0.2 Mean Y:142 61 140.4 61.8 1.6 -0.8 1.6 0.8 1.193333333146 61 143.8 62.4 2.2 -1.4 2.2 1.4148 63 148 63.8 0 -0.8 0 0.8 Maximum X:150 65 150.8 65.8 -0.8 -0.8 0.8 0.8 4154 69 153.4 68.4 0.6 0.6 0.6 0.6156 71 156.4 72.2 -0.4 -1.2 0.4 1.2 Maximum Y:159 74 158.8 76.2 0.2 -2.2 0.2 2.2 4.2163 82 160.2 80.4 2.8 1.6 2.8 1.6162 85 160.8 85.2 1.2 -0.2 1.2 0.2 Sum X:161 90 160.2 90.8 0.8 -0.8 0.8 0.8 75.53333333159 95 157.8 95.4 1.2 -0.4 1.2 0.4156 102 155.6 99.8 0.4 2.2 0.4 2.2 Sum Y:151 105 152.8 104 -1.8 1 1.8 1 83.53333333151 107 149.6 107.6 1.4 -0.6 1.4 0.6147 111 146.2 110 0.8 1 0.8 1 RMS X:143 113 143.2 112 -0.2 1 0.2 1 1.378358812139 114 138.8 112.8 0.2 1.2 0.2 1.2136 115 134.8 112.6 1.2 2.4 1.2 2.4 RMS Y:129 111 131.2 111.8 -2.2 -0.8 2.2 0.8 1.465692317127 110 128.2 110 -1.2 0 1.2 0125 109 125.2 107 -0.2 2 0.2 2124 105 123.4 104.4 0.6 0.6 0.6 0.6121 100 121.6 101.2 -0.6 -1.2 0.6 1.2120 98 119.4 97.4 0.6 0.6 0.6 0.6118 94 117.4 92.2 0.6 1.8 0.6 1.8114 90 116.6 86.6 -2.6 3.4 2.6 3.4114 79 115.8 78.8 -1.8 0.2 1.8 0.2117 72 115.8 71.4 1.2 0.6 1.2 0.6116 59 115.8 63.2 0.2 -4.2 0.2 4.2118 57 116 56 2 1 2 1114 49 114.4 48.4 -0.4 0.6 0.4 0.6115 43 112 42.8 3 0.2 3 0.2109 34 107.8 37 1.2 -3 1.2 3104 31 103 33 1 -2 1 2

97 28 97.6 30.2 -0.6 -2.2 0.6 2.290 29 92.6 29.6 -2.6 -0.6 2.6 0.688 29 87.4 31 0.6 -2 0.6 284 31 82.8 34 1.2 -3 1.2 378 38 79.4 37.8 -1.4 0.2 1.4 0.274 43 76.6 43.4 -2.6 -0.4 2.6 0.473 48 75.2 49.6 -2.2 -1.6 2.2 1.674 57 75 55 -1 2 1 277 62 77 61 0 1 0 177 65 79.8 66.8 -2.8 -1.8 2.8 1.884 73 83.8 71 0.2 2 0.2 287 77 88 74.6 -1 2.4 1 2.4

Page 203: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

94 78 93.4 77.4 0.6 0.6 0.6 0.698 80 98.4 78.2 -0.4 1.8 0.4 1.8

104 79 103.8 77.8 0.2 1.2 0.2 1.2109 77 109.4 76.2 -0.4 0.8 0.4 0.8114 75 114.6 73.8 -0.6 1.2 0.6 1.2122 70 119.8 70.6 2.2 -0.6 2.2 0.6124 68 125 67.6 -1 0.4 1 0.4130 63 129.6666667 64.33333333 0.333333333 -1.333333333 0.333333333 1.333333333135 62 135 62 0 0 0 0

Page 204: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

50 70 90 110 130 150 17020

40

60

80

100

120

140

Figure 8 Trial 3 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

50 70 90 110 130 150 17020

40

60

80

100

120

140

Figure 8 Trial 3 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 205: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 4:

Kalman Filter Data:

Kalman:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:103 83 103 83 0 0 0 0 0.920187793109 80 109 79.33333333 0 0.666666667 0 0.666666667115 75 114.2 75.4 0.8 -0.4 0.8 0.4 Maximum:120 71 119.2 71.8 0.8 -0.8 0.8 0.8 3124 68 123.8 68 0.2 0 0.2 0128 65 128 64.6 0 0.4 0 0.4 Sum:132 61 132.2 61.8 -0.2 -0.8 0.2 0.8 130.6666667136 58 136.4 59.6 -0.4 -1.6 0.4 1.6141 57 140 58.4 1 -1.4 1 1.4 RMS:145 57 143.4 58.6 1.6 -1.6 1.6 1.6 1.144244631146 59 146.4 60.2 -0.4 -1.2 0.4 1.2149 62 148.6 62.8 0.4 -0.8 0.4 0.8 Mean X:151 66 150.4 66.2 0.6 -0.2 0.6 0.2 0.792488263152 70 152.4 70.2 -0.4 -0.2 0.4 0.2154 74 154.4 74.6 -0.4 -0.6 0.4 0.6 Mean Y:156 79 156 79.6 0 -0.6 0 0.6 1.047887324159 84 157 85.6 2 -1.6 2 1.6159 91 157.2 92.2 1.8 -1.2 1.8 1.2 Maximum X:157 100 156 99 1 1 1 1 2.2155 107 152.8 106 2.2 1 2.2 1150 113 148.4 112.2 1.6 0.8 1.6 0.8 Maximum Y:143 119 143.2 116.8 -0.2 2.2 0.2 2.2 3137 122 137.4 119.4 -0.4 2.6 0.4 2.6131 123 131.6 120 -0.6 3 0.6 3 Sum X:126 120 126.6 119 -0.6 1 0.6 1 56.26666667121 116 122.4 115.8 -1.4 0.2 1.4 0.2118 114 118.6 111 -0.6 3 0.6 3 Sum Y:116 106 115.2 105.2 0.8 0.8 0.8 0.8 74.4112 99 112.8 98.6 -0.8 0.4 0.8 0.4109 91 111 90.6 -2 0.4 2 0.4 RMS X:109 83 109.6 83.2 -0.6 -0.2 0.6 0.2 0.994381714109 74 109.4 76.2 -0.4 -2.2 0.4 2.2109 69 110.4 70 -1.4 -1 1.4 1 RMS Y:111 64 111.8 64.8 -0.8 -0.8 0.8 0.8 1.276634857114 60 113.4 60.4 0.6 -0.4 0.6 0.4116 57 114.8 56.4 1.2 0.6 1.2 0.6117 52 115.6 52.8 1.4 -0.8 1.4 0.8116 49 115 48.8 1 0.2 1 0.2115 46 113 44.6 2 1.4 2 1.4111 40 110 40.8 1 -0.8 1 0.8106 36 106 37.4 0 -1.4 0 1.4102 33 100.8 34.4 1.2 -1.4 1.2 1.4

96 32 95.4 32.8 0.6 -0.8 0.6 0.889 31 90.4 32 -1.4 -1 1.4 184 32 85.4 32.6 -1.4 -0.6 1.4 0.681 32 81 34.4 0 -2.4 0 2.477 36 77.6 37.2 -0.6 -1.2 0.6 1.274 41 75 40.6 -1 0.4 1 0.472 45 73 45.4 -1 -0.4 1 0.471 49 72.2 50.8 -1.2 -1.8 1.2 1.871 56 72.4 56.2 -1.4 -0.2 1.4 0.273 63 73.6 62 -0.6 1 0.6 175 68 75.6 67.8 -0.6 0.2 0.6 0.278 74 78.6 72.6 -0.6 1.4 0.6 1.481 78 82 76.4 -1 1.6 1 1.686 80 86 79.4 0 0.6 0 0.690 82 90.4 81 -0.4 1 0.4 195 83 95.4 81.4 -0.4 1.6 0.4 1.6

Page 206: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

100 82 100.2 80.8 -0.2 1.2 0.2 1.2106 80 105.8 78.8 0.2 1.2 0.2 1.2110 77 111.8 75.4 -1.8 1.6 1.8 1.6118 72 118 71.4 0 0.6 0 0.6125 66 124.4 67.4 0.6 -1.4 0.6 1.4131 62 131.2 64.2 -0.2 -2.2 0.2 2.2138 60 137.8 62.8 0.2 -2.8 0.2 2.8144 61 143.6 63.4 0.4 -2.4 0.4 2.4151 65 148.8 66 2.2 -1 2.2 1154 69 152.8 69.8 1.2 -0.8 1.2 0.8157 75 155.4 74 1.6 1 1.6 1158 79 157.3333333 78.66666667 0.666666667 0.333333333 0.666666667 0.333333333157 82 157 82 0 0 0 0

Page 207: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y:

DifferenceX:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:104 84 104 84 0 0 0 0 1.11971831

109 80 109.6666667 79.66666667 -0.6666666670.33333333

3 0.666666667 0.333333333116 75 114.6 76 1.4 -1 1.4 1 Maximum:120 71 119.4 72.4 0.6 -1.4 0.6 1.4 5124 70 124 68.6 0 1.4 0 1.4128 66 128.2 65.2 -0.2 0.8 0.2 0.8 Sum:132 61 132.4 62.8 -0.4 -1.8 0.4 1.8 159137 58 136.6 60.6 0.4 -2.6 0.4 2.6141 59 140 59.8 1 -0.8 1 0.8 RMS:

145 59 143.4 60.6 1.6 -1.6 1.6 1.61.42491797

5145 62 146.2 62.8 -1.2 -0.8 1.2 0.8149 65 148.2 65.4 0.8 -0.4 0.8 0.4 Mean X:

151 69 150.2 68.8 0.8 0.2 0.8 0.20.98403755

9151 72 152.6 72.4 -1.6 -0.4 1.6 0.4155 76 154.6 76.6 0.4 -0.6 0.4 0.6 Mean Y:

157 80 156.2 81.6 0.8 -1.6 0.8 1.61.25539906

1159 86 157 87.8 2 -1.8 2 1.8159 94 156.6 94.2 2.4 -0.2 2.4 0.2 Maximum X:155 103 154.8 100.8 0.2 2.2 0.2 2.2 2.6153 108 151 107.4 2 0.6 2 0.6148 113 146.4 112.6 1.6 0.4 1.6 0.4 Maximum Y:140 119 141.6 116 -1.6 3 1.6 3 5136 120 136.2 117.6 -0.2 2.4 0.2 2.4131 120 130.8 117.6 0.2 2.4 0.2 2.4 Sum X:

126 116 127 116.4 -1 -0.4 1 0.469.8666666

7121 113 123.2 112.8 -2.2 0.2 2.2 0.2121 113 119.6 108 1.4 5 1.4 5 Sum Y:

117 102 116.4 102.8 0.6 -0.8 0.6 0.889.1333333

3113 96 114.4 96.4 -1.4 -0.4 1.4 0.4110 90 112.4 88.4 -2.4 1.6 2.4 1.6 RMS X:

111 81 111 82.2 0 -1.2 0 1.21.22350480

4111 73 111.2 76.2 -0.2 -3.2 0.2 3.2110 71 112.6 70.6 -2.6 0.4 2.6 0.4 RMS Y:

114 66 113.6 66.2 0.4 -0.2 0.4 0.21.60119282

6117 62 114.8 62 2.2 0 2.2 0116 59 115.8 57.8 0.2 1.2 0.2 1.2117 52 115.8 54 1.2 -2 1.2 2115 50 114.2 49.6 0.8 0.4 0.8 0.4114 47 111.8 45 2.2 2 2.2 2109 40 108.6 41.6 0.4 -1.6 0.4 1.6104 36 104.6 38.4 -0.6 -2.4 0.6 2.4101 35 99.4 35.6 1.6 -0.6 1.6 0.6

95 34 94.6 34.4 0.4 -0.4 0.4 0.488 33 90.4 34 -2.4 -1 2.4 185 34 85.6 34.8 -0.6 -0.8 0.6 0.883 34 81.8 37 1.2 -3 1.2 377 39 79 39.8 -2 -0.8 2 0.876 45 76.6 43 -0.6 2 0.6 274 47 74.8 48.2 -0.8 -1.2 0.8 1.273 50 74.6 53.2 -1.6 -3.2 1.6 3.274 60 74.8 58 -0.8 2 0.8 276 64 76 63.6 0 0.4 0 0.477 69 78 69 -1 0 1 0

Page 208: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

80 75 80.8 72.8 -0.8 2.2 0.8 2.283 77 84 76 -1 1 1 188 79 88 78.6 0 0.4 0 0.492 80 92.2 79.6 -0.2 0.4 0.2 0.497 82 97 79.8 0 2.2 0 2.2

101 80 101.6 79 -0.6 1 0.6 1107 78 107.2 77 -0.2 1 0.2 1111 75 113 73.4 -2 1.6 2 1.6120 70 119 69.8 1 0.2 1 0.2126 64 125.4 66.6 0.6 -2.6 0.6 2.6131 62 132.2 64.6 -1.2 -2.6 1.2 2.6139 62 138.4 64.4 0.6 -2.4 0.6 2.4145 65 143.8 66.2 1.2 -1.2 1.2 1.2151 69 148.6 69.4 2.4 -0.4 2.4 0.4153 73 152 73.2 1 -0.2 1 0.2155 78 153.8 76.4 1.2 1.6 1.2 1.6156 81 155 80 1 1 1 1154 81 154 81 0 0 0 0

Page 209: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

50 70 90 110 130 150 17020

40

60

80

100

120

140

Figure 8 Trial 4 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

50 70 90 110 130 150 17020

40

60

80

100

120

140

Figure 8 Trial 4 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 210: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 5:

Kalman Filter Data:

Kalman:

Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:

106 30 106 30 0 0 0 00.71421568

6

103 29 102.6666667 290.33333333

3 0 0.333333333 099 28 98.8 28.4 0.2 -0.4 0.2 0.4 Maximum:95 28 95.4 28 -0.4 0 0.4 0 2.891 27 92.2 28 -1.2 -1 1.2 189 28 89.4 28.4 -0.4 -0.4 0.4 0.4 Sum:

87 29 86.8 28.8 0.2 0.2 0.2 0.297.1333333

385 30 84.4 29.8 0.6 0.2 0.6 0.282 30 81.8 31 0.2 -1 0.2 1 RMS:

79 32 79 32.6 0 -0.6 0 0.60.91052308

576 34 76.2 34.8 -0.2 -0.8 0.2 0.873 37 73.4 37.8 -0.4 -0.8 0.4 0.8 Mean X:

71 41 71 40.8 0 0.2 0 0.20.69019607

868 45 69.2 44.8 -1.2 0.2 1.2 0.267 47 68.4 49.4 -1.4 -2.4 1.4 2.4 Mean Y:

67 54 68.2 54 -1.2 0 1.2 00.73823529

469 60 69 58.8 0 1.2 0 1.270 64 70.2 63.8 -0.2 0.2 0.2 0.2 Maximum X:72 69 72.2 68 -0.2 1 0.2 1 2.473 72 74.4 71.6 -1.4 0.4 1.4 0.477 75 77.4 75 -0.4 0 0.4 0 Maximum Y:80 78 81 78 -1 0 1 0 2.885 81 85.6 80.8 -0.6 0.2 0.6 0.290 84 90.2 82.6 -0.2 1.4 0.2 1.4 Sum X:

96 86 95 83.2 1 2.8 1 2.846.9333333

3100 84 99.4 82.6 0.6 1.4 0.6 1.4104 81 103.6 81 0.4 0 0.4 0 Sum Y:107 78 107.4 78.4 -0.4 -0.4 0.4 0.4 50.2111 76 110.8 75.2 0.2 0.8 0.2 0.8115 73 114.4 71.8 0.6 1.2 0.6 1.2 RMS X:

117 68 118.4 68.4 -1.4 -0.4 1.4 0.40.84845110

7122 64 123 65.2 -1 -1.2 1 1.2127 61 128 62.4 -1 -1.4 1 1.4 RMS Y:

134 60 133.6 60.6 0.4 -0.6 0.4 0.60.96862546

6140 59 139.2 60 0.8 -1 0.8 1145 59 144.6 60.6 0.4 -1.6 0.4 1.6150 61 149.4 62.4 0.6 -1.4 0.6 1.4154 64 153.4 65.6 0.6 -1.6 0.6 1.6158 69 156.6 69.8 1.4 -0.8 1.4 0.8160 75 158.6 74.8 1.4 0.2 1.4 0.2161 80 159.2 80.2 1.8 -0.2 1.8 0.2160 86 158.6 85.6 1.4 0.4 1.4 0.4157 91 156.4 91 0.6 0 0.6 0155 96 152.6 96.2 2.4 -0.2 2.4 0.2149 102 147.8 100.6 1.2 1.4 1.2 1.4142 106 142.4 104.2 -0.4 1.8 0.4 1.8136 108 136.4 106.4 -0.4 1.6 0.4 1.6130 109 130.8 107 -0.8 2 0.8 2125 107 126 106 -1 1 1 1

Page 211: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

121 105 121.8 103.6 -0.8 1.4 0.8 1.4118 101 118.4 99.8 -0.4 1.2 0.4 1.2115 96 116 95 -1 1 1 1113 90 114.4 89.4 -1.4 0.6 1.4 0.6113 83 113.8 83.6 -0.8 -0.6 0.8 0.6113 77 114 77.6 -1 -0.6 1 0.6115 72 114.8 71.8 0.2 0.2 0.2 0.2116 66 115.6 66 0.4 0 0.4 0117 61 116.4 60.4 0.6 0.6 0.6 0.6117 54 116.4 54.8 0.6 -0.8 0.6 0.8117 49 115.8 49.6 1.2 -0.6 1.2 0.6115 44 114.4 44.4 0.6 -0.4 0.6 0.4113 40 112 40 1 0 1 0110 35 108.8 36.2 1.2 -1.2 1.2 1.2105 32 104.8 33.2 0.2 -1.2 0.2 1.2101 30 100.4 31 0.6 -1 0.6 1

95 29 95.8 29.8 -0.8 -0.8 0.8 0.891 29 91 29 0 0 0 087 29 87 29 0 0 0 0

Page 212: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:

107 31 107 31 0 0 0 00.80098039

2

102 29 102.6666667 29.66666667

-0.66666666

7

-0.66666666

7 0.666666667 0.66666666799 29 98.8 29 0.2 0 0.2 0 Maximum:95 28 95.4 28.6 -0.4 -0.6 0.4 0.6 491 28 92.6 28.8 -1.6 -0.8 1.6 0.890 29 90 29.2 0 -0.2 0 0.2 Sum:

88 30 87.6 29.8 0.4 0.2 0.4 0.2108.933333

386 31 85 30.8 1 0.2 1 0.283 31 82.2 32.2 0.8 -1.2 0.8 1.2 RMS:78 33 79.2 34 -1.2 -1 1.2 1 1.0854955876 36 76.4 36.2 -0.4 -0.2 0.4 0.273 39 73.6 39.4 -0.6 -0.4 0.6 0.4 Mean X:

72 42 71.8 42.2 0.2 -0.2 0.2 0.20.80980392

269 47 70.2 46.6 -1.2 0.4 1.2 0.469 47 70 51 -1 -4 1 4 Mean Y:

68 58 70 55.6 -2 2.4 2 2.40.79215686

372 61 71 60 1 1 1 172 65 72 65 0 0 0 0 Maximum X:74 69 74.2 68.4 -0.2 0.6 0.2 0.6 2.674 72 76 71.8 -2 0.2 2 0.279 75 79.2 75 -0.2 0 0.2 0 Maximum Y:81 78 82.6 78 -1.6 0 1.6 0 488 81 87.4 80.6 0.6 0.4 0.6 0.491 84 91.6 81.8 -0.6 2.2 0.6 2.2 Sum X:

98 85 96.2 82 1.8 3 1.8 355.0666666

7100 81 100 81.2 0 -0.2 0 0.2104 79 104 79.4 0 -0.4 0 0.4 Sum Y:

107 77 107.4 76.8 -0.4 0.2 0.4 0.253.8666666

7111 75 110.8 73.8 0.2 1.2 0.2 1.2115 72 114.8 70.8 0.2 1.2 0.2 1.2 RMS X:

117 66 119 67.8 -2 -1.8 2 1.81.03954489

8124 64 124.2 65.2 -0.2 -1.2 0.2 1.2128 62 129.2 62.8 -1.2 -0.8 1.2 0.8 RMS Y:

137 62 134.8 61.8 2.2 0.2 2.2 0.21.12957855

5140 60 140.2 61.8 -0.2 -1.8 0.2 1.8145 61 145.2 62.8 -0.2 -1.8 0.2 1.8151 64 149.4 65 1.6 -1 1.6 1153 67 153.2 68.4 -0.2 -1.4 0.2 1.4158 73 156 72.6 2 0.4 2 0.4159 77 157.4 77.2 1.6 -0.2 1.6 0.2159 82 157.8 82.2 1.2 -0.2 1.2 0.2158 87 156.8 86.8 1.2 0.2 1.2 0.2155 92 154.2 92 0.8 0 0.8 0153 96 150.4 96.6 2.6 -0.6 2.6 0.6146 103 145.6 100.4 0.4 2.6 0.4 2.6140 105 140.6 103.4 -0.6 1.6 0.6 1.6134 106 135.2 105.2 -1.2 0.8 1.2 0.8130 107 130.6 105 -0.6 2 0.6 2126 105 126.4 103.8 -0.4 1.2 0.4 1.2123 102 123 101.2 0 0.8 0 0.8119 99 120 97.4 -1 1.6 1 1.6

Page 213: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

117 93 117.8 92.8 -0.8 0.2 0.8 0.2115 88 116.4 87.8 -1.4 0.2 1.4 0.2115 82 116 82.6 -1 -0.6 1 0.6116 77 116.2 77.2 -0.2 -0.2 0.2 0.2117 73 116.6 71.8 0.4 1.2 0.4 1.2118 66 117 66.2 1 -0.2 1 0.2117 61 117.2 60.8 -0.2 0.2 0.2 0.2117 54 116.6 55.2 0.4 -1.2 0.4 1.2117 50 115.6 50.2 1.4 -0.2 1.4 0.2114 45 114 45.2 0 -0.2 0 0.2113 41 111.2 41.2 1.8 -0.2 1.8 0.2109 36 107.8 37.6 1.2 -1.6 1.2 1.6103 34 103.8 34.8 -0.8 -0.8 0.8 0.8100 32 99.6 32.8 0.4 -0.8 0.4 0.8

94 31 95.2 31.8 -1.2 -0.8 1.2 0.892 31 91 31 1 0 1 087 31 87 31 0 0 0 0

Page 214: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

50 70 90 110 130 150 17020

40

60

80

100

120

140

Figure 8 Trial 5 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

50 70 90 110 130 150 17020

40

60

80

100

120

140

Figure 8 Trial 5 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 215: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Heart 1:

Page 216: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 1:

Kalman Filter Data:

Kalman:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:148 47 148 47 0 0 0 0 0.620599251154 43 152.6666667 42.66666667 1.333333333 0.333333333 1.333333333 0.333333333156 38 156.8 38 -0.8 0 0.8 0 Maximum:161 33 161.2 34 -0.2 -1 0.2 1 2.4165 29 165 30.6 0 -1.6 0 1.6170 27 169.2 28.2 0.8 -1.2 0.8 1.2 Sum:173 26 173.2 27 -0.2 -1 0.2 1 110.4666667177 26 177 26.6 0 -0.6 0 0.6181 27 180.8 27 0.2 0 0.2 0 RMS:184 27 184.8 27.8 -0.8 -0.8 0.8 0.8 0.792678734189 29 188.8 29 0.2 0 0.2 0193 30 192.4 30.6 0.6 -0.6 0.6 0.6 Mean X:197 32 195.6 33 1.4 -1 1.4 1 0.522097378199 35 198.4 36.4 0.6 -1.4 0.6 1.4200 39 200.8 40.4 -0.8 -1.4 0.8 1.4 Mean Y:203 46 202.4 44.8 0.6 1.2 0.6 1.2 0.719101124205 50 203.6 49.4 1.4 0.6 1.4 0.6205 54 204.2 54 0.8 0 0.8 0 Maximum X:205 58 204 58.2 1 -0.2 1 0.2 1.8203 62 203 62.8 0 -0.8 0 0.8202 67 201.8 68 0.2 -1 0.2 1 Maximum Y:200 73 200.6 74 -0.6 -1 0.6 1 2.4199 80 199.8 80.8 -0.8 -0.8 0.8 0.8199 88 199 87.4 0 0.6 0 0.6 Sum X:199 96 198.2 93.6 0.8 2.4 0.8 2.4 46.46666667198 100 197.2 99 0.8 1 0.8 1196 104 195.8 103.8 0.2 0.2 0.2 0.2 Sum Y:194 107 194 107.8 0 -0.8 0 0.8 64192 112 192 111.4 0 0.6 0 0.6190 116 189.6 115.2 0.4 0.8 0.4 0.8 RMS X:188 118 187 119.4 1 -1.4 1 1.4 0.684079706184 123 184 123.6 0 -0.6 0 0.6181 128 180.6 128 0.4 0 0.4 0 RMS Y:177 133 176.8 132.6 0.2 0.4 0.2 0.4 0.888095776173 138 173 137.2 0 0.8 0 0.8169 141 168.8 141.6 0.2 -0.6 0.2 0.6165 146 164.4 145.6 0.6 0.4 0.6 0.4160 150 160 148.8 0 1.2 0 1.2155 153 155.6 151.4 -0.6 1.6 0.6 1.6151 154 151.2 152.6 -0.2 1.4 0.2 1.4147 154 147.2 152.4 -0.2 1.6 0.2 1.6143 152 143.6 151.2 -0.6 0.8 0.6 0.8140 149 140 149.2 0 -0.2 0 0.2137 147 136.4 147 0.6 0 0.6 0133 144 133 144.4 0 -0.4 0 0.4129 143 129.2 141.8 -0.2 1.2 0.2 1.2126 139 125 138.4 1 0.6 1 0.6121 136 120.8 134.6 0.2 1.4 0.2 1.4116 130 116.6 130 -0.6 0 0.6 0112 125 112.2 125.4 -0.2 -0.4 0.2 0.4108 120 108.4 120.4 -0.4 -0.4 0.4 0.4104 116 105.2 115.4 -1.2 0.6 1.2 0.6102 111 102 110.2 0 0.8 0 0.8100 105 99 105 1 0 1 0

96 99 96.4 100 -0.4 -1 0.4 193 94 93.8 95.2 -0.8 -1.2 0.8 1.291 91 91.6 90.6 -0.6 0.4 0.6 0.489 87 90.2 86 -1.2 1 1.2 1

Page 217: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

89 82 89 81.2 0 0.8 0 0.889 76 88.2 75.8 0.8 0.2 0.8 0.287 70 87.4 70 -0.4 0 0.4 087 64 87 64 0 0 0 085 58 86.8 58.2 -1.8 -0.2 1.8 0.287 52 87.8 52.8 -0.8 -0.8 0.8 0.888 47 89.6 47.8 -1.6 -0.8 1.6 0.892 43 92.6 43.4 -0.6 -0.4 0.6 0.496 39 95.8 39.4 0.2 -0.4 0.2 0.4

100 36 99.4 36 0.6 0 0.6 0103 32 103 33 0 -1 0 1106 30 106.8 30.4 -0.8 -0.4 0.8 0.4110 28 110.4 28.2 -0.4 -0.2 0.4 0.2115 26 114.2 27 0.8 -1 0.8 1118 25 118 26.6 0 -1.6 0 1.6122 26 121.8 26.6 0.2 -0.6 0.2 0.6125 28 125.4 27.4 -0.4 0.6 0.4 0.6129 28 128.8 29.2 0.2 -1.2 0.2 1.2133 30 132.2 31.6 0.8 -1.6 0.8 1.6135 34 135.8 34.4 -0.8 -0.4 0.8 0.4139 38 139.2 37.8 -0.2 0.2 0.2 0.2143 42 142 41 1 1 1 1146 45 144.6 44 1.4 1 1.4 1147 46 146.8 46.8 0.2 -0.8 0.2 0.8148 49 148.6 49.6 -0.6 -0.6 0.6 0.6150 52 149.8 52.2 0.2 -0.2 0.2 0.2152 56 150.8 54.4 1.2 1.6 1.2 1.6152 58 151.2 56 0.8 2 0.8 2152 57 150.6 56.6 1.4 0.4 1.4 0.4150 57 149.6666667 56.33333333 0.333333333 0.666666667 0.333333333 0.666666667147 55 147 55 0 0 0 0

Page 218: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:149 47 149 47 0 0 0 0 0.856179775155 43 153.3333333 42.66666667 1.666666667 0.333333333 1.666666667 0.333333333156 38 157.6 38.4 -1.6 -0.4 1.6 0.4 Maximum:162 34 162 34.8 0 -0.8 0 0.8 3.2166 30 165.6 31.8 0.4 -1.8 0.4 1.8171 29 169.8 30 1.2 -1 1.2 1 Sum:173 28 173.6 29 -0.6 -1 0.6 1 152.4177 29 177.2 28.8 -0.2 0.2 0.2 0.2181 29 181 29 0 0 0 0 RMS:184 29 185 29.6 -1 -0.6 1 0.6 1.085254706190 30 189.2 30.4 0.8 -0.4 0.8 0.4193 31 192.6 32.2 0.4 -1.2 0.4 1.2 Mean X:198 33 195.6 34.6 2.4 -1.6 2.4 1.6 0.829213483198 38 198.4 38.6 -0.4 -0.6 0.4 0.6199 41 200.8 42.4 -1.8 -1.4 1.8 1.4 Mean Y:204 50 202 46.8 2 3.2 2 3.2 0.883146067205 50 203.2 50.8 1.8 -0.8 1.8 0.8204 55 203.6 55 0.4 0 0.4 0 Maximum X:204 58 203.2 58.6 0.8 -0.6 0.8 0.6 3.2201 62 202.2 63.6 -1.2 -1.6 1.2 1.6202 68 201.2 69 0.8 -1 0.8 1 Maximum Y:200 75 200.6 75.4 -0.6 -0.4 0.6 0.4 3.2199 82 200.2 82.4 -1.2 -0.4 1.2 0.4201 90 199.4 88.6 1.6 1.4 1.6 1.4 Sum X:199 97 198.6 94 0.4 3 0.4 3 73.8198 99 197.4 99 0.6 0 0.6 0196 102 195.6 103.6 0.4 -1.6 0.4 1.6 Sum Y:193 107 193.8 107.2 -0.8 -0.2 0.8 0.2 78.6192 113 191.8 111 0.2 2 0.2 2190 115 189.2 115.4 0.8 -0.4 0.8 0.4 RMS X:188 118 186.8 120 1.2 -2 1.2 2 1.041006907183 124 183.6 124.2 -0.6 -0.2 0.6 0.2181 130 180.2 128.8 0.8 1.2 0.8 1.2 RMS Y:176 134 176.4 133.4 -0.4 0.6 0.4 0.6 1.127767784173 138 172.8 138 0.2 0 0.2 0169 141 168.4 142 0.6 -1 0.6 1165 147 164.2 145.8 0.8 1.2 0.8 1.2159 150 160 148.8 -1 1.2 1 1.2155 153 155.8 151 -0.8 2 0.8 2152 153 151.6 151.4 0.4 1.6 0.4 1.6148 152 148 150.8 0 1.2 0 1.2144 149 144.6 149.4 -0.6 -0.4 0.6 0.4141 147 140.8 147.6 0.2 -0.6 0.2 0.6138 146 137.2 145.8 0.8 0.2 0.8 0.2133 144 133.6 143.8 -0.6 0.2 0.6 0.2130 143 129.4 141.4 0.6 1.6 0.6 1.6126 139 125.2 138 0.8 1 0.8 1120 135 121.2 134 -1.2 1 1.2 1117 129 116.8 129.4 0.2 -0.4 0.2 0.4113 124 112.4 125 0.6 -1 0.6 1108 120 109.4 120.2 -1.4 -0.2 1.4 0.2104 117 106.2 115.4 -2.2 1.6 2.2 1.6105 111 102.6 110.4 2.4 0.6 2.4 0.6101 105 99.8 105.4 1.2 -0.4 1.2 0.4

95 99 97.4 100.6 -2.4 -1.6 2.4 1.694 95 94.6 95.8 -0.6 -0.8 0.6 0.892 93 92.6 91.2 -0.6 1.8 0.6 1.891 87 91.6 86.4 -0.6 0.6 0.6 0.691 82 90.4 81.4 0.6 0.6 0.6 0.690 75 89.6 75.6 0.4 -0.6 0.4 0.688 70 88.4 69.8 -0.4 0.2 0.4 0.2

Page 219: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

88 64 88.2 64 -0.2 0 0.2 085 58 88.2 58.6 -3.2 -0.6 3.2 0.690 53 89.6 53.4 0.4 -0.4 0.4 0.490 48 91.8 48.8 -1.8 -0.8 1.8 0.895 44 95.2 44.8 -0.2 -0.8 0.2 0.899 41 97.8 40.8 1.2 0.2 1.2 0.2

102 38 101.2 37.6 0.8 0.4 0.8 0.4103 33 104.4 34.8 -1.4 -1.8 1.4 1.8107 32 107.8 32 -0.8 0 0.8 0111 30 111.2 29.8 -0.2 0.2 0.2 0.2116 27 115 29 1 -2 1 2119 27 118.8 28.6 0.2 -1.6 0.2 1.6122 29 122.6 28.4 -0.6 0.6 0.6 0.6126 30 126 29.4 0 0.6 0 0.6130 29 129.2 31.2 0.8 -2.2 0.8 2.2133 32 132.8 33.4 0.2 -1.4 0.2 1.4135 36 136.2 36 -1.2 0 1.2 0140 40 139.4 39.2 0.6 0.8 0.6 0.8143 43 142 42 1 1 1 1146 45 144.6 44.6 1.4 0.4 1.4 0.4146 46 146.6 47.2 -0.6 -1.2 0.6 1.2148 49 148.4 50 -0.4 -1 0.4 1150 53 149.6 52.4 0.4 0.6 0.4 0.6152 57 150.6 54.4 1.4 2.6 1.4 2.6152 57 150.8 55.8 1.2 1.2 1.2 1.2151 56 150 56 1 0 1 0149 56 148.6666667 55.33333333 0.333333333 0.666666667 0.333333333 0.666666667146 54 146 54 0 0 0 0

Page 220: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

50 70 90 110 130 150 170 190 210 23020

40

60

80

100

120

140

160

Heart 1 Trial 1 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

50 70 90 110 130 150 170 190 210 23020

40

60

80

100

120

140

160

Heart 1 Trial 1 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 221: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 2:

Kalman Filter:

Kalman:

Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:

146 55 146 55 0 0 0 00.61351351

4

148 56 147.3333333 560.66666666

7 0 0.666666667 0148 57 146.6 54.6 1.4 2.4 1.4 2.4 Maximum:148 55 145.2 52.4 2.8 2.6 2.8 2.6 2.8143 50 142.4 49 0.6 1 0.6 1139 44 138.8 44.2 0.2 -0.2 0.2 0.2 Sum:134 39 134.4 39.2 -0.4 -0.2 0.4 0.2 136.2130 33 130 35 0 -2 0 2126 30 125.6 31.6 0.4 -1.6 0.4 1.6 RMS:

121 29 121.2 29.2 -0.2 -0.2 0.2 0.20.81802760

9117 27 116.6 28 0.4 -1 0.4 1112 27 112 27.4 0 -0.4 0 0.4 Mean X:

107 27 107.4 27.6 -0.4 -0.6 0.4 0.60.54774774

8103 27 103 28.8 0 -1.8 0 1.8

98 30 99 30.2 -1 -0.2 1 0.2 Mean Y:

95 33 95.8 32.6 -0.8 0.4 0.8 0.40.67927927

992 34 92.8 36.4 -0.8 -2.4 0.8 2.491 39 90.2 41 0.8 -2 0.8 2 Maximum X:88 46 87.8 46.4 0.2 -0.4 0.2 0.4 2.885 53 86.2 52.6 -1.2 0.4 1.2 0.483 60 85 58.6 -2 1.4 2 1.4 Maximum Y:84 65 84.8 64 -0.8 1 0.8 1 2.685 69 85.6 68.6 -0.6 0.4 0.6 0.487 73 87.6 73.2 -0.6 -0.2 0.6 0.2 Sum X:89 76 90.2 78 -1.2 -2 1.2 2 60.893 83 93.4 83.4 -0.4 -0.4 0.4 0.497 89 97 89.2 0 -0.2 0 0.2 Sum Y:

101 96 100.6 95.8 0.4 0.2 0.4 0.2 75.4105 102 103.8 102.2 1.2 -0.2 1.2 0.2107 109 106.4 108 0.6 1 0.6 1 RMS X:

109 115 108.8 113.6 0.2 1.4 0.2 1.40.73248654

4110 118 110.8 118.8 -0.8 -0.8 0.8 0.8113 124 113 123.2 0 0.8 0 0.8 RMS Y:

115 128 115.2 126.8 -0.2 1.2 0.2 1.20.89543386

2118 131 117.6 130.4 0.4 0.6 0.4 0.6120 133 120 133.4 0 -0.4 0 0.4122 136 122.8 136.4 -0.8 -0.4 0.8 0.4125 139 125.6 139.4 -0.6 -0.4 0.6 0.4129 143 129 142.6 0 0.4 0 0.4132 146 132.6 145.6 -0.6 0.4 0.6 0.4137 149 136.2 148.4 0.8 0.6 0.8 0.6140 151 139.4 151 0.6 0 0.6 0143 153 142.6 153.4 0.4 -0.4 0.4 0.4145 156 145.2 155.4 -0.2 0.6 0.2 0.6148 158 147.2 157 0.8 1 0.8 1150 159 148.8 157.8 1.2 1.2 1.2 1.2150 159 150.2 157.8 -0.2 1.2 0.2 1.2151 157 151.2 157.2 -0.2 -0.2 0.2 0.2152 156 152.2 156.2 -0.2 -0.2 0.2 0.2153 155 153.4 155 -0.4 0 0.4 0155 154 155 154 0 0 0 0

Page 222: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

156 153 157 152.8 -1 0.2 1 0.2159 152 159.4 151.2 -0.4 0.8 0.4 0.8162 150 162.2 149 -0.2 1 0.2 1165 147 165.4 146.2 -0.4 0.8 0.4 0.8169 143 168.4 143 0.6 0 0.6 0172 139 171.4 139.8 0.6 -0.8 0.6 0.8174 136 173.6 136.6 0.4 -0.6 0.4 0.6177 134 175.8 133.4 1.2 0.6 1.2 0.6176 131 177.6 130.4 -1.6 0.6 1.6 0.6180 127 179.8 127.4 0.2 -0.4 0.2 0.4181 124 182 124.4 -1 -0.4 1 0.4185 121 185 121.6 0 -0.6 0 0.6188 119 187.4 119.2 0.6 -0.2 0.6 0.2191 117 190.2 116.6 0.8 0.4 0.8 0.4192 115 192.4 114 -0.4 1 0.4 1195 111 194.4 111 0.6 0 0.6 0196 108 196.2 107.8 -0.2 0.2 0.2 0.2198 104 198 104.2 0 -0.2 0 0.2200 101 199.6 100.8 0.4 0.2 0.4 0.2201 97 201.2 97.4 -0.2 -0.4 0.2 0.4203 94 202.6 94.2 0.4 -0.2 0.4 0.2204 91 203.4 91 0.6 0 0.6 0205 88 204.2 87.8 0.8 0.2 0.8 0.2204 85 204.8 84.2 -0.8 0.8 0.8 0.8205 81 205.2 80.4 -0.2 0.6 0.2 0.6206 76 205.6 75.6 0.4 0.4 0.4 0.4206 72 206.2 70.4 -0.2 1.6 0.2 1.6207 64 206.6 64.8 0.4 -0.8 0.4 0.8207 59 206.8 59 0.2 0 0.2 0207 53 206.4 52.8 0.6 0.2 0.6 0.2207 47 205.8 47.2 1.2 -0.2 1.2 0.2204 41 204.6 41.8 -0.6 -0.8 0.6 0.8204 36 202.8 36.8 1.2 -0.8 1.2 0.8201 32 200.4 32.6 0.6 -0.6 0.6 0.6198 28 198 29.2 0 -1.2 0 1.2195 26 194.8 26.8 0.2 -0.8 0.2 0.8192 24 191.4 25.2 0.6 -1.2 0.6 1.2188 24 187.8 24.6 0.2 -0.6 0.2 0.6184 24 183.6 24.8 0.4 -0.8 0.4 0.8180 25 179.4 25.4 0.6 -0.4 0.6 0.4174 27 175.2 26.4 -1.2 0.6 1.2 0.6171 27 171 27.8 0 -0.8 0 0.8167 29 167.4 29.2 -0.4 -0.2 0.4 0.2163 31 163.8 31.4 -0.8 -0.4 0.8 0.4162 32 160 34.6 2 -2.6 2 2.6156 38 156.4 38.4 -0.4 -0.4 0.4 0.4152 43 153.2 42.4 -1.2 0.6 1.2 0.6149 48 150.2 46.4 -1.2 1.6 1.2 1.6147 51 148.6 49.6 -1.6 1.4 1.6 1.4147 52 147.8 51.8 -0.8 0.2 0.8 0.2148 54 147.8 53 0.2 1 0.2 1148 54 148.2 53.2 -0.2 0.8 0.2 0.8149 54 148.6 53.2 0.4 0.8 0.4 0.8149 52 148.8 52.6 0.2 -0.6 0.2 0.6149 52 149 52.2 0 -0.2 0 0.2149 51 149 51.6 0 -0.6 0 0.6149 52 149 51.4 0 0.6 0 0.6149 51 148.8 50.6 0.2 0.4 0.2 0.4

149 51 148.6666667 500.33333333

3 1 0.333333333 1148 48 148 48 0 0 0 0

Page 223: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:147 56 147 56 0 0 0 0 0.921321321147 56 147.3333333 56.33333333 -0.333333333 -0.333333333 0.333333333 0.333333333148 57 146 54.2 2 2.8 2 2.8 Maximum:147 55 144.2 51.6 2.8 3.4 2.8 3.4 4.6141 47 141.4 48 -0.4 -1 0.4 1138 43 137.6 43 0.4 0 0.4 0 Sum:133 38 133.4 38.4 -0.4 -0.4 0.4 0.4 204.5333333129 32 129.4 35.2 -0.4 -3.2 0.4 3.2126 32 125.4 32.4 0.6 -0.4 0.6 0.4 RMS:121 31 121.2 30.6 -0.2 0.4 0.2 0.4 1.206887143118 29 116.6 30 1.4 -1 1.4 1112 29 112.2 29.4 -0.2 -0.4 0.2 0.4 Mean X:106 29 107.6 29.8 -1.6 -0.8 1.6 0.8 0.855255255104 29 103.2 31 0.8 -2 0.8 2

98 33 99.6 32.2 -1.6 0.8 1.6 0.8 Mean Y:96 35 97 34.6 -1 0.4 1 0.4 0.98738738794 35 93.8 38.8 0.2 -3.8 0.2 3.893 41 91.2 43.4 1.8 -2.4 1.8 2.4 Maximum X:88 50 89 48.8 -1 1.2 1 1.2 3.685 56 87.4 54.8 -2.4 1.2 2.4 1.285 62 86.2 60.4 -1.2 1.6 1.2 1.6 Maximum Y:86 65 86.4 64.8 -0.4 0.2 0.4 0.2 4.687 69 87.6 68.8 -0.6 0.2 0.6 0.289 72 89.6 73.4 -0.6 -1.4 0.6 1.4 Sum X:91 76 92.2 78.6 -1.2 -2.6 1.2 2.6 94.9333333395 85 95.2 84.4 -0.2 0.6 0.2 0.699 91 98.6 90.6 0.4 0.4 0.4 0.4 Sum Y:

102 98 101.8 97.4 0.2 0.6 0.2 0.6 109.6106 103 104.6 103.6 1.4 -0.6 1.4 0.6107 110 106.6 108.6 0.4 1.4 0.4 1.4 RMS X:109 116 109 114 0 2 0 2 1.111087087109 116 111 118.8 -2 -2.8 2 2.8114 125 113.4 123 0.6 2 0.6 2 RMS Y:116 127 115.6 126.2 0.4 0.8 0.4 0.8 1.295622877119 131 118.4 130.2 0.6 0.8 0.6 0.8120 132 120.8 133 -0.8 -1 0.8 1123 136 123.6 136.4 -0.6 -0.4 0.6 0.4126 139 126.4 139.6 -0.4 -0.6 0.4 0.6130 144 130.2 143 -0.2 1 0.2 1133 147 133.6 146 -0.6 1 0.6 1139 149 137 148.8 2 0.2 2 0.2140 151 140 151.2 0 -0.2 0 0.2143 153 143.2 153.4 -0.2 -0.4 0.2 0.4145 156 145.4 155.2 -0.4 0.8 0.4 0.8149 158 147.2 156.6 1.8 1.4 1.8 1.4150 158 148.6 157.2 1.4 0.8 1.4 0.8149 158 150 157 -1 1 1 1150 156 151 156.4 -1 -0.4 1 0.4152 155 152.2 155.8 -0.2 -0.8 0.2 0.8154 155 153.6 154.8 0.4 0.2 0.4 0.2156 155 155.8 154 0.2 1 0.2 1156 153 158.2 153 -2.2 0 2.2 0161 152 160.6 151.4 0.4 0.6 0.4 0.6164 150 163.4 148.8 0.6 1.2 0.6 1.2166 147 166.6 146 -0.6 1 0.6 1170 142 169.2 143 0.8 -1 0.8 1172 139 171.8 140 0.2 -1 0.2 1174 137 173.6 137 0.4 0 0.4 0177 135 175.8 134 1.2 1 1.2 1175 132 177.6 131.2 -2.6 0.8 2.6 0.8181 127 180.2 128.2 0.8 -1.2 0.8 1.2

Page 224: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

181 125 182.6 125.2 -1.6 -0.2 1.6 0.2187 122 185.8 122.4 1.2 -0.4 1.2 0.4189 120 188 120.2 1 -0.2 1 0.2191 118 190.8 117.2 0.2 0.8 0.2 0.8192 116 192.6 114.4 -0.6 1.6 0.6 1.6195 110 194.4 111.2 0.6 -1.2 0.6 1.2196 108 196.4 107.8 -0.4 0.2 0.4 0.2198 104 198.2 104.2 -0.2 -0.2 0.2 0.2201 101 199.8 101 1.2 0 1.2 0201 98 201.4 97.8 -0.4 0.2 0.4 0.2203 94 202.8 94.8 0.2 -0.8 0.2 0.8204 92 203.2 91.8 0.8 0.2 0.8 0.2205 89 204.2 88.2 0.8 0.8 0.8 0.8203 86 205 84.6 -2 1.4 2 1.4206 80 205.4 80.6 0.6 -0.6 0.6 0.6207 76 206 75.2 1 0.8 1 0.8206 72 206.6 69.8 -0.6 2.2 0.6 2.2208 62 207 64.4 1 -2.4 1 2.4206 59 207 58.6 -1 0.4 1 0.4208 53 206.4 52.4 1.6 0.6 1.6 0.6207 47 205.6 47.6 1.4 -0.6 1.4 0.6203 41 204.6 42.4 -1.6 -1.4 1.6 1.4204 38 202.4 37.8 1.6 0.2 1.6 0.2201 33 199.8 34 1.2 -1 1.2 1197 30 197.6 31 -0.6 -1 0.6 1194 28 194.2 28.8 -0.2 -0.8 0.2 0.8192 26 190.8 27.4 1.2 -1.4 1.2 1.4187 27 187.4 26.8 -0.4 0.2 0.4 0.2184 26 183.2 27 0.8 -1 0.8 1180 27 179.2 27.4 0.8 -0.4 0.8 0.4173 29 175.2 28.2 -2.2 0.8 2.2 0.8172 28 171.2 29.4 0.8 -1.4 0.8 1.4167 31 168 30.4 -1 0.6 1 0.6164 32 164.2 33.2 -0.2 -1.2 0.2 1.2164 32 160.4 36.6 3.6 -4.6 3.6 4.6154 43 157 40.4 -3 2.6 3 2.6153 45 154 44 -1 1 1 1150 50 151.2 47.8 -1.2 2.2 1.2 2.2149 50 150.4 49.8 -1.4 0.2 1.4 0.2150 51 149.8 51.4 0.2 -0.4 0.2 0.4150 53 149.8 52 0.2 1 0.2 1150 53 150 52.2 0 0.8 0 0.8150 53 150 52.6 0 0.4 0 0.4150 51 149.8 52.2 0.2 -1.2 0.2 1.2150 53 149.6 52.2 0.4 0.8 0.4 0.8149 51 149.6 51.8 -0.6 -0.8 0.6 0.8149 53 149.4 51.8 -0.4 1.2 0.4 1.2150 51 149 50.4 1 0.6 1 0.6149 51 149 49.33333333 0 1.666666667 0 1.666666667148 46 148 46 0 0 0 0

Page 225: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

50 70 90 110 130 150 170 190 210 23020

40

60

80

100

120

140

160

Heart 1 Trial 2 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

50 70 90 110 130 150 170 190 210 23020

40

60

80

100

120

140

160

Heart 1 Trial 2 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 226: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 3:

Kalman Filter Data:

Kalman:

Moving Average X:

Moving Average Y:

Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:

143 56 143 56 0 0 0 00.77216494

8

145 57 144 56.66666667 10.33333333

3 1 0.333333333144 57 144.8 56.8 -0.8 0.2 0.8 0.2 Maximum:146 57 145.8 56 0.2 1 0.2 1 3.8146 57 147 54.2 -1 2.8 1 2.8148 52 148.8 51.8 -0.8 0.2 0.8 0.2 Sum:151 48 150.2 49 0.8 -1 0.8 1 149.8153 45 152.4 45.4 0.6 -0.4 0.6 0.4153 43 155.2 41.8 -2.2 1.2 2.2 1.2 RMS:

157 39 158.4 38 -1.4 1 1.4 11.03005243

5162 34 161.8 34.4 0.2 -0.4 0.2 0.4167 29 165.8 31 1.2 -2 1.2 2 Mean X:

170 27 169.8 28.4 0.2 -1.4 0.2 1.40.72783505

2173 26 173.2 26.8 -0.2 -0.8 0.2 0.8177 26 176.2 26.2 0.8 -0.2 0.8 0.2 Mean Y:

179 26 179.2 26.2 -0.2 -0.2 0.2 0.20.81649484

5182 26 182.2 26.4 -0.2 -0.4 0.2 0.4185 27 184.8 26.6 0.2 0.4 0.2 0.4 Maximum X:188 27 187.4 27 0.6 0 0.6 0 2.8190 27 189.4 27.8 0.6 -0.8 0.6 0.8192 28 191.2 29 0.8 -1 0.8 1 Maximum Y:192 30 192.6 30.2 -0.6 -0.2 0.6 0.2 3.8194 33 194 31.6 0 1.4 0 1.4195 33 195.2 34 -0.2 -1 0.2 1 Sum X:197 34 196.6 37 0.4 -3 0.4 3 70.6198 40 197.8 40 0.2 0 0.2 0199 45 198.8 43.8 0.2 1.2 0.2 1.2 Sum Y:200 48 200 48.2 0 -0.2 0 0.2 79.2200 52 201.4 52.2 -1.4 -0.2 1.4 0.2203 56 203.2 56.2 -0.2 -0.2 0.2 0.2 RMS X:

205 60 205 60.6 0 -0.6 0 0.60.94661632

9208 65 206.8 65.2 1.2 -0.2 1.2 0.2209 70 207.6 69.6 1.4 0.4 1.4 0.4 RMS Y:

209 75 207.8 73.6 1.2 1.4 1.2 1.41.10721884

1207 78 207 77 0 1 0 1206 80 205.4 80 0.6 0 0.6 0204 82 203.6 82.6 0.4 -0.6 0.4 0.6201 85 201.4 85.4 -0.4 -0.4 0.4 0.4200 88 199.2 88.6 0.8 -0.6 0.8 0.6196 92 196.6 92.6 -0.6 -0.6 0.6 0.6195 96 194.4 97.2 0.6 -1.2 0.6 1.2191 102 191.8 102.4 -0.8 -0.4 0.8 0.4190 108 189.6 107.6 0.4 0.4 0.4 0.4187 114 187 113 0 1 0 1185 118 185 117.8 0 0.2 0 0.2182 123 182.4 122.2 -0.4 0.8 0.4 0.8181 126 179.8 126.4 1.2 -0.4 1.2 0.4177 130 177 130.4 0 -0.4 0 0.4174 135 174 134.2 0 0.8 0 0.8171 138 170.4 138.2 0.6 -0.2 0.6 0.2167 142 166.8 142.2 0.2 -0.2 0.2 0.2

Page 227: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

163 146 162.8 146 0.2 0 0.2 0159 150 158 149.8 1 0.2 1 0.2154 154 153.4 153 0.6 1 0.6 1147 157 149.6 155.6 -2.6 1.4 2.6 1.4144 158 146.4 157.4 -2.4 0.6 2.4 0.6144 159 144.2 157.2 -0.2 1.8 0.2 1.8143 159 142.8 155.2 0.2 3.8 0.2 3.8143 153 140.8 151.4 2.2 1.6 2.2 1.6140 147 137.2 146.8 2.8 0.2 2.8 0.2134 139 132.8 141.8 1.2 -2.8 1.2 2.8126 136 127.6 137.4 -1.6 -1.4 1.6 1.4121 134 122 133.6 -1 0.4 1 0.4117 131 116.4 130.2 0.6 0.8 0.6 0.8112 128 111.6 126.4 0.4 1.6 0.4 1.6106 122 106.8 121.8 -0.8 0.2 0.8 0.2102 117 102.2 116.6 -0.2 0.4 0.2 0.4

97 111 98.2 111 -1.2 0 1.2 094 105 94.4 105.2 -0.4 -0.2 0.4 0.292 100 91.2 99 0.8 1 0.8 187 93 89 92.6 -2 0.4 2 0.486 86 87.2 85.8 -1.2 0.2 1.2 0.286 79 85.6 78.2 0.4 0.8 0.4 0.885 71 85 71.2 0 -0.2 0 0.284 62 84.8 64.6 -0.8 -2.6 0.8 2.684 58 85.2 58.8 -1.2 -0.8 1.2 0.885 53 86.4 53.4 -1.4 -0.4 1.4 0.488 50 88.8 48.6 -0.8 1.4 0.8 1.491 44 92.4 43.6 -1.4 0.4 1.4 0.496 38 96.8 38.6 -0.8 -0.6 0.8 0.6

102 33 101.4 33.8 0.6 -0.8 0.6 0.8107 28 106.4 30.2 0.6 -2.2 0.6 2.2111 26 111.6 28 -0.6 -2 0.6 2116 26 116.8 27 -0.8 -1 0.8 1122 27 122 27.2 0 -0.2 0 0.2128 28 127 28.2 1 -0.2 1 0.2133 29 131.6 29.8 1.4 -0.8 1.4 0.8136 31 135.6 32 0.4 -1 0.4 1139 34 139.4 35 -0.4 -1 0.4 1142 38 142.6 38.4 -0.6 -0.4 0.6 0.4147 43 146 42.8 1 0.2 1 0.2149 46 149.2 47.6 -0.2 -1.6 0.2 1.6153 53 152 52 1 1 1 1155 58 153.8 55.6 1.2 2.4 1.2 2.4156 60 154.6 58.4 1.4 1.6 1.4 1.6

156 61 155 60.33333333 10.66666666

7 1 0.666666667153 60 153 60 0 0 0 0

Page 228: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:

144 57 144 57 0 0 0 01.05773195

9144 57 144 57 0 0 0 0144 57 144.8 57 -0.8 0 0.8 0 Maximum:146 57 145.8 55.6 0.2 1.4 0.2 1.4 5146 57 147.6 53.6 -1.6 3.4 1.6 3.4149 50 149.4 51 -0.4 -1 0.4 1 Sum:153 47 150.8 48.4 2.2 -1.4 2.2 1.4 205.2153 44 153.4 44.6 -0.4 -0.6 0.4 0.6153 44 156.6 41.2 -3.6 2.8 3.6 2.8 RMS:

159 38 159.6 37.6 -0.6 0.4 0.6 0.41.42125898

1165 33 163.2 34.6 1.8 -1.6 1.8 1.6168 29 167.2 31.6 0.8 -2.6 0.8 2.6 Mean X:

171 29 170.8 29.6 0.2 -0.6 0.2 0.61.01305841

9173 29 173.6 28.6 -0.6 0.4 0.6 0.4177 28 176.4 28.2 0.6 -0.2 0.6 0.2 Mean Y:

179 28 179.2 28 -0.2 0 0.2 01.10240549

8182 27 182.4 27.8 -0.4 -0.8 0.4 0.8185 28 185 27.8 0 0.2 0 0.2 Maximum X:189 28 187.4 28 1.6 0 1.6 0 4.4190 28 189.4 29 0.6 -1 0.6 1191 29 191.4 30.2 -0.4 -1.2 0.4 1.2 Maximum Y:192 32 192.6 31.2 -0.6 0.8 0.6 0.8 5195 34 194 32.6 1 1.4 1 1.4195 33 195.4 35.4 -0.4 -2.4 0.4 2.4 Sum X:

197 35 197 38.4 0 -3.4 0 3.498.2666666

7198 43 198 41.4 0 1.6 0 1.6200 47 199 45.2 1 1.8 1 1.8 Sum Y:

200 49 200.6 49.6 -0.6 -0.6 0.6 0.6106.933333

3200 52 202 53 -2 -1 2 1205 57 204 56.8 1 0.2 1 0.2 RMS X:

205 60 205.8 61.4 -0.8 -1.4 0.8 1.41.36873952

3210 66 207.4 66.2 2.6 -0.2 2.6 0.2209 72 207.6 70.2 1.4 1.8 1.4 1.8 RMS Y:208 76 207.6 74 0.4 2 0.4 2 1.47190567206 77 206.2 77.2 -0.2 -0.2 0.2 0.2205 79 204.6 80 0.4 -1 0.4 1203 82 203 82.4 0 -0.4 0 0.4201 86 200.8 85.8 0.2 0.2 0.2 0.2200 88 199 89.2 1 -1.2 1 1.2195 94 196.6 93.8 -1.6 0.2 1.6 0.2196 96 194.8 98.6 1.2 -2.6 1.2 2.6191 105 192 104 -1 1 1 1192 110 190.2 108.8 1.8 1.2 1.8 1.2186 115 187.4 114.2 -1.4 0.8 1.4 0.8186 118 185.6 118.2 0.4 -0.2 0.4 0.2182 123 182.6 122.4 -0.6 0.6 0.6 0.6182 125 180.2 126.4 1.8 -1.4 1.8 1.4177 131 177.2 130.4 -0.2 0.6 0.2 0.6174 135 174.2 134.4 -0.2 0.6 0.2 0.6171 138 170.4 138.6 0.6 -0.6 0.6 0.6167 143 166.8 142.6 0.2 0.4 0.2 0.4163 146 162.8 146.4 0.2 -0.4 0.2 0.4159 151 157.8 150.2 1.2 0.8 1.2 0.8154 154 153.6 153 0.4 1 0.4 1

Page 229: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

146 157 150.4 155.4 -4.4 1.6 4.4 1.6146 157 147.8 156.8 -1.8 0.2 1.8 0.2147 158 145.8 155.8 1.2 2.2 1.2 2.2146 158 144.6 153 1.4 5 1.4 5144 149 141.8 149 2.2 0 2.2 0140 143 137 144.8 3 -1.8 3 1.8132 137 132 140.4 0 -3.4 0 3.4123 137 126.8 137 -3.8 0 3.8 0121 136 121.4 134 -0.4 2 0.4 2118 132 116 130.6 2 1.4 2 1.4113 128 112 126.6 1 1.4 1 1.4105 120 107.4 121.6 -2.4 -1.6 2.4 1.6103 117 103 116.2 0 0.8 0 0.8

98 111 99.2 110.6 -1.2 0.4 1.2 0.496 105 95.6 105.2 0.4 -0.2 0.4 0.294 100 92.6 99 1.4 1 1.4 187 93 90.8 92.6 -3.8 0.4 3.8 0.488 86 88.8 85.6 -0.8 0.4 0.8 0.489 79 87 78 2 1 2 186 70 86.6 71.4 -0.6 -1.4 0.6 1.485 62 86.6 65.2 -1.6 -3.2 1.6 3.285 60 86.8 59.8 -1.8 0.2 1.8 0.288 55 88.4 54.6 -0.4 0.4 0.4 0.490 52 91.2 49.8 -1.2 2.2 1.2 2.294 44 95 44.4 -1 -0.4 1 0.499 38 99 39.4 0 -1.4 0 1.4

104 33 103.2 34.8 0.8 -1.8 0.8 1.8108 30 108 31.8 0 -1.8 0 1.8111 29 112.8 30.2 -1.8 -1.2 1.8 1.2118 29 117.8 29.8 0.2 -0.8 0.2 0.8123 30 123 30 0 0 0 0129 31 127.8 30.6 1.2 0.4 1.2 0.4134 31 132 32 2 -1 2 1135 32 135.8 34 -0.8 -2 0.8 2139 36 139.6 36.8 -0.6 -0.8 0.6 0.8142 40 142.6 40 -0.6 0 0.6 0148 45 146.2 44.8 1.8 0.2 1.8 0.2149 47 149.4 49.4 -0.4 -2.4 0.4 2.4153 56 152 53.2 1 2.8 1 2.8155 59 153.2 56 1.8 3 1.8 3155 59 153.6 58.2 1.4 0.8 1.4 0.8

154 59 153.3333333 58.666666670.66666666

70.33333333

3 0.666666667 0.333333333151 58 151 58 0 0 0 0

Page 230: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

50 70 90 110 130 150 170 190 210 23020

40

60

80

100

120

140

160

Heart 1 Trial 4 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

50 70 90 110 130 150 170 190 210 23020

40

60

80

100

120

140

160

Heart 1 Trial 4 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 231: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 4:

Kalman Filter Data:

Kalman:Moving Average X:

Moving Average Y:

Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:144 56 144 56 0 0 0 0 0.953703704145 56 144.3333333 54 0.666666667 2 0.666666667 2144 50 143.2 48.8 0.8 1.2 0.8 1.2 Maximum:143 44 141.4 44.2 1.6 -0.2 1.6 0.2 3.8140 38 138.6 38.8 1.4 -0.8 1.4 0.8135 33 134.8 34 0.2 -1 0.2 1 Sum:131 29 130 30.2 1 -1.2 1 1.2 137.3333333125 26 124.6 27.6 0.4 -1.6 0.4 1.6119 25 119.4 26 -0.4 -1 0.4 1 RMS:113 25 114 25.6 -1 -0.6 1 0.6 1.256612141109 25 108.8 26.4 0.2 -1.4 0.2 1.4104 27 103.8 28 0.2 -1 0.2 1 Mean X:

99 30 99.2 30.4 -0.2 -0.4 0.2 0.4 0.81111111194 33 94.8 34.8 -0.8 -1.8 0.8 1.890 37 90.8 40.8 -0.8 -3.8 0.8 3.8 Mean Y:87 47 87.8 48.2 -0.8 -1.2 0.8 1.2 1.09629629684 57 86 57.2 -2 -0.2 2 0.284 67 85.4 67.6 -1.4 -0.6 1.4 0.6 Maximum X:85 78 85.8 77.8 -0.8 0.2 0.8 0.2 387 89 87.2 87.4 -0.2 1.6 0.2 1.689 98 89.4 96.6 -0.4 1.4 0.4 1.4 Maximum Y:91 105 92.4 104.8 -1.4 0.2 1.4 0.2 3.895 113 95.8 112 -0.8 1 0.8 1

100 119 100.4 118.6 -0.4 0.4 0.4 0.4 Sum X:104 125 106.2 124.8 -2.2 0.2 2.2 0.2 58.4112 131 112.8 130.2 -0.8 0.8 0.8 0.8120 136 120.4 136 -0.4 0 0.4 0 Sum Y:128 140 128.6 142 -0.6 -2 0.6 2 78.93333333138 148 136 147.6 2 0.4 2 0.4145 155 142 152.4 3 2.6 3 2.6 RMS X:149 159 146.4 156.4 2.6 2.6 2.6 2.6 1.063769213150 160 148.8 158.8 1.2 1.2 1.2 1.2150 160 149.8 159.6 0.2 0.4 0.2 0.4 RMS Y:150 160 150 159.4 0 0.6 0 0.6 1.423567072150 159 149.8 158.8 0.2 0.2 0.2 0.2150 158 150 157.8 0 0.2 0 0.2149 157 150.6 156.4 -1.6 0.6 1.6 0.6151 155 151.8 154.8 -0.8 0.2 0.8 0.2153 153 154 152.8 -1 0.2 1 0.2156 151 157.4 150 -1.4 1 1.4 1161 148 161.4 146.6 -0.4 1.4 0.4 1.4166 143 166 142.8 0 0.2 0 0.2171 138 171 137.8 0 0.2 0 0.2176 134 175.8 132.4 0.2 1.6 0.2 1.6181 126 180.4 126.8 0.6 -0.8 0.6 0.8185 121 184.8 120.8 0.2 0.2 0.2 0.2189 115 189.2 114.4 -0.2 0.6 0.2 0.6193 108 193.2 108.4 -0.2 -0.4 0.2 0.4198 102 197 101.8 1 0.2 1 0.2201 96 200.2 95.2 0.8 0.8 0.8 0.8204 88 202.8 89 1.2 -1 1.2 1205 82 204.8 82 0.2 0 0.2 0206 77 206 74 0 3 0 3208 67 206 65.8 2 1.2 2 1.2207 56 205.2 56.4 1.8 -0.4 1.8 0.4204 47 203.4 46.4 0.6 0.6 0.6 0.6201 35 200.2 37.6 0.8 -2.6 0.8 2.6197 27 195.8 30.6 1.2 -3.6 1.2 3.6

Page 232: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

192 23 190.6 25.4 1.4 -2.4 1.4 2.4185 21 184.8 22.8 0.2 -1.8 0.2 1.8178 21 178.8 22.4 -0.8 -1.4 0.8 1.4172 22 172.8 23.6 -0.8 -1.6 0.8 1.6167 25 166.8 26.4 0.2 -1.4 0.2 1.4162 29 161.4 30.4 0.6 -1.4 0.6 1.4155 35 156.4 36.2 -1.4 -1.2 1.4 1.2151 41 151.6 42.8 -0.6 -1.8 0.6 1.8147 51 147.8 49.6 -0.8 1.4 0.8 1.4143 58 145.6 55.2 -2.6 2.8 2.6 2.8143 63 144.2 59.6 -1.2 3.4 1.2 3.4144 63 143.8 61.8 0.2 1.2 0.2 1.2

144 63 144.3333333 62.66666667 -0.3333333330.33333333

3 0.333333333 0.333333333145 62 145 62 0 0 0 0

Page 233: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y:

Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:145 57 145 57 0 0 0 0 1.194907407144 56 144.3333333 54 -0.333333333 2 0.333333333 2144 49 142.8 48.8 1.2 0.2 1.2 0.2 Maximum:143 44 140.6 44.2 2.4 -0.2 2.4 0.2 5138 38 138 39 0 -1 0 1134 34 133.8 34.8 0.2 -0.8 0.2 0.8 Sum:131 30 128.8 31.6 2.2 -1.6 2.2 1.6 172.0666667123 28 123.8 29.4 -0.8 -1.4 0.8 1.4118 28 119 28.2 -1 -0.2 1 0.2 RMS:113 27 113.6 28.2 -0.6 -1.2 0.6 1.2 1.604247833110 28 109 29.2 1 -1.2 1 1.2104 30 104.4 30.4 -0.4 -0.4 0.4 0.4 Mean X:100 33 100 32.8 0 0.2 0 0.2 0.95

95 34 95.6 37.8 -0.6 -3.8 0.6 3.891 39 92 44 -1 -5 1 5 Mean Y:88 53 89.4 51.2 -1.4 1.8 1.4 1.8 1.43981481586 61 88 60.6 -2 0.4 2 0.487 69 87.6 71 -0.6 -2 0.6 2 Maximum X:88 81 88.2 80 -0.2 1 0.2 1 489 91 89.6 88.4 -0.6 2.6 0.6 2.691 98 91.6 97.4 -0.6 0.6 0.6 0.6 Maximum Y:93 103 94.4 104.8 -1.4 -1.8 1.4 1.8 597 114 97.6 111.4 -0.6 2.6 0.6 2.6

102 118 102.6 117.8 -0.6 0.2 0.6 0.2 Sum X:105 124 108.4 124.4 -3.4 -0.4 3.4 0.4 68.4116 130 115.2 129.4 0.8 0.6 0.8 0.6122 136 123 135.8 -1 0.2 1 0.2 Sum Y:131 139 130.8 142.2 0.2 -3.2 0.2 3.2 103.6666667141 150 137 147.8 4 2.2 4 2.2144 156 141.8 152 2.2 4 2.2 4 RMS X:147 158 145 155.8 2 2.2 2 2.2 1.295909995146 157 146.4 157.4 -0.4 -0.4 0.4 0.4147 158 147.4 157.8 -0.4 0.2 0.4 0.2 RMS Y:148 158 147.8 157.6 0.2 0.4 0.2 0.4 1.862213604149 158 148.4 157.4 0.6 0.6 0.6 0.6149 157 149.6 156.6 -0.6 0.4 0.6 0.4149 156 150.8 155.6 -1.8 0.4 1.8 0.4153 154 152.6 154.2 0.4 -0.2 0.4 0.2154 153 155.4 152.4 -1.4 0.6 1.4 0.6158 151 159.2 149.4 -1.2 1.6 1.2 1.6163 148 163.2 146 -0.2 2 0.2 2168 141 167.8 142.2 0.2 -1.2 0.2 1.2173 137 172.4 136.8 0.6 0.2 0.6 0.2177 134 176.8 131.4 0.2 2.6 0.2 2.6181 124 181 126.4 0 -2.4 0 2.4185 121 185.2 120.4 -0.2 0.6 0.2 0.6189 116 189.6 114 -0.6 2 0.6 2194 107 193.4 108.4 0.6 -1.4 0.6 1.4199 102 197.2 101.8 1.8 0.2 1.8 0.2200 96 200.2 95.2 -0.2 0.8 0.2 0.8204 88 202.4 89.4 1.6 -1.4 1.6 1.4204 83 204.2 81.8 -0.2 1.2 0.2 1.2205 78 205.4 73.4 -0.4 4.6 0.4 4.6208 64 205 65.2 3 -1.2 3 1.2206 54 204 55.4 2 -1.4 2 1.4202 47 202 45.6 0 1.4 0 1.4199 34 198.6 38.2 0.4 -4.2 0.4 4.2195 29 194 32.4 1 -3.4 1 3.4191 27 188.8 28 2.2 -1 2.2 1183 25 183.4 26.4 -0.4 -1.4 0.4 1.4176 25 178 26.4 -2 -1.4 2 1.4

Page 234: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

172 26 172.2 27.6 -0.2 -1.6 0.2 1.6168 29 166.6 30.4 1.4 -1.4 1.4 1.4162 33 162 34.2 0 -1.2 0 1.2155 39 157.2 40 -2.2 -1 2.2 1153 44 152.6 46.2 0.4 -2.2 0.4 2.2148 55 149.4 52 -1.4 3 1.4 3145 60 147.8 56.2 -2.8 3.8 2.8 3.8146 62 146.4 59.6 -0.4 2.4 0.4 2.4147 60 146.2 60.6 0.8 -0.6 0.8 0.6146 61 146.6666667 60.33333333 -0.666666667 0.666666667 0.666666667 0.666666667147 60 147 60 0 0 0 0

Page 235: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

50 70 90 110 130 150 170 190 210 23020

40

60

80

100

120

140

160

Heart 1 Trial 4 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

50 70 90 110 130 150 170 190 210 23020

40

60

80

100

120

140

160

Heart 1 Trial 4 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 236: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 5:

Kalman Filter Data:

Kalman:

Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:

119 131 119 131 0 0 0 01.02791666

7128 137 128 137 0 0 0 0137 143 135.2 141.6 1.8 1.4 1.8 1.4 Maximum:144 147 141.6 145.6 2.4 1.4 2.4 1.4 4.4148 150 146.4 148.6 1.6 1.4 1.6 1.4151 151 149.4 150.4 1.6 0.6 1.6 0.6 Sum:

152 152 151 151.4 1 0.6 1 0.6164.466666

7152 152 151.8 151.8 0.2 0.2 0.2 0.2152 152 152.4 152.6 -0.4 -0.6 0.4 0.6 RMS:

152 152 153.4 153.4 -1.4 -1.4 1.4 1.41.28297614

1154 155 154.6 154.4 -0.6 0.6 0.6 0.6157 156 156 155.4 1 0.6 1 0.6 Mean X:158 157 157.4 156.4 0.6 0.6 0.6 0.6 0.9375159 157 158.4 156.8 0.6 0.2 0.6 0.2159 157 158.6 156.8 0.4 0.2 0.4 0.2 Mean Y:

159 157 158.8 156.4 0.2 0.6 0.2 0.61.11833333

3158 156 159 155.6 -1 0.4 1 0.4

159 155 160 154.2 -1 0.8 1 0.8Maximum X:

160 153 162.4 151.6 -2.4 1.4 2.4 1.4 2.8164 150 166 148.2 -2 1.8 2 1.8

171 144 170.2 143.6 0.8 0.4 0.8 0.4Maximum Y:

176 139 175.4 137.6 0.6 1.4 0.6 1.4 4.4180 132 181 130.4 -1 1.6 1 1.6186 123 186.2 122.6 -0.2 0.4 0.2 0.4 Sum X:192 114 191.2 113.8 0.8 0.2 0.8 0.2 75197 105 195.6 105 1.4 0 1.4 0201 95 199.2 96.6 1.8 -1.6 1.8 1.6 Sum Y:

202 88 201.6 89.4 0.4 -1.4 0.4 1.489.4666666

7204 81 203 81.6 1 -0.6 1 0.6204 78 203.4 73.6 0.6 4.4 0.6 4.4 RMS X:

204 66 203 64.6 1 1.4 1 1.41.14170924

5203 55 201.4 55.2 1.6 -0.2 1.6 0.2200 43 199 45.2 1 -2.2 1 2.2 RMS Y:

196 34 195.8 37.2 0.2 -3.2 0.2 3.21.41016153

5192 28 191.8 31.2 0.2 -3.2 0.2 3.2188 26 187.6 27.4 0.4 -1.4 0.4 1.4183 25 183.2 25.8 -0.2 -0.8 0.2 0.8179 24 178.4 26 0.6 -2 0.6 2174 26 173.4 27.2 0.6 -1.2 0.6 1.2168 29 168.6 29.6 -0.6 -0.6 0.6 0.6163 32 163.6 33.6 -0.6 -1.6 0.6 1.6159 37 158.8 38 0.2 -1 0.2 1154 44 154.8 43.2 -0.8 0.8 0.8 0.8150 48 151.6 48.4 -1.6 -0.4 1.6 0.4148 55 149.4 53 -1.4 2 1.4 2147 58 148.4 56 -1.4 2 1.4 2148 60 148.2 58 -0.2 2 0.2 2149 59 148.2 58.2 0.8 0.8 0.8 0.8149 58 148.2 57.4 0.8 0.6 0.8 0.6

Page 237: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

148 56 147.6 55 0.4 1 0.4 1147 54 146 52 1 2 1 2145 48 143.6 48 1.4 0 1.4 0141 44 140.2 43.2 0.8 0.8 0.8 0.8137 38 135.4 38 1.6 0 1.6 0131 32 129.6 33.6 1.4 -1.6 1.4 1.6123 28 123.8 30.2 -0.8 -2.2 0.8 2.2116 26 117.6 28.4 -1.6 -2.4 1.6 2.4112 27 111.6 28.4 0.4 -1.4 0.4 1.4106 29 106.4 30.2 -0.4 -1.2 0.4 1.2101 32 101.8 33.6 -0.8 -1.6 0.8 1.6

97 37 97.2 38.2 -0.2 -1.2 0.2 1.293 43 93.2 44 -0.2 -1 0.2 189 50 89.8 50 -0.8 0 0.8 086 58 87.6 57.4 -1.6 0.6 1.6 0.684 62 86.4 65.6 -2.4 -3.6 2.4 3.686 74 86.6 73.8 -0.6 0.2 0.6 0.287 84 88 82.4 -1 1.6 1 1.690 91 90.8 91.6 -0.8 -0.6 0.8 0.693 101 94 99.4 -1 1.6 1 1.698 108 97.8 106.4 0.2 1.6 0.2 1.6

102 113 101.6 112.4 0.4 0.6 0.4 0.6106 119 106.4 117.8 -0.4 1.2 0.4 1.2109 121 111.8 123 -2.8 -2 2.8 2117 128 118.4 128.6 -1.4 -0.6 1.4 0.6125 134 126 134.2 -1 -0.2 1 0.2135 141 134.2 140.2 0.8 0.8 0.8 0.8144 147 141.6 145.4 2.4 1.6 2.4 1.6150 151 147.6 149.6 2.4 1.4 2.4 1.4

154 154 153 153.3333333 10.66666666

7 1 0.666666667155 155 155 155 0 0 0 0

Page 238: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw:

Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:120 132 120 132 0 0 0 0 1.31125

128 137128.666666

7137.333333

3

-0.66666666

7

-0.33333333

30.66666666

70.33333333

3138 143 135.2 141.6 2.8 1.4 2.8 1.4 Maximum:143 147 141 145 2 2 2 2 8.4147 149 145.4 147.8 1.6 1.2 1.6 1.2149 149 147.8 149.2 1.2 -0.2 1.2 0.2 Sum:150 151 149.2 150 0.8 1 0.8 1 209.8150 150 150.2 150.8 -0.2 -0.8 0.2 0.8150 151 151.4 152.2 -1.4 -1.2 1.4 1.2 RMS:

152 153 153 153.2 -1 -0.2 1 0.21.76215619

2155 156 154.8 154.6 0.2 1.4 0.2 1.4158 156 156.6 155.8 1.4 0.2 1.4 0.2 Mean X:

159 157 157.8 156.4 1.2 0.6 1.2 0.61.15166666

7159 157 158.6 156.6 0.4 0.4 0.4 0.4158 156 158.6 156.6 -0.6 -0.6 0.6 0.6 Mean Y:

159 157 158.6 156.2 0.4 0.8 0.4 0.81.47083333

3158 156 159 155.2 -1 0.8 1 0.8

159 155 160.8 153.8 -1.8 1.2 1.8 1.2Maximum X:

161 152 164 150.6 -3 1.4 3 1.4 5.6167 149 167.8 147 -0.8 2 0.8 2

175 141 172 142.2 3 -1.2 3 1.2Maximum Y:

177 138 177.6 135.8 -0.6 2.2 0.6 2.2 8.4180 131 182.8 128.4 -2.8 2.6 2.8 2.6189 120 187.2 121.2 1.8 -1.2 1.8 1.2 Sum X:

193 112 191.8 112.4 1.2 -0.4 1.2 0.492.1333333

3197 105 195.8 104 1.2 1 1.2 1200 94 198.6 96.4 1.4 -2.4 1.4 2.4 Sum Y:

200 89 200.6 90.4 -0.6 -1.4 0.6 1.4117.666666

7203 82 201.8 81.8 1.2 0.2 1.2 0.2203 82 202 73.6 1 8.4 1 8.4 RMS X:

203 62 201.6 64 1.4 -2 1.4 21.48327715

2201 53 200 54.6 1 -1.6 1 1.6198 41 197.4 44.6 0.6 -3.6 0.6 3.6 RMS Y:

195 35 194.2 38.2 0.8 -3.2 0.8 3.22.00256779

6190 32 190.4 33.4 -0.4 -1.4 0.4 1.4187 30 186.6 30.8 0.4 -0.8 0.4 0.8182 29 182.4 29.6 -0.4 -0.6 0.4 0.6179 28 177.8 29.8 1.2 -1.8 1.2 1.8174 29 173 30.8 1 -1.8 1 1.8167 33 168.6 33 -1.6 0 1.6 0163 35 163.6 36.8 -0.6 -1.8 0.6 1.8160 40 159.2 40.8 0.8 -0.8 0.8 0.8154 47 155.8 45.6 -1.8 1.4 1.8 1.4152 49 153.2 50.2 -1.2 -1.2 1.2 1.2150 57 151.4 53.8 -1.4 3.2 1.4 3.2150 58 151 55.8 -1 2.2 1 2.2151 58 150.6 57.2 0.4 0.8 0.4 0.8152 57 150.2 56.8 1.8 0.2 1.8 0.2150 56 149.8 55.6 0.2 0.4 0.2 0.4148 55 148.6 53.2 -0.6 1.8 0.6 1.8148 52 146 50.4 2 1.6 2 1.6145 46 143.4 46.6 1.6 -0.6 1.6 0.6

Page 239: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

139 43 139.4 41.8 -0.4 1.2 0.4 1.2137 37 134 37.2 3 -0.2 3 0.2128 31 128.2 33.8 -0.2 -2.8 0.2 2.8121 29 123 31.2 -2 -2.2 2 2.2116 29 116.8 30.4 -0.8 -1.4 0.8 1.4113 30 111.6 31.2 1.4 -1.2 1.4 1.2106 33 107 33.6 -1 -0.6 1 0.6102 35 102.8 37 -0.8 -2 0.8 2

98 41 98 41.6 0 -0.6 0 0.695 46 94.2 47.2 0.8 -1.2 0.8 1.289 53 91.2 52.4 -2.2 0.6 2.2 0.687 61 89.4 60 -2.4 1 2.4 187 61 88.4 68 -1.4 -7 1.4 789 79 89 75.6 0 3.4 0 3.490 86 90.8 83.8 -0.8 2.2 0.8 2.292 91 93.4 93 -1.4 -2 1.4 296 102 96.2 99.6 -0.2 2.4 0.2 2.4

100 107 99.8 106 0.2 1 0.2 1103 112 103 111.4 0 0.6 0 0.6108 118 108 117.2 0 0.8 0 0.8108 118 113.6 122.8 -5.6 -4.8 5.6 4.8121 131 120.8 128.8 0.2 2.2 0.2 2.2128 135 128.2 134.6 -0.2 0.4 0.2 0.4139 142 136.6 141 2.4 1 2.4 1145 147 142.8 145.2 2.2 1.8 2.2 1.8150 150 147.6 148.8 2.4 1.2 2.4 1.2

152 152151.333333

3151.666666

70.66666666

70.33333333

30.66666666

70.33333333

3152 153 152 153 0 0 0 0

Page 240: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

50 70 90 110 130 150 170 190 210 23020

40

60

80

100

120

140

160

Heart 1 Trial 5 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

50 70 90 110 130 150 170 190 210 23020

40

60

80

100

120

140

160

Heart 1 Trial 5 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 241: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Lightning 1:

Page 242: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 1:

Kalman Filter Data:

Kalman:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:87 56 87 56 0 0 0 0 1.05162601692 58 91.66666667 57.66666667 0.333333333 0.333333333 0.333333333 0.33333333396 59 96.6 59.8 -0.6 -0.8 0.6 0.8 Maximum:

102 62 101.4 62.2 0.6 -0.2 0.6 0.2 4.2106 64 106 64.6 0 -0.6 0 0.6111 68 110.6 67.2 0.4 0.8 0.4 0.8 Sum:115 70 114.2 69.6 0.8 0.4 0.8 0.4 172.4666667119 72 117 71.6 2 0.4 2 0.4120 74 118.8 72.8 1.2 1.2 1.2 1.2 RMS:120 74 119.2 73.8 0.8 0.2 0.8 0.2 1.373135592120 74 118.2 74.8 1.8 -0.8 1.8 0.8117 75 116.6 75.8 0.4 -0.8 0.4 0.8 Mean X:114 77 114.4 77.4 -0.4 -0.4 0.4 0.4 1.201626016112 79 112.4 79.4 -0.4 -0.4 0.4 0.4109 82 111.4 81.4 -2.4 0.6 2.4 0.6 Mean Y:110 84 112 83.6 -2 0.4 2 0.4 0.901626016112 85 114 85.8 -2 -0.8 2 0.8117 88 117.8 88 -0.8 0 0.8 0 Maximum X:122 90 122.6 90.6 -0.6 -0.6 0.6 0.6 3.2128 93 127.6 93.6 0.4 -0.6 0.4 0.6134 97 132.2 96.4 1.8 0.6 1.8 0.6 Maximum Y:137 100 136.4 99.2 0.6 0.8 0.6 0.8 4.2140 102 139.8 101.8 0.2 0.2 0.2 0.2143 104 142 104 1 0 1 0 Sum X:145 106 143 105.4 2 0.6 2 0.6 98.53333333145 108 142.8 107 2.2 1 2.2 1142 107 141.4 108.4 0.6 -1.4 0.6 1.4 Sum Y:139 110 139.8 109.8 -0.8 0.2 0.8 0.2 73.93333333136 111 138.6 111.2 -2.6 -0.2 2.6 0.2137 113 138.6 113.4 -1.6 -0.4 1.6 0.4 RMS X:139 115 140 115.6 -1 -0.6 1 0.6 1.459999629142 118 143.2 118.2 -1.2 -0.2 1.2 0.2146 121 147.4 121.4 -1.4 -0.4 1.4 0.4 RMS Y:152 124 152.2 124.6 -0.2 -0.6 0.2 0.6 1.280392047158 129 157.2 127.8 0.8 1.2 0.8 1.2163 131 162.4 131 0.6 0 0.6 0167 134 167.6 134 -0.6 0 0.6 0172 137 172.6 136.4 -0.6 0.6 0.6 0.6178 139 177.6 139 0.4 0 0.4 0183 141 183.2 141.4 -0.2 -0.4 0.2 0.4188 144 188.8 143.6 -0.8 0.4 0.8 0.4195 146 194 145.8 1 0.2 1 0.2200 148 198.4 147.6 1.6 0.4 1.6 0.4204 150 201.8 148.6 2.2 1.4 2.2 1.4205 150 203.2 148.8 1.8 1.2 1.8 1.2205 149 203 147.8 2 1.2 2 1.2202 147 200.8 145.2 1.2 1.8 1.2 1.8199 143 197.4 141 1.6 2 1.6 2193 137 192.6 135.4 0.4 1.6 0.4 1.6188 129 187.4 128.8 0.6 0.2 0.6 0.2181 121 181.6 121 -0.6 0 0.6 0176 114 176 113.2 0 0.8 0 0.8170 104 171 106.4 -1 -2.4 1 2.4165 98 167.4 101.2 -2.4 -3.2 2.4 3.2163 95 165 97.4 -2 -2.4 2 2.4163 95 164.6 95.4 -1.6 -0.4 1.6 0.4

Page 243: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

164 95 165.8 94.2 -1.8 0.8 1.8 0.8168 94 167.8 93.6 0.2 0.4 0.2 0.4171 92 169.8 92.8 1.2 -0.8 1.2 0.8173 92 171.2 91.6 1.8 0.4 1.8 0.4173 91 170.6 89.4 2.4 1.6 2.4 1.6171 89 168 86.2 3 2.8 3 2.8165 83 163.4 81.6 1.6 1.4 1.6 1.4158 76 157.6 76 0.4 0 0.4 0150 69 151.4 70.2 -1.4 -1.2 1.4 1.2144 63 146.4 65.4 -2.4 -2.4 2.4 2.4140 60 143.2 61.6 -3.2 -1.6 3.2 1.6140 59 142.2 58.8 -2.2 0.2 2.2 0.2142 57 143 57 -1 0 1 0145 55 144.8 55.6 0.2 -0.6 0.2 0.6148 54 146 53.4 2 0.6 2 0.6149 53 145.8 50.6 3.2 2.4 3.2 2.4146 48 143.8 46.4 2.2 1.6 2.2 1.6141 43 139.8 40.8 1.2 2.2 1.2 2.2135 34 134.4 33.6 0.6 0.4 0.6 0.4128 26 129.2 26.6 -1.2 -0.6 1.2 0.6122 17 124.2 20.6 -2.2 -3.6 2.2 3.6120 13 119.6 17.2 0.4 -4.2 0.4 4.2116 13 114.8 16.4 1.2 -3.4 1.2 3.4112 17 109.6 18.4 2.4 -1.4 2.4 1.4104 22 104 22 0 0 0 0

96 27 96 27 0 0 0 0

Page 244: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:

Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:88 57 88 57 0 0 0 0 1.19349593592 58 92 58 0 0 0 096 59 97 60.4 -1 -1.4 1 1.4 Maximum:

103 63 101.6 62.8 1.4 0.2 1.4 0.2 5106 65 106.2 65.4 -0.2 -0.4 0.2 0.4111 69 110.8 68 0.2 1 0.2 1 Sum:115 71 114 70.2 1 0.8 1 0.8 195.7333333119 72 116.4 71.8 2.6 0.2 2.6 0.2119 74 117.8 72.8 1.2 1.2 1.2 1.2 RMS:118 73 117.8 73.8 0.2 -0.8 0.2 0.8 1.563593249118 74 116.6 75 1.4 -1 1.4 1115 76 115.2 76.2 -0.2 -0.2 0.2 0.2 Mean X:113 78 113.4 78.2 -0.4 -0.2 0.4 0.2 1.263414634112 80 112.4 80.2 -0.4 -0.2 0.4 0.2109 83 112.4 82 -3.4 1 3.4 1 Mean Y:113 84 113.8 84.2 -0.8 -0.2 0.8 0.2 1.123577236115 85 116.6 86.4 -1.6 -1.4 1.6 1.4120 89 121 88.6 -1 0.4 1 0.4 Maximum X:126 91 125.4 91.4 0.6 -0.4 0.6 0.4 3.6131 94 129.8 94.4 1.2 -0.4 1.2 0.4135 98 133.8 97 1.2 1 1.2 1 Maximum Y:137 100 137.2 99.6 -0.2 0.4 0.2 0.4 5140 102 139.8 102.2 0.2 -0.2 0.2 0.2143 104 141.4 104 1.6 0 1.6 0 Sum X:144 107 142 105 2 2 2 2 103.6143 107 141.4 107 1.6 0 1.6 0140 105 139.8 108.4 0.2 -3.4 0.2 3.4 Sum Y:137 112 138.8 109.8 -1.8 2.2 1.8 2.2 92.13333333135 111 138.6 111.6 -3.6 -0.6 3.6 0.6139 114 139.6 114.4 -0.6 -0.4 0.6 0.4 RMS X:142 116 142 116.6 0 -0.6 0 0.6 1.558923957145 119 145.8 119.4 -0.8 -0.4 0.8 0.4149 123 150 122.8 -1 0.2 1 0.2 RMS Y:154 125 154.4 125.8 -0.4 -0.8 0.4 0.8 1.568248639160 131 158.8 128.8 1.2 2.2 1.2 2.2164 131 163.6 131.6 0.4 -0.6 0.4 0.6167 134 168.8 134.4 -1.8 -0.4 1.8 0.4173 137 173.4 136.6 -0.4 0.4 0.4 0.4180 139 178.4 139.2 1.6 -0.2 1.6 0.2183 142 184.2 141.8 -1.2 0.2 1.2 0.2189 144 189.8 144 -0.8 0 0.8 0196 147 194.4 146.2 1.6 0.8 1.6 0.8201 148 198.4 147.6 2.6 0.4 2.6 0.4203 150 201.2 148.4 1.8 1.6 1.8 1.6203 149 201.8 148 1.2 1 1.2 1203 148 200.8 146.6 2.2 1.4 2.2 1.4199 145 198.4 143.6 0.6 1.4 0.6 1.4196 141 195 139 1 2 1 2191 135 190.4 133.2 0.6 1.8 0.6 1.8186 126 185.6 127 0.4 -1 0.4 1180 119 180.4 119.2 -0.4 -0.2 0.4 0.2175 114 175.4 112 -0.4 2 0.4 2170 102 171.2 106.6 -1.2 -4.6 1.2 4.6166 99 168.4 102.6 -2.4 -3.6 2.4 3.6165 99 166.8 99.4 -1.8 -0.4 1.8 0.4166 99 167.2 98 -1.2 1 1.2 1167 98 168.8 96.8 -1.8 1.2 1.8 1.2172 95 170.4 95.6 1.6 -0.6 1.6 0.6174 93 171.6 94.2 2.4 -1.2 2.4 1.2173 93 172 92.4 1 0.6 1 0.6

Page 245: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

172 92 169.8 89.4 2.2 2.6 2.2 2.6169 89 166 85.6 3 3.4 3 3.4161 80 161 80.6 0 -0.6 0 0.6155 74 155.2 74.6 -0.2 -0.6 0.2 0.6148 68 149.8 69.2 -1.8 -1.2 1.8 1.2143 62 146.4 65.6 -3.4 -3.6 3.4 3.6142 62 144.6 62.4 -2.6 -0.4 2.6 0.4144 62 144.8 60 -0.8 2 0.8 2146 58 146.4 58.6 -0.4 -0.6 0.4 0.6149 56 147.8 57 1.2 -1 1.2 1151 55 147.8 54 3.2 1 3.2 1149 54 146 50.6 3 3.4 3 3.4144 47 142.8 45.6 1.2 1.4 1.2 1.4137 41 137.8 39.4 -0.8 1.6 0.8 1.6133 31 132.4 31.8 0.6 -0.8 0.6 0.8126 24 128 25.6 -2 -1.6 2 1.6122 16 124 21 -2 -5 2 5122 16 119.8 19.6 2.2 -3.6 2.2 3.6117 18 115 20.2 2 -2.2 2 2.2112 24 109.6 23.2 2.4 0.8 2.4 0.8102 27 103 27.33333333 -1 -0.333333333 1 0.333333333

95 31 95 31 0 0 0 0

Page 246: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

50 70 90 110 130 150 170 190 210 23010

30

50

70

90

110

130

150

Lightning 1 Trial 1 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

50 70 90 110 130 150 170 190 210 23010

30

50

70

90

110

130

150

Lightning 1 Trial 1 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 247: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 2:

Kalman Filter Data:

Kalman:

Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:121 14 121 14 0 0 0 0 0.931464174122 14 121.3333333 14.33333333 0.666666667 -0.333333333 0.666666667 0.333333333121 15 122.2 15.4 -1.2 -0.4 1.2 0.4 Maximum:122 16 123.8 17.2 -1.8 -1.2 1.8 1.2 4.6125 18 126.4 19.8 -1.4 -1.8 1.4 1.8129 23 130 23.2 -1 -0.2 1 0.2 Sum:135 27 134.6 27.8 0.4 -0.8 0.4 0.8 199.3333333139 32 139.6 33.2 -0.6 -1.2 0.6 1.2145 39 144.2 38.8 0.8 0.2 0.8 0.2 RMS:150 45 147.8 44.4 2.2 0.6 2.2 0.6 1.239317804152 51 150.6 49.4 1.4 1.6 1.4 1.6153 55 151.8 53 1.2 2 1.2 2 Mean X:153 57 151.2 55.8 1.8 1.2 1.8 1.2 1.188161994151 57 149.6 57.4 1.4 -0.4 1.4 0.4147 59 147.6 58.6 -0.6 0.4 0.6 0.4 Mean Y:144 59 145.8 59.6 -1.8 -0.6 1.8 0.6 0.674766355143 61 144.8 61.6 -1.8 -0.6 1.8 0.6144 62 145.6 63.8 -1.6 -1.8 1.6 1.8 Maximum X:146 67 148.2 67 -2.2 0 2.2 0 4.6151 70 152 70.6 -1 -0.6 1 0.6157 75 156.6 75 0.4 0 0.4 0 Maximum Y:162 79 161.6 79.4 0.4 -0.4 0.4 0.4 2.8167 84 166.6 84 0.4 0 0.4 0171 89 170.4 87.8 0.6 1.2 0.6 1.2 Sum X:176 93 172.8 91 3.2 2 3.2 2 127.1333333176 94 173.4 93.4 2.6 0.6 2.6 0.6174 95 172.2 95 1.8 0 1.8 0 Sum Y:170 96 169.6 96 0.4 0 0.4 0 72.2165 97 166.8 97.2 -1.8 -0.2 1.8 0.2163 98 165 99 -2 -1 2 1 RMS X:162 100 165.2 102.4 -3.2 -2.4 3.2 2.4 1.49556631165 104 167.2 106.8 -2.2 -2.8 2.2 2.8171 113 170.4 111.8 0.6 1.2 0.6 1.2 RMS Y:175 119 174.8 117.8 0.2 1.2 0.2 1.2 0.913837321179 123 179.4 124 -0.4 -1 0.4 1184 130 183.6 129.4 0.4 0.6 0.4 0.6188 135 187.2 134.2 0.8 0.8 0.8 0.8192 140 190.8 139.4 1.2 0.6 1.2 0.6193 143 194 144 -1 -1 1 1197 149 196.8 147.8 0.2 1.2 0.2 1.2200 153 198.8 150.6 1.2 2.4 1.2 2.4202 154 200 152.4 2 1.6 2 1.6202 154 199.8 152.4 2.2 1.6 2.2 1.6199 152 197.4 150.6 1.6 1.4 1.6 1.4196 149 193.4 147.6 2.6 1.4 2.6 1.4188 144 188.2 143.8 -0.2 0.2 0.2 0.2182 139 182.2 139.6 -0.2 -0.6 0.2 0.6176 135 175.6 135 0.4 0 0.4 0169 131 169 130.4 0 0.6 0 0.6163 126 162.6 126 0.4 0 0.4 0155 121 156.2 121.8 -1.2 -0.8 1.2 0.8150 117 150.4 117.6 -0.4 -0.6 0.4 0.6144 114 144.6 113.6 -0.6 0.4 0.6 0.4140 110 139.8 110 0.2 0 0.2 0134 106 136.2 107.2 -2.2 -1.2 2.2 1.2131 103 134.2 104.8 -3.2 -1.8 3.2 1.8132 103 133.8 103.2 -1.8 -0.2 1.8 0.2134 102 135 102.4 -1 -0.4 1 0.4

Page 248: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

138 102 138 102 0 0 0 0140 102 141.2 101.4 -1.2 0.6 1.2 0.6146 101 144.2 101 1.8 0 1.8 0148 100 145.6 100.2 2.4 -0.2 2.4 0.2149 100 145.4 98.6 3.6 1.4 3.6 1.4145 98 142.8 96.6 2.2 1.4 2.2 1.4139 94 138.4 94.4 0.6 -0.4 0.6 0.4133 91 132.6 91.6 0.4 -0.6 0.4 0.6126 89 126.4 88.8 -0.4 0.2 0.4 0.2120 86 120.8 86.2 -0.8 -0.2 0.8 0.2114 84 115.6 83.8 -1.6 0.2 1.6 0.2111 81 111.4 81.4 -0.4 -0.4 0.4 0.4107 79 108.6 79.4 -1.6 -0.4 1.6 0.4105 77 107.6 77.8 -2.6 -0.8 2.6 0.8106 76 107.8 76.4 -1.8 -0.4 1.8 0.4109 76 109.6 75 -0.6 1 0.6 1112 74 112.6 73.8 -0.6 0.2 0.6 0.2116 72 115.8 72.4 0.2 -0.4 0.2 0.4120 71 119 70.8 1 0.2 1 0.2122 69 121.8 69.6 0.2 -0.6 0.2 0.6125 68 122.8 68 2.2 0 2.2 0126 68 121.4 65.8 4.6 2.2 4.6 2.2121 64 118.6 63.4 2.4 0.6 2.4 0.6113 60 113.8 60.8 -0.8 -0.8 0.8 0.8108 57 107.4 57.6 0.6 -0.6 0.6 0.6101 55 100.8 54.8 0.2 0.2 0.2 0.2

94 52 94.8 52.2 -0.8 -0.2 0.8 0.288 50 89.6 49.8 -1.6 0.2 1.6 0.283 47 85 47 -2 0 2 082 45 81.2 44.6 0.8 0.4 0.8 0.478 41 78.6 42.4 -0.6 -1.4 0.6 1.475 40 77.6 40.8 -2.6 -0.8 2.6 0.875 39 77.8 39.4 -2.8 -0.4 2.8 0.478 39 79.6 38.8 -1.6 0.2 1.6 0.283 38 83.2 38.2 -0.2 -0.2 0.2 0.287 38 87.8 37.6 -0.8 0.4 0.8 0.493 37 92.8 36.6 0.2 0.4 0.2 0.498 36 97.4 35.4 0.6 0.6 0.6 0.6

103 34 102 33.8 1 0.2 1 0.2106 32 105.6 32 0.4 0 0.4 0110 30 109 30.2 1 -0.2 1 0.2111 28 112.2 28.4 -1.2 -0.4 1.2 0.4115 27 115.4 26.6 -0.4 0.4 0.4 0.4119 25 118.4 24.6 0.6 0.4 0.6 0.4122 23 121.6 22.6 0.4 0.4 0.4 0.4125 20 124.2 20.4 0.8 -0.4 0.8 0.4127 18 125.8 18.6 1.2 -0.6 1.2 0.6128 16 127.3333333 16.66666667 0.666666667 -0.666666667 0.666666667 0.666666667127 16 127 16 0 0 0 0

Page 249: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:122 15 122 15 0 0 0 0 1.19470405121 14 121.3333333 15 -0.333333333 -1 0.333333333 1121 16 122.8 16.2 -1.8 -0.2 1.8 0.2 Maximum:123 16 124.6 18.2 -1.6 -2.2 1.6 2.2 5.4127 20 127.8 21.2 -0.8 -1.2 0.8 1.2131 25 131.8 24.6 -0.8 0.4 0.8 0.4 Sum:137 29 136.4 29.8 0.6 -0.8 0.6 0.8 255.6666667141 33 141.2 35.2 -0.2 -2.2 0.2 2.2146 42 145.2 40.4 0.8 1.6 0.8 1.6 RMS:151 47 148.2 45.6 2.8 1.4 2.8 1.4 1.540785089151 51 150.2 50 0.8 1 0.8 1152 55 150.6 52.6 1.4 2.4 1.4 2.4 Mean X:151 55 149.2 55 1.8 0 1.8 0 1.351401869148 55 147.6 56.4 0.4 -1.4 0.4 1.4144 59 146 57.6 -2 1.4 2 1.4 Mean Y:143 58 145 59.2 -2 -1.2 2 1.2 1.038006231144 61 145.2 62 -1.2 -1 1.2 1146 63 147.2 64.6 -1.2 -1.6 1.2 1.6 Maximum X:149 69 150.8 68.2 -1.8 0.8 1.8 0.8 5.4154 72 154.6 72.2 -0.6 -0.2 0.6 0.2161 76 159 76.6 2 -0.6 2 0.6 Maximum Y:163 81 163.6 80.8 -0.6 0.2 0.6 0.2 3.6168 85 168 85.2 0 -0.2 0 0.2172 90 170.6 88.4 1.4 1.6 1.4 1.6 Sum X:176 94 172.2 90.8 3.8 3.2 3.8 3.2 144.6174 92 171.8 93 2.2 -1 2.2 1171 93 170 94.2 1 -1.2 1 1.2 Sum Y:166 96 167.4 95 -1.4 1 1.4 1 111.0666667163 96 165.2 96.8 -2.2 -0.8 2.2 0.8163 98 165 99.6 -2 -1.6 2 1.6 RMS X:163 101 166.8 103.8 -3.8 -2.8 3.8 2.8 1.718496288170 107 169.6 108.8 0.4 -1.8 0.4 1.8175 117 173.2 113.8 1.8 3.2 1.8 3.2 RMS Y:177 121 177.8 119.8 -0.8 1.2 0.8 1.2 1.339704405181 123 181.4 125.4 -0.4 -2.4 0.4 2.4186 131 184.8 130.2 1.2 0.8 1.2 0.8188 135 187.8 134.2 0.2 0.8 0.2 0.8192 141 191.4 139.8 0.6 1.2 0.6 1.2192 141 194.2 144 -2.2 -3 2.2 3199 151 196.8 147.4 2.2 3.6 2.2 3.6200 152 198.4 149.8 1.6 2.2 1.6 2.2201 152 199.4 151.4 1.6 0.6 1.6 0.6200 153 198.4 150.4 1.6 2.6 1.6 2.6197 149 195.2 148.2 1.8 0.8 1.8 0.8194 146 191 145.4 3 0.6 3 0.6184 141 186 141.8 -2 -0.8 2 0.8180 138 180.2 138.2 -0.2 -0.2 0.2 0.2175 135 174 134.2 1 0.8 1 0.8168 131 168 130.2 0 0.8 0 0.8163 126 162.4 126 0.6 0 0.6 0154 121 156.4 122.2 -2.4 -1.2 2.4 1.2152 117 151.2 118 0.8 -1 0.8 1145 116 145.4 114 -0.4 2 0.4 2142 110 141.2 110.8 0.8 -0.8 0.8 0.8134 106 138 108.6 -4 -2.6 4 2.6133 105 136.6 106.2 -3.6 -1.2 3.6 1.2136 106 136.6 104.8 -0.6 1.2 0.6 1.2138 104 138.2 104.2 -0.2 -0.2 0.2 0.2142 103 141.4 103.6 0.6 -0.6 0.6 0.6142 103 144 102.6 -2 0.4 2 0.4149 102 145.8 101.8 3.2 0.2 3.2 0.2

Page 250: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

149 101 145.6 100.8 3.4 0.2 3.4 0.2147 100 144.4 98.8 2.6 1.2 2.6 1.2141 98 140.8 96.6 0.2 1.4 0.2 1.4136 93 135.8 94.2 0.2 -1.2 0.2 1.2131 91 130.2 91.6 0.8 -0.6 0.8 0.6124 89 125 89 -1 0 1 0119 87 120.2 86.6 -1.2 0.4 1.2 0.4115 85 115.6 84.4 -0.6 0.6 0.6 0.6112 81 112.4 82.2 -0.4 -1.2 0.4 1.2108 80 110.4 80.4 -2.4 -0.4 2.4 0.4108 78 110 78.8 -2 -0.8 2 0.8109 78 110.6 77.6 -1.6 0.4 1.6 0.4113 77 112.8 76 0.2 1 0.2 1115 75 115.4 74.6 -0.4 0.4 0.4 0.4119 72 118.2 73 0.8 -1 0.8 1121 71 120.6 71.4 0.4 -0.4 0.4 0.4123 70 122.6 70.2 0.4 -0.2 0.4 0.2125 69 122 68.4 3 0.6 3 0.6125 69 119.6 65.8 5.4 3.2 5.4 3.2116 63 116.2 63.2 -0.2 -0.2 0.2 0.2109 58 111 60.6 -2 -2.6 2 2.6106 57 104.6 57.4 1.4 -0.4 1.4 0.4

99 56 99 54.8 0 1.2 0 1.293 53 94.2 52.6 -1.2 0.4 1.2 0.488 50 90 50.6 -2 -0.6 2 0.685 47 86 47.4 -1 -0.4 1 0.485 47 82.6 45 2.4 2 2.4 279 40 80.6 43.2 -1.6 -3.2 1.6 3.276 41 80 41.8 -4 -0.8 4 0.878 41 80.6 40.2 -2.6 0.8 2.6 0.882 40 82.8 40 -0.8 0 0.8 088 39 86.8 39.2 1.2 -0.2 1.2 0.290 39 91 38.4 -1 0.6 1 0.696 37 95.4 37.2 0.6 -0.2 0.6 0.299 37 98.8 35.8 0.2 1.2 0.2 1.2

104 34 103 34 1 0 1 0105 32 105.8 32.4 -0.8 -0.4 0.8 0.4111 30 109.2 30.6 1.8 -0.6 1.8 0.6110 29 112.4 28.8 -2.4 0.2 2.4 0.2116 28 115.8 27.2 0.2 0.8 0.2 0.8120 25 118.6 25.2 1.4 -0.2 1.4 0.2122 24 121.8 23 0.2 1 0.2 1125 20 124 20.8 1 -0.8 1 0.8126 18 125.2 19.4 0.8 -1.4 0.8 1.4127 17 126.3333333 17.66666667 0.666666667 -0.666666667 0.666666667 0.666666667126 18 126 18 0 0 0 0

Page 251: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

50 70 90 110 130 150 170 190 21010

30

50

70

90

110

130

150

Lightning 1 Trial 2 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

50 70 90 110 130 150 170 190 21010

30

50

70

90

110

130

150

Lightning 1 Trial 2 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 252: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 3:

Kalman Filter Data:

Kalman:Moving Average X:

Moving Average Y:

Difference X:

DifferenceY:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:72 37 72 37 0 0 0 0 1.28888888980 41 77.66666667 39.33333333 2.333333333 1.666666667 2.333333333 1.66666666781 40 79.4 40 1.6 0 1.6 0 Maximum:81 41 82.2 41 -1.2 0 1.2 0 4.283 41 84.4 42 -1.4 -1 1.4 186 42 87.4 43.8 -1.4 -1.8 1.4 1.8 Sum:91 46 91.8 46.4 -0.8 -0.4 0.8 0.4 224.266666796 49 97.2 49.8 -1.2 -0.8 1.2 0.8

103 54 103.6 53.8 -0.6 0.2 0.6 0.2 RMS:110 58 109.8 57.2 0.2 0.8 0.2 0.8 1.658770568118 62 115.4 60.8 2.6 1.2 2.6 1.2122 63 119.6 64.2 2.4 -1.2 2.4 1.2 Mean X:124 67 121.8 67.2 2.2 -0.2 2.2 0.2 1.54559387124 71 121.6 70.2 2.4 0.8 2.4 0.8121 73 119.4 73.6 1.6 -0.6 1.6 0.6 Mean Y:117 77 116.6 76.6 0.4 0.4 0.4 0.4 1.032183908111 80 114.4 78.8 -3.4 1.2 3.4 1.2110 82 114.2 81.6 -4.2 0.4 4.2 0.4 Maximum X:113 82 116.4 84.6 -3.4 -2.6 3.4 2.6 4.2120 87 121.6 88.2 -1.6 -1.2 1.6 1.2128 92 128.4 92 -0.4 0 0.4 0 Maximum Y:137 98 135.4 96.4 1.6 1.6 1.6 1.6 3.8144 101 141.2 100.4 2.8 0.6 2.8 0.6148 104 144.8 104 3.2 0 3.2 0 Sum X:149 107 145.6 106.8 3.4 0.2 3.4 0.2 134.4666667146 110 144.4 109.6 1.6 0.4 1.6 0.4141 112 142.4 112 -1.4 0 1.4 0 Sum Y:138 115 141 114.4 -3 0.6 3 0.6 89.8138 116 141.8 116.6 -3.8 -0.6 3.8 0.6142 119 145.6 119.6 -3.6 -0.6 3.6 0.6 RMS X:150 121 151.6 122.8 -1.6 -1.8 1.6 1.8 1.877867327160 127 159.8 126.6 0.2 0.4 0.2 0.4168 131 169.4 130.8 -1.4 0.2 1.4 0.2 RMS Y:179 135 179 135.4 0 -0.4 0 0.4 1.405935239190 140 188.2 139.8 1.8 0.2 1.8 0.2198 144 196.6 144.2 1.4 -0.2 1.4 0.2206 149 203.2 148 2.8 1 2.8 1210 153 207.8 150.6 2.2 2.4 2.2 2.4212 154 210.4 152 1.6 2 1.6 2213 153 210.6 151.6 2.4 1.4 2.4 1.4211 151 209 149.4 2 1.6 2 1.6207 147 205.8 145 1.2 2 1.2 2202 142 201.2 139 0.8 3 0.8 3196 132 196 131.4 0 0.6 0 0.6190 123 190.2 122.8 -0.2 0.2 0.2 0.2185 113 184.2 113.6 0.8 -0.6 0.8 0.6178 104 178.2 105.2 -0.2 -1.2 0.2 1.2172 96 172.8 98.4 -0.8 -2.4 0.8 2.4166 90 168.4 93.6 -2.4 -3.6 2.4 3.6163 89 165.8 90.6 -2.8 -1.6 2.8 1.6163 89 165.2 89.2 -2.2 -0.2 2.2 0.2165 89 166.2 89 -1.2 0 1.2 0169 89 168 89 1 0 1 0171 89 169.2 88.4 1.8 0.6 1.8 0.6172 89 168.6 86.6 3.4 2.4 3.4 2.4169 86 166.2 83.6 2.8 2.4 2.8 2.4162 80 162.2 79.6 -0.2 0.4 0.2 0.4157 74 157 74.2 0 -0.2 0 0.2

Page 253: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

151 69 151.8 68.6 -0.8 0.4 0.8 0.4146 62 147.8 63.8 -1.8 -1.8 1.8 1.8143 58 145.2 60 -2.2 -2 2.2 2142 56 144.6 57.2 -2.6 -1.2 2.6 1.2144 55 145.6 55.8 -1.6 -0.8 1.6 0.8148 55 147.4 55 0.6 0 0.6 0151 55 149.2 54.2 1.8 0.8 1.8 0.8152 54 150 53 2 1 2 1151 52 149 50 2 2 2 2148 49 146.6 45.2 1.4 3.8 1.4 3.8143 40 143 38.8 0 1.2 0 1.2139 31 138.4 31.6 0.6 -0.6 0.6 0.6134 22 133.6 24.6 0.4 -2.6 0.4 2.6128 16 129.2 19.4 -1.2 -3.4 1.2 3.4124 14 124.2 17 -0.2 -3 0.2 3121 14 119 17.4 2 -3.4 2 3.4114 19 113.4 20 0.6 -1 0.6 1108 24 107.2 23.6 0.8 0.4 0.8 0.4100 29 100.4 27.8 -0.4 1.2 0.4 1.2

93 32 93.6 31.8 -0.6 0.2 0.6 0.287 35 86.8 35.6 0.2 -0.6 0.2 0.680 39 80.8 39.2 -0.8 -0.2 0.8 0.274 43 76.2 42.4 -2.2 0.6 2.2 0.670 47 73.2 45.2 -3.2 1.8 3.2 1.870 48 72.6 47.6 -2.6 0.4 2.6 0.472 49 74.2 49.2 -2.2 -0.2 2.2 0.277 51 77.4 50.2 -0.4 0.8 0.4 0.882 51 81.66666667 51.33333333 0.333333333 -0.333333333 0.333333333 0.33333333386 52 86 52 0 0 0 0

Page 254: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:73 38 73 38 0 0 0 0 1.483141762

80 41 77.66666667 39.666666672.33333333

3 1.333333333 2.333333333 1.33333333380 40 79.2 40.2 0.8 -0.2 0.8 0.2 Maximum:80 41 82 41.2 -2 -0.2 2 0.2 5.683 41 84.8 42.6 -1.8 -1.6 1.8 1.687 43 88.4 44.8 -1.4 -1.8 1.4 1.8 Sum:94 48 93.4 48 0.6 0 0.6 0 258.066666798 51 99.2 51.6 -1.2 -0.6 1.2 0.6

105 57 105.8 55.4 -0.8 1.6 0.8 1.6 RMS:112 59 111 58.2 1 0.8 1 0.8 1.851587976120 62 116 61.8 4 0.2 4 0.2120 62 119.2 64.6 0.8 -2.6 0.8 2.6 Mean X:123 69 120.4 67.4 2.6 1.6 2.6 1.6 1.675862069121 71 119 70.8 2 0.2 2 0.2118 73 116.6 74.4 1.4 -1.4 1.4 1.4 Mean Y:113 79 114.4 76.8 -1.4 2.2 1.4 2.2 1.290421456108 80 113.6 78.8 -5.6 1.2 5.6 1.2112 81 115.2 82.2 -3.2 -1.2 3.2 1.2 Maximum X:117 81 119.2 85 -2.2 -4 2.2 4 5.6126 90 125.8 89 0.2 1 0.2 1133 93 132.4 93 0.6 0 0.6 0 Maximum Y:141 100 138.4 97.6 2.6 2.4 2.6 2.4 4.4145 101 142.4 100.8 2.6 0.2 2.6 0.2147 104 144.2 104.2 2.8 -0.2 2.8 0.2 Sum X:146 106 143.2 106.6 2.8 -0.6 2.8 0.6 145.8142 110 141.6 109.4 0.4 0.6 0.4 0.6136 112 140 111.8 -4 0.2 4 0.2 Sum Y:137 115 140.2 114.4 -3.2 0.6 3.2 0.6 112.2666667139 116 142.8 116.8 -3.8 -0.8 3.8 0.8147 119 148.6 120.4 -1.6 -1.4 1.6 1.4 RMS X:155 122 155.4 123.8 -0.4 -1.8 0.4 1.8 2.014798381165 130 164 127.8 1 2.2 1 2.2171 132 173 132.2 -2 -0.2 2 0.2 RMS Y:182 136 181.6 136.8 0.4 -0.8 0.4 0.8 1.672526099192 141 189.8 140.6 2.2 0.4 2.2 0.4198 145 197.2 144.8 0.8 0.2 0.8 0.2206 149 202.4 148.2 3.6 0.8 3.6 0.8208 153 206 150 2 3 2 3208 153 207.8 150.8 0.2 2.2 0.2 2.2210 150 207.2 150 2.8 0 2.8 0207 149 205.6 147.2 1.4 1.8 1.4 1.8203 145 202.6 142 0.4 3 0.4 3200 139 198.4 136.2 1.6 2.8 1.6 2.8193 127 193.8 128.6 -0.8 -1.6 0.8 1.6189 121 188.8 120.4 0.2 0.6 0.2 0.6184 111 183 111.8 1 -0.8 1 0.8178 104 177.8 105 0.2 -1 0.2 1171 96 173 99.4 -2 -3.4 2 3.4167 93 169.6 95.8 -2.6 -2.8 2.6 2.8165 93 167.8 93.4 -2.8 -0.4 2.8 0.4167 93 168.2 92.4 -1.2 0.6 1.2 0.6169 92 169.4 92 -0.4 0 0.4 0173 91 171 91.2 2 -0.2 2 0.2173 91 170.8 89.6 2.2 1.4 2.2 1.4173 89 168.6 86.6 4.4 2.4 4.4 2.4166 85 165.2 83 0.8 2 0.8 2158 77 160.4 78.6 -2.4 -1.6 2.4 1.6156 73 155.2 72.8 0.8 0.2 0.8 0.2149 69 151 67.6 -2 1.4 2 1.4147 60 148.4 64 -1.4 -4 1.4 4

Page 255: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

145 59 146.8 61 -1.8 -2 1.8 2145 59 147.4 58.8 -2.4 0.2 2.4 0.2148 58 148.8 58.2 -0.8 -0.2 0.8 0.2152 58 150.2 57.4 1.8 0.6 1.8 0.6154 57 151.2 55.8 2.8 1.2 2.8 1.2152 55 150.8 53.8 1.2 1.2 1.2 1.2150 51 148.6 49.4 1.4 1.6 1.4 1.6146 48 145.4 43.6 0.6 4.4 0.6 4.4141 36 141.6 36.8 -0.6 -0.8 0.6 0.8138 28 137 30 1 -2 1 2133 21 132.8 24 0.2 -3 0.2 3127 17 129 20.6 -2 -3.6 2 3.6125 18 124 20 1 -2 1 2122 19 119 21.4 3 -2.4 3 2.4113 25 113.2 24.6 -0.2 0.4 0.2 0.4108 28 106.8 27.6 1.2 0.4 1.2 0.4

98 33 100 30.8 -2 2.2 2 2.293 33 93.4 33.8 -0.4 -0.8 0.4 0.888 35 86.8 37.2 1.2 -2.2 1.2 2.280 40 81.8 40 -1.8 0 1.8 075 45 78 42.8 -3 2.2 3 2.273 47 75.6 45.4 -2.6 1.6 2.6 1.674 47 76.2 47.6 -2.2 -0.6 2.2 0.676 48 78.4 48.8 -2.4 -0.8 2.4 0.883 51 81.2 49.8 1.8 1.2 1.8 1.2

86 51 85.33333333 51.333333330.66666666

7 -0.333333333 0.666666667 0.33333333387 52 87 52 0 0 0 0

Page 256: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

50 70 90 110 130 150 170 190 210 23010

30

50

70

90

110

130

150

Lightning 1 Trial 3 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

50 70 90 110 130 150 170 190 210 23010

30

50

70

90

110

130

150

Lightning 1 Trial 3 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 257: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 4:

Kalman Filter:

Kalman:

Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:

162 124 162 124 0 0 0 01.11931464

2

163 125 164.3333333 125.6666667

-1.33333333

3

-0.66666666

7 1.333333333 0.666666667168 128 167.6 128 0.4 0 0.4 0 Maximum:171 131 170.6 129.8 0.4 1.2 0.4 1.2 4.4174 132 174.2 131.4 -0.2 0.6 0.2 0.6177 133 177.4 132.8 -0.4 0.2 0.4 0.2 Sum:

181 133 180.4 133.8 0.6 -0.8 0.6 0.8239.533333

3184 135 184 135 0 0 0 0186 136 187.4 136.4 -1.4 -0.4 1.4 0.4 RMS:192 138 190.6 138.4 1.4 -0.4 1.4 0.4 1.53576267194 140 194.2 140.8 -0.2 -0.8 0.2 0.8197 143 197.8 143.2 -0.8 -0.2 0.8 0.2 Mean X:202 147 200 145 2 2 2 2 1.32211838204 148 200.6 145 3.4 3 3.4 3203 147 199.2 142.6 3.8 4.4 3.8 4.4 Mean Y:

197 140 195.2 137.4 1.8 2.6 1.8 2.60.91651090

3190 131 189 130 1 1 1 1182 121 181.4 120.8 0.6 0.2 0.6 0.2 Maximum X:173 111 174.2 111.4 -1.2 -0.4 1.2 0.4 4.4165 101 168 103.2 -3 -2.2 3 2.2161 93 163.4 96.8 -2.4 -3.8 2.4 3.8 Maximum Y:159 90 161.2 92.4 -2.2 -2.4 2.2 2.4 4.4159 89 161 90.2 -2 -1.2 2 1.2162 89 162.2 89.6 -0.2 -0.6 0.2 0.6 Sum X:

164 90 164.4 89.4 -0.4 0.6 0.4 0.6141.466666

7167 90 167 89.4 0 0.6 0 0.6170 89 169 89.2 1 -0.2 1 0.2 Sum Y:

172 89 170.2 88.4 1.8 0.6 1.8 0.698.0666666

7172 88 169.4 86.8 2.6 1.2 2.6 1.2170 86 166.6 83.8 3.4 2.2 3.4 2.2 RMS X:

163 82 161.2 78.4 1.8 3.6 1.8 3.61.68669584

9156 74 154.4 71.8 1.6 2.2 1.6 2.2145 62 147.6 65.2 -2.6 -3.2 2.6 3.2 RMS Y:

138 55 142.2 59.4 -4.2 -4.4 4.2 4.41.36828033

3136 53 139 55 -3 -2 3 2136 53 138.8 53 -2.8 0 2.8 0140 52 140.8 52.2 -0.8 -0.2 0.8 0.2144 52 143.8 51.8 0.2 0.2 0.2 0.2148 51 147.2 51.4 0.8 -0.4 0.8 0.4151 51 149.6 51.2 1.4 -0.2 1.4 0.2153 51 151 50.6 2 0.4 2 0.4152 51 150.8 49.6 1.2 1.4 1.2 1.4151 49 148.8 47.4 2.2 1.6 2.2 1.6147 46 145.8 44 1.2 2 1.2 2141 40 141.8 39 -0.8 1 0.8 1138 34 137 32.6 1 1.4 1 1.4132 26 132.2 26 -0.2 0 0.2 0127 17 128.2 20.6 -1.2 -3.6 1.2 3.6123 13 124.4 16.6 -1.4 -3.6 1.4 3.6

Page 258: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

121 13 121.2 15 -0.2 -2 0.2 2119 14 117.8 16 1.2 -2 1.2 2116 18 113.8 18.6 2.2 -0.6 2.2 0.6110 22 109 22 1 0 1 0103 26 103.4 25.8 -0.4 0.2 0.4 0.2

97 30 97.2 29.6 -0.2 0.4 0.2 0.491 33 91 33 0 0 0 085 37 85.8 35.8 -0.8 1.2 0.8 1.279 39 82 38 -3 1 3 177 40 80.2 40 -3.2 0 3.2 078 41 80.6 41.8 -2.6 -0.8 2.6 0.882 43 83.2 44 -1.2 -1 1.2 187 46 87.6 46.6 -0.6 -0.6 0.6 0.692 50 93.4 49.6 -1.4 0.4 1.4 0.499 53 100.2 53 -1.2 0 1.2 0

107 56 107 56.2 0 -0.2 0 0.2116 60 113.6 58.8 2.4 1.2 2.4 1.2121 62 118.6 61 2.4 1 2.4 1125 63 121.8 62.6 3.2 0.4 3.2 0.4124 64 122.4 64 1.6 0 1.6 0123 64 121.2 65.4 1.8 -1.4 1.8 1.4119 67 118.4 67 0.6 0 0.6 0115 69 115.4 68.8 -0.4 0.2 0.4 0.2111 71 112.4 70.8 -1.4 0.2 1.4 0.2109 73 110.2 72.4 -1.2 0.6 1.2 0.6108 74 110.2 74 -2.2 0 2.2 0108 75 112.4 76 -4.4 -1 4.4 1115 77 116.4 78.4 -1.4 -1.4 1.4 1.4122 81 122.2 81 -0.2 0 0.2 0129 85 128.8 84.4 0.2 0.6 0.2 0.6137 87 134.8 88.2 2.2 -1.2 2.2 1.2141 92 140.2 91.8 0.8 0.2 0.8 0.2145 96 144.2 95 0.8 1 0.8 1149 99 146.2 98.2 2.8 0.8 2.8 0.8149 101 146.8 100.8 2.2 0.2 2.2 0.2147 103 146 103 1 0 1 0144 105 144 105 0 0 0 0141 107 141.6 106.8 -0.6 0.2 0.6 0.2139 109 139.6 108.2 -0.6 0.8 0.6 0.8137 110 138.4 109.4 -1.4 0.6 1.4 0.6137 110 138.6 110.8 -1.6 -0.8 1.6 0.8138 111 140.6 112.4 -2.6 -1.4 2.6 1.4142 114 144.6 114.6 -2.6 -0.6 2.6 0.6149 117 149.4 117.2 -0.4 -0.2 0.4 0.2157 121 155 120 2 1 2 1161 123 160.8 122.8 0.2 0.2 0.2 0.2166 125 166 125.6 0 -0.6 0 0.6171 128 170.8 128 0.2 0 0.2 0175 131 175.8 130.2 -0.8 0.8 0.8 0.8181 133 180.4 132.4 0.6 0.6 0.6 0.6186 134 185 134.2 1 -0.2 1 0.2189 136 189.2 135.8 -0.2 0.2 0.2 0.2194 137 192.8 137.2 1.2 -0.2 1.2 0.2196 139 196 138.8 0 0.2 0 0.2199 140 198.6 140.4 0.4 -0.4 0.4 0.4202 142 200 141.6 2 0.4 2 0.4

202 144 201.6666667 1430.33333333

3 1 0.333333333 1201 143 201 143 0 0 0 0

Page 259: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:163 125 163 125 0 0 0 0 1.308411215

163 125 165 126.3333333 -2-

1.333333333 2 1.333333333169 129 168.2 128.6 0.8 0.4 0.8 0.4 Maximum:172 132 171 130.2 1 1.8 1 1.8 7174 132 174.8 131.8 -0.8 0.2 0.8 0.2177 133 178 133 -1 0 1 0 Sum:182 133 180.8 133.8 1.2 -0.8 1.2 0.8 280185 135 184.8 135.2 0.2 -0.2 0.2 0.2186 136 188.2 136.8 -2.2 -0.8 2.2 0.8 RMS:194 139 191.2 139.2 2.8 -0.2 2.8 0.2 1.779822293194 141 195 141.8 -1 -0.8 1 0.8197 145 198.4 144.2 -1.4 0.8 1.4 0.8 Mean X:204 148 199.8 145.2 4.2 2.8 4.2 2.8 1.475389408203 148 199.4 144.2 3.6 3.8 3.6 3.8201 144 197.2 140.4 3.8 3.6 3.8 3.6 Mean Y:192 136 192.2 134.4 -0.2 1.6 0.2 1.6 1.141433022186 126 185.6 126.6 0.4 -0.6 0.4 0.6179 118 178.4 117.6 0.6 0.4 0.6 0.4 Maximum X:170 109 172.6 109.2 -2.6 -0.2 2.6 0.2 5.2165 99 168 102.8 -3 -3.8 3 3.8163 94 164.8 97.8 -1.8 -3.8 1.8 3.8 Maximum Y:163 94 164 94.8 -1 -0.8 1 0.8 7163 93 164.4 93.6 -1.4 -0.6 1.4 0.6166 94 165.6 93.2 0.4 0.8 0.4 0.8 Sum X:167 93 167.4 92.4 -0.4 0.6 0.4 0.6 157.8666667169 92 169.2 91.6 -0.2 0.4 0.2 0.4172 90 170.4 90.4 1.6 -0.4 1.6 0.4 Sum Y:172 89 170.6 89 1.4 0 1.4 0 122.1333333172 88 168.6 86.6 3.4 1.4 3.4 1.4168 86 164.8 82.6 3.2 3.4 3.2 3.4 RMS X:159 80 158.6 76.2 0.4 3.8 0.4 3.8 1.894715831153 70 151.8 69.8 1.2 0.2 1.2 0.2141 57 146.2 64 -5.2 -7 5.2 7 RMS Y:138 56 142.6 59.4 -4.6 -3.4 4.6 3.4 1.656981203140 57 141 56.4 -1 0.6 1 0.6141 57 142.4 55.8 -1.4 1.2 1.4 1.2145 55 144.8 55 0.2 0 0.2 0148 54 147.2 54 0.8 0 0.8 0150 52 149.6 53 0.4 -1 0.4 1152 52 150.8 52.4 1.2 -0.4 1.2 0.4153 52 151 51.4 2 0.6 2 0.6151 52 149.8 49.8 1.2 2.2 1.2 2.2149 49 147.2 46.8 1.8 2.2 1.8 2.2144 44 144.2 43 -0.2 1 0.2 1139 37 140.2 37.2 -1.2 -0.2 1.2 0.2138 33 135.6 30.4 2.4 2.6 2.4 2.6131 23 131.6 24.8 -0.6 -1.8 0.6 1.8126 15 128.4 21 -2.4 -6 2.4 6124 16 125 18 -1 -2 1 2123 18 122.2 18 0.8 0 0.8 0121 18 118.4 20 2.6 -2 2.6 2117 23 114 22.6 3 0.4 3 0.4107 25 108.6 25.4 -1.6 -0.4 1.6 0.4102 29 102.6 28.6 -0.6 0.4 0.6 0.4

96 32 96.2 31.4 -0.2 0.6 0.2 0.691 34 90.8 34.2 0.2 -0.2 0.2 0.285 37 86.4 36.2 -1.4 0.8 1.4 0.880 39 83.8 37.8 -3.8 1.2 3.8 1.280 39 83 39.8 -3 -0.8 3 0.883 40 84.2 42 -1.2 -2 1.2 2

Page 260: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

87 44 87.4 44.6 -0.4 -0.6 0.4 0.691 48 91.8 47.6 -0.8 0.4 0.8 0.496 52 97.4 51 -1.4 1 1.4 1

102 54 103.6 54.2 -1.6 -0.2 1.6 0.2111 57 109.6 57 1.4 0 1.4 0118 60 115 59 3 1 3 1121 62 118.6 60.8 2.4 1.2 2.4 1.2123 62 120.4 62 2.6 0 2.6 0120 63 120 63.8 0 -0.8 0 0.8120 63 118.4 65.4 1.6 -2.4 1.6 2.4116 69 115.8 67.4 0.2 1.6 0.2 1.6113 70 113.6 69.6 -0.6 0.4 0.6 0.4110 72 111.4 71.6 -1.4 0.4 1.4 0.4109 74 110.4 72.8 -1.4 1.2 1.4 1.2109 73 112 74.4 -3 -1.4 3 1.4111 75 115.2 76.6 -4.2 -1.6 4.2 1.6121 78 120 79 1 -1 1 1126 83 126 82 0 1 0 1133 86 132 85.8 1 0.2 1 0.2139 88 136.6 89.6 2.4 -1.6 2.4 1.6141 94 141.2 92.8 -0.2 1.2 0.2 1.2144 97 144 95.8 0 1.2 0 1.2149 99 145 98.6 4 0.4 4 0.4147 101 145 100.8 2 0.2 2 0.2144 102 144 103 0 -1 0 1141 105 142 105 -1 0 1 0139 108 140.2 106.6 -1.2 1.4 1.2 1.4139 109 139 108 0 1 0 1138 109 138.8 109.2 -0.8 -0.2 0.8 0.2138 109 140.2 110.8 -2.2 -1.8 2.2 1.8140 111 143 112.8 -3 -1.8 3 1.8146 116 147.6 115.4 -1.6 0.6 1.6 0.6153 119 152.2 118.2 0.8 0.8 0.8 0.8161 122 157.6 121.2 3.4 0.8 3.4 0.8161 123 162.6 123.6 -1.6 -0.6 1.6 0.6167 126 167 126.2 0 -0.2 0 0.2171 128 171.2 128.4 -0.2 -0.4 0.2 0.4175 132 176.2 130.4 -1.2 1.6 1.2 1.6182 133 180.6 132.4 1.4 0.6 1.4 0.6186 133 185.2 134.4 0.8 -1.4 0.8 1.4189 136 189.2 135.8 -0.2 0.2 0.2 0.2194 138 192.6 137.2 1.4 0.8 1.4 0.8195 139 195.8 139.2 -0.8 -0.2 0.8 0.2199 140 198.2 141 0.8 -1 0.8 1202 143 199 141.6 3 1.4 3 1.4201 145 200.3333333 143 0.666666667 2 0.666666667 2198 141 198 141 0 0 0 0

Page 261: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

50 70 90 110 130 150 170 190 210 23010

30

50

70

90

110

130

150

Lightning 1 Trial 4 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

50 70 90 110 130 150 170 190 210 23010

30

50

70

90

110

130

150

Lightning 1 Trial 4 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 262: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Trial 5:

Kalman Filter Data:

Kalman:Moving Average X:

Moving Average Y:

Difference X:

Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:109 27 109 27 0 0 0 0 1.230930931108 28 105.6666667 29 2.333333333 -1 2.333333333 1100 32 99.6 31.4 0.4 0.6 0.4 0.6 Maximum:

93 34 94.4 34 -1.4 0 1.4 0 4.288 36 88.6 37.6 -0.6 -1.6 0.6 1.683 40 84.2 40.8 -1.2 -0.8 1.2 0.8 Sum:79 46 81 44 -2 2 2 2 273.266666778 48 78.8 47 -0.8 1 0.8 177 50 77.8 49.2 -0.8 0.8 0.8 0.8 RMS:77 51 78.6 49.2 -1.6 1.8 1.6 1.8 1.5341178 51 80.8 48 -2.8 3 2.8 383 46 84.4 45.8 -1.4 0.2 1.4 0.2 Mean X:89 42 89.2 42.6 -0.2 -0.6 0.2 0.6 1.46246246295 39 94.4 38.8 0.6 0.2 0.6 0.2

101 35 99.2 35.2 1.8 -0.2 1.8 0.2 Mean Y:104 32 103.4 32.2 0.6 -0.2 0.6 0.2 0.999399399107 28 106.8 29.4 0.2 -1.4 0.2 1.4110 27 109.6 27.2 0.4 -0.2 0.4 0.2 Maximum X:112 25 112.6 25.2 -0.6 -0.2 0.6 0.2 4.2115 24 115.6 23.8 -0.6 0.2 0.6 0.2119 22 118.4 21.8 0.6 0.2 0.6 0.2 Maximum Y:122 21 121 20 1 1 1 1 3.6124 17 122.8 18.2 1.2 -1.2 1.2 1.2125 16 123.6 17.4 1.4 -1.4 1.4 1.4 Sum X:124 15 122.8 17.8 1.2 -2.8 1.2 2.8 162.3333333123 18 120.4 20 2.6 -2 2.6 2118 23 116.4 22.8 1.6 0.2 1.6 0.2 Sum Y:112 28 111.8 26.4 0.2 1.6 0.2 1.6 110.9333333105 30 106.4 29.8 -1.4 0.2 1.4 0.2101 33 101 33 0 0 0 0 RMS X:

96 35 96 35.4 0 -0.4 0 0.4 1.76084655291 39 91.8 37.6 -0.8 1.4 0.8 1.487 40 87.8 40 -0.8 0 0.8 0 RMS Y:84 41 84.2 42.4 -0.2 -1.4 0.2 1.4 1.26744088981 45 81.4 44.2 -0.4 0.8 0.4 0.878 47 80 46.2 -2 0.8 2 0.877 48 80 48.2 -3 -0.2 3 0.280 50 81.8 50 -1.8 0 1.8 084 51 86.2 52.4 -2.2 -1.4 2.2 1.490 54 92.6 55.4 -2.6 -1.4 2.6 1.4

100 59 100 58.8 0 0.2 0 0.2109 63 107.8 63.2 1.2 -0.2 1.2 0.2117 67 115.4 67.6 1.6 -0.6 1.6 0.6123 73 120.8 71.4 2.2 1.6 2.2 1.6128 76 123.8 74.6 4.2 1.4 4.2 1.4127 78 124.2 76.8 2.8 1.2 2.8 1.2124 79 122.4 78.2 1.6 0.8 1.6 0.8119 78 119.4 79 -0.4 -1 0.4 1114 80 117 79.8 -3 0.2 3 0.2113 80 116.4 81 -3.4 -1 3.4 1115 82 117.8 83 -2.8 -1 2.8 1121 85 121.6 85.4 -0.6 -0.4 0.6 0.4126 88 126.8 88.2 -0.8 -0.2 0.8 0.2133 92 132.6 91.2 0.4 0.8 0.4 0.8139 94 138.2 94.6 0.8 -0.6 0.8 0.6144 97 143.6 98.2 0.4 -1.2 0.4 1.2149 102 147.8 101.4 1.2 0.6 1.2 0.6153 106 150.2 104.2 2.8 1.8 2.8 1.8

Page 263: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

154 108 150.6 106.6 3.4 1.4 3.4 1.4151 108 149.4 108.4 1.6 -0.4 1.6 0.4146 109 147.8 110.2 -1.8 -1.2 1.8 1.2143 111 147 112.6 -4 -1.6 4 1.6145 115 148 115.8 -3 -0.8 3 0.8150 120 150.8 119.2 -0.8 0.8 0.8 0.8156 124 155.2 122.8 0.8 1.2 0.8 1.2160 126 160.2 126.2 -0.2 -0.2 0.2 0.2165 129 165.4 129.2 -0.4 -0.2 0.4 0.2170 132 171.8 132.6 -1.8 -0.6 1.8 0.6176 135 178.8 136.6 -2.8 -1.6 2.8 1.6188 141 186 140.8 2 0.2 2 0.2195 146 192.8 145.4 2.2 0.6 2.2 0.6201 150 198.8 149.6 2.2 0.4 2.2 0.4204 155 202.4 152.6 1.6 2.4 1.6 2.4206 156 204 153.8 2 2.2 2 2.2206 156 204 152.6 2 3.4 2 3.4203 152 202 148.4 1 3.6 1 3.6201 144 198.2 142 2.8 2 2.8 2194 134 193.2 133.8 0.8 0.2 0.8 0.2187 124 187.4 124.8 -0.4 -0.8 0.4 0.8181 115 181.4 116 -0.4 -1 0.4 1174 107 176 108.4 -2 -1.4 2 1.4171 100 171.6 102.6 -0.6 -2.6 0.6 2.6167 96 168.4 98.6 -1.4 -2.6 1.4 2.6165 95 167.2 96.2 -2.2 -1.2 2.2 1.2165 95 167.4 94.8 -2.4 0.2 2.4 0.2168 95 168.8 93.8 -0.8 1.2 0.8 1.2172 93 170.8 92.8 1.2 0.2 1.2 0.2174 91 172.2 91.4 1.8 -0.4 1.8 0.4175 90 172 89 3 1 3 1172 88 169.2 85.6 2.8 2.4 2.8 2.4167 83 164.4 81.6 2.6 1.4 2.6 1.4158 76 158.4 76.6 -0.4 -0.6 0.4 0.6150 71 152.4 71.2 -2.4 -0.2 2.4 0.2145 65 147.4 66.4 -2.4 -1.4 2.4 1.4142 61 144.6 62.8 -2.6 -1.8 2.6 1.8142 59 144.4 60 -2.4 -1 2.4 1144 58 146 58.4 -2 -0.4 2 0.4149 57 148.4 57.4 0.6 -0.4 0.6 0.4153 57 150.6 56.4 2.4 0.6 2.4 0.6154 56 151.8 54.6 2.2 1.4 2.2 1.4153 54 151.2 51.8 1.8 2.2 1.8 2.2150 49 148.4 47.4 1.6 1.6 1.6 1.6146 43 144.6 42 1.4 1 1.4 1139 35 140.4 36.2 -1.4 -1.2 1.4 1.2135 29 136 31 -1 -2 1 2132 25 132 26.6 0 -1.6 0 1.6128 23 129.4 23.8 -1.4 -0.8 1.4 0.8126 21 127.2 21.6 -1.2 -0.6 1.2 0.6126 21 125.2 19.8 0.8 1.2 0.8 1.2124 18 124 18.33333333 0 -0.333333333 0 0.333333333122 16 122 16 0 0 0 0

Page 264: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Raw Measurement Data:

Raw:Moving Average X:

Moving Average Y: Difference X: Difference Y:

Difference X (abs):

Difference Y (abs): Statistics:

X Y Mean:109 28 109 28 0 0 0 0 1.402102102109 28 105.6666667 29.66666667 3.333333333 -1.666666667 3.333333333 1.666666667

99 33 99.8 32 -0.8 1 0.8 1 Maximum:93 34 94.8 34.8 -1.8 -0.8 1.8 0.8 5.289 37 89.2 38.8 -0.2 -1.8 0.2 1.884 42 85.6 41.8 -1.6 0.2 1.6 0.2 Sum:81 48 83 45 -2 3 2 3 311.266666781 48 81 47.6 0 0.4 0 0.480 50 80.2 49 -0.2 1 0.2 1 RMS:79 50 81.6 48 -2.6 2 2.6 2 1.75208727280 49 84 46.4 -4 2.6 4 2.688 43 87.6 44 0.4 -1 0.4 1 Mean X:93 40 92.2 40.8 0.8 -0.8 0.8 0.8 1.59879879998 38 97 37.4 1 0.6 1 0.6

102 34 100.8 34.6 1.2 -0.6 1.2 0.6 Mean Y:104 32 104.2 32.2 -0.2 -0.2 0.2 0.2 1.205405405107 29 107 29.8 0 -0.8 0 0.8110 28 109.6 28 0.4 0 0.4 0 Maximum X:112 26 112.8 26.2 -0.8 -0.2 0.8 0.2 5.2115 25 115.8 24.8 -0.8 0.2 0.8 0.2120 23 118.6 22.4 1.4 0.6 1.4 0.6 Maximum Y:122 22 121 20.6 1 1.4 1 1.4 4124 16 122.6 19 1.4 -3 1.4 3124 17 122.8 18.8 1.2 -1.8 1.2 1.8 Sum X:123 17 121.4 19.8 1.6 -2.8 1.6 2.8 177.4666667121 22 118.6 22.6 2.4 -0.6 2.4 0.6115 27 114.4 25.4 0.6 1.6 0.6 1.6 Sum Y:110 30 110 28.6 0 1.4 0 1.4 133.8103 31 105 31.2 -2 -0.2 2 0.2101 33 100.2 33.8 0.8 -0.8 0.8 0.8 RMS X:

96 35 95.8 35.8 0.2 -0.8 0.2 0.8 1.98621878691 40 92.2 37.8 -1.2 2.2 1.2 2.288 40 88.6 40.4 -0.6 -0.4 0.6 0.4 RMS Y:85 41 85.2 43 -0.2 -2 0.2 2 1.48140290183 46 82.8 44.6 0.2 1.4 0.2 1.479 48 82 46.6 -3 1.4 3 1.479 48 82.8 48.6 -3.8 -0.6 3.8 0.684 50 85 50.6 -1 -0.6 1 0.689 51 90.2 53.2 -1.2 -2.2 1.2 2.294 56 96.8 56.4 -2.8 -0.4 2.8 0.4

105 61 103.6 60.2 1.4 0.8 1.4 0.8112 64 110.4 64.8 1.6 -0.8 1.6 0.8118 69 117 68.8 1 0.2 1 0.2123 74 120.6 72 2.4 2 2.4 2127 76 122 74.6 5 1.4 5 1.4123 77 121.4 76.2 1.6 0.8 1.6 0.8119 77 119.2 77.4 -0.2 -0.4 0.2 0.4115 77 116.8 78.2 -1.8 -1.2 1.8 1.2112 80 115.8 79.4 -3.8 0.6 3.8 0.6115 80 117.2 81.4 -2.2 -1.4 2.2 1.4118 83 120 83.8 -2 -0.8 2 0.8126 87 124.8 86.4 1.2 0.6 1.2 0.6129 89 130 89.4 -1 -0.4 1 0.4136 93 135.4 92.4 0.6 0.6 0.6 0.6141 95 140.2 95.6 0.8 -0.6 0.8 0.6145 98 145 99.2 0 -1.2 0 1.2150 103 148.2 102 1.8 1 1.8 1153 107 149.4 104.4 3.6 2.6 3.6 2.6152 107 148.8 106.6 3.2 0.4 3.2 0.4147 107 147.2 108.4 -0.2 -1.4 0.2 1.4142 109 146.2 110.4 -4.2 -1.4 4.2 1.4

Page 265: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

142 112 146.8 113.4 -4.8 -1.4 4.8 1.4148 117 149.4 117 -1.4 0 1.4 0155 122 153.2 120.4 1.8 1.6 1.8 1.6160 125 158 123.8 2 1.2 2 1.2161 126 162.6 126.8 -1.6 -0.8 1.6 0.8166 129 167.2 129.4 -1.2 -0.4 1.2 0.4171 132 173.8 133.2 -2.8 -1.2 2.8 1.2178 135 180.8 137.4 -2.8 -2.4 2.8 2.4193 144 187.8 141.8 5.2 2.2 5.2 2.2196 147 194 146.4 2 0.6 2 0.6201 151 199.2 150.2 1.8 0.8 1.8 0.8202 155 201.2 152.2 0.8 2.8 0.8 2.8204 154 202 152.6 2 1.4 2 1.4203 154 201.6 150 1.4 4 1.4 4200 149 199.2 145 0.8 4 0.8 4199 138 195.4 138.2 3.6 -0.2 3.6 0.2190 130 190.6 130.4 -0.6 -0.4 0.6 0.4185 120 185.4 122 -0.4 -2 0.4 2179 115 180.2 114.8 -1.2 0.2 1.2 0.2174 107 175.8 108.6 -1.8 -1.6 1.8 1.6173 102 172.4 104.4 0.6 -2.4 0.6 2.4168 99 170 101.2 -2 -2.2 2 2.2168 99 169.8 99.4 -1.8 -0.4 1.8 0.4167 99 170.2 97.6 -3.2 1.4 3.2 1.4173 98 171.6 96 1.4 2 1.4 2175 93 173 94.4 2 -1.4 2 1.4175 91 173.6 92.2 1.4 -1.2 1.4 1.2175 91 171.8 88.8 3.2 2.2 3.2 2.2170 88 167.6 85 2.4 3 2.4 3164 81 162.2 80.8 1.8 0.2 1.8 0.2154 74 156.2 75.6 -2.2 -1.6 2.2 1.6148 70 151.2 70.4 -3.2 -0.4 3.2 0.4145 65 147.4 66.6 -2.4 -1.6 2.4 1.6145 62 146.4 63.8 -1.4 -1.8 1.4 1.8145 62 147.4 61.6 -2.4 0.4 2.4 0.4149 60 149.6 60.4 -0.6 -0.4 0.6 0.4153 59 151.2 59.4 1.8 -0.4 1.8 0.4156 59 152.6 57.6 3.4 1.4 3.4 1.4153 57 152.4 55.2 0.6 1.8 0.6 1.8152 53 150.4 51.6 1.6 1.4 1.6 1.4148 48 146.4 46.4 1.6 1.6 1.6 1.6143 41 142.8 40.8 0.2 0.2 0.2 0.2136 33 139 35.4 -3 -2.4 3 2.4135 29 135.2 31 -0.2 -2 0.2 2133 26 132.2 27.4 0.8 -1.4 0.8 1.4129 26 130.6 25.4 -1.6 0.6 1.6 0.6128 23 128.6 23.2 -0.6 -0.2 0.6 0.2128 23 126.6 21.4 1.4 1.6 1.4 1.6125 18 125.3333333 19.33333333 -0.333333333 -1.333333333 0.333333333 1.333333333123 17 123 17 0 0 0 0

Page 266: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

50 70 90 110 130 150 170 190 210 23010

30

50

70

90

110

130

150

Lightning 1 Trial 5 Kalman Filter

Moving AverageKalman Filter

X-coordinate

Y-co

ordin

ate

50 70 90 110 130 150 170 190 210 23010

30

50

70

90

110

130

150

Lightning 1 Trial 5 Raw Measurement

Moving AverageRaw Data

X-coordinate

Y-co

ordin

ate

Page 267: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Appendix C: Source Code

Page 268: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

// Interactive Human Interfacing Device Software – Alan Cheng 2013-2014// Build 1 Version: Zeta// // [NOTE] This code has not been cleaned-up, so it may be messy and hard to read

#include "stdafx.h"#include <iostream>#include <fstream>#include <vector>#include <math.h>#include "C:/Program Files/OpenNI2/Include/OpenNI.h"#include "opencv2/opencv.hpp"#include <opencv2/core/core.hpp>#include <opencv2/highgui/highgui.hpp>#include <opencv2/imgproc/imgproc.hpp>#include <Windows.h>

using namespace std;using namespace openni;using namespace cv;

#define CVX_RED CV_RGB(0xff,0x00,0x00)#define CVX_GREEN CV_RGB(0x00,0xff,0x00)#define CVX_BLUE CV_RGB(0x00,0x00,0xff)

void fingerDetection(cv::Mat binaryImg, cv::Mat colorImg, vector<int> OldOutX, vector<int> OldOutY);void Calibration(cv::Mat binaryImg, cv::Mat colorImg, int* centerxOut, int* centeryOut, int* length, int* width);

void MouseMove(int x, int y);void LeftClick();

vector<int> OutX, OutY;int UpX, UpY, DownX, DownY, CurX, CurY;int UpXCali, UpYCali, DownXCali, DownYCali, CurXCali, CurYCali;double ratioX, ratioY;int displacementX, displacementY;

//Detect and return mouse position//Initialization loop variantvoid CallBackFunc(int event, int x, int y, int flags, void* userdata){

CurX = x;CurY = y;

if ( event == EVENT_LBUTTONDOWN ) {

UpX = x;UpY = y;

} else if ( event == EVENT_RBUTTONDOWN ) {

DownX = x;DownY = y;

}}

Page 269: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

//Calibration Loop Variantvoid CallBackFuncCali(int event, int x, int y, int flags, void* userdata){

CurXCali = x;CurYCali = y;

if ( event == EVENT_LBUTTONDOWN ) {

UpXCali = x;UpYCali = y;

} else if ( event == EVENT_RBUTTONDOWN ) {

DownXCali = x;DownYCali = y;

}}

//Main Functionint _tmain(int argc, _TCHAR* argv[]){

OpenNI::initialize();

Device devAnyDevice;devAnyDevice.open(ANY_DEVICE);

//----------------[Define Video Settings]-------------------//Set Properties of Depth StreamVideoMode mModeDepth;mModeDepth.setResolution( 640, 480 );mModeDepth.setFps( 30 );mModeDepth.setPixelFormat( PIXEL_FORMAT_DEPTH_100_UM );

//Set Properties of Color StreamVideoMode mModeColor;mModeColor.setResolution( 640, 480 );mModeColor.setFps( 30 );mModeColor.setPixelFormat( PIXEL_FORMAT_RGB888 );//----------------------------------------------------------

if( devAnyDevice.isImageRegistrationModeSupported(IMAGE_REGISTRATION_DEPTH_TO_COLOR ) )

{devAnyDevice.setImageRegistrationMode( IMAGE_REGISTRATION_DEPTH_TO_COLOR );

}

//----------------------[Initial Streams]---------------------VideoStream streamInitDepth;streamInitDepth.create( devAnyDevice, SENSOR_DEPTH );

VideoStream streamInitColor;streamInitColor.create( devAnyDevice, SENSOR_COLOR );

streamInitDepth.setVideoMode( mModeDepth );streamInitColor.setVideoMode( mModeColor );

namedWindow( "Depth Image (Init)", CV_WINDOW_AUTOSIZE );

Page 270: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

namedWindow( "Color Image (Init)", CV_WINDOW_AUTOSIZE );

VideoFrameRef frameDepthInit;VideoFrameRef frameColorInit;

streamInitDepth.start();streamInitColor.start();cv::Mat BackgroundFrame;

int avgDist = 0;int iMaxDepthInit = streamInitDepth.getMaxPixelValue();//------------------------------------------------------------//--------------------[Initiation Process]--------------------while( true ){

streamInitDepth.readFrame( &frameDepthInit );streamInitColor.readFrame( &frameColorInit );

const cv::Mat mImageDepth( frameDepthInit.getHeight(), frameDepthInit.getWidth(),

CV_16UC1, (void*)frameDepthInit.getData());

cv::Mat mScaledDepth; mImageDepth.convertTo( mScaledDepth, CV_8U, 255.0 / iMaxDepthInit );

cv::imshow( "Depth Image (Init)", mScaledDepth );

const cv::Mat mImageRGB(frameColorInit.getHeight(), frameColorInit.getWidth(),

CV_8UC3, (void*)frameColorInit.getData());

cv::Mat cImageBGR; cv::cvtColor( mImageRGB, cImageBGR, CV_RGB2BGR );

//--------------------[Get Average Distance]---------------------int depthVal = 0;int frameHeight = frameDepthInit.getHeight();int frameWidth = frameDepthInit.getWidth();//---------------------------------------------------------------

int initCount = 0;for(int i = 0; i < frameHeight; i++){

for(int j = 0; j < frameWidth; j++){

depthVal = mImageDepth.at<unsigned short>(i, j) + depthVal;initCount++;

}}

avgDist = depthVal / ((frameHeight) * (frameWidth));

cout << "Average Distance: " << avgDist << endl;//---------------------------------------------------------------char CurrentXY[8], UpXY[8], DownXY[8];sprintf(CurrentXY, "%d,%d", CurX, CurY);sprintf(UpXY, "%d,%d", UpX, UpY);sprintf(DownXY, "%d,%d", DownX, DownY);

Page 271: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

setMouseCallback("Color Image (Init)", CallBackFunc, NULL);putText(cImageBGR, CurrentXY, cvPoint(frameWidth-100, frameHeight-30),

FONT_HERSHEY_COMPLEX_SMALL, 1, cvScalar(255, 255, 0));putText(cImageBGR, UpXY, cvPoint(UpX, UpY-20),

FONT_HERSHEY_COMPLEX_SMALL, 1, cvScalar(255, 255, 0));putText(cImageBGR, DownXY, cvPoint(DownX, DownY-20),

FONT_HERSHEY_COMPLEX_SMALL, 1, cvScalar(255, 255, 0));rectangle(cImageBGR, cvPoint(UpX, UpY), cvPoint(DownX, DownY),

cvScalar(255,255,0), 2);//---------------------------------------------------------------

cv::imshow( "Color Image (Init)", cImageBGR );

if( cv::waitKey(1) == 'q'){

mImageDepth.copyTo(BackgroundFrame);break;

}}

streamInitDepth.destroy();streamInitColor.destroy();

destroyWindow( "Depth Image (Init)" );destroyWindow( "Color Image (Init)" );

//------------------------------------------------------------//------------------------------------------------------------VideoStream streamCaliDepth;streamCaliDepth.create( devAnyDevice, SENSOR_DEPTH );

VideoStream streamCaliColor;streamCaliColor.create( devAnyDevice, SENSOR_COLOR );

streamCaliDepth.setVideoMode( mModeDepth );streamCaliColor.setVideoMode( mModeColor );

streamCaliDepth.start();streamCaliColor.start();

//------------------------------------------------------------//-------------------[Calibration Process]--------------------//Block = 77mm by 21mm

namedWindow( "Depth Image (Init)", CV_WINDOW_AUTOSIZE );namedWindow( "Color Image (Init)", CV_WINDOW_AUTOSIZE );namedWindow( "Depth Image (Init-Binary)", CV_WINDOW_AUTOSIZE );

VideoFrameRef frameDepthCali;VideoFrameRef frameColorCali;

streamCaliDepth.start();streamCaliColor.start();

int centerA[2][1], centerB[2][1], blockDL, blockDW, blockRL, blockRW;double ratioDL, ratioDW, ratioRL, ratioRW;

Page 272: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

vector<Mat> frameRGB;

while( true ){

streamCaliDepth.readFrame( &frameDepthCali );streamCaliColor.readFrame( &frameColorCali );

const cv::Mat mImageColor( frameDepthCali.getHeight(), frameDepthCali.getWidth(),

CV_8UC3, (void*)frameColorCali.getData());const cv::Mat mImageDepth( frameDepthCali.getHeight(),

frameDepthCali.getWidth(),CV_16UC1, (void*)frameDepthCali.getData());

cv::Mat mScaledDepth;mImageDepth.convertTo( mScaledDepth, CV_8U, 255.0 / iMaxDepthInit );

cv::imshow( "Depth Image (Init)", mScaledDepth );

const cv::Mat mImageRGB(frameColorCali.getHeight(), frameColorCali.getWidth(),

CV_8UC3, (void*)frameColorCali.getData());

cv::Mat cImageBGR;cv::cvtColor( mImageRGB, cImageBGR, CV_RGB2BGR );

cv::Mat mImageThres( frameDepthCali.getHeight(), frameDepthCali.getWidth(), CV_8UC1 );

for(int i = UpY; i < DownY; i++){

for(int j = UpX; j < DownX; j++){

avgDist = BackgroundFrame.at<unsigned short>(i, j);int depthVal = mImageDepth.at<unsigned short>(i, j);

if((depthVal <= avgDist-10) && (depthVal > 100)){

mImageThres.data[mImageThres.step[0]*i + mImageThres.step[1]*j] = 0;

}else{

mImageThres.data[mImageThres.step[0]*i + mImageThres.step[1]*j] = 255;

}}

}

//---------------------------------------------------------------char CurrentXYCali[8], UpXYCali[8], DownXYCali[8];sprintf(CurrentXYCali, "%d,%d", CurXCali, CurYCali);sprintf(UpXYCali, "%d,%d", UpXCali, UpYCali);sprintf(DownXYCali, "%d,%d", DownXCali, DownYCali);

setMouseCallback("Color Image (Init)", CallBackFuncCali, NULL);putText(cImageBGR, CurrentXYCali, cvPoint(540, 450),

FONT_HERSHEY_COMPLEX_SMALL, 1, cvScalar(255, 255, 0));putText(cImageBGR, UpXYCali, cvPoint(UpXCali, UpYCali-20),

Page 273: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

FONT_HERSHEY_COMPLEX_SMALL, 1, cvScalar(255, 255, 0));putText(cImageBGR, DownXYCali, cvPoint(DownXCali, DownYCali-20),

FONT_HERSHEY_COMPLEX_SMALL, 1, cvScalar(255, 255, 0));rectangle(cImageBGR, cvPoint(UpXCali, UpYCali), cvPoint(DownXCali,

DownYCali), cvScalar(255,255,0), 1);

//---------------------------------------------------------------//---------------------------------------------------------------cv::imshow( "Depth Image (Init-Binary)", mImageThres );//---------------------------------------------------------------Calibration( mImageThres, cImageBGR, &centerB[0][0], &centerB[1][0],

&blockDL, &blockDW );

cv::imshow( "Color Image (Init)", cImageBGR );

centerA[0][0] = abs(UpXCali - DownXCali)/2+UpXCali;centerA[1][0] = abs(UpYCali - DownYCali)/2+UpYCali;blockRL = abs(UpXCali - DownXCali);blockRW = abs(UpYCali - DownYCali);

cout << blockDL << ", " << blockDW << " " << blockRL << ", " << blockRW << " ";

if((blockDL > 0) && (blockDW > 0) && (blockRL > 0) && (blockRW > 0)){

ratioDL = 21/(double)blockDL;ratioDW = 77/(double)blockDW;ratioRL = 21/(double)blockRL;ratioRW = 77/(double)blockRW;ratioX = ratioRL/ratioDL;ratioY = ratioRW/ratioDW;

}

cout << ratioDL << ", " << ratioDW << " " << ratioRL << ", " << ratioRW << endl;

if( cv::waitKey(1) == 'q'){

break;}

}

streamCaliDepth.destroy();streamCaliColor.destroy();

displacementX = centerA[0][0] - centerB[0][0];displacementY = centerA[1][0] - centerB[1][0];

destroyWindow( "Depth Image (Init)" );destroyWindow( "Color Image (Init)" );destroyWindow( "Depth Image (Init-Binary)" );

//------------------------------------------------------------//------------------------------------------------------------

VideoStream streamDepth;

Page 274: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

streamDepth.create( devAnyDevice, SENSOR_DEPTH );

VideoStream streamColor;streamColor.create( devAnyDevice, SENSOR_COLOR );

streamDepth.setVideoMode( mModeDepth );streamColor.setVideoMode( mModeColor );

streamDepth.start();streamColor.start();

namedWindow( "Depth Image", CV_WINDOW_AUTOSIZE );namedWindow( "Color Image", CV_WINDOW_AUTOSIZE );namedWindow( "Thresholded Image", CV_WINDOW_AUTOSIZE );

namedWindow( "Mouse Canvas", CV_WINDOW_AUTOSIZE );

int iMaxDepth = streamDepth.getMaxPixelValue();

VideoFrameRef frameColor;VideoFrameRef frameDepth;

OutX.clear();OutY.clear();

vector<int> OldOutX, OldOutY;OldOutX.clear();OldOutY.clear();

//------------------------------------------------------------// Kalman Filter:KalmanFilter KF(4, 2, 0);Mat_<float> state(4, 1); /* (x, y, Vx, Vy) */Mat processNoise(4, 1, CV_32F);Mat_<float> measurement(2,1); measurement.setTo(Scalar(0));char code = (char)-1;int averageX = 0;int averageY = 0;int resetCounter = 0;

KF.statePre.at<float>(0) = averageX;KF.statePre.at<float>(1) = averageY;KF.statePre.at<float>(2) = 0;KF.statePre.at<float>(3) = 0;KF.transitionMatrix = *(Mat_<float>(4, 4) << 1,0,1,0, 0,1,0,1, 0,0,1,0,

0,0,0,1);

setIdentity(KF.measurementMatrix);setIdentity(KF.processNoiseCov, Scalar::all(1e-4));setIdentity(KF.measurementNoiseCov, Scalar::all(/*1e-1*/1e-4));setIdentity(KF.errorCovPost, Scalar::all(.1));

vector<Point> mousev,kalmanv;

mousev.clear();kalmanv.clear();

Mat canvas(768, 1366, CV_8UC3);

Page 275: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

//------------------------------------------------------------//-----------------------[Main Process]-----------------------while( true )

{streamDepth.readFrame( &frameDepth );streamColor.readFrame( &frameColor );

const cv::Mat mImageDepth( frameDepth.getHeight(), frameDepth.getWidth(), CV_16UC1,

(void*)frameDepth.getData());

cv::Mat mScaledDepth;mImageDepth.convertTo( mScaledDepth, CV_8U, 255.0 / iMaxDepth );

//////////////////////////////////////////////////////////////////////////////////

/---------------------[Downsampling]----------------------------------------------

double min;double max;cv::minMaxIdx(mImageDepth, &min, &max);cv::Mat adjMap;

float scale = 255 / (max-min);mImageDepth.convertTo(adjMap,CV_8UC1, scale, -min*scale);

cv::Mat falseColorsMap;applyColorMap(adjMap, falseColorsMap, cv::COLORMAP_AUTUMN);

cv::imshow("Out", falseColorsMap);//-------------------------------------------------------------------------

-------

//////////////////////////////////////////////////////////////////////////////////

cv::imshow( "Depth Image", mScaledDepth );cv::imshow( "Depth Image2", adjMap );

const cv::Mat mImageRGB(frameColor.getHeight(), frameColor.getWidth(), CV_8UC3,

(void*)frameColor.getData());

cv::Mat cImageBGR;cv::cvtColor( mImageRGB, cImageBGR, CV_RGB2BGR );

//-------------[Threshold]-----------------cv::Mat mImageThres( frameDepth.getHeight(), frameDepth.getWidth(), CV_8UC1

);

int backgroundPxlCount = 0;for(int i = UpY; i < DownY; i++){

for(int j = UpX; j < DownX; j++){

int depthVal = mImageDepth.at<unsigned short>(i, j);

Page 276: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

avgDist = BackgroundFrame.at<unsigned short>(i, j);

int xShift = j;int yShift = i;int frameHeight = frameDepth.getHeight();int frameWidth = frameDepth.getWidth();

if((depthVal > (avgDist-12)) && (depthVal <= (avgDist-6))){

if((frameWidth > xShift ) && (xShift >= 0) && (frameHeight > yShift) &&

(yShift >= 0))

mImageThres.data[mImageThres.step[0]*yShift + mImageThres.step[1]*xShift]

= 255;}else{

if((frameWidth > xShift ) && (xShift >= 0) && (frameHeight > yShift) &&

(yShift >= 0))

mImageThres.data[mImageThres.step[0]*yShift + mImageThres.step[1]*xShift]

= 0;}

backgroundPxlCount++;}

}GaussianBlur( mImageThres, mImageThres, Size(3,3), 0, 0 );

fingerDetection( mImageThres, cImageBGR, OldOutX, OldOutY );

cv::imshow("Thresholded Image", mImageThres);//----------------------------------------

if( cv::waitKey(1) == 'q'){

break;}

//----------------------------------

//----------------------------------//Simulate mouse/touch here:

double windowLength = DownX - UpX;double windowHeight = DownY - UpY;

if(OutX.size() > 0){

resetCounter = 0;averageX = 0;averageY = 0;

Page 277: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

for( int i = 0; i < OutX.size(); i++ ){

averageX += OutX[i];averageY += OutY[i];

}

averageX = windowLength - ((averageX / OutX.size()) - UpX);averageY = ((averageY / OutY.size()) - UpY);

}else if(resetCounter > 10){

KF.statePre.at<float>(0) = averageX;KF.statePre.at<float>(1) = averageY;

}else{

resetCounter++;}

//------------------------------------------------Mat prediction = KF.predict();Point predictPt(prediction.at<float>(0),prediction.at<float>(1));

measurement(0) = averageX;measurement(1) = averageY;

Point measPt(measurement(0),measurement(1));mousev.push_back(measPt);

Mat estimated = KF.correct(measurement);

if(estimated.at<float>(0) > windowLength){

estimated.at<float>(0) = windowLength;}if(estimated.at<float>(1) > windowHeight){

estimated.at<float>(1) = windowHeight;}

Point statePt(estimated.at<float>(0),estimated.at<float>(1));kalmanv.push_back(statePt);

for (int i = 0; i < kalmanv.size()-1; i++){

line(canvas, kalmanv[i], kalmanv[i+1], Scalar(0,255,0), 1);}

//------------------------------------------------

//1366 x 768double shiftX, shiftY;shiftX = 1366/windowLength;shiftY = 768/windowHeight;

int ptX = statePt.x *shiftX;

Page 278: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

int ptY = statePt.y *shiftY;

int MptX = measPt.x *shiftX;int MptY = measPt.y *shiftY;

canvas = Scalar::all(0);

line( canvas, Point( ptX - 5, ptY - 5 ), Point( ptX + 5, ptY + 5 ), Scalar(255,255,255), 2, CV_AA, 0);

line( canvas, Point( ptX + 5, ptY - 5 ), Point( ptX - 5, ptY + 5 ),Scalar(255,255,255), 2, CV_AA, 0 );

line( canvas, Point( MptX - 5, MptY - 5 ), Point( MptX + 5, MptY + 5 ), Scalar(0,0,255), 2, CV_AA, 0);

line( canvas, Point( MptX + 5, MptY - 5 ), Point( MptX - 5, MptY + 5 ),Scalar(0,0,255), 2, CV_AA, 0 );

for (int i = 0; i < mousev.size()-1; i++){

line(canvas, Point(mousev[i].x*shiftX, mousev[i].y*shiftY),Point(mousev[i+1].x*shiftX, mousev[i+1].y*shiftY), Scalar(255,255,0),

1);}for (int i = 0; i < kalmanv.size()-1; i++){

line(canvas, Point(kalmanv[i].x*shiftX, kalmanv[i].y*shiftY),Point(kalmanv[i+1].x*shiftX, kalmanv[i+1].y*shiftY), Scalar(0,255,0),

1);}

imshow( "Mouse Canvas", canvas );

if(OutX.size() > 0){

MouseMove( statePt.x *shiftX, statePt.y *shiftY );}

cv::imshow( "Color Image", cImageBGR );//----------------------------------OldOutX.clear();OldOutY.clear();OldOutX = OutX;OldOutY = OutY;OutX.clear();OutY.clear();

}//------------------------------------------------------------//------------------------------------------------------------

streamDepth.destroy();streamColor.destroy();

devAnyDevice.close();

openni::OpenNI::shutdown();

return 0;

Page 279: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

}

//----------------------------------------------------------------//----------------------------------------------------------------

void fingerDetection(cv::Mat binaryImg, cv::Mat colorImg, vector<int> OldOutX, vector<int> OldOutY){

vector<vector<Point> > contours;vector<Vec4i> hierarchy;Mat counterImg = binaryImg.clone();findContours( counterImg, contours, hierarchy, CV_RETR_TREE,

CV_CHAIN_APPROX_SIMPLE, Point(0, 0) );

// Get the momentsvector<Moments> mu(contours.size() );for( int i = 0; i < contours.size(); i++ ){

mu[i] = moments( contours[i], false );}

// Get the mass centers:vector<Point2f> mc( contours.size() );for( int i = 0; i < contours.size(); i++ ){

mc[i] = Point2f( mu[i].m10/mu[i].m00 , mu[i].m01/mu[i].m00 );}

//Draw contours. NOW IN COLORRNG rng(12345);

double area;

for( int i = 0; i< contours.size(); i++ ){

int centerx = (mu[i].m10/mu[i].m00);int centery = (mu[i].m01/mu[i].m00); area = contourArea(contours[i], false);Point2f encCenter;int perimeter = (int)arcLength(contours[i], true);double circularity = (4 * 3.14 * area)/(perimeter * perimeter);

if((area > 50) && (area < 300) && (circularity > 0.2) && (circularity < 0.8))

{float furthestReach;/////////////////////////////////////////////////////////Draw stuffPoint pt, ptx1, ptx2, pty1, pty2;centerx = (double)centerx*1.3/*ratioX*/+(double)displacementX;centery = (double)centery*1.05/*ratioY*/+(double)displacementY;pt.x = centerx;pt.y = centery;ptx1.x = centerx - 10; ptx1.y = centery;ptx2.x = centerx + 10; ptx2.y = centery;pty1.x = centerx; pty1.y = centery - 10;

Page 280: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

pty2.x = centerx; pty2.y = centery + 10;

Scalar color = Scalar( rng.uniform(0, 255), rng.uniform(0,255), rng.uniform(0,255) );

Scalar white = (255, 255, 255);Scalar black = (0, 0, 0);

if( OldOutX.size() > 0){

for(int k = 0; k < OldOutX.size(); k++){

int distance = sqrt(pow((double)(centerx-OldOutX[k]),2) + pow((double)(centery-OldOutY[k]),2) );

if(distance < 20){

drawContours( colorImg, contours, i, color, 2, 8, hierarchy, 0,

Point() );line( colorImg, ptx1, ptx2, color, 2 );

line( colorImg, pty1, pty2, color, 2 );

///////////////////////////////////////////////////////OutX.push_back(centerx);OutY.push_back(centery);

}}

}else{

OutX.push_back(centerx);OutY.push_back(centery);

}}

}}

//----------------------------------------------------------------//----------------------------------------------------------------

void Calibration(cv::Mat binaryImg, cv::Mat colorImg, int* centerxOut, int* centeryOut, int* length, int* width){

vector<vector<Point> > contours;vector<Vec4i> hierarchy;Mat counterImg = binaryImg.clone();findContours( counterImg, contours, hierarchy, CV_RETR_TREE,

CV_CHAIN_APPROX_SIMPLE, Point(0, 0) );

// Get the momentsvector<Moments> mu(contours.size() );// Get the mass centers:vector<Point2f> mc( contours.size() );

vector<vector<Point> > contours_poly( contours.size() );vector<Rect> boundRect(contours.size() );

Page 281: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

for( int i = 0; i < contours.size(); i++ ){

mu[i] = moments( contours[i], false );mc[i] = Point2f( mu[i].m10/mu[i].m00 , mu[i].m01/mu[i].m00 );

}

for( int i = 0; i < contours_poly.size(); i++ ){

approxPolyDP( Mat(contours[i]), contours_poly[i], 3, true );boundRect[i] = boundingRect( Mat(contours_poly[i]) );

}

//Draw contours. NOW IN COLORRNG rng(12345);

double area;

for( int i = 1; i< contours.size(); i++ ){

int centerx = (mu[i].m10/mu[i].m00);int centery = (mu[i].m01/mu[i].m00);

if(( boundRect[i].width < 640) && ( boundRect[i].height < 480)&& ( boundRect[i].width > 1) && ( boundRect[i].height > 1))

{*length = boundRect[i].width;*width = boundRect[i].height;

}

area = contourArea(contours[i], false);Point2f encCenter;

if((area > 500) && (area < 5000)){

/////////////////////////////////////////////////////////Draw stuffPoint pt, ptx1, ptx2, pty1, pty2;*centerxOut = centerx;*centeryOut = centery;pt.x = centerx;pt.y = centery;ptx1.x = centerx - 10; ptx1.y = centery;ptx2.x = centerx + 10; ptx2.y = centery;pty1.x = centerx; pty1.y = centery - 10;pty2.x = centerx; pty2.y = centery + 10;

Scalar color = Scalar( rng.uniform(0, 255), rng.uniform(0,255), rng.uniform(0,255) );

drawContours( colorImg, contours, i, color, 2, 8, hierarchy, 0, Point() );

rectangle( colorImg, boundRect[i], color, 2, 8 );

line( colorImg, ptx1, ptx2, color, 2 );line( colorImg, pty1, pty2, color, 2 );///////////////////////////////////////////////////////

}}

}

Page 282: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

//-------------[Mouse emulation]---------------void MouseMove ( int x, int y ){

double fScreenWidth = ::GetSystemMetrics( SM_CXSCREEN )-1; double fScreenHeight = ::GetSystemMetrics( SM_CYSCREEN )-1; double fx = x*(65535.0f/fScreenWidth);double fy = y*(65535.0f/fScreenHeight);INPUT Input={0};Input.type = INPUT_MOUSE;Input.mi.dwFlags = MOUSEEVENTF_MOVE|MOUSEEVENTF_ABSOLUTE;Input.mi.dx = fx;Input.mi.dy = fy;::SendInput(1,&Input,sizeof(INPUT));

}

void LeftClick ( ){

INPUT Input={0};// left down Input.type = INPUT_MOUSE;Input.mi.dwFlags = MOUSEEVENTF_LEFTDOWN;::SendInput(1,&Input,sizeof(INPUT));

// left up::ZeroMemory(&Input,sizeof(INPUT));Input.type = INPUT_MOUSE;Input.mi.dwFlags = MOUSEEVENTF_LEFTUP;::SendInput(1,&Input,sizeof(INPUT));

}

void LeftHold ( ){

INPUT Input={0};// left down Input.type = INPUT_MOUSE;Input.mi.dwFlags = MOUSEEVENTF_LEFTDOWN;::SendInput(1,&Input,sizeof(INPUT));

}

void LeftRelease( ){

// left upINPUT Input={0};Input.type = INPUT_MOUSE;Input.mi.dwFlags = MOUSEEVENTF_LEFTUP;::SendInput(1,&Input,sizeof(INPUT));

}

Page 283: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Appendix D: Source Code Version 2.0 - SVM

Page 284: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

// Interactive Human Interfacing Device Software – Alan Cheng 2014// Build 2 Version: Beta// // [NOTE] This code has not been cleaned-up, so it may be messy and hard to read

#include "stdafx.h"#include <iostream>#include <fstream>#include <vector>#include <math.h>#include "C:/Program Files/OpenNI2/Include/OpenNI.h"#include "opencv2/opencv.hpp"#include <opencv2/core/core.hpp>#include <opencv2/highgui/highgui.hpp>#include <opencv2/imgproc/imgproc.hpp>#include <Windows.h>

using namespace std;using namespace openni;using namespace cv;

#define CVX_RED CV_RGB(0xff,0x00,0x00)#define CVX_GREEN CV_RGB(0x00,0xff,0x00)#define CVX_BLUE CV_RGB(0x00,0x00,0xff)

void fingerDetection(cv::Mat binaryImg, cv::Mat colorImg, vector<int> OldOutX, vector<int> OldOutY);void Calibration(cv::Mat binaryImg, cv::Mat colorImg, int* centerxOut, int* centeryOut);void shapeFeatures(cv::Mat binaryImg, cv::Mat colorImg, int classtype);void sendCoord();

void MouseMove(int x, int y);void LeftClick();void LeftHold();void LeftRelease();

vector<int> OutX, OutY;int UpX, UpY, DownX, DownY, CurX, CurY;CvSVM SVMFinger;

void CallBackFunc(int event, int x, int y, int flags, void* userdata){

CurX = x;CurY = y;

if ( event == EVENT_LBUTTONDOWN ) {

UpX = x;UpY = y;

} else if ( event == EVENT_RBUTTONDOWN ) {

DownX = x;DownY = y;

}}

Page 285: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

int _tmain(int argc, _TCHAR* argv[]){

FILE *fptrI = fopen("C:\\Users\\Alan\\Documents\\ShapeFeatures.csv","w");fprintf(fptrI, "Classtype, Area, Perimeter, Circularity, Extent\n");fclose(fptrI);

Mat input = imread("C:\\Users\\Alan\\Pictures\\Science Fair 2014\\SVM\\Shape Features\\Fingers.bmp", 1);

Mat input2 = imread("C:\\Users\\Alan\\Pictures\\Science Fair 2014\\SVM\\Shape Features\\NotFingers.bmp", 1);

Mat inputF = imread("C:\\Users\\Alan\\Pictures\\Science Fair 2014\\SVM\\Shape Features\\ImageFeaturesBinaryF.bmp", 1);

Mat gray(input.rows, input.cols, CV_8UC3);Mat gray2(input.rows, input.cols, CV_8UC3);Mat grayF(input.rows, input.cols, CV_8UC3);cvtColor(input, gray, CV_BGR2GRAY);cvtColor(input2, gray2, CV_BGR2GRAY);cvtColor(inputF, grayF, CV_BGR2GRAY);shapeFeatures(gray, input, 1);shapeFeatures(gray2, input2, 2);namedWindow("Image");imshow("Image", input);namedWindow("Image2");imshow("Image2", input2);

//------------------------------------------------------//--------[SVM]--------// Read input data from file created abovedouble parameters[5];vector<double> svmI, svmA, svmP, svmC, svmE;int size = 1;double index = 0; double area = 0; double perimeter = 0; double circularity

= 0;char buffer[1024];char *record, *lineData;FILE* fptrR = fopen("C:\\Users\\Alan\\Documents\\ShapeFeatures.csv", "r");fscanf(fptrR, "%*[^\n]\n", NULL);

svmI.resize(size); svmA.resize(size); svmP.resize(size); svmC.resize(size);

while((lineData=fgets(buffer, sizeof(buffer), fptrR))!=NULL){

size++;svmI.resize(size);svmA.resize(size);svmP.resize(size);svmC.resize(size);svmE.resize(size);

record = strtok(lineData, ";");for(int i = 0; i < 5; i++);{

double value = atoi(record);record = strtok(lineData,";");

}char *lineCopy = record;char *pch;

Page 286: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

pch = strtok(lineCopy, ",");parameters[0] = atoi(pch);

int j = 1;while( j < 5 ){

pch = strtok (NULL, ",");parameters[j] = atof(pch);j++;

}svmI[size-1] = parameters[0];svmA[size-1] = parameters[1];svmP[size-1] = parameters[2];svmC[size-1] = parameters[3];svmE[size-1] = parameters[4];

}fclose(fptrR);//---------------------

// Set up training datafloat labels[1000];for(int i = 0; i < svmI.size()-1; i++){

labels[i] = svmI[i+1];}

Mat labelsMat(1000, 1, CV_32FC1, labels);

float trainingData[1000][4];for(int i = 0; i < svmE.size()-1; i++){

trainingData[i][0] = svmE[i+1];trainingData[i][1] = svmC[i+1];trainingData[i][2] = svmA[i+1];trainingData[i][3] = svmP[i+1];

} Mat trainingDataMat(1000, 4, CV_32FC1, trainingData);

// Set up SVM's parameters CvSVMParams params; params = SVMFinger.get_params();

// Train the SVM SVMFinger.train_auto(trainingDataMat, labelsMat, Mat(), Mat(), params);

waitKey(); destroyWindow("Image"); destroyWindow("Image2");

//------------------------------------------

OpenNI::initialize();

Device devAnyDevice; devAnyDevice.open(ANY_DEVICE);

//----------------[Define Video Settings]------------------- //Set Properties of Depth Stream VideoMode mModeDepth; mModeDepth.setResolution( 640, 480 );

Page 287: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

mModeDepth.setFps( 30 ); mModeDepth.setPixelFormat( PIXEL_FORMAT_DEPTH_100_UM );

//Set Properties of Color Stream VideoMode mModeColor; mModeColor.setResolution( 640, 480 ); mModeColor.setFps( 30 ); mModeColor.setPixelFormat( PIXEL_FORMAT_RGB888 ); //----------------------------------------------------------

//----------------------[Initial Streams]--------------------- VideoStream streamInitDepth; streamInitDepth.create( devAnyDevice, SENSOR_DEPTH );

VideoStream streamInitColor; streamInitColor.create( devAnyDevice, SENSOR_COLOR );

streamInitDepth.setVideoMode( mModeDepth ); streamInitColor.setVideoMode( mModeColor );

namedWindow( "Depth Image (Init)", CV_WINDOW_AUTOSIZE ); namedWindow( "Color Image (Init)", CV_WINDOW_AUTOSIZE );

VideoFrameRef frameDepthInit; VideoFrameRef frameColorInit;

streamInitDepth.start(); streamInitColor.start(); cv::Mat BackgroundFrame;

int avgDist = 0; int iMaxDepthInit = streamInitDepth.getMaxPixelValue(); //------------------------------------------------------------ //--------------------[Initiation Process]-------------------- while( true ) { streamInitDepth.readFrame( &frameDepthInit );

streamInitColor.readFrame( &frameColorInit );

const cv::Mat mImageDepth( frameDepthInit.getHeight(), frameDepthInit.getWidth(), CV_16UC1, (void*)frameDepthInit.getData());

cv::Mat mScaledDepth; mImageDepth.convertTo( mScaledDepth, CV_8U, 255.0 / iMaxDepthInit );

cv::imshow( "Depth Image (Init)", mScaledDepth );

const cv::Mat mImageRGB(frameColorInit.getHeight(), frameColorInit.getWidth(), CV_8UC3, (void*)frameColorInit.getData());

cv::Mat cImageBGR; cv::cvtColor( mImageRGB, cImageBGR, CV_RGB2BGR );

//--------------------[Get Average Distance]--------------------- int depthVal = 0;

int frameHeight = frameDepthInit.getHeight(); int frameWidth = frameDepthInit.getWidth();

Page 288: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

//------------ //--------------------------------------------------------------- int initCount = 0; for(int i = 0; i < frameHeight; i++) {

for(int j = 0; j < frameWidth; j++){

depthVal = mImageDepth.at<unsigned short>(i, j) + depthVal;initCount++;

} } avgDist = depthVal / ((frameHeight) * (frameWidth)); cout << "Average Distance: " << avgDist << endl; //--------------------------------------------------------------- char CurrentXY[8], UpXY[8], DownXY[8]; sprintf(CurrentXY, "%d,%d", CurX, CurY); sprintf(UpXY, "%d,%d", UpX, UpY); sprintf(DownXY, "%d,%d", DownX, DownY);

setMouseCallback("Color Image (Init)", CallBackFunc, NULL); putText(cImageBGR, CurrentXY, cvPoint(frameWidth-100, frameHeight-30),

FONT_HERSHEY_COMPLEX_SMALL, 1, cvScalar(255, 255, 0)); putText(cImageBGR, UpXY, cvPoint(UpX, UpY-20),

FONT_HERSHEY_COMPLEX_SMALL, 1, cvScalar(255, 255, 0)); putText(cImageBGR, DownXY, cvPoint(DownX, DownY-20),

FONT_HERSHEY_COMPLEX_SMALL, 1, cvScalar(255, 255, 0)); rectangle(cImageBGR, cvPoint(UpX, UpY), cvPoint(DownX, DownY),

cvScalar(255,255,0), 2); //--------------------------------------------------------------- cv::imshow( "Color Image (Init)", cImageBGR );

if( cv::waitKey(1) == 'q') { mImageDepth.copyTo(BackgroundFrame);

break; }

}

streamInitDepth.destroy(); streamInitColor.destroy();

destroyWindow( "Depth Image (Init)" ); destroyWindow( "Color Image (Init)" );

//------------------------------------------------------------ UpY = UpY - 20; UpX = UpX - 40; DownY = DownY - 5; DownX = DownX + 0; //------------------------------------------------------------

VideoStream streamDepth; streamDepth.create( devAnyDevice, SENSOR_DEPTH );

VideoStream streamColor; streamColor.create( devAnyDevice, SENSOR_COLOR );

streamDepth.setVideoMode( mModeDepth );

Page 289: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

streamColor.setVideoMode( mModeColor );

streamDepth.start(); streamColor.start();

namedWindow( "Depth Image", CV_WINDOW_AUTOSIZE ); namedWindow( "Color Image", CV_WINDOW_AUTOSIZE ); namedWindow( "Thresholded Image", CV_WINDOW_AUTOSIZE );

namedWindow( "Mouse Canvas", CV_WINDOW_AUTOSIZE );

int iMaxDepth = streamDepth.getMaxPixelValue();

VideoFrameRef frameColor; VideoFrameRef frameDepth;

OutX.clear(); OutY.clear();

vector<int> OldOutX, OldOutY; OldOutX.clear(); OldOutY.clear(); //------------------------------------------------------------ // Kalman Filter: KalmanFilter KF(4, 2, 0); Mat_<float> state(4, 1); /* (x, y, Vx, Vy) */ Mat processNoise(4, 1, CV_32F); Mat_<float> measurement(2,1); measurement.setTo(Scalar(0)); char code = (char)-1; int averageX = 0; int averageY = 0; int resetCounter = 0;

KF.statePre.at<float>(0) = averageX; KF.statePre.at<float>(1) = averageY; KF.statePre.at<float>(2) = 0; KF.statePre.at<float>(3) = 0; KF.transitionMatrix = *(Mat_<float>(4, 4) << 1,0,1,0, 0,1,0,1, 0,0,1,0, 0,0,0,1);

setIdentity(KF.measurementMatrix); setIdentity(KF.processNoiseCov, Scalar::all(1e-4)); setIdentity(KF.measurementNoiseCov, Scalar::all(/*1e-1*/1e-3)); setIdentity(KF.errorCovPost, Scalar::all(.1));

vector<Point> mousev,kalmanv;

mousev.clear(); kalmanv.clear();

Mat canvas(768, 1366, CV_8UC3);

bool mouseClick = false;

//------------------------------------------------------------ //-----------------------[Main Process]----------------------- while( true ) {

Page 290: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

streamDepth.readFrame( &frameDepth ); streamColor.readFrame( &frameColor );

const cv::Mat mImageDepth( frameDepth.getHeight(), frameDepth.getWidth(), CV_16UC1, (void*)frameDepth.getData());

cv::Mat mScaledDepth; mImageDepth.convertTo( mScaledDepth, CV_8U, 255.0 / iMaxDepth );

const cv::Mat mImageRGB(frameColor.getHeight(), frameColor.getWidth(), CV_8UC3, (void*)frameColor.getData());

cv::Mat cImageBGR; cv::cvtColor( mImageRGB, cImageBGR, CV_RGB2BGR );

//-------------[Threshold]-----------------cv::Mat mImageThres( frameDepth.getHeight(), frameDepth.getWidth(), CV_8UC1

);

int backgroundPxlCount = 0;for(int i = UpY; i < DownY; i++){

for(int j = UpX; j < DownX; j++){

int depthVal = mImageDepth.at<unsigned short>(i, j);avgDist = BackgroundFrame.at<unsigned short>(i, j)-2;if((depthVal > (avgDist-15)) && (depthVal <= (avgDist-5))){

mImageThres.at<uchar>(i, j) = 255;}else{

mImageThres.at<uchar>(i, j) = 0;}

backgroundPxlCount++;}

}GaussianBlur( mImageThres, mImageThres, Size(3,3), 0, 0 );

fingerDetection( mImageThres, cImageBGR, OldOutX, OldOutY);

cv::imshow("Thresholded Image", mImageThres);//----------------------------------------

sendCoord();

if( cv::waitKey(1) == 'q'){

break;}

//----------------------------------

//----------------------------------//Simulate mouse/touch here:

Page 291: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

double windowLength = DownX - UpX;double windowHeight = DownY - UpY;

if((OutX.size() > 0) && (resetCounter < 1)){

resetCounter = 0;averageX = 0;averageY = 0;for( int i = 0; i < OutX.size(); i++ ){

averageX += OutX[i];averageY += OutY[i];

}

averageX = windowLength - ((averageX / OutX.size()) - UpX);averageY = ((averageY / OutY.size()) - UpY);averageX = windowLength - averageX;averageY = windowHeight - averageY;

}else if((resetCounter > 1) && (OutX.size() > 0)){

resetCounter = 0;averageX = 0;averageY = 0;for( int i = 0; i < OutX.size(); i++ ){

averageX += OutX[i];averageY += OutY[i];

}averageX = windowLength - ((averageX / OutX.size()) - UpX);averageY = ((averageY / OutY.size()) - UpY);averageX = windowLength - averageX;averageY = windowHeight - averageY;KF.statePre.at<float>(0) = averageX;KF.statePre.at<float>(1) = averageY;KF.statePre.at<float>(2) = 0;KF.statePre.at<float>(3) = 0;setIdentity(KF.measurementMatrix);setIdentity(KF.processNoiseCov, Scalar::all(1e-4));setIdentity(KF.measurementNoiseCov, Scalar::all(/*1e-1*/1e-3));setIdentity(KF.errorCovPost, Scalar::all(.1));mousev.clear();kalmanv.clear();

}else if(resetCounter > 1){

LeftRelease( );mouseClick = false;

}else{

resetCounter++;}

//------------------------------------------------Mat prediction = KF.predict();

Point predictPt(prediction.at<float>(0),prediction.at<float>(1));

measurement(0) = averageX;

Page 292: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

measurement(1) = averageY;

Point measPt(measurement(0),measurement(1));mousev.push_back(measPt);

// generate measurement //measurement += KF.measurementMatrix*state;

Mat estimated = KF.correct(measurement);if(estimated.at<float>(0) > windowLength){

estimated.at<float>(0) = windowLength;}if(estimated.at<float>(1) > windowHeight){

estimated.at<float>(1) = windowHeight;}

Point statePt(estimated.at<float>(0),estimated.at<float>(1));kalmanv.push_back(statePt);

for (int i = 0; i < kalmanv.size()-1; i++) {line(canvas, kalmanv[i], kalmanv[i+1], Scalar(0,255,0), 1);}//------------------------------------------------//1366 x 768double shiftX, shiftY;shiftX = 1366/windowLength;shiftY = 768/windowHeight;

int ptX = statePt.x *shiftX;int ptY = statePt.y *shiftY;

int MptX = measPt.x *shiftX;int MptY = measPt.y *shiftY;

canvas = Scalar::all(0);

line( canvas, Point( ptX - 5, ptY - 5 ), Point( ptX + 5, ptY + 5 ), Scalar(255,255,255), 2, CV_AA, 0);

line( canvas, Point( ptX + 5, ptY - 5 ), Point( ptX - 5, ptY + 5 ),Scalar(255,255,255), 2, CV_AA, 0 );

line( canvas, Point( MptX - 5, MptY - 5 ), Point( MptX + 5, MptY + 5 ), Scalar(0,0,255), 2, CV_AA, 0);

line( canvas, Point( MptX + 5, MptY - 5 ), Point( MptX - 5, MptY + 5 ),Scalar(0,0,255), 2, CV_AA, 0 );

for (int i = 0; i < mousev.size()-1; i++){

line(canvas, Point(mousev[i].x*shiftX, mousev[i].y*shiftY),Point(mousev[i+1].x*shiftX, mousev[i+1].y*shiftY),

Scalar(255,255,0), 1);}for (int i = 0; i < kalmanv.size()-1; i++){

line(canvas, Point(kalmanv[i].x*shiftX, kalmanv[i].y*shiftY),Point(kalmanv[i+1].x*shiftX, kalmanv[i+1].y*shiftY),

Scalar(0,255,0), 1);

Page 293: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

}

imshow( "Mouse Canvas", canvas );

if(OutX.size() > 0){

MouseMove( (/*1366 - */(statePt.x *shiftX)), (/*768 - */(statePt.y *shiftY)) );

if(mouseClick == false){

LeftHold( );mouseClick = true;

}}

cv::imshow( "Color Image", cImageBGR );//----------------------------------OldOutX.clear();OldOutY.clear();OldOutX = OutX;OldOutY = OutY;OutX.clear();OutY.clear();

} //------------------------------------------------------------ //------------------------------------------------------------

streamDepth.destroy(); streamColor.destroy();

devAnyDevice.close();

openni::OpenNI::shutdown();

return 0;}

//----------------------------------------------------------------//----------------------------------------------------------------

void fingerDetection(cv::Mat binaryImg, cv::Mat colorImg, vector<int> OldOutX, vector<int> OldOutY){

vector<vector<Point> > contours;vector<Vec4i> hierarchy;Mat counterImg = binaryImg.clone();findContours( counterImg, contours, hierarchy, CV_RETR_TREE,

CV_CHAIN_APPROX_SIMPLE, Point(0, 0) );

/// Get the momentsvector<Moments> mu(contours.size() );for( int i = 0; i < contours.size (); i++ ){

mu[i] = moments( contours[i], false );}

/// Get the mass centers:vector<Point2f> mc( contours.size() );

Page 294: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

for( int i = 0; i < contours.size(); i++ ){

mc[i] = Point2f( mu[i].m10/mu[i].m00 , mu[i].m01/mu[i].m00 );}

//Draw contours. NOW IN COLORRNG rng(12345);

double area;

for( int i = 0; i< contours.size(); i++ ) {

int centerx = (mu[i].m10/mu[i].m00);int centery = (mu[i].m01/mu[i].m00);

area = contourArea(contours[i], false);Point2f encCenter;int perimeter = (int)arcLength(contours[i], true);double circularity = (4 * 3.14 * area)/(perimeter * perimeter);

CvBox2D box;box = minAreaRect(contours[i]);double extent = (area/(box.size.height*box.size.width));

//if((area > 15) && (area < 200) && (circularity > 0.1) && (circularity < 0.9))

Mat sampleMat = (Mat_<float>(1,4) << extent, circularity, area, perimeter);

float response = SVMFinger.predict(sampleMat);if((response == 1) && (area > 30) && (area < 400)){

float furthestReach;/////////////////////////////////////////////////////////Draw stuffPoint pt, ptx1, ptx2, pty1, pty2;centerx = (double)centerx*0.97+3/*+10*/;centery = (double)centery*1.02+20/*+10*/;pt.x = centerx;pt.y = centery;ptx1.x = centerx - 10; ptx1.y = centery;ptx2.x = centerx + 10; ptx2.y = centery;pty1.x = centerx; pty1.y = centery - 10;pty2.x = centerx; pty2.y = centery + 10;

Scalar color = Scalar( rng.uniform(0, 255), rng.uniform(0,255), rng.uniform(0,255) );

Scalar white = (255, 255, 255);Scalar black = (0, 0, 0);

if( OldOutX.size() > 0){

for(int k = 0; k < OldOutX.size(); k++){

int distance = sqrt(pow((double)(centerx-OldOutX[k]),2) + pow((double)(centery-OldOutY[k]),2) );

if(distance < 20){

Page 295: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

drawContours( colorImg, contours, i, color, 2, 8, hierarchy, 0, Point() );

line( colorImg, ptx1, ptx2, color, 2 );line( colorImg, pty1, pty2, color, 2 );

///////////////////////////////////////////////////////OutX.push_back(centerx);OutY.push_back(centery);

}}

}else{

OutX.push_back(centerx);OutY.push_back(centery);

}}

}}

void sendCoord(){

for(int i = 0; i < OutX.size(); i++){

int centerX = OutX[i];int centerY = OutY[i];

}

}

//-------------[Mouse emulation]---------------void MouseMove ( int x, int y ){

double fScreenWidth = ::GetSystemMetrics( SM_CXSCREEN )-1; double fScreenHeight = ::GetSystemMetrics( SM_CYSCREEN )-1; double fx = x*(65535.0f/fScreenWidth);double fy = y*(65535.0f/fScreenHeight);INPUT Input={0};Input.type = INPUT_MOUSE;Input.mi.dwFlags = MOUSEEVENTF_MOVE|MOUSEEVENTF_ABSOLUTE;Input.mi.dx = fx;Input.mi.dy = fy;::SendInput(1,&Input,sizeof(INPUT));

}

void LeftClick ( ){

INPUT Input={0};// left down Input.type = INPUT_MOUSE;Input.mi.dwFlags = MOUSEEVENTF_LEFTDOWN;::SendInput(1,&Input,sizeof(INPUT));

// left up::ZeroMemory(&Input,sizeof(INPUT));Input.type = INPUT_MOUSE;Input.mi.dwFlags = MOUSEEVENTF_LEFTUP;

Page 296: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

::SendInput(1,&Input,sizeof(INPUT));}

void LeftHold( ){

INPUT Input={0};// left down Input.type = INPUT_MOUSE;Input.mi.dwFlags = MOUSEEVENTF_LEFTDOWN;::SendInput(1,&Input,sizeof(INPUT));

}

void LeftRelease( ){

// left upINPUT Input={0};Input.type = INPUT_MOUSE;Input.mi.dwFlags = MOUSEEVENTF_LEFTUP;::SendInput(1,&Input,sizeof(INPUT));

}

void shapeFeatures(cv::Mat binaryImg, cv::Mat colorImg, int classtype){

FILE *fptr = fopen("C:\\Users\\Alan\\Documents\\ShapeFeatures.csv","a");

vector<vector<Point> > contours;vector<Vec4i> hierarchy;Mat counterImg = binaryImg.clone();findContours( counterImg, contours, hierarchy, CV_RETR_TREE,

CV_CHAIN_APPROX_SIMPLE, Point(0, 0) );

/// Get the momentsvector<Moments> mu(contours.size() );for( int i = 0; i < contours.size(); i++ ){

mu[i] = moments( contours[i], false );}

/// Get the mass centers:vector<Point2f> mc( contours.size() );for( int i = 0; i < contours.size(); i++ ){

mc[i] = Point2f( mu[i].m10/mu[i].m00 , mu[i].m01/mu[i].m00 );}

//Draw contours. NOW IN COLORRNG rng(12345);

double area;

for( int i = 0; i< contours.size(); i++ ){

int centerx = (mu[i].m10/mu[i].m00);int centery = (mu[i].m01/mu[i].m00);

area = contourArea(contours[i], false);

Page 297: A Novel Human Machine Interface Using 3D Vision and Kalman Filter Optimization

Point2f encCenter;int perimeter = (int)arcLength(contours[i], true);double circularity = (4 * 3.14 * area)/(perimeter * perimeter);CvBox2D box;box = minAreaRect(contours[i]);double extent = (area/(box.size.height*box.size.width));

float furthestReach;/////////////////////////////////////////////////////////Draw stuffScalar color = Scalar( rng.uniform(0, 255), rng.uniform(0,255),

rng.uniform(0,255) );drawContours( colorImg, contours, i, Scalar(0, 0, 255), 2, 8,

hierarchy, 0, Point() );///////////////////////////////////////////////////////fprintf(fptr, "%d,%f,%d,%f,%f\n", classtype, area, perimeter,

circularity, extent);}fclose(fptr);

}