- baljeet aulakh -arnold csok -jared shepherd - amandeep singh

31
-Baljeet Aulakh -Arnold Csok -Jared Shepherd -Amandeep Singh EEC 490 Spring 2012 Kinect Fitness Trainer 1

Upload: neci

Post on 25-Feb-2016

63 views

Category:

Documents


0 download

DESCRIPTION

- Baljeet Aulakh -Arnold Csok -Jared Shepherd - Amandeep Singh. EEC 490 Spring 2012 Kinect Fitness Trainer. Project Overview. Recognition of Body Movements and Exercises Everything is Voice Controlled Ability to Calculate Velocity And Acceleration - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

1

-Baljeet Aulakh -Arnold Csok

-Jared Shepherd-Amandeep Singh

EEC 490 Spring 2012

Kinect Fitness Trainer

Page 2: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

2

Project OverviewRecognition of Body Movements and

ExercisesEverything is Voice ControlledAbility to Calculate Velocity And

Acceleration Write Statistics of Movements to an

Output File User Interface Implementation

Page 3: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

3

Requirements

Hardware: Kinect + Laptop(2.66Hz above)

Software: Kinect SDK + Microsoft Speech Platform + C# 2010

Environment: Quite Room(voice) + Only one thing moving at a time

Page 4: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

4

User Interface

Page 5: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

5

Step by Step

Voice Recognition Detection Algorithms Velocity and Acceleration Ability to Write Statistics to an

Output file Documentation and Website

Page 6: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

6

Voice Recognition

Allow User To Control Kinect By Voice

Ease of Access to the Program

As Simple as- Start && Stop

Say Which Exercise to Count

Page 7: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

7

How Voice Recognition WorkInitialize the Audio Source from the

SensorsInitialize Speech Recognition by

Speech RecognizerCreate a Speech Recognition Engine

with Exercise NamesListen to User SpeechRespond to User Speech

Page 8: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

8

Problem With Voice RecognitionThe First Release of Language Pack

Doesn't Have a Reliable Confidence Model

Kinect Tries To Match Every Audio Source It Picks Up

Problem with Matching the Right Exercise Because of This

Page 9: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

9

Possible Solutions For Voice Recognition

Test Confidence Interval for Best Accuracy

Use Fitness Trainer in a Quiet Environment

Introduce Noise Cancelation Use of Headset/BluetoothWait for a More Reliable Language Pack

Page 10: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

10

Detection Algorithms

Resources Recognition algorithms List of Recognized Body Movements An Example Velocity and Acceleration

Page 11: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

11

20 Joint Skeleton SystemProvided By Microsoft Research Kinect

Page 12: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

12

Resources 20 Joint Skeleton System Each joint gives x, y and z valuesVector Math to find the angleTimer functionalityVoice recognition functionalityFlag Variables in programming.Counters to store the repetition.

Page 13: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

13

Recognition algorithms Simple cases:• Displacement of the Joints• Displacement in the X, Y, Z Direction Moderate cases:• Calculating Angles Between Joints• Setting Threshold For Some Angles Complex cases:• Set a Step by Step Routine using all of the

above to detect a motion

Page 14: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

14

Recognized ExercisesThe Following Exercises Are Recognized: 1. Squats2. Upper Left Punch3. Upper Right Punch4. Right Punch5. Left Punch6. Right Arm Curl7. Left Arm Curl

Page 15: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

15

Recognized Exercises: Continued

8. Left Kick9. Right Kick10. Bowling11. Hip Abduction12. Lateral Weight Shift13. Hamstring Stretch

Counter For All Of These Exercises

Page 16: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

16

An example: Bowling motion

Page 17: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

17

Bowling final position

Page 18: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

JOINT STATISTICS

Each exercise has the average acceleration and velocity of all joints calculated.

The statistics are time stamped and saved into a text file.

Page 19: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

Two points

Distance between them

Time to travel between them

AVERAGE VELOCITY

Page 20: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

AVERAGE VELOCITY

Initial and final position of the joint

The distance formula

A Stopwatch

Page 21: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

INITIAL AND FINAL POSITION

Save reference position(point) of skeleton for use in all calculations in defaultPosition[20].

getDisplayPosition(data.Joints[JointID.HipCenter]);

getDisplayPosition(data.Joints[JointID.Spine]);

The order is very important

Page 22: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

DISTANCE FORMULAJoints lie on a Cartesian plane

However there is a caveat; The position is measured in pixels

Pixels Are Converted Into Centimeters with the conversion factor of 72 DPIx2 = (x2 * 2.54) / (72);

Page 23: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

TEXT FILE

For every joint returned from the array generated from GetValues

foreach (JointID joint in Enum.GetValues(typeof(JointID)))

JointID Joint = (JointID)i;When int is typecasted to JointID it returns the

name of the joint at that position, NOT A STRING

Page 24: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

24

Display / Interface

The display on the bottom of the screen shows what exercise is being done by the user

The state: Start / StopExercise counterExercise to be detected Exercise that was detected

Page 25: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

25

Known Limitations

1. Blocked by an object2. Overlapping joints3. Distance from the Kinect Throws Off X, Y, Z

coordinates4. Two People In Front Of Kinect5. Joints Move During Angle Calculation

These Limitations Make The Accuracy of the Exercise Recognition Difficult

E.g. Crossed arms...

Page 26: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

26

Future ApproachesFix Voice Recognition problemHave a new accurate Skeleton SystemBe able to record and replay exercisesMore complex body motionsIntroduce more exercisesFind solutions to the Limitations for

example a solution to the overlapping joints and blocking objects.

Page 27: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

27

Conclusion

Learned about Kinect programming Use with windows Use of Kinect in different areas other

than gaming Learned and getting used to C#

Page 28: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

28

Website And Facebook Page

Link To Our Webpage:“http://www.baljeetaulakh.com” Link To Our Facebook Page:“https://www.facebook.com/pages/Kinect-

Fitness-Trainer/236918233072748”

Page 29: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

29

Questions?????OR

Suggestions?

Page 30: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

30

End of Slides

Page 31: - Baljeet Aulakh -Arnold  Csok -Jared Shepherd - Amandeep  Singh

31

Fun Time!!!!!

Demonstration!!!!!