- baljeet aulakh -arnold csok -jared shepherd - amandeep singh

1

-Baljeet Aulakh -Arnold Csok

-Jared Shepherd-Amandeep Singh

EEC 490 Spring 2012

Kinect Fitness Trainer

2

Project OverviewRecognition of Body Movements and

ExercisesEverything is Voice ControlledAbility to Calculate Velocity And

Acceleration Write Statistics of Movements to an

Output File User Interface Implementation

3

Requirements

Hardware: Kinect + Laptop(2.66Hz above)

Software: Kinect SDK + Microsoft Speech Platform + C# 2010

Environment: Quite Room(voice) + Only one thing moving at a time

4

User Interface

5

Step by Step

Voice Recognition Detection Algorithms Velocity and Acceleration Ability to Write Statistics to an

Output file Documentation and Website

6

Voice Recognition

Allow User To Control Kinect By Voice

Ease of Access to the Program

As Simple as- Start && Stop

Say Which Exercise to Count

7

How Voice Recognition WorkInitialize the Audio Source from the

SensorsInitialize Speech Recognition by

Speech RecognizerCreate a Speech Recognition Engine

with Exercise NamesListen to User SpeechRespond to User Speech

8

Problem With Voice RecognitionThe First Release of Language Pack

Doesn't Have a Reliable Confidence Model

Kinect Tries To Match Every Audio Source It Picks Up

Problem with Matching the Right Exercise Because of This

9

Possible Solutions For Voice Recognition

Test Confidence Interval for Best Accuracy

Use Fitness Trainer in a Quiet Environment

Introduce Noise Cancelation Use of Headset/BluetoothWait for a More Reliable Language Pack

10

Detection Algorithms

Resources Recognition algorithms List of Recognized Body Movements An Example Velocity and Acceleration

11

20 Joint Skeleton SystemProvided By Microsoft Research Kinect

12

Resources 20 Joint Skeleton System Each joint gives x, y and z valuesVector Math to find the angleTimer functionalityVoice recognition functionalityFlag Variables in programming.Counters to store the repetition.

13

Recognition algorithms Simple cases:• Displacement of the Joints• Displacement in the X, Y, Z Direction Moderate cases:• Calculating Angles Between Joints• Setting Threshold For Some Angles Complex cases:• Set a Step by Step Routine using all of the

above to detect a motion

14

Recognized ExercisesThe Following Exercises Are Recognized: 1. Squats2. Upper Left Punch3. Upper Right Punch4. Right Punch5. Left Punch6. Right Arm Curl7. Left Arm Curl

15

Recognized Exercises: Continued

8. Left Kick9. Right Kick10. Bowling11. Hip Abduction12. Lateral Weight Shift13. Hamstring Stretch

Counter For All Of These Exercises

16

An example: Bowling motion

17

Bowling final position

JOINT STATISTICS

Each exercise has the average acceleration and velocity of all joints calculated.

The statistics are time stamped and saved into a text file.

Two points

Distance between them

Time to travel between them

AVERAGE VELOCITY

AVERAGE VELOCITY

Initial and final position of the joint

The distance formula

A Stopwatch

INITIAL AND FINAL POSITION

Save reference position(point) of skeleton for use in all calculations in defaultPosition[20].

getDisplayPosition(data.Joints[JointID.HipCenter]);

getDisplayPosition(data.Joints[JointID.Spine]);

The order is very important

DISTANCE FORMULAJoints lie on a Cartesian plane

However there is a caveat; The position is measured in pixels

Pixels Are Converted Into Centimeters with the conversion factor of 72 DPIx2 = (x2 * 2.54) / (72);

TEXT FILE

For every joint returned from the array generated from GetValues

foreach (JointID joint in Enum.GetValues(typeof(JointID)))

JointID Joint = (JointID)i;When int is typecasted to JointID it returns the

name of the joint at that position, NOT A STRING

24

Display / Interface

The display on the bottom of the screen shows what exercise is being done by the user

The state: Start / StopExercise counterExercise to be detected Exercise that was detected

25

Known Limitations

1. Blocked by an object2. Overlapping joints3. Distance from the Kinect Throws Off X, Y, Z

coordinates4. Two People In Front Of Kinect5. Joints Move During Angle Calculation

These Limitations Make The Accuracy of the Exercise Recognition Difficult

E.g. Crossed arms...

26

Future ApproachesFix Voice Recognition problemHave a new accurate Skeleton SystemBe able to record and replay exercisesMore complex body motionsIntroduce more exercisesFind solutions to the Limitations for

example a solution to the overlapping joints and blocking objects.

27

Conclusion

Learned about Kinect programming Use with windows Use of Kinect in different areas other

than gaming Learned and getting used to C#

28

Website And Facebook Page

Link To Our Webpage:“http://www.baljeetaulakh.com” Link To Our Facebook Page:“https://www.facebook.com/pages/Kinect-

Fitness-Trainer/236918233072748”

29

Questions?????OR

Suggestions?

30

End of Slides

31

Fun Time!!!!!

Demonstration!!!!!