pykinect: body iteration application development using python
DESCRIPTION
by 上官林傑 (ericsk)TRANSCRIPT
![Page 1: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/1.jpg)
Eric ShangKuan (ericsk)
Technical Evangelist Microsoft Taiwan Corporation
![Page 2: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/2.jpg)
Agenda
• Kinect for Windows
• PyKinect: How-to
• References
![Page 3: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/3.jpg)
Kinect for Windows (K4W)
![Page 4: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/4.jpg)
Kinect for Windows
![Page 5: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/5.jpg)
How Kinect Works
![Page 6: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/6.jpg)
Skeleton Tracking
![Page 7: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/7.jpg)
Detailed Spec
Resolution 1280 x 960 (RGB Camera)
Viewing angle 43° vertical by 57° horizontal field of view
Tilt range ±27°
Frame rate 30fps
Audio format 16KHz, 24-bit PCM
Audio input A four-microphone array with 24-bit analog-to-digital converter (ADC) and Kinect-resident signal processing including acoustic echo cancellation and noise suppression
# of tracking users 4
# of tracking points 20
![Page 8: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/8.jpg)
Different from XBox360 Kinect
• Near mode
– Enables the camera to see objects as close as
40 centimeters in front of the device without
losing accuracy or precision, with graceful
degradation out to 3 meters.
• Shortening USB cable and small dongle
• Support and software updates
![Page 9: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/9.jpg)
Kinect Software Development
• Windows 7/8 RP
• Visual Studio 2010
• C++ or C# (w/ .NET 4.0)
• Kinect for Windows SDK (1.5)
• (optional) DirectX
• (optional) Microsoft Speech Platform Runtime
• (optional) Kinect for Windows Language Pack
![Page 10: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/10.jpg)
K4W HW/SW Interaction
![Page 11: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/11.jpg)
K4W SDK Architecture
1. Kinect hardware 2. Kinect drivers 3. Audio and Video Components 4. DirectX Media Object (DMO) for microphone array beam-forming and audio source
localization. 5. Win32 standard APIs
![Page 12: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/12.jpg)
PyKinect
![Page 13: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/13.jpg)
PyKinect Project
• A Python bindings of K4W SDK.
– K4W SDK is required.
• A part of Python Tools for Visual Studio (PTVS)
open source project.
• http://pytools.codeplex.com/wikipage?title=PyKi
nect
![Page 14: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/14.jpg)
Enable PyKinect
• CPython 2.7 (32-bit)
• Kinect for Windows SDK (1.5 is OK)
• 2 ways to install PyKinect
– Install PTVS and PTVS – PyKinect Sample , then install PyKinect
through PTVS menu.
– Install through PyPI: http://pypi.python.org/pypi/pykinect/1.0
• (optional) PyGame
![Page 15: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/15.jpg)
How to use PyKinect?
from pykinect import nui
# Create a kinect instance
kinect = nui.Runtime()
# Enable the skeleton engine
kinect.skeleton_engine.enabled = True
# skeleton frame ready event handling
kinect.skeleton_frame_ready += skeleton_frame_ready
# depth frame ready event handling
kinect.depth_frame_ready += depth_frame_ready
# video frame ready event handling
kinect.video_frame_ready += video_frame_ready
![Page 16: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/16.jpg)
Skeleton Tracking
• Each joint (關節) is represented by 3-axis
coordination (x, y, z)
– RH axis
• Use JointId to ensure where the joint is.
![Page 17: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/17.jpg)
Get Skeleton Data
def skeleton_frame_ready(skeleton_frame): skeletons = skeleton_frame.SkeletonData # index for users for index, data in enumerate(skeletons): # get head position head_positions = data.SkeletonPositions[JointId.Head] # handle limbs - left arm LEFT_ARM = (JointId.ShoulderCenter, JointId.ShoulderLeft, JointId.ElbowLeft, JointId.WristLeft, JointId.HandLeft) left_arm_positions = [data.SkeletonPositions[x] for x in LEFT_ARM]
![Page 18: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/18.jpg)
Demo – Draw Skeletons
![Page 19: PyKinect: Body Iteration Application Development Using Python](https://reader030.vdocuments.site/reader030/viewer/2022020709/5555ffaed8b42a8a5f8b53ea/html5/thumbnails/19.jpg)
References
• Kinect for Windows SDK Programming Guide &
References:
http://msdn.microsoft.com/en-us/library/hh855347
• Check-out the PyKinect source:
http://pytools.codeplex.com/SourceControl/list/changes
ets