![Page 1: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/1.jpg)
1
EEC-492/592EEC-492/592Kinect Application Kinect Application
DevelopmentDevelopmentLecture 2Lecture 2
Wenbing ZhaoWenbing Zhao
[email protected]@ieee.org
![Page 2: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/2.jpg)
22
Outline
Reminder No class next Monday: Martin Luther King Day
Components of Kinect sensor Kinect for Windows SDK Building first Kinect application
![Page 3: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/3.jpg)
33
Components of Kinect for Windows Color camera Infrared (IR) emitter IR depth sensor Tilt motor Microphone array LED
![Page 4: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/4.jpg)
44
The Color Camera
Red, blue, green color images (RGB)
Frame rate: 30 FPS at 640x480 12 FPS at 1280x960
Viewing range: 43 degrees vertical by
57 degrees horizontal
![Page 5: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/5.jpg)
55
IR Emitter & IR Depth Sensor IR emitter emits IR light in a pseudo-random dot pattern Dotted light reflects off different objects IR depth sensor reads reflected light => depth info Depth stream resolution
640x480 320x240 80x60
![Page 6: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/6.jpg)
66
Depth Data Processing
![Page 7: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/7.jpg)
77
Tilt Motor Used to change the camera and sensor’s angles to
get the correct position of the human skeleton Can tilt vertically (upwards/downwards) up to 27
degrees
![Page 8: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/8.jpg)
88
Microphone Array 4 different microphones: locate the direction of
the audio wave, not just capture the sound Enhanced noise suppression, echo cancellation,
beam-forming
![Page 9: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/9.jpg)
99
LED
Placed between camera and IR emitter Used for indicating the status of the Kinect Green indicates Kinect drivers loaded
properly
![Page 10: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/10.jpg)
1010
Kinect for Xbox vs. Kinect for Windows Kinect for Windows
Near mode: as close as 40cm away from Kinect Can be used for commercial applications Can be used in a Win7 virtual machine: HyperV,
VMWare, Parallels For both
Track motion up to 12 feet (4 meters) away Identical resolution
![Page 11: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/11.jpg)
1111
Kinect for Windows SDK Supported operating systems
Windows 7, Windows 8, Windows Embedded 7 Hardware requirements
Dual core 2.66 GHz or faster CPU Dedicated USB 2.0 bus 2GB or higher RAM
Software requirement Microsoft Visual Studio 2010 or higher Kinect for Windows SDK: current version 1.8
![Page 12: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/12.jpg)
1212
Kinect for Windows SDK
![Page 13: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/13.jpg)
1313
Kinect for Windows SDK
![Page 14: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/14.jpg)
1414
How Applications Interact with Kinect
![Page 15: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/15.jpg)
1515
Classification of Kinect SDK APIs
![Page 16: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/16.jpg)
1616
Kinect Driver The Kinect driver controls the camera, depth
sensor, audio microphone array, and motion Data passes between the sensor and the app in
the form of data streams Color data stream Depth data stream Audio data stream Infrared data stream (for low light environment) Accelerameter data
![Page 17: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/17.jpg)
1717
The Near Mode Track a human upper body as close as 40 cm Only Kinect for Windows supports the near mode
![Page 18: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/18.jpg)
1818
Tracking Human Skeleton and Joint Movement Kinect SDK does the magic to
extract the skeleton and joint positions from the data streams received
Can track up to two users at the same time with full joint positions
Each skeleton consists of 20 joints
![Page 19: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/19.jpg)
1919
Building First Kinect App Modified KinectInfoBox app Steps
Create a new project: C# WPF application Adding Kinect reference Draw the GUI Modify MainWindow.xaml Adding code
![Page 20: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/20.jpg)
2020
Creating a New Project
![Page 21: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/21.jpg)
2121
Adding Kinect Libraries
![Page 22: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/22.jpg)
2222
Create User Interface
![Page 23: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/23.jpg)
2323
Modify MainWindow.xaml
![Page 24: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/24.jpg)
2424
Adding Code (MainWindow.xaml.cs) Setting up name space
using Microsoft.Kinect;
Setting up state variable for the Kinect sensor An instance of Microsoft.Kinect.KinectSensor class It represents the complete runtime pipeline for the Kinect sensor
during life cycle of the apppublic partial class MainWindow : Window
{KinectSensor sensor;
// remaining code goes here
}
![Page 25: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/25.jpg)
2525
Adding Code private void WindowLoaded(object sender, RoutedEventArgs e) {
if (KinectSensor.KinectSensors.Count > 0) {
this.sensor = KinectSensor.KinectSensors[0];
if (this.sensor != null && !this.sensor.IsRunning) {
this.sensor.Start();
displayInfo();
}
}
else {
MessageBox.Show("No device is connected with system!");
this.Close();
}
}
private void WindowClosing(object sender, System.ComponentModel.CancelEventArgs e) {
if (this.sensor != null && this.sensor.IsRunning) {
this.sensor.Stop();
}
![Page 26: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/26.jpg)
2626
Adding Code
Double click “Increase” button, the template for the button1_Click() method will be automatically created
Double click “Decrease” button, the template for the button2_Click() method will be automatically created
private void displayInfo() { this.textBlock1.Text = this.sensor.DeviceConnectionId; this.textBlock2.Text = this.sensor.Status.ToString(); this.textBlock3.Text = this.sensor.ElevationAngle.ToString(); }
private void button1_Click(object sender, RoutedEventArgs e) { this.sensor.ElevationAngle = this.sensor.ElevationAngle + 1; displayInfo(); }
private void button2_Click(object sender, RoutedEventArgs e) { this.sensor.ElevationAngle = this.sensor.ElevationAngle - 1; displayInfo(); }
![Page 27: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/27.jpg)
2727
More In-depth Stuff Starting Kinect sensor: sensor.Start() In the Start() method
First check the status of Kinect, if no sensor connected, runtime will throw an InvalidOperationException object with the KinectNotReady message
Init the sensor if connected
Init options None: default option UseDepthAndPlayerIndex UseColor UseSkeletonTracking UseDepth UseAudio
![Page 28: 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao wenbing@ieee.org](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649ebd5503460f94bc6ef8/html5/thumbnails/28.jpg)
2828
More In-depth Stuff Open data streams
ColorImageStream: this.sensor.ColorStream.Enable(); DepthImageStream: this.sensor.ColorStream.Enable(); SkeletonStream: this.sensor.SkeletonStream.Enable();
Stopping the Kinect sensor this.sensor.Stop(); Stops the color, depth, and skeleton data stream Stops audio source, if open Kills all threads spawned Shuts down Kinect sensor and sets init option to None