leveraging the kinect sdk to control a remote device
DESCRIPTION
Copy of the Presentation used at a talk at the State of North Carolina Undergraduate Research Symposium at UNC Charlotte in November 2013TRANSCRIPT
Leveraging the Kinect SDK to Control a Remote Device
Akhil Acharya and Sean FreemermanSummer Ventures in Science and Mathematics
Appalachian State University
Computing Today
Objective
● Original research on novel interaction techniques
● Develop new methodologies to interface the Kinect Software Development Kit with an Infrared (IR) transmitter
● Investigate the technology used by the Kinect to accurately track body parts
● Determine the viability of the Kinect platform
Tools
● Kinect○ Tracks movement
● IR Toy○ Sends IR signals
● WinLIRC software package○ "Middle man" between Kinect Application and IR toy
● 2 AirSwimmer Remote Controlled Balloons○ System scalable to any IR device
Kinect SDK Application
WinLIRC (Always Running)
Position Data
Pass command to WinLIRC
Serial information over USB
IR signals
Microsoft Kinect
● Announced: 2009● Released: 2010● Full body motion controller for Xbox 360● Scatters multiple IR beams
○ Readings at discrete points○ Saves on processing power
● Tracks depth by measuring depth of focus ● Kinect SDK allows developers to create
applications using Kinect
Why Kinect?
● Novel form of human-computer interaction● Relatively cheap
○ $150 to get started● Hands-Free● 3D Capabilities
Kinect SDK Application
WinLIRC (Always Running)
Position Data
Pass command to WinLIRC
Serial information over USB
IR signals
Application Design
● Built using example application "Skeleton-Basics WPF"
● Communicates with the Kinect
Control System (Kinect)
● Relative distance measured○ Distance between right shoulder and right hand○ User doesn't need to stand in the center of the
Kinect's FOV● Radius of 0.2 units - "null space"
○ Better differentiate commands○ Space to rest hand without performing action
SystemKinect SDK Application
WinLIRC (Always Running)
Position Data
Pass command to WinLIRC
Serial information over USB
IR signals
Control System (WinLIRC)
● If X and Y values exceed "null space" boundaries, command is sent to WinLIRC○ Done every 30 Frames (1 second)
■ Prevents WinLIRC from being overloaded○ Commands
■ Left/Right (X Values)■ Up/Down (Y Values)
● All commands defined as bytes in AirSwimmers.cfg file.
AirSwimmer
● Two models○ Shark ("Bruce")○ Clownfish ("Nemo")
● Lightweight control system○ Microcontroller with IR receiver○ Weighted ballast○ Servo○ IR remote
Challenges
● Translating information● Debugging● Documentation
Results
● It works! ● Movement occurs in near real time● Caveats:
○ Balloon requires line of sight■ Limited movement capability
○ Remotes avoid this by having higher power LEDs
Next Steps
● Higher emission IR transmitter● Full-on voice control
○ Partially implemented already● Replicate with other motion control devices
○ Second Generation Kinect (Late 2013)○ Leap Motion
● Untapped potential○ Controlling non-IR devices○ Potential to control any device
Kinect SDK Application
WinLIRC (Always Running)
Position Data
Pass command to WinLIRC
Serial information over USB
IR signals
Kinect SDK Application
WinLIRC (Always Running)
Position Data
Pass command to WinLIRC
Serial information over USB
IR signals
Kinect SDK Application
Position Data
Lessons
● Value of documentation● Powerful Kinect API● Potential of Gesture-based computing
Acknowledgements
● Mentors: Dr. Rahman Tashakkori, Mr. Luke Rice, Ms. Bahar Akhtar
● Appalachian State University● AirSwimmer and IR Toy donated by Dan
Thyer.● UNC-Charlotte and the State of North
Carolina Undergraduate Research Symposium
Thank You