Computer Vision: Parallelize or Paralyze

Team Purple ThreadsCSE Capstone 2012

April 2012

Abstract

• Purple Threads• Project Description• Motivation• System Overview• First Steps• Target Drone Platform• Turret System• Final Status

Purple Threads

Top Row: Duc Tran, Aviral Shrivastava(Sponsor), Gabriel SilvaBottom Row: Nicholas Moulin, Craig Hartmann, Anthony Russo, Nadim Hoque

Project Description

• This project is a real world system of robots that visually demonstrates the benefits of parallelization for computer vision applications

Motivation

• Computer vision systems have a wide variety of applications, but are very processor intensive

• Parallelization allows the implementation of more advanced computer vision techniques by removing the bottleneck on the processor

System Overview

• Project consists of two different robots– Target Drone Platform

• Detect & track projectiles• Avoid projectiles

– Turret System• Detect & track the target • Hit target with foam darts

First Steps

• Requirement Elicitation– High Level– Low Level

• Hardware/Software Specification• Risk Management• Project Timeline• Budget• Configuring Development Environment

Target Drone Platform

• Hardware– Traxxas Slash VXL*– ArduPilot Mega w/ IMU Shield– ION Intel Atom Motherboard*– Microsoft Xbox 360 Kinect Sensor*

*Other names and brands may be claimed as the property of others

Target Drone Platform

• Software– EMGU CV 2.3.0

• Open CV Wrapper (C#)– ArduRover

• Code for ArduPilot– Microsoft Robotics Developer Studio 4*– Kinect for Windows SDK v1.0*

*Other names and brands may be claimed as the property of others

Target Drone Platform

• Implementation– Assemble Target Drone

• Hardware Components• Power System

– Setup Wireless Access to Drone– Configure Drone Software Stack– Implemented Platform Services through

RDS

Target Drone Platform

• Implementation– Obtain RGB & Depth Image Data From

Kinect*– Projectile Detection

• Color Threshold– Projectile Tracking– Collision Avoidance Algorithm

*Other names and brands may be claimed as the property of others

Target Drone Platform

• Setbacks– Foam dart too small to track accurately– Hardware too heavy for original shocks– Depth frame coordinates are offset from

RGB frame coordinates with no translation function

– Dr. Shrivastava crashed the car into a tree!

Turret System

• Hardware– USB Missile Turret– Microsoft Xbox360 Kinect Sensor*– Arduino Uno– Servos

• Software– OpenCV 2.3.1– Libfreenect– Ubuntu 10.04*

*Other names and brands may be claimed as the property of others

Turret System

• Implementation– Obtain RGB/Depth Images from Kinect*

Sensor– Object Detection

• Color Threshold• Cascade Classification

– C USB Turret Driver– Program Arduino to Control Servos– C NetDuino Driver

*Other names and brands may be claimed as the property of others

Turret System

• Implementation– Equations Created to Track Target

• Depth Disparity to Depth(ft)– d_feet = (disparity-824.8)/15.2– d_feet = (0.1236*tan((disparity/2842.5)+1.1863))*3.2808399

• Pixels per Foot depending on Depth– Pixels/Foot = -55.91836*log(d_feet)+177.49225974457

Turret System

• Setbacks– Original turret configuration not precise

enough– Unable to implement libfreenect driver on

PS3*– Turret caught on fire– Turret mounting unstable

*Other names and brands may be claimed as the property of others

Turret System

• Turret Tracking and Hitting a Moving Car

• http://youtu.be/r7ccvPGcY8U

Turret System

• Effect of Parallelization on Turret System

• http://youtu.be/D5yTRKgOykY

Final StatusMajor Deliverables

Deadline Status

Project Plan 9/26/2011 100%

Unparallelized Turret System

1/31/2012 100%

Unparallelized Target Drone

2/7/2012 100%

Parallelized Robots

2/20/2012 50%

Benchmark Results

2/28/2012 50%

Product Documentation

3/15/2012 100%

Final Delivery of System

3/15/2012 75%

Questions

?

Backup Slides

Turret System

• Implementation– Equations To Track Target

• Horizontal Rotation• Vertical Rotation