Joe Cohenhttp://compsci.snc.edu/cs460/cohejr

Presentation Overview

Project definition and requirements Solution process and explanation Methodology Demonstration Development process Final remarks

Project Definition Mount a laptop and webcam atop a

robot (iRobot Create) Develop an application to control the

Create via the laptop Develop two separate applications:

Server: Run on laptop, take user input and control Create, capture webcam video and publish to client

Client: Run on separate computer, have full user interface to control the Create, allow for the viewing of video feed

Project Definition

Connect the two applications via sockets

Develop psuedo-language to communicate between the two applications

Run the two applications over the same (wireless) router

Solution ProcessCreate Control

Develop a VB application to control the Create robot

The language built into Create takes a set of codes and parameters to execute commands

Develop user interface for client application

Solution ProcessCreate Control

Operation Code # of Bytes

Parameter Meaning

Drive 137 4 Velocity (2) Radius (2)

Store Song 140 x Number, Length, note 1, note 1 length, etc…

Play Song 141 1 Song Number

Read Sensors 142 1 Packet Number

Wait 155 1 Time to wait in 1/10 second

Start OI 128 0

Safe Mode 131 0

Solution ProcessPseudo-Language

Develop a pseudo-language to communicate between server and client applications

Client application takes user input, encodes and sends to server. Server decodes and executes command.

Data/Control socket uses numerical codes

Solution ProcessPseudo-Language

Action Code

Drive straight forward at ## speed value 1##

Drive straight backward at ## speed value 0##

Change direction 0## or 1##

Stop 200 or 201

Turn 300, 301, 302, 310, 311, 312

Reset Robot 320

Solution ProcessSocket Connections

Sockets used to communicate between two computer/programs

Server application establishes socket and waits for client to connect to it

Connection made by IP address and port number

Solution ProcessClient/Server Application

Separate current application into client and serverClient: Keep user interface, instead of

calling Create control functions, encode and sent command over socket

Server: Decode code and execute Create commands

Solution ProcessWireless Network

Allow the applications to run over a wireless network rather than the internet

Benefits:Faster transmission of commandsWireless movement of robotWider range

Solution ProcessVideo Capture

Use Intel’s OpenCV camera library to initialize camera and capture frames

Once frames are located in memory a bitmap file is created

Using a memory dump the bitmap is displayed in a PictureBox on the screen

Solution ProcessVideo Feed

Once the bitmap structure is built, it is sent down a second socket to the client application

The server constantly captures and displays the video but only sends to client when it is requested

Video demonstration

Solution ProcessKeyboard Control

Incorporate another control method for the user via the keyboard

Allows for basic functionality using the keyboard

Perhaps a better way to control the robot

Solution Process

Project easily broke into three parts Develop separate solutions for

each part and piece together to make final product

Robot control grouped into three functions: straight, stop, and turn

Solution ProcessResources

iRobot Create Open Interface manual

Previous projects by John Moss, and Bryan Jorgensen & Adam Ziegler

Internet development forums All CS faculty

Solution ProcessCS Courses & Concepts

Operating systems – COM Ports, sockets, numerical conversions

Databases – VB 6 user interface development

Event Programming – Structure and flow of windows applications

Solution ProcessShortcomings

Populate COM ports on server connection form based on active COM ports

Allow for resizing of the video input window

Create respond to sensor events

Solution ProcessFuture Extensions

Improve on shortcomings Improve user interface and

application feedback Wireless camera and Create control Incorporate video algorithms:

motion tracking, object detection, etc.

Possibly integrate into .Net framework for further development

Solution ProcessAdvice to Future Seniors

Work early, and work often BACKUPS! Divide and conquer approach, don’t

get hung up on one part Ask for help and meet with as

many people as possible Define project early and voice

possible frustrations Develop passion for project

Thank you…

All CS professors Other CS seniors My father and the other fine

employees at Manitowoc Ice Bryan Jorgensen & Adam Ziegler Paul Deibele

Live demo Questions ?