Stanley – RC Car

Introduction

• The idea of this project – develop Remote Controlled Car that will be controlled

by the computer.• Initial Idea– Control Stanley by the keyboard– Control Stanley by the mouse– Draw the path

• Add on – Direction by laser pointer– Follow the Light point– Follow the object based on the colour.

The Hardware

• Remote Controlled Car– Digital controller

• PhidgetInterfaceKit 0/0/4 board– USB-based controller, – with 4 digital outputs

• USB Webcam• IP Webcam

Connecting the Car

• Controller – Switches for controlling

directions– One switch = two points

(ground and active)– Press switch closes circuit

• Identify the points– Different colour wires for different direction– Solder on group of two points

• PhidgetInterfaceKit Board– 4 Relay Outputs, each

relay has 3 connections:• #C = Common• NO – Normally Open• NC – Normally Close

• Screw the wires– Grouped by the colour – to each relay• #C, and NO• Port no = #

Connecting the Car

Connecting the Camera

• Power connection wireless– Power connector (2mm out diameter1.3 inner diameter).– Battery Holder– Wires– 4 AA CHARGABLE batteries at 1.2V each, with

capacity rating over 2000mAh.

The Software

• Visual Studio 2008• Microsoft Robotics Studio• Phidget Libraries• Aforge Libraries

The Software

• Microsoft Robotics Studio– Microsoft Robotics Studio Runtime have two main components;

• Concurrency and Coordinate Runtime (CCR) • Managed code Library

• The Decentralized System Services (DSS)• Decentralized Software Service Protocol (DSSP) –to interact with the services

• XML/Soap• Subscribe to service

• Contains state information, identification and has ports that can accept messages and service handlers to process these messages

– Supports wide range of programming Languages; VB.NET, C#, Microsoft Visual Programming Language (VPL) (provided as part of the Microsoft Robotics Studio)

– Create Service• Command prompt

– dssnewservice/service: RemoteCar• Folder with project and Solution Files

– RemoteCar.cs (contains implementation of the remote car service)– RemoteCar.manifest.xml (indicate a set of service that are to be started)

The Software

• Phidget Libraries– PhitgetInterfaceKit is managed behind this library– MRS includes a built-in service for PhidgetInterfaceKit– Communicate with the PhidgetInterface

• Add Reference• Import Library

private phidgetinterfacekit.PhidgetInterfaceKitBoardsOperations _ikPort = new phidgetinterfacekit.PhidgetInterfaceKitBoardsOperations();

• Declare Port private phidgetinterfacekit.PhidgetInterfaceKitBoardsOperations _ikPort = new phidgetinterfacekit.PhidgetInterfaceKitBoardsOperations();

• Subscribe To the Service_ikPort.SelectiveSubscribe(null,new phidgetinterfacekit.PhidgetInterfaceKitBoardsOperations());

The Software

public enum Direction { Forward = 0, Backward, Left, Right}public void SetOutput(Direction dir, bool enabled) { if (dir == Direction.None) return; // create a new request to send to the Phidgets service phidgetinterfacekit.SetOutputRequestType req = new

phidgetinterfacekit.SetOutputRequestType(); // assign the index and state (true/false) req.Index = (int)dir; req.State = enabled; // send the request to the port _ikPort.SetOutput(req); }

The Software

• Aforge.NET Libraries– Developed by Andrew Kirillow– Written in C#– For Computer Vision– 6 main libraries:

• AForge.Imaging is library for processing routines and filters.• AForge.Neuro is neural networks computation library.• AForge.Genetic is evaluation programming library.• AForge.Vision is computer vision library.• AForge.Machine Learning is machine learning library.• AForge.Robotics.Lego is for manipulation of Lego Mindstorm NXT

devices.– Classes for accessing local video capture devices (DirectShow),

and motion JPEG stream.

Stanley without camera

• Control the car by keyboard–Testing tool–ProcessCmdKey/keyUp

• Control the car on the screen–MouseUp/Down

• Control the car by the mouse–Compare last x and y coordinates with new

(count every 5th)

Stanley without camera

• Control the car by the mouse–Compare last x and y coordinates with new

(count every 5th)

Stanley without camera

• Follow the Path• Compare first x and y points with last• Calculate line length between points (Line Length = √ ((x2-x1)*(x2-x1)) + ((y2-y1)*(y2-y1))

• Calculate Angle

• Compare angles ==> • Measure Stanley movements

Stanley with camera

• Control on the screen With camera– Testing tool– MouseDown/Up

• Follow direction from the laser pointer– Look trough each pixel in image– Convert each pixel in corresponding colour from grey scale by

using the Greyscale BT709 algorithm– red * 0.2125 + green * 0.7154 + blue * 0.0721 = 0-255– Value >240– Record x and y points– Determine direction – Same algorithm like in “mouse move”

Stanley with camera

• Follow the Lightest point• Find the lightest pixel – same as in previous

example• Record the x and y points• Compare the points with camera position• Determine direction• Determine Distance (stop/go)• Search for the object

• Track the object based on the colour•Filter Image:

•Blur•Colour Filter –RGB – range 0-255 •Convert to Greyscale•Threshold

•Convert image into a 1-bit binary image•Black and white (depending on the pixel value

•Get Image size and position•BlobCounter / getObjectRectangles();

•Find the centre of the object •Record the x and y points•Compare the points with •camera position•Determine direction•Determine Distance (stop/go)•Search for the object

Stanley with camera

Deploy the Stanley

• DSS Deploy Tool • Creates files containing all MRS

components needed for running the application

• dssdeploy /p /m:“RemoteCar\RemoteCar.manifest.xml" InstallStanley.exe

• Start • MRS not needed• Trough Standard Command Prompt• cd /d C:/InstallStanley" bin\dsshost /p:50000 /m:" RemoteCar\

RemoteCar.manifest.xml"