VISPS Overview

• Capture visual data from two cameras• Find laser• Triangulate to find X, Y, Z coordinates of laser• Output coordinates to RS-232 serial output

RS-232 Out

Talk Overview

• VISPS goals and requirements – Cephas• Design (Theory + Architecture) – Steve• Design (Details + Communication) – Cephas• Design analysis – Kevin• Issues and testing – Kevin• Schedule – Cephas• Conclusion – Cephas

GOAL• Produce X, Y, Z coordinates of laser marker

relative to known reference frame

Requirements, use:• 630-680nm laser pointer• 2 RC-2 black and white cameras• XSV300 Development Board• PIC Microcontroller

VISPS Purpose

High Level Design Goals

• Produce X, Y, Z from visual data

• Simple interface

• Easy to use

• General, Accurate

• Design headroom

RequirementsTable 3 – VISPS operating parameter

Parameter Value

Coordinate Accuracy @ 63 inches the coordinates will be within 0.1 inches of its true location.

Data Output RS-232 Protocol (19.2 Kbps / 8 data bits / 1 stop bit / no parity)

Voltage Requires 9 and 12 V supply

Maximum Operating Distance

60 feet

Laser Flare Wavelength 630-680 nm

Laser Flare Max Output < 1 mW

Constraints From Requirements

• Code size on PIC C < 20kB

• Enough pins on XSV300 for two cameras input and communication with PIC C

How Are Design Goals Achieved?

• Mathematics and theory

• Logical implementation and architecture

• Implementation details and subsystem communication

Triangulation Problem Summary

Side ViewTop View

tantan

tantan

DZ tanZY

tantan

tan

DX

Visual Data

Left Camera Right Camera

Pixel coordinates are available.

Camera Calibration

Pixel Distances

• h is a constant 340 pixels at any distance

Angle Derivation From Pixels

a

htan

Triangulation By Pixels

a

htan

b

htan

h

ctan

ba

Da

ah

bh

ah

DD

X

tantan

tan

Math And Theory Summary

• Y, and Z found Similar to X

• Found X, Y, and Z using in terms of pixel coordinates

• No trigonometry or complex math

•No multiplication of error

ba

DaX

ba

DhZ

ba

DcY

Design Implementation

• Goals (Output points at 30fps, Preserve design headroom)

• Black Box design outline

• Component design specifics and I/O details

PIC µC

• Compute angles from pixel data

• Compute X, Y, Z from angles

• Output coordinates using RS-232 protocol

XSV300

•Locate Marker in pixel data

RoboCam RC-2

• Deliver pixel data

Generic Laser Pointer

• Project marker

RS-232 Coordinate output

Generic Red Filters• Remove non-laser lightVISPS

System Pin-outs

First saturated point

Last saturated point

Average of coordinates

Finding Center Of Marker

XSV300-PIC Protocol

NEW

READY

BUSY

C1hi C1lo C2hi C2lo Rhi RloDATA

Design Analysis

• Goals met

• Requirements satisfied

• Error analysis

• Lab results

Goals Met

• The design will correctly produce X, Y, and Z coordinates.

• We managed to avoid solutions that would have made using VISPS overly cumbersome or complicated.

• VISPS can output points at a full 30 points per second, the same rate as the camera

• Significant upgrades to VISPS accuracy from higher resolution cameras would not require a major design change

Requirements Satisfied

• Reworking our math enabled us to fit all the needed code on the PIC C

• With current cameras, VISPS is accurate to within 0.1 inches at 60 inches

• Moving to XSV300 from XS40 gave us the extra I/O pins we needed

Error Sources and Analysis

• Miscalibration of cameras

• Camera alignment

• Camera leveling

• Camera resolution

• Output resolution

Miscalibration Of Cameras

• All our mathematics are based on initial measurements at Z = 160cm

• Introduction of systematic error in Z if h does not really equal 340 pixels

• Spend extra time taking careful measurements

• Integer value for h is encouraging• Do sanity checks against manual

measurements

Camera Alignment• All our mathematics are based on the assumption

that the two cameras are perfectly parallel• Introduction of systematic error if cameras are not

parallel• Measure D at a distance check

Camera Leveling

• To know the reference frame for the X, Y, Z values produced, camera assembly must be perfectly leveled

• Use bubble levels mounted on camera assembly

• Do sanity checks against manual measurements

• Pixel size increases with distance

• Error is equal to ½ pixel width as a function of Z

• This error is inherent in VISPS and cannot be reduced except by increasing camera resolution

• Not bad. Even at 340ft, error only = 5.5 inches!

Camera Resolution

18.063

_

Zwidthpixel

Output Resolution

• Output is in tenths of inches

• Not a factor after 63 inches

• This error is inherent in VISPS

• Error can be reduced by reducing the max output distance

Lab Results

• Cameras filtered with red filter produce good, noise free data

• Able to find laser marker quickly, accurately, and consistently

• PIC C output shows that math is correct

• Unable to test system accuracy until parts are integrated

Problem Areas and Testing

• Resolved issues

• Remaining issues

• Reviewers comments

• Testing

Resolved Issues

• Noise filtering

• Interfering light sources

• Complex math and PIC C memory constraints

Remaining Issues

• Porting FPGA design to XSV300

• System accuracy

Reviewers CommentsProblem Their Suggestion What We DidSingle Pixel Noise In Camera Data

9 or 3 pixel median filter. 2 pixel neighbor intensity check.

Noise is not saturated.

Interfering Light Sources

Strobe laser using spinning wheel and take difference.

Red Filter to block non-laser light.

Jittery Or Unstable Coordinates

Average laser location over 4 frames for anti-jitter.

Nothing. We want the jitter.

Limited Memory of XS40

Move FPGA design to XSV300.

Moved FPGA design to XSV300

Floating-Point Math Bloat

Use LUT. Reworked math to avoid trigonometry.

Testing• Do lots of sanity checks against manual measurements• Test validity of math theories• Visually inspect VGA output of laser finding module

and camera perspective• Logic analyzer to check FPGA output• Extra PIC C test environment for PIC C • View serial output on Microsoft Hyperterminal• Output using printf() to test X, Y, Z production

independently of coordinate output protocol

Current Status And RoadmapCompleted:Find laser in visual dataModule to talk to PIC C PIC C mathPIC C, FPGA and RS-232 I/O

Remaining• Port FPGA design to XSV300• Integrate components and test• Win32 App for demo

Port FPGA Designto XSV300.

Integrate Components,test, and debug.

More debugging, test,demo.

Week 8

Week 9

Week 10

Conclusion And Summary

• Our design will work

• Design goals are met