Variable Resolution Vision Variable Resolution Vision System in Mobile RoboticsSystem in Mobile Robotics

Armando SousaArmando Sousa asousa@fe.up.pt

Paulo Costa, António MoreiraPaulo Costa, António MoreiraFaculdade de Engenharia da Universidade do Porto (FEUP)

Instituto de Sistemas e Robótica

R. Dr. Roberto Frias, S/N / 4200-465 Porto / Portugal

Problem: Robot VisionProblem: Robot Vision

Onboard cameras are the main sensor– Quality Cameras Large Image– Larger Image Comp. Power

Embedded Robot Vision– Real Time Constraints– Limited Computing Power Available

Vision ProblemsVision Problems

Given several Vision Problems

(a) Camera Onboard (b) Static External Camera

(c) generic object moves in

3D over a plane

Objects Nearer are Always Larger

Goal StatementGoal Statement

Structured Vision

– Objects Closer are Larger– Distance to Vision Plane is

Known

Not every pixel is essential

to find close objects

Sub Sample the Image !!!

Projection Effect

Pin-Hole Camera ModelPin-Hole Camera Model

Pin-Hole Camera Model

F o c a l D is ta n c e

O p tic a l A x isL e n s C e n te r

Im a g e P la n e

P

p

Z

u ’

X

Y

v ’

F o c a l D is ta n c e

L e n s C e n te r

Im a g e P la n e

P

p

u

v’

u’

v

(Ou,Ov)

Y / Z f v'

Y / X f = u'

Objects far away occupy less pixels

Lens Distortion ModelLens Distortion Model

) rd k 1 ( Sv )Oy - vd( v'

) rd k 1 ( Su )Ox - ud ( u'2

1

21

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

)yd (xd rd 22

Lens distortion model (barrel / pin-cushion)

Math ResultsMath Results

DistanceZc

2( )( )sin ( ) Ov v S ( )cos f

2

( ) ( )cos S Ov ( )cos S v ( )sin f 2 =

Zc2

( ) Ou u2

S2

( ) ( )cos S Ov ( )cos S v ( )sin f2

Zc2

2

Sub Sample image with density proportional to distance

* No lens distortion correction

Dens = Min(Distance/Horizon,1)

Distance GraphDistance Graph

Offline Densities BitmapOffline Densities Bitmap

Distance Graph GeyScale

DistanceEncoding

Dithered Bitmap

using Floyd Steinberg error diffusion)

ExampleExample

Actual images relating to the5dpo-2000 Robotic Soccer Team

Pixel WeightPixel Weight

Error in center of cluster due to Center of mass calculations

Taking local densities into consideration

(important for tall objects)

Review of AlgorithmReview of Algorithm

Generate Offline Densities Bitmap: – distance formulae pixel densities grayscale bitmap B&W

Pixel Classification: – black pixels run pixel color calibration

Clustering & Merging:– Cluster neighboring pixels together

Iterate Image:– Iterate for the whole image to take advantage of cache

optimisations

Tech Compare ChartTech Compare Chart

Pixelcount

Time(Cel 450 MHz)

W/O WeighedCenters

192x144x15 100% 100%(7ms)

-

384x288x15 400% 440%(31ms)

-

384x288x15(with sub-sampling)

130% 160%(11ms)

130%(9ms)

Results relating to the5dpo-2000 Robotic Soccer Team (BT878, PAL, 25 fps)

ConclusionsConclusions

Main advantage of the method is that objects far away are more clearly seen

The presented method is suited to Real Time applications Method involves re-sampling the image!

– interesting if computing power is not enough to process the whole image at maximum resolution

Generation of offline Dither Bitmap implies all camera parameters must be known and constant

Bitmap densities parameters:– min density min object size and required precision

– Greatly affects execution time

Future WorkFuture Work

Adapt the algorithm for moving cameras

Measurement's accuracy can be improved by rescanning the boundaries of the objects at full resolution

Other considerations for distance model may be taken into consideration