Robot Vision with CNNs:a Practical Example

P. VitulloP. Campolucci

G. ApicellaL. Pompeo

D. BellachiomaS. Graziani

M. BalsiDep. of Electronic Engineering

“La Sapienza” Univ. of Rome, Italy

X. Vilasís–CardonaS. LuengoJ. SolsonaR. Funosas

A. MaraschiniA. Aznar

V. GiovenaleP. Giangrossi

Barcelona, 19/2/03

Framework of this work

• completely autonomous robot• simple (cheap) hardware• vision-based guidance

– short term: line following– longer term: navigation in a real environment

Architecture

• Cellular Neural Networks to handle all the image processing

• Fuzzy-rule-based navigation

Cellular Neural Networks

• Fully parallel analog vision chips• Capable of real-time nonlinear image

processing and feature detection

• Algorithmically programmable to implement complex operations

• On-board image acquisition (focal-plane processing)

Cellular Neural Networks

• Recurrent Neural (?) Network• Locally connected VLSI-friendly• Space-invariant synapses (cloning

templates)– small number of parameters: explicit design

• Continuous variables – analog computing (discrete-time model for digital)

TopologyLocally connected VLSISpace-invariant synapses

Discrete–time model

• Binary state variable• Analog or binary input depending

on implementation

IuB

nxAsignnx

ijNklklljki

ijNklklljkiij

;

;1

Application• Input ports: analog arrays u, x(0)• Output port: binary array x()• “Analog instruction”: {A,B,I} (cloning

template)• Feature detection (nonlinear image

filtering)

IuB

nxAsignnx

ijNklklljki

ijNklklljkiij

;

;1

CNN “Universal” Machine

• Local memory• Global control (broadcasting cloning

templates and memory transfer commands)

• “Analogic” computing: stored-program analog/logic algorithms

Task: line following

• The robot is to follow a maze of straight lines crossing at approximately right angles

• Functions required by vision module:

Acquiring image, cleaning, thinning linesMeasuring orientation/displacement of lines

Image processing algorithm

• Image acquisition

• Binarization

• Line thinning

Image processing algorithm (ctd.)

• Directional line filtering

• Projection

Fuzzy control

Simulation

y (m) z vs. x (m)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

el cochecito(Barcelona)

control (386)

CNN emul. (DSP)

Visibilia (Rome)

PAL B/WCAMERA

FPGA-based CNN emulatorCeloxica RC-100 board

Xilinx Spartan II 200Kgates

microcontroller

Jackrabbit BL1810

PIC 16F84

SERVOMOTOR

(steering)

LCD

PS/2 mouse port

Rabbit2000microcontroller

Parallel port E

Parallel port ASerial port D

STEPPERMOTOR

(advancing)

STEPPER MOTOR

CONTROLLER

Celoxica RC-100

VGA

Jackrabbit BL1810

drivingstart

vert

hor

follow vert

horY

Y

N

N

horY

N

normal driving

crossing

timer:=0

timer>10s N

Y

store left avail.

turn left if avail.else right

diag (L/R)

Y

follow diagY

N

Continuation of the work

more realistic tasks:• obstacle avoidance• navigation in a real-life environment

Obstacle avoidance• using other sensors together with

vision, e.g. ultrasound• monocular range evaluation• local path-finding strategies

Hybrid (topological/metric) navigation

door recognition

Robot Vision with CNNs:a Practical Example

M. BalsiDep. of Electronic Engineering

“La Sapienza” Univ. of Rome, Italy

balsi@uniroma1.it

Barcelona, 19/2/03