Robot Compagnion Robot Compagnion Localization Localization

at home and in the officeat home and in the office

Arnoud Visser, Arnoud Visser, Jürgen SturmJürgen Sturm, , Frans GroenFrans Groen

University of AmsterdamInformatics Institute

OverviewOverview Mobile roboticsMobile robotics

Robot localizationRobot localization

Presentation of the panorama approachPresentation of the panorama approach

ResultsResults

Demonstration videosDemonstration videos

Mobile roboticsMobile robotics

SICO at Kosair Children's Hospital Dometic, Louisville, Kentucky

Sony Aibos playing soccerCinekids, De Balie, Amsterdam

Robot cranes and trucks unloading shipsPort of Rotterdam

RC3000, the robocleanerKärcher

The localization problemThe localization problem

Robot localizationRobot localization.. is the problem of estimating the robot’s .. is the problem of estimating the robot’s

pose relative to a map of the pose relative to a map of the environment.environment.

Position trackingPosition tracking Global localizationGlobal localization Kidnapping problemKidnapping problem

LocalizationLocalization

SensorsSensors Odometry, GPS, Laserscanner, Camera..Odometry, GPS, Laserscanner, Camera.. Feature spaceFeature space

World representationWorld representation Topological graphs, grid-based mapsTopological graphs, grid-based maps

FiltersFilters Kalman filters, particle filtersKalman filters, particle filters

Classical approachesClassical approaches

Special environmentsSpecial environments (Visual) landmarks(Visual) landmarks (Electro-magnetic) guiding lines(Electro-magnetic) guiding lines

Special sensorsSpecial sensors GPSGPS Laser-scannersLaser-scanners Omni-directional camerasOmni-directional cameras

Special requirementsSpecial requirements Computationally heavy (offline Computationally heavy (offline

computation)computation)

New approachNew approach

Natural environmentsNatural environments Human environmentsHuman environments Unstructured and/or unknown for the Unstructured and/or unknown for the

robotrobot Normal sensorsNormal sensors

CameraCamera Reasonable requirementsReasonable requirements

Real-timeReal-time Moderate hardware requirementsModerate hardware requirements

Platform: Sony AiboPlatform: Sony Aibo

Internal camera•30fps•208x160 pixels

Computer•64bit RISC processor•567 MHz•64 MB RAM•16 MB memorystick•WLAN

Actuators•Legs: 4 x 3 joints•Head: 3 joints

Demo: CompassDemo: Compass

Library, University of Amsterdam

SynopsisSynopsis

.. .... ..

Raw image Color class imageSector-based

feature extraction

Previously learnedworld panoramafor a given spot

Alignment step

Odometry data Post filtering

rotation, translation,confidence range

Color segmentationColor segmentation

Sidetrack: Color Sidetrack: Color CalibrationCalibration

Robot collects colors Robot collects colors from environmentfrom environment

Colors are clustered Colors are clustered using an EM algorithmusing an EM algorithm

Color-to-Colorclass Color-to-Colorclass lookup table is created lookup table is created for faster accessfor faster access

Raw image

Color class image

MathematicsMathematics

rotation

translation

feature vector

ideal world model

learned world model

360;..;0[for

,

RRfP

TRfP

f

T

R

spot

Feature space conversionFeature space conversion

Feature vectors and Feature vectors and world modelworld model

World model distribution

Feature vector consists of color transition counts between the n color classes

nnn

n

ff

ff

f

1

111

n

i

n

jijspotspot RfPRfP

1 1

Feature space conversion Feature space conversion (2)(2)

Raw imageColor class

imageSector-based

feature vectors

LearningLearning

)1(

12-2

-11

1 22 if

22 if

2 if

1

binsij

-binsbinsij

ijij

ijij

bins

k

kij

ijspot

fm

fm

fm

mRfP

Update distribution of single color class transition

by updating the constituting counters bins

ijij mm ;...;1

MatchingMatching

Likelihood ofSingle sector

Rotation estimate

Confidence estimate

sssspot

R

sssspot

sssspot

R

sssspot

spot

RRfP

RRfPC

RRfPR

RfP

RfP

mean

ˆˆ

maxargˆ

Adjacent sectors

Post-processing: Post-processing: CompassCompass

Idea: smooth rotational estimate over Idea: smooth rotational estimate over multiple framesmultiple frames

+ removes outliers+ removes outliers

+ stabilizes estimate+ stabilizes estimate

+ integrates (rotational) odometry+ integrates (rotational) odometry

Results: CompassResults: Compass

Brightly illuminated living room

Results: CompassResults: Compass

Daylight office environment

Results: CompassResults: Compass

Outdoor soccer field

Results: CompassResults: Compass

Robocup 4-Legged soccer field

Signal degradation (w.r.t. Signal degradation (w.r.t. distance)distance)

Robocup 4-Legged soccer field

0

10

20

30

40

50

60

70

80

90

0 50 100 150 200 250Distance from learned spot (centimeters)

deg

rees

confidence range

error in rotation estimate

Post-processing: Grid Post-processing: Grid localizationlocalization

Idea: learn multiple spots, then use Idea: learn multiple spots, then use confidence value to estimate the confidence value to estimate the robot‘s position in betweenrobot‘s position in between

– – fixed grid fixed grid (better: self-learned graph based on (better: self-learned graph based on

confidence)confidence)

– – difficult to integrate odometrydifficult to integrate odometry

+ proof of concept+ proof of concept

Demo: Grid localizationDemo: Grid localization

Robocup 4-Legged soccer field

Results: Grid localizationResults: Grid localization

Robocup 4-Legged soccer field

-100

-75

-50

-25

0

25

50

75

100

-100 -75 -50 -25 0 25 50 75 100

x [cm]

y [c

m] Positioning

accuracy

cm

cm

cm

cm

37.15

73.16

09.12

30.22

Robot walks back to center after kidnap

ConclusionsConclusions Novel approach to localization:Novel approach to localization:

Works in unstructured Works in unstructured environmentsenvironments

Tested on various locationsTested on various locations

Interesting approach for mobile Interesting approach for mobile robots robots at home and in the officeat home and in the office

Questions?Questions?