Intelligent Vision Systems 1

Intelligent Vision Systems

@ Institute of Computer Science III

Jens Behley, Ali Borji, Armin B. Cremers,

Simone Frintrop, Dominik Klein, Volker Steinhage

8.10.2009

Intelligent Vision Systems 2

Team

now Ph.D.

Intelligent Vision Systems 3

Computational Visual Attention• Visual attention:

concept of human vision

• Computational attention systems: simulate this behaviour

• Useful especially for complex systems:

– quickly determine region of interest,

– restrict time-consuming processes to small parts

Computational attention system

Input image

Saliency map

Intelligent Vision Systems 4

Attention System VOCUS

Intensity OrientationColor

FeatureMaps

input image

ConspicuityMaps

[Frintrop, Springer LNAI, 2005]SaliencyMap

greenblueoff on

0o 45o

90o 135o

yellowred

Uniquenessweigt

Uniquenessweigt

Intelligent Vision Systems 5

Attentive Robot Localization• Use visual landmarks to

localize robot

• tracking and redetection of discriminative landmarks

• attention systems to find salient landmarks

Where am I?

landmark 1

landmark 2

landmark 3

match!

match!

match!

I must be in the kitchen!

[Frintrop,Jensfelt:IEEE Trans. on Robotics 2008][Frintrop: ECCV workshop, 2008]

Intelligent Vision Systems 6

Attentive Robot LocalizationCan also be integrated into visual SLAM (simultaneous localization and mapping):

Video at:http://www.informatik.uni-bonn.de/~frintrop/research/aslam.html

Intelligent Vision Systems 7

Visual Tracking• Challenges:

– moving camera– real-time constraints– illumination changes– unknown environment– quick online learning

of objects desired

• We use probabilistic particlefilter approach with a cognitiveobservation model

[Yukie Nagai, University Bielefeld]

[Frintrop,Kessel, ICRA 2009][Frintrop, Königs, Hoeller, Schulz: J. on Social Robotics 2010]

Intelligent Vision Systems 8

Results

[Frintrop,Kessel, ICRA 2009][Frintrop, Königs, Hoeller, Schulz: J. on Social Robotics 2010]

Considerably better resultsthan standard color histogramtracking

Current work:

- consider spatial layout of target to compute component-based descriptor:

- adapt target descriptoraccording to backgroundchanges

[Frintrop, submitted]

[Borji, Frintrop, submitted]

Intelligent Vision Systems 9

Adapting Feature Descriptors forBackground Change

• Before: target descriptor from first frame is used in particles

• Now: target descriptor for the cluster of each background is used

• How : A number of background clusters from a train sequence is first derived. Then over a test sequence, cluster of the current background is determined and particles are updated with the descriptor of this cluster

Comparing approaches 1) first frame, 2) clustering 3) frame by frame and 4) ground truth object positions

Results in finer tracking than previous approach[Borji, Frintrop, submitted]

Intelligent Vision Systems 10

Multi-Sensor Object Classification• smart combination of sensors• Compute a spatial mapping of

sensors• AdaBoost to learn optimal

combination of sensors.

→ see our poster

[Klein,Schulz,Frintrop, ICVS 2009]

Intelligent Vision Systems 11

Object Classification & Tracking• co-operation with Fraunhofer FKIE• dense 3D laserscans (Velodyne HDL64-E)

current work:• classification in dynamic scenes

using CRFs and kNN• tracking of multiple objects

with particle filters

Intelligent Vision Systems 12

Generation of 3D City ModelsStrategies:

• Recognition by Components

• Data Fusion(Areal images, DSM, GIS)

→ see our poster

[Behley, Steinhage - ICVS 2009]

Intelligent Vision Systems 13

Maintenance of 3D City ModelsStrategy:

• Open Source Components

• SQL queries

• Point & window queries

[Steinhage, Behley, Meisel - submitted]

select Window(Geometry)from Buildingswhere RoofType =“HipRoof” and

OverlapsRect (Footprint,@Rectangle);

Intelligent Vision Systems 14

Taxon Identification in SystematicsStrategy:

• Fingerprinting Bees:

– Automatic Extraction of Morphological Features

– Non-linear Kernel Discriminant Analysis

– Multi-Media Database

[Steinhage et al. - ATIS, Taylor & Francis 2007]

[Francoy et al. - Genetics and Molecular Research 2009]

Intelligent Vision Systems 15

Taxon Identification in SystematicsStrategy:

• Fingerprinting Orthoptera:

– Automatic Extraction of Morphological Features

– Multi-Image Analysis

[Steinhage et al. - ATIS, Taylor & Francis 2007]

Intelligent Vision Systems 16

Thank You