Download - 3D SLAM for Omni-directional Camera
Yuttana SuttasupaAdvisor: Asst.Prof. Dr.Attawith Sudsang
3D SLAM for Omni-directional Camera
2
Outline Introduction
Localization, Mapping, SLAM, SLAM Application
Related Work Vision SLAM, SLAM with Omni-directional Camera
Our Problem Challenge of problem, Propose method, SLAM
Algorithm
Etc. Scopes, Work plan
3
Localization Robot can estimate its location with respects to
landmarks in an environment
4
Mapping Robot can reconstruct the position of landmarks
that its encounter in an environment
5
Simultaneous localization and mapping (SLAM) Robot build up a map and localize itself
simultaneously while traversing in an unknown environment
[Paul Newman, 06]
6
Objective Introduce SLAM method for a hand-held omni-
directional Camera moving freely in an unknown environment
Algorithm can reconstruct 3D camera path and 3D environment map
7
SLAM Challenges (Why is SLAM hard?) “Chicken and Egg” problem
Robot needs map to localize itself Robot needs to know its location to reconstruct
landmark positions Uncertainty problem
Sensor noise Data association
8
SLAM SLAM Problem
How can a robot localize its own position and build up a map in an unknown environment
SLAM Algorithm A procedure to solve a SLAM Problem Using a probabilistic approach to solve Find an appropriate representation for the observation
model and motion model SLAM Application
How to applied the observation model in our interest environment
What is a motion model for our interest robot It’s our work
9
Simultaneous localization and mapping (SLAM)
10
SLAM Application Indoor Environment Outdoor Environment At Home Under water UAV Navigation
[Ahn et al.,07; Davison et al,06; ...]
[Miro et al.,06; Newman, 07; Han et al., 07; ...]
[Choi et al.,06; Ahn et al,06; Motard et al.,07]
[Williams et al.,01; Ribas et al.,06]
[Jonghyuk Kim et al.,07]
11
Sensor
Sonar Laser range finder Omni-directional CameraVideo Camera
Sensor Price Accuracy Data typeSonar cheap low rangeLaser Range Finder
expensive high range & bearing
Camera vary (up to image quality)
moderate bearing
12
Vision SLAM Stereo Vision Based SLAM
[Thanh et al.,06; Schleicher et al.,06; Schleicher et al.,07; Lemaire et al.,07; Marzorati et al.,07; Han et al.,07, ...]
13
Vision SLAM Monocular SLAM
[Davison et al.,03; Eade et al.,06; Sunderhauf et al.,07; ...]
14
Omni-direction Camera Advantage
Large field of view (360° field of view) Real-time information @ 29.97Hz (NTSC) Vision data with color information Inexpensive
Disadvantage low resolution compare to the FOV
15
Omni-direction Camera
16
SLAM with Omni-directional Camera SLAM with Omni-directional Stereo Vision Sensor [Kim et al., 03] Visual SLAM by Single-Camera Catadioptric Stereo [Kim et al., 06]
17
SLAM with Omni-directional Camera Localization of mobile robots with omnidirectional vision
using particle filter and iterative SIFT [Tamimi et al., 05] Localization for Mobile Robots using Panoramic Vision,
Local Features and Particle Filter [Andreasson et al., 05]
18
SLAM with Omni-directional Camera Incremental Topological Mapping Using Omnidirectional
Vision [Valgren et al., 06] Appearance-based SLAM with Map Loop Closing Using
an Omnidirectional Camera [Saedan et al., 07]
19
SLAM with Omni-directional Camera SLAM in Indoor Environments using Omni-directional
Vertical and Horizontal Line Features [Kim et al., 08]
20
Our Problem Propose SLAM method for a hand-held omni-
directional Camera Omni-directional camera can move freely in an
unknown indoor environment Reconstruct 3D camera path and 3D
environment map (landmark-based) No need any initial information or predefine data
21
Challenge Irregular Sensor
Uncommon sensor’s measurement model What features should we use to be a measurement
Insufficient information Bearing-only data Unpredictable camera trajectory Don’t have any initial information
A high dimensional state Need to estimate 3D camera path Need to estimate 3D environment map
Proposed Method
23
Localization Concept
x
bel(x)
x
bel(x)
x
bel(x)
x
bel(x)
Initial state
Measurement-update
Measurement-update
Time-update
observation
observation
24
SLAM Algorithm Probabilistic SLAM [Smith and Cheeseman, 86]
The probability distribution of robot state and landmark locations
The observation model
The motion model
25
SLAM Algorithm SLAM recursive algorithm
Time-update
Measurement Update
26
Using SLAM in our problem Use EKF SLAM to solve our SLAM problem
Find an observation modelhow to measurement landmarkshow to detect features from an omni-image
Find a motion modeldetermine how a camera move
Find a state representationhow to represent a camera statehow to represent landmarks state
[Moutarlier and Chatila, 89]
27
An observation model Detect features from an
omni-image Point features, Line features
Features association How features associate with
landmarks
Feature measurements Observation model
landmark
camera
28
A motion model Camera can move freely in an environment
Don’t know camera motion model Cannot predict camera trajectory Too many state variables to estimate with SLAM only
May need to pre-estimate camera state Using pre-estimate camera state to predict camera
state
time k time k + m
camera
landmark
29
Result evaluation Localization evaluation
Using ground truth data Using global localization
Mapping evaluation Using ground truth data from structural environment
30
Scope of the research Develop SLAM method for an omni-directional
Camera Develop an algorithm to detect features from
omni-directional image A camera can move freely in 3D environment
without knowledge of motion model Algorithm can reconstruct 3D camera path and
3D environment map Test a system in an indoor environment No dynamic objects
31
Work Plan Study the works in the related fields Develop algorithms Test the system Evaluate a result Prepare and engage in a thesis defense
Thank you
33
SLAM Solution Solutions to the SLAM Problem
EKF SLAM - using the extended Kalman filter (EKF) to solve the SLAM problem [Moutarlier and Chatila, 89]
Fast SLAM - using the Rao-Blackwellized particle filter to solve the SLAM problem [Montemerlo et al., 02]
Etc.
34
Extended Kalman Filter Kalman Filter
An efficient algorithm for state estimation problems Based on linear dynamical systems A Hidden Markov models with Gaussian distributions
Extended Kalman Filter Nonlinear version of the Kalman filter Using Jacobian to linearize nonlinear functions
35
Extended Kalman Filter - Predict
x
P(x)
Initial state
x
bel(x)
Predict
Predicted state
Predicted estimate covariance
state transition function
control vectorprevious state
state transition jacobian previous error covariance
process noise covariance
36
Extended Kalman Filter - Update
x
P(x)
Update
Updated state estimate
Updated estimate covariance
predict state
Estimate measurement
Innovation covariance
Optimal Kalman gain
Innovation
innovation
kalman gain
observation function
mesurement
estimate measurement
kalman gain
innovation covariancepredicted estimate covariance
observation jacobian
observation noise covariance
37
A state representation represent a camera state and landmarks state
correspond to a motion model and an observation model
38
Sensor Sonar Laser range finder Video Camera Omni-directional Camera Etc.
Sonar
Laser range finder Omni-directional CameraVideo Camera
39
How SLAM will be used Detect good features from image Find a good state representation for camera
state and landmarks state Find a good observation model from image
features for SLAM Find a good motion model for SLAM
Don’t know camera motion model Need camera motion estimation
40
Feature Detection Point Detector
Harris Detector Scale & Affine invariant
point detectors
Line Detector Hough (line) transform
[Ying and Hu, 04]
41
Feature Detection Object Detector
Template matching SIFT [Lowe, 04]
42
Feature Associations Feature associations from Image
SIFT Optical flow
Feature associations from measurement Mahalanobis distance
Estimate Measurement
Real Measurement
Landmark state
43
Camera Motion Estimation Camera can move freely in an environment
Cannot predict camera trajectory Too many state variables to estimate with SLAM only
Need pre-estimate camera state Using relation of two period omni images Non-linear Least Square Iterative Extended kalman filter
44
Camera Motion Estimation Known landmark positions related to a reference
frame (from SLAM state) Known current landmark mesurement (from
current image) Can estimate a camera state related to a
reference frame
time k time k + m
camera
landmark
45
System Coordinate World Frame Reference Frames Camera Frame
landmark
World Frame
Camera Frame
Reference Frame
Camera
Camera trajectory
46
SLAM Prediction
Use a camera estimation state from previous
Measurement Update Use measurements from current and old features
Landmark Augmentation Add a new landmark when a camera encounter a new
feature
Reference Frame Augmentation Select new reference frame from old suitable camera
state
47
SLAM Application Indoor Environment
Outdoor Environment[Ahn et al.,07; Davison et al,06; ...]
[Bailey,02; Asmar et al., 06; Miro et al.,06; Newman, 07; Han et al., 07; ...]
48
SLAM Application At Home
Under water[Choi et al.,06; Ahn et al,06; Motard et al.,07]
[Williams et al.,01; Ribas et al.,06]
49
SLAM Application UAV Navigation
Agricultural Robotics[Jonghyuk Kim et al.,07]
50
Type of SLAM Grid maps or scans
Landmark-based
[Lu & Milios, 97; Gutmann, 98: Thrun 98; Burgard, 99; Konolige & Gutmann, 00; Thrun, 00; Arras, 99; Haehnel, 01;...]
[Leonard et al., 98; Castelanos et al., 99: Dissanayake et al., 2001; Montemerlo et al., 2002;...]
51
Type of SLAM Topology
Appearance
[Mark Cummins and Paul Newman, 07]
[Motard et al., 07; Valgren et al, 06]
52
Problems in SLAM Costs
Computational/Storage, Use of Information form based estimators.
Sensing Issues Interpretation of Sensor Data/Association.
Environmental Representations Feature based, grids, Sum-of-Gaussians, Scan Matching.
Loop Closing Recognising where you have been before.
Scaling Large Environments.
Observability Estimating a high dimensional state with low dimensional
measurements.
53
Video Camera Advantage
Information rich Real-time information Compact Inexpensive
Disadvantage Hard to deal with image raw data Not provide depth information Need a lot of computation time
54
Our Approach Feature Detection and measurement / Feature
Associations Camera Motion Estimation SLAM