real-time dense visual odometry for quadrocopters christian kerl 11.05.20121
TRANSCRIPT
1
Real-time Dense Visual Odometry for Quadrocopters
Christian Kerl
11.05.2012
2
Outline
• Motivation• Hardware & Software• Approach• Problems• Ideas
11.05.2012
3
Motivation
• Quadrocopters need sensors to fly in unknown environments– Motion– Position– Obstacles
• Restricted on-board sensors– IMU– Visual navigation (no GPS)
• Restricted computing resources Autonomous system11.05.2012
4
Motivation
• Standard approach to visual odometry:– Sparse feature tracking in intensity / color images– Examples: Jakob, ETH Zurich, TU Graz, MIT– On-board frame rates 10 Hz
• Our approach:– Using full RGB-D image information– No feature tracking
11.05.2012
5
Hardware – Asctec Pelican
11.05.2012
6
Hardware – Asctec Pelican
IMU
AutoPilot
Board
Atom Board
11.05.2012
7
Hardware – Asctec Pelican
• IMU – 3 axis magnetometer, gyroscope, accelerometer
• AutoPilot Board– Highlevel + Lowlevel Processor (ARM)
• Atom Board – Intel Atom Z530 1.6 GHz– 1 GB RAM– 7 Mini-USB Ports– WirelessLAN
• 600 g payload11.05.2012
8
Software – Asctec Pelican
• ROS drivers for Asctec Pelican from ETH Zurich• Nonlinear dynamic inversion for position control• Luenberger Observer for data fusion • Updated version using Extended Kalman Filter to
be presented on ICRA 2012• Needs absolute position input from external
source• Allows to command accelerations, velocities or
positions
11.05.2012
9
Hardware – Asus Xtion Pro Live
• 24 bit RGB image• 16 bit depth image• 640x480 @ 30 Hz• 150 g
+ On-camera RGB and depth image registration+ Time synchronized depth and RGB image- Rolling shutter- Auto exposure11.05.2012
10
Approach
• Estimate transformation minimizing squared intensity error (energy minimization)
11.05.2012
11
Approach
• Linearization
• with
• minimize
=> solve normal equations
11.05.2012
12
Analysis
• Estimate transformation minimizing squared intensity error (energy minimization)
X translation Y translation11.05.2012
13
• Hovering
Image Data from Quadrocopter
11.05.2012
14
• Trajectory along camera z-axis
Image Data from Quadrocopter
11.05.2012
15
Problems
• Motion blur• Auto exposure• Dynamic objects (humans)
11.05.2012
16
Problems – Motion Blur
11.05.2012
17
Problems – Motion Blur
11.05.2012
18
Problems – Motion Blur
11.05.2012
19
Problems – Motion Blur
11.05.2012
20
Problems – Motion Blur
11.05.2012
21
Problems – Motion Blur
11.05.2012
22
Problems – Motion Blur
11.05.2012
23
Problems – Auto Exposure
11.05.2012
24
Problems – Auto Exposure
11.05.2012
25
Problems – Dynamic Objects
11.05.2012
26
Ideas
• Weighted Least Squares• Initial motion estimate between 2 consecutive
frames from IMU data fusion• Multiple iterations per level, convergence
checks• Regularization term to minimize / constrain
least squares solution• Minimization of intensity and depth error
11.05.2012
27
Ideas – Weighted Least Squares
• Assign smaller weight to residual outliers
=>
• Weight calculation
11.05.2012
28
Ideas – Weighted Least Squares
• Influence function– Tukey weight
– Huber weight
11.05.2012
29
Ideas – Weighted Least Squares
• Weighted error
11.05.2012
30
Ideas – Weighted Least Squares
• Influence on energy function
X translation w/o weights X translation w/ Huber weights11.05.2012
31
Ideas – Weighted Least Squares
• Influence on energy function
Y translation w/o weights Y translation w/ Huber weights11.05.2012
32
Ideas – Weighted Least Squares
11.05.2012
33
Ideas – Weighted Least Squares
• Robustification with respect to dynamic objects
• Slightly degrades tracking performance
• How to choose parameter b?
11.05.2012
34
Ideas – Initialization from IMU
• Use transformation from IMU data fusion as initial estimate
11.05.2012
35
Ideas – Initialization from IMU
• Use transformation from IMU data fusion as initial estimate
11.05.2012
36
Ideas – Initialization from IMU
• Use transformation from IMU data fusion as initial estimate
11.05.2012
37
Ideas – Multiple Iterations
• Perform multiple optimization steps per image pyramid level
• Stop when increment below threshold• Bad frames / diverging
results can be recognized and skipped
11.05.2012
38
Summary/Discussion
• Weighted Least Squares needs more work (especially weight calculation)
• Initialization from IMU promising• Multiple Iterations for increased accuracy and
divergence detection promising, but computationally expensive
• Jumps in trajectory are really problematic!=> Ideas welcome!
11.05.2012