motion from image and inertial measurements
DESCRIPTION
Motion from image and inertial measurements. Dennis Strelow Honeywell Advanced Technology Lab. On the web. Related materials: these slides related papers movies VRML models at: http://www.dennis-strelow.com/umn. Introduction (1). From an image sequence, we can recover: - PowerPoint PPT PresentationTRANSCRIPT
Motion from image and inertial measurements
Dennis Strelow
Honeywell Advanced Technology Lab
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 2
On the web
Related materials:
these slides
related papers
movies
VRML models
at:
http://www.dennis-strelow.com/umn
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 3
Introduction (1)
From an image sequence, we can recover:
6 degree of freedom (DOF) camera motion
without knowledge of the camera’s surroundings
without GPS
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 4
Introduction (2)
Fitzgibbon
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 5
Introduction (3)
Potential applications include:
modeling from video
Yuji Uchida
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 6
Introduction (4)
micro air vehicles (MAVs)
AeroVironment Black Widow AeroVironment Microbat
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 7
Introduction (5)
rover navigation
Hyperion Nister, et al.
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 8
Introduction (6)
search and rescue robots
Rhex (movies: http://ai.eecs.umich.edu/Rhex/Movies)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 9
Introduction (7)
NASA Personal Satellite Assistant (PSA)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 10
Introduction (8)
For these problems, we want:
6 DOF motion
in unknown environments
without GPS or other absolute positioning
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 11
Introduction (8)
For these problems, we want:
6 DOF motion
in unknown environments
without GPS or other absolute positioning
using small, light, and cheap sensors
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 12
Introduction (8)
For these problems, we want:
6 DOF motion
in unknown environments
without GPS or other absolute positioning
using small, light, and cheap sensors
over the long term
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 13
Introduction (9)
Long-term motion estimation:
absolute distance or time is long
only a small fraction of the scene is visible at any one time
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 14
Introduction (10)
given these requirements, cameras are promising sensors…
…and many algorithms for motion from images already exist
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 15
Introduction (11)
But, where are the systems for estimating the motion of:
over the long term?
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 16
Introduction (12)
…and for automatically modeling
rooms
buildings
cities
from a handheld camera?
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 17
Introduction (13)
Motion from images suffers from some long-standing difficulties
This work attacks these problems by…
exploiting image and inertial measurements
robust image feature tracking
recognizing previously mapped locations
exploiting omnidirectional images
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 18
Outline
Motion from images
refresher
bundle adjustment
difficulties
Motion from image and inertial measurements
Robust image feature tracking
Long-term motion estimation
Conclusion
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 19
Motion from images: refresher (1)
A two-step process is common:
sparse feature tracking
estimation
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 20
Motion from images: refresher (1)
A two-step process is common:
sparse feature tracking
estimation
Sparse feature tracking:
inputs: raw images
outputs: projections
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 21
Motion from images: refresher (2)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 22
Motion from images: refresher (3)
Template matching:
correlation tracking
Lucas-Kanade (Lucas and Kanade, 1981)
Extraction and matching:
Harris features (Harris, 1992)
Scale Invariant Feature Transform (SIFT) keypoints (Lowe, 2004)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 23
Motion from images: refresher (4)
The second step is estimation:
inputs:
projections
outputs:
6 DOF camera position at the time of each image
3D position of each tracked point
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 24
Motion from images: refresher (5)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 25
Motion from images: refresher (6)
bundle adjustment (various, 1950’s)
Kalman filtering (Broida, Chandrashekhar, and Chellappa, 1990)
variable state dimension filter (VSDF) (McLauchlan, 1996)
two- and three-frame methods(Hartley and Zisserman, 2000; Nister, et al. 2004)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 26
Motion from images: bundle adjustment (1)
From tracking, we have the image locations xij for each point j and each image i
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 27
Motion from images: bundle adjustment (2)
Suppose we also have estimates of:
the camera rotation ρi and translation ti at time of each image
3D point positions Xj of each tracked point
Then, we can compute reprojections:
))(( iji tXR
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 28
Motion from images: bundle adjustment (3)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 29
Motion from images: bundle adjustment (4)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 30
Motion from images: bundle adjustment (5)
So, minimize:
with respect to all the ρi, ti, Xj
)),)(((,
image ijijji
i xtXRDE
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 31
Motion from images: bundle adjustment (5)
So, minimize:
with respect to all the ρi, ti, Xj
)),)(((,
image ijijji
i xtXRDE
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 32
Motion from images: difficulties (1)
Estimation step can be very sensitive to…
incorrect or insufficient image feature tracking
camera modeling and calibration errors
outlier detection thresholds
sequences with degenerate camera motions
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 33
Motion from images: difficulties (2)
Iterative batch methods have poor convergence or may fail to converge if:
observations are missing
the initial estimate is poor
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 34
Motion from images: difficulties (3)
Recursive methods suffer from:
poor prior assumptions on the motion
poor approximations in state error modeling
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 35
Motion from images: difficulties (4)
Resulting errors are:
gross local errors
long term drift
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 36
Motion from images: difficulties (5)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 37
Motion from images: difficulties (6)
151 images, 23 pointsmanually corrected Lucas-Kanade
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 38
Motion from images: difficulties (7)
squares: ground truth points dash-dotted line: accurate estimate solid line: image-only, bundle adjustment estimate
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 39
Outline
Motion from images
Motion from image and inertial measurements
inertial sensors
algorithms and results
related work
Robust image feature tracking
Long-term motion estimation
Conclusion
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 40
Motion from image and inertial measurements: inertial sensors (1)
inertial sensors can be integrated to estimate six degree of freedom motion
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 41
Motion from image and inertial measurements: inertial sensors (2)
But many applications require small, light, and cheap sensors
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 42
Motion from image and inertial measurements: inertial sensors (3)
Integrating the outputs of these low grade sensors will produce drifting motion because of:
noise
unmodeled nonlinearities
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 43
Motion from image and inertial measurements: inertial sensors (4)
And, we can’t even integrate until we can separate the effects of…
rotation ρ
gravity g
acceleration a
slowly changing bias ba
noise n
…in the accelerometer measurements
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 44
Motion from image and inertial measurements: inertial sensors (5)
Image and inertial measurements are highly complementary
With inertial measurements we can:
decrease sensitivity in image-only estimates
establish two rotation angles without drift
establish the global scale
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 45
Motion from image and inertial measurements: inertial sensors (5)
Image and inertial measurements are highly complementary
With inertial measurements we can:
decrease sensitivity in image-only estimates
establish two rotation angles without drift
establish the global scale
…even with our low-grade sensors
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 46
Motion from image and inertial measurements: inertial sensors (6)
With image measurements, we can:
reduce the drift in integrating inertial data
distinguish between… rotation ρ
gravity g
acceleration a
bias ba
noise n
…in accelerometer measurements
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 47
Motion from image and inertial measurements: algorithms and results (1)
This work has developed both:
batch
recursive
algorithms for motion from image and inertial measurements
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 48
Motion from image and inertial measurements: algorithms and results (2)
Gyro measurements:
ω’, ω: measured and actual angular velocity
bω: gyro bias
n: gaussian noise
nb '
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 49
Motion from image and inertial measurements: algorithms and results (3)
Accelerometer measurements:
ρ: rotation
a’, a: measured and actual acceleration
g: gravity vector
ba: accelerometer bias
n: gaussian noise
nbgaRa' aT )((
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 50
Motion from image and inertial measurements: algorithms and results (4)
batch algorithm minimizes a combined error:
inertialimagecombined EEE
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 51
Motion from image and inertial measurements: algorithms and results (5)
image term Eimage is the same as before
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 52
Motion from image and inertial measurements: algorithms and results (6)
inertial error term Einertial is:
1
1n,translatio
1
1velocity,
1
1rotation,
ntranslatiovelocityrotationinertial
f
ii
f
ii
f
ii
E
E
E
EEEE
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 53
Motion from image and inertial measurements: algorithms and results (6)
inertial error term Einertial is:
1
1n,translatio
1
1velocity,
1
1rotation,
ntranslatiovelocityrotationinertial
f
ii
f
ii
f
ii
E
E
E
EEEE
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 54
Motion from image and inertial measurements: algorithms and results (6)
inertial error term Einertial is:
1
1n,translatio
1
1velocity,
1
1rotation,
ntranslatiovelocityrotationinertial
f
ii
f
ii
f
ii
E
E
E
EEEE
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 55
Motion from image and inertial measurements: algorithms and results (7)
,...))(,( 1n,translatio itii tItDE
timeτi-1 (time of image i - 1)
ti-1
ti
I(ti-1, …)
τi (time of image i)
tran
slat
ion
( : translation estimate for image i – 1)
( : translation estimate for image i)
( : translation integrated from previous estimate)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 56
Motion from image and inertial measurements: algorithms and results (8)
timeτ0
tran
slat
ion
τ1 τ2 τ5 τ3 τ4 τf-3 τf-2 τf-1
1
1n,translationtranslatio
f
iiEE
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 57
Motion from image and inertial measurements: algorithms and results (9)
1
1n,translatio
1
1velocity,
1
1rotation,
ntranslatiovelocityrotationinertial
f
ii
f
ii
f
ii
E
E
E
EEEE
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 58
Motion from image and inertial measurements: algorithms and results (10)
It(τi-1, τi ,…, ti-1) depends on:
τi-1, τi (known)
all inertial measurements for times τi-1< τ < τi (known)
ρi-1, ti-1
g
bω, ba
camera linear velocities: vi
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 59
Motion from image and inertial measurements: algorithms and results (12)
dash-dotted line: batch estimate from image and inertial solid line: image-only, bundle adjustment estimate squares: ground truth points
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 60
Motion from image and inertial measurements: algorithms and results (13)
IEKF for the same sensors, unknowns dash-dotted line: batch estimate solid line: IEKF estimate
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 61
Motion from image and inertial measurements: algorithms and results (14)
Difficulties with IEKF for this application:
prior assumptions about motion smoothness
cannot model relative error between adjacent camera positions
So, converting the batch algorithm into a variable state dimension filter (VSDF) is a promising future direction
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 62
Motion from image and inertial measurements: algorithms and results (15)
IEKF assumptions on motion smoothness dash-dotted line: batch estimate solid line: IEKF estimate
right: IEKF propagation variances too strict
left: IEKF propagation variances just right
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 63
Motion from image and inertial measurements
Recap:
image, gyro, and accelerometer measurements
batch algorithm
recursive algorithm
experiments
evaluate batch and recursive algorithms
establish basic facts about motion from image and inertial measurements
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 64
Outline
Motion from images
Motion from image and inertial measurements
Robust image feature tracking
Lucas-Kanade and real sequences
The “smalls” tracker
Long-term motion estimation
Conclusion
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 65
Robust image feature tracking: Lucas-Kanade and real sequences (1)
Combining image and inertial measurements improves our situation, but…
we still need accurate feature tracking tracking
some sequences do not come with inertial measurements
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 66
Robust image feature tracking: Lucas-Kanade and real sequences (2)
better feature tracking for improved 6 DOF motion estimation
remaining results will be image-only
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 67
Robust image feature tracking: Lucas-Kanade and real sequences (3)
Lucas-Kanade has been the go-to feature tracker for shape-from-motion
minimizes a correlation-like matching error
using general minimization
evaluates the matching error at only a few locations
subpixel resolution
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 68
Robust image feature tracking: Lucas-Kanade and real sequences (4)
Additional heuristics used to apply Lucas-Kanade to shape-from-motion:
task: heuristic:
choose features to track high image texture
identify mistracked, occluded, no-longer-visible
convergence, matching error
handle large motions image pyramid
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 69
Robust image feature tracking: Lucas-Kanade and real sequences (5)
But Lucas-Kanade performs poorly on many real sequences…
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 70
Robust image feature tracking: the “smalls” tracker (1)
smalls is a new feature tracker targeted at 6 DOF motion estimation
exploits the rigid scene assumption
eliminates the heuristics normally used with Lucas-Kanade
SIFT is an enabling technology here
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 71
Robust image feature tracking: the “smalls” tracker (2)
First step: epipolar geometry estimation
use SIFT to establish matches between the two images
get the 6 DOF camera motion between the two images
get the epipolar geometry relating the two images
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 72
Robust image feature tracking: the “smalls” tracker (3)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 73
Robust image feature tracking: the “smalls” tracker (4)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 74
Robust image feature tracking: the “smalls” tracker (5)
Second step: track along epipolar lines
use nearby SIFT matches to get initial position on epipolar line
exploits the rigid scene assumption
eliminates heuristic: pyramid
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 75
Robust image feature tracking: the “smalls” tracker (6)
Third step: prune features
geometrically inconsistent features are marked as mistracked and removed
clumped features are pruned
eliminates heuristic: detecting mistracked features based on convergence, error
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 76
Robust image feature tracking: the “smalls” tracker (7)
Fourth step: extract new features
spatial image coverage is the main criterion
required texture is minimal when tracking is restricted to the epipolar lines
eliminates heuristic: extracting only textured features
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 77
Robust image feature tracking: the “smalls” tracker (8)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 78
Robust image feature tracking: the “smalls” tracker (9)
left: odometry only right: images only
average error: 1.74 m
maximum error: 5.14 m
total distance: 230 m
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 79
Robust image feature tracking: the “smalls” tracker (10)
Recap:
exploits the rigid scene and eliminates heuristics
allows hands-free tracking for real sequences
can still be defeated by textureless areas or repetitive texture
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 80
Outline
Motion from images
Motion from image and inertial measurements
Robust image feature tracking
Long-term motion estimation
proof of concept system
experiment
Conclusion
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 81
Long-term motion estimation: proof of concept system (1)
Image-based motion estimates from any system will drift:
if the features we see are always changing
given sufficient time
if we don’t recognize when we’ve revisited a location
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 82
Long-term motion estimation: proof of concept system (2)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 83
Long-term motion estimation: proof of concept system (3)
To limit drift:
recognize when we’ve returned to a previous location
exploit the return
A proof of concept system demonstrates these capabilities
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 84
Long-term motion estimation: proof of concept system (4)
“smalls” tracker state: 2D feature history for images in I
variable state dimension filter (VSDF) state for images in I: 6 DOF camera positions, covariances for images in I 3D positions for features visible in I
SIFT keypoints for image in
system state S
image indices: I = {i1, …, in}
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 85
Long-term motion estimation: proof of concept system (5)
0 1 2 3 4 5 6 7 8
{0, 1}
{0} {0, 1, 2} {0, 1, …, 8}
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 86
rollback
Long-term motion estimation: proof of concept system (6)
0 1 2 3 4 5 6 7 8
{0, 1}
{0} {0, 1, 2} {0, 1, …, 8}
non-rollback States:
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 87
rollback
Long-term motion estimation: proof of concept system (7)
0 1 2 3 4 5 6 7 8
{0, 1}
{0} {0, 1, 2} {0, 1, …, 8}
8
non-rollback States:
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 88
rollback
Long-term motion estimation: proof of concept system (8)
0 1 2 3 4 5 6 7 8
{0, 1}
{0} {0, 1, 2}
8
{0, 1, 2, 3, 8}
non-rollback pruned States:
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 89
rollback
Long-term motion estimation: proof of concept system (9)
0 1 2 3 4 5 6 7 8
8 9 10 11
11 12 13 14
14 15 16 17
17 18 19 20
{0, …, 6, 11, 12, 17, …, 20}
non-rollback pruned States:
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 90
Long-term motion estimation: proof of concept system (10)
When to “roll back”?
examine the camera covariances for the current state and the candidate rollback state
check the number of SIFT matches
extend from the candidate state
examine the camera covariances for the current state and the resulting extended state
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 91
Long-term motion estimation: experiment (1)
CMU FRC highbay views; 945 images total
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 92
Long-term motion estimation: experiment (2)
CMU FRC highbay
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 93
Long-term motion estimation: experiment (2)
CMU FRC highbay
(first forward pass: images 0-213)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 94
Long-term motion estimation: experiment (2)
CMU FRC highbay
(first forward pass: images 0-213)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 95
Long-term motion estimation: experiment (2)
CMU FRC highbay
(first forward pass: images 0-213)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 96
Long-term motion estimation: experiment (2)
CMU FRC highbay
(first backward pass: images 214-380)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 97
rollback
Long-term motion estimation: experiment (3)
0 1 2 3 4 5 6 7 8
8 9 10 11
11 12 13 14
14 15 16 17
17 18 19 20
non-rollback pruned States:
normally, the system produces a general tree of states
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 98
Long-term motion estimation: experiment (4)
…0 1 2 3 4 5 6 7
13 14 15 14
16 17 18 17
non-rollback rollback pruned States:
for this example, the “rollback” states are restricted to the first forward pass
8 9
10 11 12 14
14
213
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 99
Long-term motion estimation: experiment (5)
movie…bottom half is smalls output:
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 100
Long-term motion estimation: experiment (6)
movie…top half is motion estimates:
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 101
Long-term motion estimation: experiment (7)
movie…top half is motion estimates:
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 102
Outline
Motion from images
Motion from image and inertial measurements
Robust image feature tracking
Long-term motion estimation
Conclusion
remaining issues
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 103
Conclusion: remaining issues
all: system is experimental, not optimized for speed
image and inertial: VSDF
“smalls”: integration of gyro, more robustness to poor texture needed
long-term: “roll back” space, computation grow with sequence length
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 104
Thanks!
Related materials:
these slides
related papers
movies
VRML models
at:
http://www.dennis-strelow.com/umn
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 105
Motion from omnidirectional images (1)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 106
Motion from omnidirectional images (2)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 107
Motion from omnidirectional images (3)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 108
Motion from omnidirectional images (4)
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 109
Motion from omnidirectional images (5)
left: non-rigid camera right: rigid camera
squares: ground truth points solid: image-only estimates
dash-dotted: image-and-inertial estimates
Dennis Strelow -- Motion estimation from image and inertial measurements -- March 22. 2006 110
Motion from omnidirectional images (6)
In this experiment:
omni images
conventional images + inertial
have roughly the same advantages
But in general:
inertial has some advantages that omni images alone can’t produce
omni images can be harder to use