motion from image and inertial measurements (additional slides)
DESCRIPTION
Motion from image and inertial measurements (additional slides). Dennis Strelow Carnegie Mellon University. Outline. Robust image feature tracking (in detail) Lucas-Kanade and real sequences The “smalls” tracker Motion from omnidirectional images. - PowerPoint PPT PresentationTRANSCRIPT
Motion from image and inertial measurements
(additional slides)
Dennis Strelow
Carnegie Mellon University
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 2
Outline
Robust image feature tracking (in detail)
Lucas-Kanade and real sequences
The “smalls” tracker
Motion from omnidirectional images
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 3
Robust image feature tracking: Lucas-Kanade and real sequences (1)
Combining image and inertial measurements improves our situation, but…
we still need accurate feature tracking tracking
some sequences do not come with inertial measurements
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 4
Robust image feature tracking: Lucas-Kanade and real sequences (2)
better feature tracking for improved 6 DOF motion estimation
remaining results will be image-only
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 5
Robust image feature tracking: Lucas-Kanade and real sequences (3)
Lucas-Kanade has been the go-to feature tracker for shape-from-motion
minimizes a correlation-like matching error
using general minimization
evaluates the matching error at only a few locations
subpixel resolution
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 6
Robust image feature tracking: Lucas-Kanade and real sequences (4)
Additional heuristics used to apply Lucas-Kanade to shape-from-motion:
task: heuristic:
choose features to track high image texture
identify mistracked, occluded, no-longer-visible
convergence, matching error
handle large motions image pyramid
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 7
Robust image feature tracking: Lucas-Kanade and real sequences (5)
But Lucas-Kanade performs poorly on many real sequences…
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 8
Robust image feature tracking: the “smalls” tracker (1)
smalls is a new feature tracker targeted at 6 DOF motion estimation
exploits the rigid scene assumption
eliminates the heuristics normally used with Lucas-Kanade
SIFT is an enabling technology here
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 9
Robust image feature tracking: the “smalls” tracker (2)
First step: epipolar geometry estimation
use SIFT to establish matches between the two images
get the 6 DOF camera motion between the two images
get the epipolar geometry relating the two images
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 10
Robust image feature tracking: the “smalls” tracker (3)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 11
Robust image feature tracking: the “smalls” tracker (4)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 12
Robust image feature tracking: the “smalls” tracker (5)
Second step: track along epipolar lines
use nearby SIFT matches to get initial position on epipolar line
exploits the rigid scene assumption
eliminates heuristic: pyramid
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 13
Robust image feature tracking: the “smalls” tracker (6)
Third step: prune features
geometrically inconsistent features are marked as mistracked and removed
clumped features are pruned
eliminates heuristic: detecting mistracked features based on convergence, error
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 14
Robust image feature tracking: the “smalls” tracker (7)
Fourth step: extract new features
spatial image coverage is the main criterion
required texture is minimal when tracking is restricted to the epipolar lines
eliminates heuristic: extracting only textured features
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 15
Robust image feature tracking: the “smalls” tracker (8)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 16
Robust image feature tracking: the “smalls” tracker (9)
left: odometry only right: images only
average error: 1.74 m
maximum error: 5.14 m
total distance: 230 m
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 17
Robust image feature tracking: the “smalls” tracker (10)
Recap:
exploits the rigid scene and eliminates heuristics
allows hands-free tracking for real sequences
can still be defeated by textureless areas or repetitive texture
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 18
Outline
Robust image feature tracking (in detail)
Motion from omnidirectional images
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 19
Motion from omnidirectional images (1)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 20
Motion from omnidirectional images (2)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 21
Motion from omnidirectional images (3)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 22
Motion from omnidirectional images (4)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 23
Motion from omnidirectional images (5)
left: non-rigid camera right: rigid camera
squares: ground truth points solid: image-only estimates
dash-dotted: image-and-inertial estimates
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 24
Motion from omnidirectional images (6)
In this experiment:
omni images
conventional images + inertial
have roughly the same advantages
But in general:
inertial has some advantages that omni images alone can’t produce
omni images can be harder to use