visual odometry for ground vehicle applications david nister, oleg naroditsky, james bergen sarnoff...
Post on 21-Dec-2015
216 views
TRANSCRIPT
![Page 1: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/1.jpg)
Visual Odometry for Ground Vehicle Applications
David Nister, Oleg Naroditsky, James Bergen
Sarnoff Corporation, CN5300 Princeton, NJ 08530
CPSC 643, Presentation 4
Journal of Field Robotics.Vol. 23, No. 1, pp. 3-200, 2006.
![Page 2: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/2.jpg)
Mostly Related Works
• Stereo Visual Odometry – Agrawal 2007• Monocular Visual Odometry – Campbell 2005• A Visual Odometry System – Olson 2003• Previous Work of this Paper – Nister 2004
![Page 3: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/3.jpg)
Mostly Related Works
• Integrate with IMU/GPS• Bundle Adjustment
• Stereo Visual Odometry – Agrawal 2007• Monocular Visual Odometry – Campbell 2005• A Visual Odometry System – Olson 2003• Previous Work of this Paper – Nister 2004
![Page 4: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/4.jpg)
Mostly Related Works
• 3-DOF Camera• Optical Flow Method
• Stereo Visual Odometry – Agrawal 2007• Monocular Visual Odometry – Campbell 2005• A Visual Odometry System – Olson 2003• Previous Work of this Paper – Nister 2004
![Page 5: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/5.jpg)
Mostly Related Works
• Stereo Visual Odometry – Agrawal 2007• Monocular Visual Odometry – Campbell 2005• A Visual Odometry System – Olson 2003• Previous Work of this Paper – Nister 2004
• With absolute orientation sensor
• Forstner interest operator in the left Image, matches from left to right
• Use approximate prior knowledge
• Iteratively select landmark points
![Page 6: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/6.jpg)
Mostly Related Works
• Stereo Visual Odometry – Agrawal 2007• Monocular Visual Odometry – Campbell 2005• A Visual Odometry System – Olson 2003• Previous Work of this Paper – Nister 2004
• Estimates ego-motion using a hand-held Camera• Real-time algorithm based on RANSIC
![Page 7: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/7.jpg)
Other Related Works
• Simultaneous Localization and Mapping (SLAM)
• Robot Navigation with Omnidirectional Camera
![Page 8: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/8.jpg)
Motivation
Olson:
With absolute orientation sensor
Forstner interest operator in the left Image, matches from left toright
Use approximate prior knowledge
Iteratively select landmark points
Nister:
Use pure visual information
Use Harris corner detection in allimages, track feature to feature
No prior knowledge
RANSIC based estimation in real-time
![Page 9: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/9.jpg)
Feature Detection
Harris Corner Detection
Search for the local maxima of the corner strength .
determinant, trance, constant, window area,
derivatives of input image, weight function.
( , )s x yd t k
,x yI I w,a b
max
![Page 10: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/10.jpg)
Feature Detection
Four Sweeps to Calculate
• Compute , by filters and .• Calculate the horizontal sum by filter .• Calculate the vertical sum by filter .• Calculate corner strength .
( , )s x y
, ,x x x y y yI I I I I I [ 1,0,1] [ 1,0,1]T
[1,4,6,4,1][1,4,6,4,1]T
( , )s x y
101 11
46
1
41
4 61 4 1
( , )S x y
![Page 11: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/11.jpg)
Feature Detection
Detected Feature Points
Superimposed feature tracks through images
![Page 12: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/12.jpg)
Feature Matching
Two Directional Matching
• Calculate the normalized correlation in reign, where , are two consecutive input images.
• Match the feature points in the circular area that have the maximum correlation in two directions.
21 2 1 2
2 2 2 21 1 2 2
max ( ) ( )
n I I I I
n I I n I I
1I 2In n 11n
n n n n
![Page 13: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/13.jpg)
Robust Estimation
The Monocular Scheme
• Separate the matching points into 5-points groups.
• Treat each group as a 5-poi-nt relative pose problem.
• Use RANSAC to select well matched groups.
• Estimate camera motion us-ing the selected groups.
• Put the current estimation in to the coordinate of the prev-ious one.
![Page 14: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/14.jpg)
Robust Estimation
The Stereo Scheme
• Match the feature points in stereo images, then triangulatethem into 3D points.
• Estimation the camera motion using RANSAC and the 3D points in consecutive frames.
3D point
![Page 15: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/15.jpg)
Experiments
Different Platforms
![Page 16: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/16.jpg)
Experiments
Speed and Accuracy
![Page 17: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/17.jpg)
Experiments
Visual Odometry vs. Differential GPS
![Page 18: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/18.jpg)
Experiments
Visual Odometry vs. Inertial Navigation System (INS)
![Page 19: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/19.jpg)
Experiments
Visual Odometry vs. Wheel Recorder
![Page 20: Visual Odometry for Ground Vehicle Applications David Nister, Oleg Naroditsky, James Bergen Sarnoff Corporation, CN5300 Princeton, NJ 08530 CPSC 643, Presentation](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d5a5503460f94a3af09/html5/thumbnails/20.jpg)
Conclusion and Future Work
Conclusion
• A real-time ego motion estimation system.
• Work both on monocular camera and stereo head.
• Results are accurate and robust.
Future Work
• Integrate visual odometry with Kalman filter.
• Use sampling methods with multimodal distributions.