exploiting segmentation for robust 3d object matching michael krainin, kurt konolige, and dieter fox
TRANSCRIPT
![Page 1: Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox](https://reader035.vdocuments.site/reader035/viewer/2022062717/56649e1c5503460f94b0a47b/html5/thumbnails/1.jpg)
Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox
![Page 2: Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox](https://reader035.vdocuments.site/reader035/viewer/2022062717/56649e1c5503460f94b0a47b/html5/thumbnails/2.jpg)
2
Shape-based Pose Quality Metrics
Given an object model (triangle mesh) and a depth map,
Should not assume any object texture Should be robust to clutter and occlusion
![Page 3: Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox](https://reader035.vdocuments.site/reader035/viewer/2022062717/56649e1c5503460f94b0a47b/html5/thumbnails/3.jpg)
3
Outline
Progression of quality metrics Iterative Closest Point Beam-based Sensor Models Segmentation Sensor Model
Key Contributions Gradient-based optimization for beam-based models
Improves on ICP by reasoning about free-space Novel sensor model framework
Uses segmentation to relax beam-independence assumptions
Explicitly reasons about surface extents
![Page 4: Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox](https://reader035.vdocuments.site/reader035/viewer/2022062717/56649e1c5503460f94b0a47b/html5/thumbnails/4.jpg)
4
ICP Error Metric
Mean squared distance for pairs of closest points [Besl & McKay‘92]
No notion of amount of model or scene explained by the correspondences
Discards viewpoint/free-space information
![Page 5: Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox](https://reader035.vdocuments.site/reader035/viewer/2022062717/56649e1c5503460f94b0a47b/html5/thumbnails/5.jpg)
5
x xx
Beam-Based Sensor Models
Maximize data likelihood:
Typically assume pixel independence:
Measured Depth – Rendered Depth0l
p(p
ixel |
mod
el, p
ose
)
x
x
x
x
![Page 6: Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox](https://reader035.vdocuments.site/reader035/viewer/2022062717/56649e1c5503460f94b0a47b/html5/thumbnails/6.jpg)
6
Beam Model Sensor Models
Beam-based sensor model properties: Consider viewpoint Prefer to explain more pixels using the model
Use in pose estimation: 2D particle-based localization [Thrun et al. ‘05]
Coarse to fine grid search for body tracking [Ganapathi et al. ‘10]
Annealed particle filters for vehicle detection and tracking [Petrovskaya & Thrun ‘09]
Has not been used with gradient-based optimization
![Page 7: Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox](https://reader035.vdocuments.site/reader035/viewer/2022062717/56649e1c5503460f94b0a47b/html5/thumbnails/7.jpg)
7
Beam Model Optimization
We propose a gradient-based optimization
Levenberg-Marquardt using the g2o graph optimization framework [Kümmerle et al. ‘11]
Error function evaluations by re-rendering OpenGL rendering, CUDA sensor model
evaluation 1 ms per evaluation on a mid-range graphics
card ~500 evaluations per g2o optimization
ICP
Beam Optimization
observed rendered
![Page 8: Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox](https://reader035.vdocuments.site/reader035/viewer/2022062717/56649e1c5503460f94b0a47b/html5/thumbnails/8.jpg)
8
Surface Extents
Can still match to the wrong surface
Idea: Use size of surfaces to rule out matches like these
![Page 9: Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox](https://reader035.vdocuments.site/reader035/viewer/2022062717/56649e1c5503460f94b0a47b/html5/thumbnails/9.jpg)
9
Segmentation
(Over-)Segmentation as an estimate of surface extents
Connected components using depths and normals
![Page 10: Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox](https://reader035.vdocuments.site/reader035/viewer/2022062717/56649e1c5503460f94b0a47b/html5/thumbnails/10.jpg)
10
Segmentation Sensor Model
Usual beam-based sensor model:
Segmentation model:
Allows consistent classification of entire segment as being generated from the model or not
![Page 11: Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox](https://reader035.vdocuments.site/reader035/viewer/2022062717/56649e1c5503460f94b0a47b/html5/thumbnails/11.jpg)
11
Segmentation Sensor Model
Let S be a partition (segmentation) of D. Then given model M and pose T,
Let mi be an indicator for whether Si was generated from M
Segment pixels conditionally independent given classification
![Page 12: Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox](https://reader035.vdocuments.site/reader035/viewer/2022062717/56649e1c5503460f94b0a47b/html5/thumbnails/12.jpg)
12
Error Functions
Standard Beam-Based Model Segmentation Model
x
![Page 13: Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox](https://reader035.vdocuments.site/reader035/viewer/2022062717/56649e1c5503460f94b0a47b/html5/thumbnails/13.jpg)
13
Test Data
Recorded 46 cluttered scenes with ground truth poses
![Page 14: Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox](https://reader035.vdocuments.site/reader035/viewer/2022062717/56649e1c5503460f94b0a47b/html5/thumbnails/14.jpg)
14
Pose Estimation Results
Optimization algorithms:
Random restart evaluation functions (upper bounded by 93% from previous result):
ICP Beam Optimization
Successful convergence [%] (exists a correct end pose among 20 random restarts)
86 ± 3 93 ± 2
ICP Mean Squared Error
Standard Beam Model
Segmentation Beam Model
Successful matches [%](determines correct end pose among 20 restarts)
79 ± 3 67 ± 4 85 ± 3
![Page 15: Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox](https://reader035.vdocuments.site/reader035/viewer/2022062717/56649e1c5503460f94b0a47b/html5/thumbnails/15.jpg)
15
Robustness to Segmentation Granularity
Spam 1 of 5 0 degrees
8 degrees 12 degrees
x
x
x
(Fixed to 8 degrees in other experiments)
![Page 16: Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox](https://reader035.vdocuments.site/reader035/viewer/2022062717/56649e1c5503460f94b0a47b/html5/thumbnails/16.jpg)
16
Conclusions
Contributions Novel formulation for beam-based sensor models Demonstration of gradient-based approach for optimizing
beam-based sensor models All code and test data freely available
Future Work Detecting failed segmentations Extensions such as color and normal direction Directly optimizing the segmentation model