Multiple View Object Cosegmentation using Appearance and Stereo Cues
1
Multiple View Object Cosegmentation using Appearance and Stereo Cues
ECCV 2012
2014/3/25 Yichong Bai
Multiple View Object Cosegmentation using Appearance and Stereo Cues
2
Problem
• Segmenting a rigid object in multiple views, using stereo cues• Input• unordered set of images
• Output• Segmentation• (Fine-grained depth map)• Object mask
2014/3/25 Yichong Bai
Multiple View Object Cosegmentation using Appearance and Stereo Cues
3
Overview
• Piecewise planar, layer-based depth map estimation based on stereo• Robust to unreliable stereo matching• Unsupervised• Workflow• Dense piecewise planar depth map, stereo cue, color• Segment object in each image• Fuse segmentation across multiple views
2014/3/25 Yichong Bai
Multiple View Object Cosegmentation using Appearance and Stereo Cues
4
Difference with ours
• Matching separated with segmentation• Stereo matching first• Not for matching
2014/3/25 Yichong Bai
Multiple View Object Cosegmentation using Appearance and Stereo Cues
5
Model
• Notations•
• L: labeling, pixelplanes
2014/3/25 Yichong Bai
Multiple View Object Cosegmentation using Appearance and Stereo Cues
6
Model
• Plane hypotheses• From semi-dense depth map, seed and grow• Depth map needed• Steps
• Fit surfel s_p at pixel p by least square• Fit plane by surfels• Seed and grow on graph where nodes represent pixels with valid surfels and
edges are 5-degree adjacencyLargest clusters are plane hypothesis
2014/3/25 Yichong Bai
Multiple View Object Cosegmentation using Appearance and Stereo Cues
7
Model
• Formulation• Multi-label MRF optimization problem (reference?)• Approximation by Energy Min.•
2014/3/25 Yichong Bai
Multiple View Object Cosegmentation using Appearance and Stereo Cues
8
Model
• Terms• Geometric unary term E^G
•
• Appearance unary term E^A• GMM in Lab color space
• Pairwise term E_{pq}• Contrast sensitive Potts model
2014/3/25 Yichong Bai
Multiple View Object Cosegmentation using Appearance and Stereo Cues
9
Solve
• Initialization• No appearance term
2014/3/25 Yichong Bai
Multiple View Object Cosegmentation using Appearance and Stereo Cues
10
Solve
• Iteration• GMM for appearance
• EM algorithm generating GMM model for surfel
2014/3/25 Yichong Bai
Multiple View Object Cosegmentation using Appearance and Stereo Cues
11
Model
• Surface segmentation• L segmentation
2014/3/25 Yichong Bai
Multiple View Object Cosegmentation using Appearance and Stereo Cues
12
Solve
• Fusion move• Initialize by assigning different regions to each pixel• Fuse coplanar polygons in a greedy way
2014/3/25 Yichong Bai
Multiple View Object Cosegmentation using Appearance and Stereo Cues
13
Model
• Object Segmentation• F: region labeling• Objectiveness
• Object’s pixels are visible in multi-views
2014/3/25 Yichong Bai
Multiple View Object Cosegmentation using Appearance and Stereo Cues
14
Evaluation
• Experiments• Use Grabcut (interactive) as groundtruth• Intersection over union metric
2014/3/25 Yichong Bai
Multiple View Object Cosegmentation using Appearance and Stereo Cues
15
Evaluation
• Results
2014/3/25 Yichong Bai