geodesic star convexity for interactive image segmentation

Geodesic Star Convexity for interactive image segmentation Varun Gulshan † , Carsten Rother ‡ , Antonio Criminisi ‡ , Andrew Blake ‡ and Andrew Zisserman † 1. Star- convexity † Visual Geometry Group, University of Oxford, UK ‡ Microsoft Research, Cambridge, UK References p c q Let y denote a binary segmentation and S * ({c}) the set of star convex shapes wrt to center c. Star-convex constraint expressed as an energy: Factorization into pairwise terms: 2. Star-convexity -- Extensions Semantics I – visibility to atleast one center (equivalently: union of star shape sets) Semantics II – visibility to nearest star center 3. Visibility Experiment Method 1 star 2 stars ESC 4.50±0.5 8 2.85±0.3 9 GSC 4.16±0.5 4 2.22±0.3 0 4. Star-convexity in an interactive system Boykov Jolly Energy: 5. Evaluation Simulat ed user [2] Dataset: 151 Images taken from GrabCut, PASCAL VOC and the alpha matting dataset. •Energy is submodular as (1,0) labeling has infinite energy. •Only needs to be imposed for neighbouring pixels – hence efficient. c •The union constraint is not submodular. •Semantics don’t extend easily to brush strokes as star centers • Tractable to implement (submodular) •Semantics extend nicely to brush strokes as star centers User interaction Color likelihood Output - BJ Output - GSC Evaluation criteria: Measure avg. number of strokes to reach desired accuracy. Metho d Avg. Effort BJ 12.35 PP 10.66 ESC 10.57 GSC 10.23 GSCse q 9.63 Metho d Avg. Effort SP 15.14 BJ 12.35 RW 12.31 GSCse q 9.63 [1] O. Veksler. Star shape prior for graph-cut based image segmentation. ECCV, 2008 [2] H. Nickisch, P. Kohli and C. Rother. Learning an interactive segmentation system. arXiv Technical Report, Dec. 2009. [3] L. Grady. Random walks for image segmentation. IEEE PAMI, 2006 [4] X. Bai and G. Sapiro. Geodesic matting: A framework for fast interactive image and video segmentation and matting. IJCV 2009 [5] S. Vicente, V. Kolmogorov and C. Rother. Graph cut based image segmentation with connectivity priors. CVPR 2008. Euclide an Geodesi c 2.1 Multiple Stars 2.2 Geodesic Stars GSC New Brush Stroke .... Comparison: Various shape constraints Comparison: Different algorithms Dij-GC [5] BJ 4.1 Sequential system Occlusion rates Veksler, ECCV 08 [1] Visualization of geodesics computed using image gradients Original image FG stroke BG stroke User interaction Original image FG stroke BG stroke SP = Shortest Paths [4] RW = Random Walker [3] PP = Post- Processing for connected components Code and dataset available at: http://www.robots.ox.ac.uk/~vgg/research/iseg/

Upload: magda

Post on 23-Feb-2016

71 views

Category:

Documents

1 download

Report

Download

Tags:

Embed Size (px):

DESCRIPTION

Geodesic Star Convexity for interactive image segmentation. Varun Gulshan † , Carsten Rother ‡ , Antonio Criminisi ‡ , Andrew Blake ‡ and Andrew Zisserman †. † Visual Geometry Group, University of Oxford, UK ‡ Microsoft Research, Cambridge, UK. c. 1. Star-convexity. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Geodesic Star Convexity for interactive image segmentation

Geodesic Star Convexity for interactive image segmentationVarun Gulshan†, Carsten Rother‡, Antonio Criminisi‡, Andrew Blake‡ and Andrew Zisserman†

1. Star-convexity

†Visual Geometry Group, University of Oxford, UK ‡Microsoft Research, Cambridge, UK

References

Let y denote a binary segmentation and S*({c}) the set of star convex shapes wrt to center c. Star-convex constraint expressed as an energy:

Factorization into pairwise terms:

2. Star-convexity -- Extensions

Semantics I – visibility to atleast one center(equivalently: union of star shape sets)

Semantics II – visibility to nearest star center

3. Visibility Experiment

Method 1 star 2 stars

ESC 4.50±0.58 2.85±0.39

GSC 4.16±0.54 2.22±0.30

4. Star-convexity in an interactive system

Boykov Jolly Energy:

5. Evaluation

Simulated user [2]

Dataset: 151 Images taken from GrabCut, PASCAL VOC and the alpha matting dataset.

•Energy is submodular as (1,0) labeling has infinite energy.•Only needs to be imposed for neighbouring

pixels – hence efficient.

•The union constraint is not submodular.• Semantics don’t extend easily to

brush strokes as star centers

•Tractable to implement (submodular)• Semantics extend nicely to brush

strokes as star centers

User interaction Color likelihood

Output - BJ Output - GSC

Evaluation criteria:Measure avg. number of strokes to reach desired accuracy.

Method Avg. Effort

BJ 12.35

PP 10.66

ESC 10.57

GSC 10.23

GSCseq 9.63

Method Avg. Effort

SP 15.14

BJ 12.35

RW 12.31

GSCseq 9.63

[1] O. Veksler. Star shape prior for graph-cut based image segmentation. ECCV, 2008[2] H. Nickisch, P. Kohli and C. Rother. Learning an interactive segmentation system. arXiv Technical Report, Dec. 2009.[3] L. Grady. Random walks for image segmentation. IEEE PAMI, 2006

[4] X. Bai and G. Sapiro. Geodesic matting: A framework for fast interactive image and video

segmentation and matting. IJCV 2009[5] S. Vicente, V. Kolmogorov and C. Rother. Graph cut based image segmentation with connectivity priors. CVPR 2008.

Euclidean Geodesic

2.1 Multiple Stars 2.2 Geodesic Stars

GSC

New Brush Stroke

....

Comparison: Various shape constraints Comparison: Different algorithms

Dij-GC [5]BJ

4.1 Sequential system

Occlusion rates

Veksler, ECCV 08 [1]

Visualization of geodesics computed using image gradients

Original imageFG stroke