Corp. ResearchPrinceton, NJ

Computing geodesics and minimal surfaces via graph cuts

Yuri Boykov, Siemens Research, Princeton, NJ

joint work with

Vladimir Kolmogorov, Cornell University, Ithaca, NY

Corp. ResearchPrinceton, NJ

Two standard object extraction methods

Interactive Graph cuts [Boykov&Jolly ‘01]

• Discrete formulation

• Computes min-cuts on N-D grid-graphs

Geodesic active contours [Caselles et.al. ‘97, Yezzi et.al ‘97]

• Continuous formulation

• Computes geodesics in image- based N-D Riemannian spaces

Geo-cuts

• Minimal geometric artifacts

• Solved via local variational technique (level sets)

• Possible metrication errors

• Global minima

Corp. ResearchPrinceton, NJGeodesics and minimal surfaces

The shortest curve between two points is a geodesic

Riemannian metric(space varying, tensor D(p))

Geodesic contours use image-based Riemannian metric

Euclidian metric (constant)

A

B

A

B

Generalizes to 3D (minimal surfaces)

distance map distance

map

Corp. ResearchPrinceton, NJ

Graph cuts (simple example à la Boykov&Jolly, ICCV’01)

n-links

s

t a cuthard constraint

hard constraint

Minimum cost cut can be computed in polynomial time

(max-flow/min-cut algorithms)

Corp. ResearchPrinceton, NJMetrication errors on graphs

discrete metric ???

Minimum cost cut (standard 4-neighborhoods)

Continuous metric space

(no geometric artifacts!)

Minimum length geodesic contour (image-based Riemannian metric)

Corp. ResearchPrinceton, NJ

Cut Metrics :cuts impose metric properties on graphs

C

Cut metric is determined by the graph topology and by edge weights. Can a cut metric approximate a given Riemannian metric?

Ce

eC ||||||

Cost of a cut can be interpreted as a geometric “length” (in 2D) or “area” (in 3D) of the corresponding contour/surface.

Corp. ResearchPrinceton, NJOur key technical result

The main technical problem is solved via Cauchy-Crofton formula from integral geometry.

We show how to build a grid-graph such that its

cut metric approximates any given Riemannian metric

Corp. ResearchPrinceton, NJ

Integral Geometry andCauchy-Crofton formula

C

ddnC L21||||Euclidean length of C :

the number of times line L intersects C

2

0

a set of all lines L

CL

a subset of lines L intersecting contour C

Corp. ResearchPrinceton, NJ

Cut Metric on gridscan approximate Euclidean Metric

C

k

kkknC 21||||

Euclidean length

2kk

kw

gcC ||||graph cut cost

for edge weights:the number of edges of family k intersecting C

Edges of any regular neighborhood system

generate families of lines

{ , , , }

Graph nodes are imbeddedin R2 in a grid-like fashion

Corp. ResearchPrinceton, NJCut metric in Euclidean case

2kk

kw

“standard” 4-neighborhoods

(Manhattan metric)

256-neighborhoods8-neighborhoods

“Distance maps” (graph nodes “equidistant” from a given node) :

(Positive!) weights depend only on edge direction k.

Corp. ResearchPrinceton, NJReducing Metrication Artifacts

originalnoisy image

Image restoration [BVZ 1999]

restoration with “standard” 4-neighborhoods

restoration with 8-neighborhoods

using edge weights

2kk

kw

Corp. ResearchPrinceton, NJCut Metric in Riemannian case

The same technique can used to compute edge weights that approximate arbitrary Riemannian metric defined by tensor D(p)

• Idea: generalize Cauchy-Crofton formula

4-neighborhoods 8-neighborhoods 256-neighborhoods

Local “distance maps” assuming anisotropic D(p) = const

))(,()( pDkfpwk (Positive!) weights depend on

edge direction k and on location/pixel p.

Corp. ResearchPrinceton, NJ

||||||||0,0

CC gc

Convergence theorem

Theorem: For edge weights set by tensor D(p)

0|| e

C

)( pwk

Corp. ResearchPrinceton, NJ“Geo-Cuts” algorithm

image-derivedRiemannian metric

D(p) )( pwk

regular grid edge weights

Boundary conditions(hard/soft

constraints)

Global optimization

Graph-cuts[Boykov&Jolly, ICCV’01]

||ˆ||||ˆ|| CC gc

min-cut = geodesic

Corp. ResearchPrinceton, NJ

Minimal surfaces in image inducedRiemannian metric spaces (3D)

3D bone segmentation (real time screen capture)

Corp. ResearchPrinceton, NJ

Our results reveal a relation between…

Level Sets Graph Cuts [Osher&Sethian’88,…] [Greig et. al.’89, Ishikawa et. al.’98, BVZ’98,…]

Gradient descent method VS. Global minimization tool

variational optimization method for combinatorial optimization for fairly general continuous energies a restricted class of energies [e.g. KZ’02]

finds a local minimum finds a global minimum near given initial solution for a given set of boundary conditions

anisotropic metrics are harder anisotropic Riemannian metrics to deal with (e.g. slower) are as easy as isotropic ones

numerical stability has to be carefully

addressed [Osher&Sethian’88]:continuous formulation -> “finite

differences”

numerical stability is not an issue

discrete formulation ->min-cut algorithms

(restricted class of energies)

Corp. ResearchPrinceton, NJConclusions

“Geo-cuts” combines geodesic contours and graph cuts. • The method can be used as a “global” alternative to variational level-sets.

Reduction of metrication errors in existing graph cut methods• stereo [Roy&Cox’98, Ishikawa&Geiger’98, Boykov&Veksler&Zabih’98, ….]• image restoration/segmentation [Greig’86, Wu&Leahy’97,Shi&Malik’98,…]• texture synthesis [Kwatra/et.al’03]

Theoretical connection between discrete geometry of graph cuts and concepts of integral & differential geometry

Corp. ResearchPrinceton, NJ

Geo-cuts (more examples)

3D segmentation (time-lapsed)