planes, snakes, and active contours

Active Contours / Planes

Sebastian Thrun, Gary Bradski, Daniel RussakoffStanford CS223B Computer Vision

http://robots.stanford.edu/cs223b

Some slides taken from: Julien Jomier

Sebastian Thrun Stanford University CS223B Computer Vision

Outline

Snakes Planes


Introduction (1)

The active contour model, or snake, is defined as an energy-minimizing spline.

Active contours results from work of Kass et.al. in 1987.

Active contour models may be used in image segmentation and understanding.

The snake’s energy depends on its shape and location within the image.

Snakes can be closed or open


Example


Introduction (2)

Aorta segmentation using active contours


Introduction (3)

First an initial spline (snake) is placed on the image, and then its energy is minimized.

Local minima of this energy correspond to desired image properties.

Unlike most other image models, the snake is active, always minimizing its energy functional, therefore exhibiting dynamic behavior.

Also suitable for analysis of dynamic data or 3D image data.


Kass Algorithm

The snake is defined parametrically as v(s)=[x(s),y(s)], where s[0,1] is the normalized arc length along the contour. The energy functional to be minimized may be written as

Econt = snake continuity

Ecurv = snake curvature

Eimage = image forces (e.g., edge attraction)

10

10

10

10

*

))((

))(())((

))((

dssvE

dssvEdssvE

dssvEE

image

curvcont

snakesnake


Internal Energy

Continuity:

Curvature:

2

))((dsdvsvEcont

2

2

2

))((dsvdsvEcurv


Image Forces

Dark/Bright Lines

Edges

))(())(( svIsvEimage

))(())(( svIsvEimage


Trade-offs

determine trade-off

10

10

10

10

*

))((

))(())((

))((

dssvE

dssvEdssvE

dssvEE

image

curvcont

snakesnake


Numerical Algorithm

Select N initial locations p1,…, pN Update until convergence

|)(|

|2|||

1.0 211

21

k

kkk

kk

kiii

pIppp

pp

ppp

|)(||2|||2

112

1*

kk

kkkkksnake pIpppppE


Snakes, Done Right

Define Spline over p1,…, pN Optimize criterion for all points on spline Allow for corners

(optimization becomes tricky, fills entire literature)


Applications

Main Applications are: Segmentation Tracking Registration

http://www.markschulze.net/snakes/





Examples (1)

Julien Jomier


Examples (2)

Julien Jomier


Examples (3)

Heart Julien Jomier


Examples (4)3D Segmentation of the Hippocampus

Julien Jomier


Examples (5)

Julien Jomier


Example (6)

Julien Jomier


Example (7)

Julien Jomier


Problems with snakes

Snakes sometimes degenerate in shape by shrinking and flattening.

Stability and convergence of the contour deformation process unpredictable.Solution: Add some constraints

Initialization is not straightforward.Solution: Manual, Learned, Exhaustive


References

M. Kass, A. Witkin, and D. Terzopoulos. Snakes: Active contour models. In Proc. 1st ICCV, pages 259-268, June 1987. London, UK.

Yongjik Kim. A summary of Implicit Snake Formulation.

Jorgen Ahlberg. Active Contours in Three Dimensions.

M. Bertalmio, G. Sapiro and G. Randall. Morphing Active Contours. IEEE PAMI, Vol 22, No 7, July 2000


Outline

Snakes Planes


Finding Planes

Lines in Image (e.g., Hough transform) Planes in 3D space (e.g., stereo reconstruction,

SFM) Problems:

– Number of planes– Parameters of planes– Data association: which point belongs to which plane– Outlier removal (noise, non-flat surfaces)


Range Data in Multi-Planar Environment

Taken with Laser Range Finder, but similar for


Basic Idea

******


Mathematical Model: Expectation Maximization

3D Model: },,,{ 21 J

3, jjj

Planar surface in 3D

ijij zz ),dist(Distance point-surface

surface

surface normal

y

x

zdisplacement


Mixture Measurement Model

Case 1: Measurement zi caused by plane j

2

2)(21

22

1)|(

jij z

ji ezp

2

2max

2ln

21

2max

*2

11)|(

z

i ez

zp

Case 2: Measurement zi caused by something else


Measurement Model with Correspondences

J

j

jijj

zc

zc

Ji eccczp 12

2

2

2max

*)(

2ln

21

2*12

1),,,,|(

correspondence variables C:

}

J

jj

j

cc

cc

1*

*

1

}1,0{,

1

)(

2ln

21

2

12

2

2

2max

*

21),|(

i

zc

zc

J

j

jijiji

eCZp


Expected Log-Likelihood Function

1

)(

2ln

21

2

12

2

2

2max

*

21),|(

i

zc

zc

J

j

jijiji

eCZp

i

J

j

jijij

ic

zcE

zcE

J

CZpE

12

2

2

2max

*

2

)(][

21

2ln][

21

2)1(1ln

)|,(ln

…after some simple math

mapping with known data association

probabilistic data association


E-Step

Calculate expectations with fixed model :

(normalize so that )

2

2

2

2max

)(21

2ln

21

*

][

][

jij z

ij

z

i

ecE

ecE

J

jiji cEcE

1* 1][][


M-Step

Maximize

i

jijij

ic

zcE

zcE

J

CZpE

2

2

2

2max

*

2

)(][

21

2ln][

21

2)1(

1ln

)|,(ln

1 jj subject to

Is equivalent to minimizing

i

J

jjijij zcE

1

2)]([ 1 jj subject to


M-Step: Solve via Lagrange Multipliers

i

J

jjijij zcE

1

2 min)]([ 1: jjj

i

J

j

J

jjjjjijij zcEL

1 1

2)]([

Define

0)]([!

jijijj

zcEL

0)]([!

jjijijijj

zzcEL

1 jj

And observe that


M-Step: Solve via Lagrange Multipliers Solve for :

Substitute back:

Is of the linear form:

Solution: two Eigenvectors of A with smallest Eigenvalues.

kjkj

kkjkj

j cE

zcE

][

][0)]([

!

jijijj

zcEL

0)][

][]([

!

jji

kjkj

kkjkj

ijijj

zcE

zcEzcEL

0)]([!

jjijijijj

zzcEL

jjjA


Determining Number of Surfaces

J =1

First model component

*

*

J =1

E-Step

*

*

J =1

M-Step

*

*

J =3

Add model components

J =3

E-Step

J =3

M-step

J =1

Prune model

J =3

Add model components

J =3

E/M Steps

*J =2

Prune model


Choosing the “Right” Number of Planes: AIC

J=2 J=3 J=5J=0 J=1 J=4

increased data likelihood

increased prior probability

)(log)|(log)|(log JpJdpconstdJp


Model Selection

Approximately every 20 iterations of EM: Start new surfaces

– Near any set of collinear measurements Terminate unsupported surfaces

– If not supported by enough measurements– If density of measurements too low– If two planes are too close to each other


Results

C:\Documents and Settings\Administrator\My Documents\vrml\wean-map\result2.wrl


Online Robotic Mapping @ CMU


Online Robotic Mapping @ Stanford