cse 554 lecture 8: laplacian deformationtaoju/cse554/lectures/lect08_deformation.pdfcse554 laplacian...

CSE554 Laplacian Deformation Slide 1

CSE 554

Lecture 8: Laplacian Deformation

CSE 554

Lecture 8: Laplacian Deformation

Fall 2012


ReviewReview

• Alignment– Registering source to target by

rotation and translation

• Rigid-body transformations

• Methods– Aligning principle directions (PCA)

– Aligning corresponding points (SVD)

– Iterative improvement (ICP)

• Combines PCA and SVD

Input

After PCA

After ICP

Source

Target


Non-rigid RegistrationNon-rigid Registration

• Rigid alignment cannot account for shape variance

• Non-rigid deformation can give a better fit

After non-rigid deformationRigid alignment

Source

Target


Non-rigid RegistrationNon-rigid Registration

• A minimization problem– Minimizing the distance between the

deformed source and the target

• “Fitting term”

– Minimizing the distortion to the source shape

• “Distortion term”


Intrinsic vs. ExtrinsicIntrinsic vs. Extrinsic

• Intrinsic methods– Deforms points on the source curve/surface

– App: boundary curve or surface matching

• Extrinsic methods– Deforms all points on and interior to the source curve/surface

– App: image or volume matching


Laplacian-based DeformationLaplacian-based Deformation

• An intrinsic method– Simple, efficient, producing reasonable results

• Preserving local shape features

– Widely used in graphics applications for interactive deformation

Reference: “Laplacian surface editing”, by Sorkine et al., 2004 (citation ~ 500)


• Input– Source with n points: p1,…,pn

• Let the first m points be “handles”

– Target location of handles: q1,…,qm

• Output– Deformed locations of source points: p1’,…,pn’

SetupSetup

p1=q1

p3=q3

q2

p2

An example with 3 target points, two of which are stationary (red)

Source

Deformed

When deforming the source to fit a target shape: m=n and qi is the point on the target closest to pi.


OverviewOverview

• Finding deformed locations pi’ that minimize:

– Ef: fitting term

• Measures how close are the deformed source to the target

– Ed: distortion term

• Measures how much the source shape is changed

E Ef Ed


Fitting TermFitting Term

• Sum of squared distances to target handle locations

Ef i1

m

pi ' qi2

q2

p2


Distortion TermDistortion Term

• Q: How to measure shape?

• A: By “bumpiness” at each vertex– Laplacian: vector from the centroid of neighbors to the vertex

• Recall that in fairing, we reduced this vector to “smooth out” bumps

• A linear operator over point locations

Lpi pi 1

Ni

jNi

pj

pi

pi1pi2

pi1 pi2

2

where Ni are indices of neighboring vertices of pi


Distortion TermDistortion Term

• Minimizing changes in Laplacians during deformation– Over all source points

Ed n

i1

Lpi ' i2

pi

Lpi '

pi '

i

di: Laplacian at pi before deformation


Putting TogetherPutting Together

• Finding deformed locations pi’ that minimize:

– A quadratic equation in terms of variables (pix’, piy’, piz’)

• qi, di are constants

• L[] is a linear operator

E Ef Ed

i1

m

pi ' qi2 n

i1

Lpi ' i2


Quadratic MinimizationQuadratic Minimization

• A general form of quadratic minimization:

– There are s variables: x=(x1,…,xs)T

– Each a1,…, ak is a length-s column vector (linear coefficients)

– Each b1,…, bk is a scalar (constant coefficients)

– k should be greater than s (so that the problem is over-constrained)

min k

i1

aiT x bi2



• To solve:

• Re-write in matrix form:

min A x B2

where A a1

T

ª

akT

is a k by s matrix

min k

i1

aiT x bi2

B b1

ª

bk

is a length-k vector



• The minimizer is where the partial derivatives are all zero

– To solve for x in this equation:

• Taking matrix inverse (good for small s, but numerically unstable for large s)

• Using specialized linear system solver (LinearSolve in Mathematica, TNT/LAPACK in C)

0 Ax B2

x 2 AT A x 2 AT B AT A x AT B

x AT A1 AT B



• Re-writing our minimization in the general form

– In 2D, there are 2n variables: x = (p1x’,…, pnx’, p1y’,…, pny’ )T

• In 3D, there are 3n variables

– We will next re-write each quadratic term in 2D as (aix-bi)2

• Can be extended easily to 3D

E Ef Ed

i1

m

pi ' qi2 n

i1

Lpi ' i2



• The ai and bi in the fitting term

– There are 2m quadratic terms

– In the first set of m terms:

• For i=1,…,m, bi=qix, ai contains all zero, except its (i)th entry is 1.

– In the second set of m terms:

• For i=1,…,m, bi+m=qiy, ai+m contains all zero, except its (i+n)th entry is 1

Ef i1

m

pi ' qi2 i1

m

pix ' qix2 i1

m

piy ' qiy2

2 m

i1

aiT x bi2

x p1x ', …, pnx ', p2y ', …, pny '



• The ai and bi in the fitting term

– There are 2m quadratic terms

– Example with 3 vertices and 2 fitting constraints (n=3; m=2):

Ef i1

m

pi ' qi2 i1

m

pix ' qix2 i1

m

piy ' qiy2

2 m

i1

aiT x bi2

x

p1x 'p2x 'p3x 'p1y '

p2y '

p3y '

b1 q1x

b2 q2x

b3 q1y

b4 q2y

a1T 1 0 0 0 0 0

a2T 0 1 0 0 0 0

a3T 0 0 0 1 0 0

a4T 0 0 0 0 1 0



• The ai and bi in the distortion term:

– There are 2n quadratic terms

– The first set of n terms:

• For i=1,…,n, ai is all zero except the (i)th entry is 1, the (j)th entries are -1/|Ni|for all jœNi, and bi=dix

– The second set of n terms:

• For i=1,…,n, ai+n is all zero except the (i+n)th entry is 1, the (j+n)th entries are -1/|Ni| for all jœNi, and bi+n=diy

Ed n

i1

Lpi ' i2 n

i1

Lpix ' ix2 n

i1

Lpiy ' iy2

Lpi pi 1

Ni

jNi

pj

2 n

i1

aiT x bi2



• The ai and bi in the distortion term:

– There are 2n quadratic terms

– Example with 3 vertices (n=3):

Ed n

i1

Lpi ' i2 n

i1

Lpix ' ix2 n

i1

Lpiy ' iy2

Lpi pi 1

Ni

jNi

pj

x

p1x 'p2x 'p3x 'p1y '

p2y '

p3y '

b1 1x

b2 2x

b3 3x

b4 1y

b5 2y

b6 3y

2 n

i1

aiT x bi2

p1 p2

p3a1

T 1 12

12

0 0 0 a2

T 12

1 12

0 0 0 a3

T 12

12

1 0 0 0 a4

T 0 0 0 1 12

12

a5T 0 0 0 1

21 1

2

a6T 0 0 0 1

2 1

21


SummarySummary

• Compute Laplacians (di)

• Construct coefficients (ai, bi)– Stick them into matrices (A,B)

• Solve (x)– AT A x AT B


ResultsResults

Deformed

A small deformation


ResultsResults

Deformed

A larger deformation


ResultsResults

Stretching

Deformed


ResultsResults

Deformed

Shrinking


ResultsResults

Rotation

Deformed


DiscussionDiscussion

• Limitations – Local features are “skewed”, and they don’t scale with the model

• Reason: Laplacian changes with rotation or scale– Two bumps that differ by rotation or scale have different Laplacians

• Which will be penalized by our distortion term

pipi '

Lpi

Lpi 'Lpi Lpi '


A Better Distortion TermA Better Distortion Term

• Not penalizing rotation and scaling of local features– Transforming the original Laplacian vectors before comparing to the

deformed Laplacians

• Ti is a matrix that describes how the local shape around pi is deformed

Ed n

i1

Lpi ' Ti i2

pipi 'Ti


Key QuestionsKey Questions

• How to represent transformations as matrices?

• How to compute Ti?

• We will focus in the derivations of the 2D case– 3D results will be briefly presented at the end


Transformation Matrices (2D)Transformation Matrices (2D)

• Homogeneous coordinates– A 2D point: (x,y,1)

• A 2D vector: (x,y,0)

– A 3D point: (x,y,z,1)

• A 3D vector: (x,y,z,0)



• Translation– Cartesian coordinates: vector addition

– Homogeneous coordinates: matrix product

px'

py' vx

vy px

py

px'

py'

1

1 0 vx0 1 vy

0 0 1

pxpy

1



• Isotropic scaling– Cartesian coordinates: vector scaling


px'

py' s px

py

px'

py'

1

s 0 10 s 10 0 1

pxpy

1



• Rotation– Cartesian coordinates: matrix product


px'

py' Cos Sin

Sin Cos pxpy

px'

py'

1

Cos Sin 0Sin Cos 0

0 0 1

pxpy

1



• Summary of elementary similarity transformations– To combine transformations: take the product of these matrices

px'

py'

1

M pxpy

1

Trsv 1 0 vx0 1 vy

0 0 1

Scls s 0 10 s 10 0 1

Rot Cos Sin 0Sin Cos 0

0 0 1

Translation by vector v

Scaling by scalar s

Rotation by angle a


Similarity Transforms (2D)Similarity Transforms (2D)

• General similarity transformations

– The product of any set of elementary matrices can be written this way

– Any choice of (a, w, tx, ty) can be written as a sequence of rotation, isotropic scaling and translation

• Note that a and w can’t be both zero

T a w txw a ty

0 0 1

a w txw a ty

0 0 1 Trstx, ty.Scl a2 w2 .RotArcTan a

w


Computing Ti (2D)Computing Ti (2D)

• Suppose we know the deformed locations pi’

• Compute Ti as the similarity transform that best fits the neighborhood of pi to that of pi’

min Ti pi pi '2 jNi

Ti pj pj '2

pipi 'Ti



• Suppose we know the deformed locations pi’

• Compute Ti as the similarity transform that best fits the neighborhood of pi to that of pi’

• This is a quadratic minimization problem for entries of Ti

– E.g., a, w, tx, ty

min Ti pi pi '2 jNi

Ti pj pj '2



• The matrix form of the minimization is:

where is a 2|Ni|+2 by 4 matrix, and Ni={i1, i2,…} are indices of neighboring vertices of pi

C

pix piy 1 0

piy pix 0 1

pi1x pi1y 1 0

pi1y pi1x 0 1

ª ª ª ª

min C

awtxty

pix 'piy '

pi1x 'pi1y '

ª

2



• By quadratic minimization:

– Linear expressions of variables (pix’ , piy’)

awtxty

CT C1 CT

pix 'piy '

pi1x 'pi1y '

ª


Distortion Term (2D)Distortion Term (2D)

• Two parts of each distortion term:– Transformed Laplacian:

– Laplacian of the deformed locations:

Ti i D

awtxty

D CT C1 CT

pix 'piy '

pi1x 'pi1y '

ª

Lpi ' L

pix 'piy '

pi1x 'pi1y '

ª

L 1 0 1

Ni0 ...

0 1 0 1Ni

...

where

where is a 2 by 2|Ni|+2 matrix

Lpi ' Ti i2

D ix iy 0 0

iy ix 0 0



• Putting together:

– They form 2n quadratic terms (aix-bi)2 for x = (p1x’,…, pnx’, p1y’,…, pny’ )T

• All bi are zero

• Each ai can be extracted from H

H L D CT C1 CT

H1, H2where

are its rows

Ed n

i1

Lpi ' Ti i2

n

i1

H1

pix 'piy '

pi1x 'pi1y '

ª

2

n

i1

H2

pix 'piy '

pi1x 'pi1y '

ª

2

and


Results (2D)Results (2D)

New distortion term

Old distortion term



New distortion termOld distortion term



New distortion term

Old distortion term


RegistrationRegistration

• Use nearest neighbors as corresponding target locations – Assuming the source is already close to the target

• Iterative closest point (ICP)– 1. For each point on the source, assign its closest point on the target as

its corresponding point. Compute Laplacian-based deformation.

• A threshold on the closest distance can be used to throw away unlikely correspondences

– 2. Repeat step (1) until a termination criteria is met.

• Maximum iteration or minimum RMSD improvement


ResultResult

After rigid alignment 1 iteration of Laplacian

7 iterations of Laplacian Overlaying all curves


ResultResult

• Weighting the distortion term

E Ef wEd

small w

medium w

large w



• Elementary transformation matrices– To perform a sequence of transformations: take the product of these

matrices

Translation by vector v

Scaling by scalar s

Rotation by angle aaround X axis

px'

py'

pz'

1

M

pxpy

pz1

Trsv 1 0 0 vx0 1 0 vy

0 0 1 vz0 0 0 1

Scls s 0 0 10 s 0 10 0 s 10 0 0 1

RotX, 1 0 0 00 Cos Sin 00 Sin Cos 00 0 0 1



• General similarity transformations in 3D

– Approximates the product of a set of elementary matrices

• Up to a small rotation angle

• May introduce skewing for large rotations

T

s h3 h2 txh3 s h1 ty

h2 h1 s tz0 0 0 1



• Assuming known deformation, by quadratic minimization:

– Linear expressions of the deformed points pi’

• C is a 3|Ni|+3 by 7 matrix

sh1h2h3txty

tz

CT C1 CT

pix 'piy '

piz 'pi1x 'pi1y '

pi1z '

ª

C

pix 0 piz piy 1 0 0

piy piz 0 pix 0 1 0

piz piy pix 0 0 0 1

pi1x 0 pi1z pi1y 1 0 0

pi1y pi1z 0 pi1x 0 1 0

pi1z pi1y pi1x 0 0 0 1

ª ª ª ª ª ª ª

where



• Constructing transformed Laplacian:

whereTi i D

sh1h2h3txty

tz

D CT C1 CT

pix 'piy '

piz 'pi1x 'pi1y '

pi1z '

ª

D

ix 0 iz iy 1 0 0

iy iz 0 ix 0 1 0

iz iy ix 0 0 0 1where

cse 554 lecture 8: laplacian deformationtaoju/cse554/lectures/lect08_deformation.pdfcse554 laplacian...

Documents