Shape Matching and Anisotropy

Shape Matching and Anisotropy

Michael Kazhdan, Thomas Funkhouser, and Szymon

Rusinkiewicz

Princeton University

Michael Kazhdan, Thomas Funkhouser, and Szymon

Rusinkiewicz

Princeton University

Motivation

3D data is becoming more commonly available

Someday 3D models will be as common as images are today

Someday 3D models will be as common as images are today

Cheap Scanners World Wide Web3D CafeCyberware

Fast Graphics Cards

ATI

Images courtesy ofCyberware, ATI, & 3Dcafe

Motivation

When 3D models are ubiquitous, there will be a shift in research focus

Future research will ask:“How do we find 3D models?”

Future research will ask:“How do we find 3D models?”

Utah VW Bug Utah Teapot Stanford Bunny

Images courtesy ofStanford & Utah

Previous research has asked:“How do we construct 3D models?”

Previous research has asked:“How do we construct 3D models?”

Challenge

Given: A database of 3D models and a query model

Find: The k database models most similar to the query

Images courtesy ofGoogle & Princeton

Approach

To retrieve the nearest k models: Compute the distance between the query and every

database model. Sort the database models by proximity. Return the first k matches.

3D Query

Database ModelsBest Match(es)

Sorted Models

Sort by proximityQuery

comparison

Comparing 3D Models

Direct Approach: Establish pair-wise correspondences between

points on the surfaces of the two models. Define the distance between the models as

the distance between corresponding points.

2 ii qppnpn

qnqn

pn-1

qn-1qn-1q3q3

q2q1q1

p3

p2

p1

Similarity defined as distance between models Establishing correspondences is difficult and slow

Comparing 3D Models

Practical Approach: Represent each 3D model by a shape

descriptor. Define the distance between two models as

the distance between their shape descriptors.Shape Descriptors:Extended Gaussian Images, Horn Complex Extended Gaussian Images, Kang

et al.Spherical Attribute Images, Delingette et

al. Crease Histograms, BeslShape Histograms, Ankerst et al.Shape Distributions, Osada et al. Spherical Extent Functions, Vranic et al.Gaussian EDTs, Funkhouser et al. Symmetry Descriptors, Kazhdan et al.

Approximates distance between models Correspondences are implicit Comparison is easy and fast

Observation

It is not enough to consider the distance between two 3D surfaces… We also need to consider how the surfaces transform into each other.

M1

Q

Q M2M2

Q

M1

21 MQMQ ?

Match models in two steps:

1. Factor out low-frequency alignment of the models

2. Match the aligned models

Define similarity by combininglow-frequency alignment infowith high-frequency difference

Our Approach

Isotropic ModelsAnisotropy

Outline

Introduction

Aligning Anisotropic Scales• Related Work• Anisotropy Normalization• Convergence Properties

Shape Matching

Conclusion and Future Work

Point Set Alignment

Given point sets P={p1,…,pn} and Q={q1,…qn}, what is the optimal alignment A minimizing:

n

iii qAp

1

2)(

P Q

Original

[Horn, 1987]

Point Sets (Translation)

Translate so that the center is at the origin:

n

ii

n

ii qandp

11

0 0

A model can be aligned for translation independent of what it will be compared against

P Q

Original

[Horn, 1987]

Translated

P’ Q’

Point Sets (Isotropic Scale)

Scale so that mean variance from center is equal to 1:

n

ii

n

ii q

nandp

n 1

2

1

21

1 1

1

A model can be aligned for isotropic scaleindependent of what it will be compared against

P Q

Original Translated

P’ Q’ P” Q”

Scaled

[Horn, 1987]

Point Sets (Anisotropic Scale)

Scale so that the variance in every direction equal to 1: 1 1

1

1

2

vpvn

n

ii

A model can be aligned for anisotropic scaleindependent of what it will be compared against

Anisotropic Models Isotropic Models

Unit Variance

Unit Variance in Every Direction

Covariance Matrix is Identity (Covariance Ellipse is a Sphere)

Anisotropic Model

Isotropic Model

Initial Point Set

Rescaled Point Set

Covariance Ellipse

Covariance Ellipse

For point sets, transform by inverse square root of the covariance matrix

From Points to Surfaces

Points samples from a surface become isotropic but the sampled surface does not.

Point Set Model Surface Model

From Points to Surfaces

Uniform samples do not stay uniform…

Initial Point Set Isotropic Point Set

From Points to Surfaces

Uniform samples do not stay uniform…

But the model gets more isotropic.

Iteratively rescale to get models that are progressively more isotropic

Convergence of Iteration

Provably convergent Show that in the worst case smallest eigenvalue doesn’t

get smaller and largest one doesn’t get larger. Use the triangle inequality to show that at least one of the

eigenvalues has to change. In practice, converges very quickly

0

1

2

0 2 4 6 8 10Iterations

RM

S E

rro

r

Max

Average

Tested on 1890 Viewpoint models

Outline

Introduction

Aligning Anisotropic Scales

Shape Matching Extending Shape Descriptors Experimental Results

Conclusion and Future Work

Product Descriptor

For any shape descriptor, we define a new shape descriptor that is the product of: The descriptor of the isotropic model, and The anisotropic scales

Initial ModelInitial Model

Isotropic ModelIsotropic Model

Rescaling EllipseRescaling Ellipse

DescriptorDescriptor

EigenvaluesEigenvalues

New DescriptorNew Descriptor

Factored Matching

Isotropic Models Anisotropy

Parameterized family of shape metrics, as a function of anisotropy importance .Parameterized family of shape metrics, as a function of anisotropy importance .

Experimental Database

Princeton Shape Benchmark ~900 models, 90 classes

14 biplanes 50 human bipeds 7 dogs 17 fish

16 swords 6 skulls 15 desk chairs 13 electric guitars

http://shape.cs.princeton.edu/benchmark/

Example Query

Results Without Anisotropy Factorization

Results With Anisotropy Factorization (=3)

Query

1 2 3 4

8765

1 2 3 4

8765

Gaussian EDT, Funkhouser et al. 2003

Retrieval Results (=3)

Descriptor Dim Improvement

SHELLS 1D 63%

D2 1D 36%

EGI* 1D 64%

CEGI* 1D 28%

Sectors* 1D 31%

EXT* 1D 39%

REXT* 2D 16%

Voxel* 2D 23%

Sectors + Shells*

2D 35%

Gaussian EDT* 2D 4%

Rotation Invariant Descriptors

Descriptor Dim

Improvement

EGI 2D 29%

CEGI 2D 20%

Sectors 2D 5%

EXT 2D 7%

REXT 3D 4%

Voxel 3D 1%

Sectors + Shells

3D 7%

Gaussian EDT 3D 4%

Rotation Varying Descriptors

* Spherical Power Spectrum Representation

Outline

Introduction

Aligning Anisotropic Scales

Shape Matching

Conclusion and Future Work

Conclusion

Presented and iterative approach for transforming anisotropic models into isotropic ones:

Provides a method for factoring shape matching Improves matching for all descriptors Facilitates registration of models

Initial ModelInitial Model

Isotropic ModelIsotropic Model

Rescaling EllipseRescaling Ellipse

DescriptorDescriptor

EigenvaluesEigenvalues New DescriptorNew Descriptor

Conclusion

Presented and iterative approach for transforming anisotropic models into isotropic ones:

Provides a method for factoring shape matching Gives rise to improved matching retrieval results Facilitates registration of models

Anisotropic Models

Conclusion

Presented and iterative approach for transforming anisotropic models into isotropic ones:

Provides a method for factoring shape matching Gives rise to improved matching retrieval results Facilitates registration of models

Isotropic Models

Future Work

Address cross-class anisotropy variance Factor out higher order transformations

1 2 3 4

5 6 7 8

1 2 3 4

5 6 7 8

Without Anisotropy Factorization

With Anisotropy Factorization

1 2 3 4

5 6 7 8

Without Anisotropy Factorization

1 2 3 4

5 6 7 8

With Anisotropy Factorization

Future Work

Address cross-class anisotropy variance Factor out higher order transformations

TranslationAnisotropic

Scale ?Isotropic

Scale Rotation

Thank You

FundingNational Science Foundation

Source Code Dan Rockmore and Peter Kostelec

http://www.cs.dartmouth.edu/~geelong/spherehttp://www.cs.dartmouth.edu/~geelong/soft

DatabasesViewpoint Data Labs, Cacheforce, De Espona Infografica

http://www.viewpoint.comhttp://www.cacheforce.comhttp://www.deespona.com

Princeton Shape Matching GroupPatrick Min and Phil Shilane

http://shape.cs.princeton.edu

Thank You