EFFICIENT GLOBALLY OPTIMAL 2D-TO-3DDEFORMABLE SHAPE MATCHING

Zorah Lähner¹, Emanuele Rodolà1,2, Frank R. Schmidt1,

Michael M. Bronstein2, Daniel Cremers1

Published at CVPR 2016

1 Technical University Munich2 Università della Svizzera Italiana

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 2

THE TASK

Finding a continuous matchingbetween a 2D template shapeand a 3D model that matchespoints which look alike

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 3

RETRIEVAL

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 4

SKETCH-BASED MODELLING

Kraevoy et al., 2009

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 5

INPUTS

NM

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 6

PRODUCT MANIFOLD

n pointsmpoints

x

Each node in the product manifold represents a possible matchbetween a point on the contour and on the model.

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 7

PRODUCT MANIFOLD

x

i j

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 8

PRODUCT MANIFOLD

An edge between (i, j) and (k, l) exists ifboth i, k and j, l are the same orneighbors on the original shapes.

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 9

ENERGY FORMULATION

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 10

ENERGY FORMULATION

x

i jx

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 11

SLICED PRODUCT MANIFOLD

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 12

SLICED PRODUCT MANIFOLD

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 13

RUNTIME

Given a 2D template shape M and a 3D model shape N, discretized at m and nvertices, respectively, we can find a minimizer inO(mn2 log(n)) time.If n = O(m2), this leads to the subcubic runtime of O(n2.5 log(n)).

m vertices n vertices

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 14

FEATURES

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 15

SPECTRAL FEATURES

Purely based on intrinsic properties and therefore invariant toisometric deformations

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 16

SPECTRAL FEATURES

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 17

SPECTRAL FEATURES

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 18

RESULTS

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 19

RESULTS

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 20

RETRIEVAL RESULTS

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 21

CLASSIFICATION

Embedding of all Tosca Shapes into 2D

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 22

CONCLUSION

● Fully automated Matching● Provable subcubic time, even faster in normal cases● Globally optimal solution● Continuous solution● Comparable 2D-to-3D Features

Zorah Lähner - Efficient Globally Optimal 2D-to-3D Deformable Shape Matching 23

Thank you for your attention!Questions?