shape from moments - 1 - shape from moments an estimation perspective michael elad *, peyman...

Shape From Moments - 1 -

Shape from Moments An Estimation

Perspective

Michael Elad*, Peyman Milanfar**, and Gene Golub*

SIAM 2002 MeetingMS104 - Linear Algebra in Image Processing

July 12th, 2002

* The CS Department – SCCM Program Stanford University** Peyman Milanfar is with the University of California Santa Cruz

(UCSC).


Chapter A

Background


na

A.1 Davis Theorem

P

8z

2z

3z

4z

7z

6z

5z

1z

x

y{The complex plane Z}

,zfzzzz

zzzz

2i

dxdy)z(''fP

N

1nn

n1n

n1n

n1n

n1n

Theorem (Davis

1977) : For any closed

2D polygon, and for any

analytic function f(z) the

following holds


A.2 Complex Moments

If we use the analytic function , we get from

Davis Theorem that

kzzf

2k

N

1n

knn

P

2k

P

zadxdyz)1k(kdxdy)z(''f

k

2kP

2kk )1k(kdxdyz)1k(k

k

where we define


A.3 Shape From Moments

M

0k

N

1n

kn

n1n

n1n

n1n

n1nk z

zzzz

zzzz

2i

• Can we compute the vertices from these equations ?

• How many moments are required for exact recovery ?

Shape From Moments

M 0kk N 1nnz


A.4 Previous Results

• Golub et. al. (1999): – Pencil method replacing the Prony’s - better numerical stability.

– Sensitivity analysis.

• Milanfar et. al. (1995): – (2N-1) moments are theoretically sufficient for computing the N

vertices.

– Prony’s method is proposed.

• Prony’s and the Pencil approaches:– Rely strongly on the linear algebra formulation of the problem.

– Both are sensitive to perturbations in the moments.

– Both will be presented briefly.


Function of the vertices (and their order)

A.5 To Recap

Known values

N

2

1

MN

M2

M1

2N

22

21

1N

12

11

0N

02

01

M

2

1

0

a

a

a

zzz

zzz

zzz

zzz

Estimate !!! N1nnz


A.6 Our Focus

• Noisy measurements: What if the moments are

contaminated by additive noise ? How can re-

pose our problem as an estimation task and solve

it using traditional stochastic estimation tools ?

• More measurements: What if there are M>2N-1

moments ? How can we exploit them to robustify

the computation of the vertices ?


A.7 Related Problems

• It appears that there are several very different

applications where the same formulation is

obtained– Identifying an auto-regressive system from its output,

– Decomposing of a linear mixture of complex cissoids,

– Estimating the Direction Of Arrival (DOA) in array processing,

– and more ... N 1kka • Major difference – are not functions of the

unknowns but rather free parameters.

• Nevertheless, existing algorithms can be of use.


Chapter B

Prony and Pencil Based Methods


B.1 Prony’s Relation

N

1n

nNn

NN

1nn zpzzz)z(P

M

0k

N

1n

knnk za

1

1N

N

1M1NMNM

1N32

N21

1N10

M

2N

1N

N

p

p

p


B.2 Prony’s Methods

M

2N

1N

N

1

1N

N

1M1NMNM

1N32

N21

1N10

p

p

p

d. IQML, Structuted-TLS, Modified Prony, and more.

c. Hankel Constrained SVD, followed by root-finding,

b. Total-Least-Squares, followed by root-finding,

a. Regular Least-Squares, followed by root-finding,


B.3 Pencil Relation

1M1NMNM

1N32

N21

1N10

0T

M2NM1NM

2N43

1N32

N21

1T

Define

0vTzT n0n1 After some (non-trivial) manipulation we obtain

For some non zero vectors Vn.


B.4 Non-Square Pencil

T1 - zn T0 vn=0


B.5 Pencil Methods

0vTzT n0n1

a. Take square portions, solve for the

eigenvalues, and cluster the results,H0Tb. Square by left multiplication with (closely

related to LS-Prony),

c. Hua-Sarkar approach: different squaring methods

which is more robust and related to ESPRIT.


Chapter C

ML and MAP Approaches


C.1 What are we Missing ?

• We have seen a set of simple methods that

give reasonable yet inaccurate results.

N 1kka

• In our specific problem we do not exploit the

fact that are vertices-dependent.

• In all the existing methods there is no

mechanism for introducing prior-

knowledge about the unknowns.


C.2 Recall …

We have the following system of equations

Measured Function of the unknowns

zaza

a

a

zzz

zzz

zzz

zzz

t

N

2

1

MN

M2

M1

2N

22

21

1N

12

11

0N

02

01

M

2

1

0

V


C.3 Our Suggestion

• If we assume that the moments are contaminated

by zero-mean white Gaussian noise, Direct-

Maximum-Likelihood (DML) solution is given by

22

zzaztMinimize V

• Direct minimization is hard to workout, BUT

• We can use one of the above methods to obtain an

initial solution, and then iterate to minimize the

above function until getting to a local minima.


C.4 Things to Consider

• Even (complex) coordinate descent with effective

line-search can be useful and successful (in order

to avoid derivatives).

• Per each candidate solution we HAVE TO solve the

ordering problem !!!! Treatment of this problem is

discussed in Durocher (2001).

• If the initial guess is relatively good, the ordering

problem becomes easier, and the chances of the

algorithm to yield improvement are increased.


C.5 Relation to VarPro

• VarPro (Golub & Pereyra 1973)

– Proposed for minimizing

– The basic idea: Represents the a as and

use derivatives of the Pseudo-Inverse matrix.

22azt V

tza V

• Later work (1978) by Kaufman and Pereyra

covered the case where a=a(z) (linear

constraints).• We propose to exploit this or similar method, and

choose a good initial solution for our iterative

procedure.


C.6 Regularization

• Since we are minimizing (numerically) the DML

function, we can add a regularization – a penalty term

for directing the solution towards desired properties.

zgzaztMinimize 22

z V

• The minimization process is just as easy.

• This concept is actually an application of the

Maximum A-posteriori-Probability (MAP)

estimator.


C.7 MAP Possibilities

Kind of Prior Expression for g(*)

Promoting 90° angles

Promoting smoothness (1)

Promoting smoothness (2)

Promoting regularity

Promoting small area

N

1n

2

n1n

n1n

n1n

n1n 1zzzz

zzzz

21

N

1n

2n1n zz

N

1n

2n1n1n z2zz

N

1nn1n

2N

1nn1n zzImzz

N

1nn1n zzIm


Chapter D

Results


D.1 Experiment #1

• Compose the following

star-shaped polygon

(N=10 vertices),

• Compute its exact

moments (M=100),

• add noise (=1e-4),

• Estimate the vertices

using various methods.

-1 -0.5 0 0.5 1

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1


D.1 Experiment #1

LS-Prony method Squared Pencil method Hua-Sarkar method

0.0201 0.0196 0.0174

-1 -0.5 0 0.5 1

-0.8-0.6-0.4-0.2

0.20.40.60.8

-1

0

1

-1 -0.5 0 0.5 1

-0.8-0.6-0.4-0.2

0.20.40.60.8

-1

0

1

-1 -0.5 0 0.5 1

-0.8-0.6-0.4-0.2

0.20.40.60.8

-1

0

1

Mean Squared Error averaged over 100 trials


D.2 Experiment #2

• For the star-shape polygon

with noise variance =1e-

4, initialize using Hua-

Sarkar algorithm.

• Then, show the DML

function per each vertex,

assuming all other vertices

fixed.

+ Hua-Sarkar result

New local minimum

-1 -0.5 0 0.5 1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

0.1 0.2 0.3 0.4 0.5 0.6

-0.35

-0.3

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1


D.3 Experiment #3

• For the E-shape polygon

with noise variance =1e-3,

initialize using LS-Prony

algorithm.

• Then, show the DML

function per each vertex,

assuming all other vertices

fixed.

+ LS-Prony result

New local minimum

-0.4 -0.2 0 0.2 0.4-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

-0.3 -0.2 -0.1 0 0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9


D.4 Experiment #4

• For the E-shape polygon with

noise variance =1e-3, initialize

using LS-Prony algorithm.

• Then, show the MAP function per

each vertex, assuming all other

vertices fixed.

• Regularization – promote 90°

angles.

+ LS-Prony result

New local minimum

-0.4 -0.2 0 0.2 0.4-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1


0 5 10 15

10-2

10-1

D.5 Experiment #5

Error as a function of the iteration number*

DMLMAP

* Using a derivative free coordinate-descent procedure


D.6 To Conclude

• The shape-from-moments problem is formulated, showing a close resemblance to other problems in array processing, signal processing, and antenna theory.

• The existing literature offers many algorithms for estimating the “vertices” – some of them are relatively simple but also quite sensitive.

• In this work we propose methods to use these simple algorithms as initialization, followed by a refining stage based on the Direct Maximum Likelihood and the MAP estimator.

shape from moments - 1 - shape from moments an estimation perspective michael elad *, peyman...

Documents