Determining Planar Translation and Rotation

by Optical Flow

David HongNCSSM, Mini-term 2008

Example Problem 1

In:

Out: 4 0 0s s sX Y

Example Problem 2

In:

Out: 0 4 0s s sX Y

Example Problem 3

In:

Out: 0 0 6s s sX Y

Problem (Formal Statement)

In: , n nf g R

Out: , , such that

translated by and and

rotated by is

s s s

s s

s

X Y

f X Y

g

Motivation

Many Applications:– Special Effects (Movie)

– Video Compression

– Pattern Recognition

– Image Stabilization (Digital Cameras)

– Dead-reckoning (Mobile Robotics)

State of the Art

Block Motion Estimation

Demir-Ertűrk (2007)

Optical Flow

Lucas-Kanade (1985)

Optical Flow with Smoothness Constraint

Horn-Schunck (1980)

X

Y

u(x,y,t)

U(X,Y)

xy

Idea

Idea

0 0, , ,U X Y u x y t

Then the floor-coordinate is (X0,Y0) and the sensor-coordinate is (x,y) at time t.

Let us consider a point on the plane.

From there, we can see:

Idea

0 0, , ,

0

U X Y u x y t

d dU u

dt dtdx

u u udtdyx y t

dt

Differentiating on time gives us:

Idea

0

0

cos sin

sin coss s s

s s s

X Xx

Y Yy

Expressing (x,y) in terms of (X0,Y0) and the mouse position (Xs,Ys,Θs) gives us:

Idea

s

s

s

dX

dtdYu u u u u

x yx y y x dt t

d

dt

Putting the two together, we get:

This is underdetermined!

Idea

, 1 , 1 1, 1,1, 1, , 1 , 1 , ,'

2 2 2 2

s

i j i j i j i ji j i j i j i j i j i js

s

X

tu u u uu u u u u uY

i x j yx y y x t t

t

We choose a convenient unit so Δx= Δy=Δt=1

Discretizing, we get:

1, 1, , 1 , 1 , 1 , 1 1, 1, , ,2 's

i j i j i j i j i j i j i j i j s i j i j

s

X

u u u u u u i u u j Y u u

Idea

This is normally over-determined!

Now, using all points, we get:

1, 1, , 1 , 1 , 1 , 1 1, 1, , ,2 's

i j i j i j i j i j i j i j i j s i j i j

s

X

u u u u u u i u u j Y u u

Use least-squares!

Algorithm

1, 1, , 1 , 1 , 1 , 1 1, 1, , ,2 's

i j i j i j i j i j i j i j i j s i j i j

s

X

u u u u u u i u u j Y u u

uu’

Improvement by Iteration

u

u’

Iteration 1:

3

2

0

s

s

s

X

Y

u’’

Iteration 2:

1

1

0

s

s

s

X

Y

Overall Estimate:

4

1

0

s

s

s

X

Y

u’’’

It is more difficult with rotation!Lagrange Interpolation!

Improvement by Iteration

u

u’

Performance of Algorithm

Surface:

Performance of Algorithm

Good Surface: Bad Surface:

Algorithm Fails!

Performance of AlgorithmSurface:

First iteration:

Second iteration:

Performance of AlgorithmSurface:

Performance of AlgorithmSurface:

Future Work

• Smoothness Constraint

• Pre-processing the image

• Condition for Convergence

• Phase Correlation

Thank You!