Detecting Motion and Optic Flow

Cmput 610

Martin Jagersand

Today:

Image motion and optic flow More on projects Decide on project schedules

Image motion

Somehow quantify differences in image sequences.

Image differences. Optic flow 3-6 dim motion

Motion is used to:

Attention: Detect and direct using eye and head motions

Control: Locomotion, manipulation, tools

Vision: Segment, depth, trajectory

Small camera re-orientation

Note: Almost all pixels change!

Classes of motion

Still camera, single moving object Still camera, several moving objects Moving camera, still background Moving camera, moving objects

The motion field

Vector field over the image:

[u,v] = f(x,y), u,v = Vel vector, x,y = Im pos FOE, FOC Focus of Expansion, Contraction

Optic/image flow

Assumption: Image intensities from object points remain constant over time.

Im(x + î x; y+ î y; t + î t) = Im(x; y; t)

Taylor expansion of intensity variation

Keep linear terms Use constancy assumption and rewrite:

Notice: Linear constraint, but no unique solution

à@t

@Im = @x@Im; @y

@Imð ñ

á î xî y

ò ó

0= @x@Imî x + @y

@Imî y+ @t@Imî t

Im(x + î x;y+ î y; t + î t) = Im(x;y; t) + @x@Imî x + @y

@Imî y+ @t@Imî t +h:o:t:

Im(x + î x;y+ î y; t + î t) = Im(x;y; t) + @x@Imî x + @y

@Imî y+ @t@Imî t +h:o:t:

Aperture problem

Rewrite as dot product

Notice: Can only detect vectors normal to gradient direction

à@t

@Im = @x@Im; @y

@Imð ñ

á î xî y

ò ó

= r Imá î xî y

ò ó

Aperture problem 2

Using several points

Over determined equation system

Im = Mu Can be solved in e.g. least squares sense

using matlab u = M\Im

...à

@t@Im

...

0

@

1

A =

......

@x@Im

@y@Im

......

0

@

1

A î xî y

ò ó

3-6D Optic flow

Generalize to many freedooms (DOFs)

All 6 freedoms

X Y Rotation Scale Aspect Shear

M(u) = @Im=@u

M(u) = @Im=@u

Flow vectors

Norbert’s trick: Use an mpeg-card to speed up motion computation

Application: mpeg compression

Other applications:

Recursive depth recovery: Kostas and Jane Motion control (we will cover) Segmentation Tracking

Part 2:Project discussion

How to start and carry out a literature search and reading project

What is a reasonable schedule?

Literature search

Goal: Find the 10-15 most relevant and recent papers in a subarea.

Method: 1. Seed with a few relevant papers.

2. Do citation search backwards and forwards.

3. Find common “buzz words”. Do title and abstract text search.

4. Do internet search. e.g. “Cora” from justresearch

Literature search 2

Expect to:– Read the titles of hundreds of papers– Read the abstract of 30-50 of papers– Skim through dozens of papers

In order to find the 10-15 or so most relevant.

Report:

Write a review article summarizing the main contributions and comparing the results in the papers.

Challenge: Master the concepts of the papers from only reading. – Some papers are difficult to fully grasp without trying

the concepts– Some are wrong or incomplete

Alternative: Practical project

Read a few papers (Provided by Martin) Do a computer implementation and perform

experiments based on the papers Write a report on the the papers, your work and

the experimental results

Projects: rest of semester:

Decide on project, consult with Martin Write project proposal and time plan Course presentations:

– Reading project: Select one or two papers and present to class

– Practical project: Present your work.

Final report