1

Computational Vision

CSCI 363, Fall 2012Lecture 21

Motion II

2

Gradient Models

The gradient models use the "Contrast Brightness Assumption".In 1 spatial dimension, this states:

€

I(x,t) = I(x +∂x,t +∂t)

I

xx0

t0 I

xx0 + x

t0 + t

3

The Gradient Constraint Equation

€

∂I∂x∂x +

∂I

∂t∂t = 0

Let u =∂x

∂t, Ix =

∂I

∂x, I t =

∂I

∂t then

Ixu + I t = 0

The Gradient Constraint Equation in 1 dimension:

€

∂I∂x∂x +

∂I

∂y∂y+

∂I

∂t∂t = 0

Let u =∂x

∂t,v =

∂y

∂t, Ix =

∂I

∂x, Iy =

∂I

∂y, I t =

∂I

∂t then

Ixu + Iyv+ I t = 0

The Gradient Constraint Equation in 2 dimensions:

4

The Aperture Problem•The gradient constraint equation for a 2D image is 1 equation with 2 unknowns (u and v).

•To solve for u and v, we must make measurements of Ix, Iy, and It at 2 locations where they are not all identical.

•If our view is limited to an edge seen through an aperture, we cannot solve for both u and v independently. We can only find the component of motion perpendicular to the edge.

Aperture

EdgePerpendicular velocitycomponent

5

The Aperture Problem is Fundamental

•The aperture problem is a fundamental problem when one is trying to measure image velocity using local detectors.

•This is true in biological vision (neurons have local receptive fields).

•This is also true in machine vision (intensity is detected locally by photodetectors).

6

Solving the Aperture problem1. Assume pure translation of the object.

The true velocity may lie anywhere along this "constraint" line.

2. Make two separate local measurements.

v1

v2 Replot in velocity space

v2

v1vx

vy

The true velocity is at the intersection of the constraint lines.

7

The Smoothness ConstraintThe previous solution to the aperture problem requires a rigid object that is not rotating.

If the object is rotating or changing shape (deforming), we need another constraint to solve for velocity.

The "smoothness constraint" states that the velocity along a boundary (or within a 2D area of the image) varies smoothly.

Note: This is violated across the boundary of a moving object.

8

Measuring Velocity along a Contour

S

v

C

vy

vx€

∂v∂S

€

Length of ∂v

∂S is ∂v

∂S

Total variation over the curve is:

€

∂v∂S

2

C∫

To impose the smoothness constraint, we find the velocity field that minimizes the above integral.

9

Do Humans use a smoothness constraint?

The model incorporating the smoothness constraint finds the correct result for:

1) Pure translation of an object2) General motion of a rigid object with straight edges.

Nakayama and Silverman developed a stimulus that shows that humans integrate along contours and over small 2D area:

Oscillating contourLooks non-rigid

Add line-breaksLooks rigid

Add linesLooks rigid

10

Motion Illusions

The model fails for several cases where humans also do not see the correct velocity:

1) Rotating spirals (look like they are expanding)http://www.michaelbach.de/ot/mot_adaptSpiral/index.html

2) The Barberpole illusion (looks like it is moving up)http://www.123opticalillusions.com/pages/barber_pole.php

3) "Wobbling" ellipses.

The model computes velocities along the contours that are consistent with human perception.

Note: A few experiments have shown this is not exactly true all the time.

11

Rotating spirals

True velocity vectors

Initial measurements

Smoothest velocity fieldfrom initial measurements

12

Barberpole illusion

True velocity vectors

Smoothest velocity field

13

Motion Energy Models

An alternative way to think about 2D motion detection involves using spatio-temporal frequency filters.

This type of model relies on filters that are combined to detect a certain range of spatial and temporal frequencies.

Various people have developed versions of these models:van Santen & SperlingWatson & AhumadaAdelson & Bergen

14

Motion as orientation

In x-t space, motion is an oriented line. The slant depends on speed.In x-y-t space, motion becomes an oriented slab within a volume.

x-t

x-y-t

15

Orientation Detectors in Space-Time

Filters oriented in space time can detect a moving stimulus.The orientation of the filter relates to its preferred speed of motion.

These filters can detect sampled motion as well.

Oriented Spatio-temporal filters:

16

Separable Spatio-temporal filters

A Spatio-temporal filter can be created as the product between a spatial filter and a temporal filter.

Spatial impulse response = HS(x)Temporal impulse response = HT(t)Spatio-temporal impulse response: HST(x, t) = HS(x)HT(t)

17

Response to a Moving Edge

t1t2 t3

There is little response at t1 and t3.There is largest response at t2 during the edge motion

18

Oriented spatio-temporal filters

The previous filter was not selective for direction of motion.We can develop an oriented filter that is selective for direction, by creating a spatio-temporal Gabor filter:

€

g(x,t) =1

2πσ xσ t

e−

t 2

2σ t2

+x2

2σ x2

⎛

⎝ ⎜ ⎜

⎞

⎠ ⎟ ⎟

sin(2πωx x + 2πω t t)

-+

-

Filter selective for leftward motion

19

Response of oriented filter

Non-oriented

Oriented

Moving edgestimulus

Filter Response

20

Problems with Gabor filter

The Gabor filter by itself results in several problems:

1) It is phase sensitive: It depends on a particular alignment of the pattern with the filter at a given time. (The response to a drifting sinewave is an oscillation).

2) The sign of the response depends on the stimulus contrast (e.g. white on black gives opposite response to black on white).

21

Solution: Motion Energy

Motion energy filters are constructed with 2 gabor filters, one of which uses a sine and the other uses a cosine (a "quadrature pair").

If you square the outputs of the gabors and sum, the result is motion energy.

22

Motion Energy Responses

With motion energy filters:

The response is always positive.

The response is the same for a black-white edge as for a white-black edge.

The response to motion is independent of contrast.

The response is constant for a drifting sinewave.