Optical Snow and the Aperture Problem

Richard MannSchool of Computer Science

University of Waterloo

Michael Langer

School of Computer Science

McGill University

Optical flow

J.J. Gibson, The Senses Considered As Perceptual Systems, 1966

Layered motion

e.g. occlusions, transparency

Motion beyond layers

e.g. falling snow

“Optical snow”

“Optical Snow”

Lateral egomotion in a 3D cluttered scene

Optical snow

Overview of Talk

• background: - Fourier analysis of optical snow - how to estimate direction of optical snow? (Langer and Mann, ICCV ’01)

Overview of Talk

• background: - Fourier analysis of optical snow - how to estimate direction of optical snow? (Langer and Mann, ICCV ’01)

• new stuff: - aperture problem

Fourier analysis of image translation

v f + v f + f = 0x x y y t

If image patch is translating with velocity (v , v )then all power lies on a plane:

x y

fy

t

(Watson & Ahumada ’85)

f x

f t

Optical Snow

Image velocities are (α v , α v ) x y

α v f + α v f + f = 0x x y y t

ft

f x

ft

Fourier analysis of optical snow

“bowtie”

Bowtie of falling spheres

f Θ

f t

Bowtie of bush

f t

f Θ

Q: How to compute motion direction ?A: rotate a wedge and measure power

Minimum of power in wedge occurswhen wedge is aligned with the bowtie.

Computing the direction of motion

The motion direction is perpendicular to the direction of minimum of power.

motion directionminimum of power

Aperture Problem

Vertically falling cylinders appearto move in normal direction.

“normal”direction

Aperture Problem

true motiondirection

“normal” direction(max of power)

Aperture problem

falling ellipsoids

same power butrandom phase

?

Summary

• Optical snow: a new motion category

• Fourier-based method for detecting direction of motion

• Analysis of aperture problem