1 computational vision csci 363, fall 2012 lecture 31 heading models

1

Computational Vision

CSCI 363, Fall 2012Lecture 31

Heading Models

2

Motion over a Ground Plane

3

Image motion for Translation plus Rotation

TRANSLATION WITH ROTATION

4

Models for Computing Observer Motion

€

vx −−TX + xTZ

Z

⎛ ⎝ ⎜

⎞ ⎠ ⎟

⎛ ⎝ ⎜

⎞ ⎠ ⎟∑2

+ vy −−TY + yTZ

Z

⎛ ⎝ ⎜

⎞ ⎠ ⎟

⎛ ⎝ ⎜

⎞ ⎠ ⎟2

1. Error minimization: Minimize the equation

2. Template models:Use a group of "templates" that correspond to the flow fields that

would be created by a given set of translation and rotation parameters. Find the template that matches the flow field best.

3. Motion Parallax models:Make use of the fact that image velocities due to translation are

dependent on Z. Image velocities due to rotation are not.

5

Image Velocities

vx = (-TX + x TZ) / Z - (1 + x2) RY + xy RX + y RZvy = (-TY + y TZ) / Z + (1 + y2) RX - xy RY - x RZ

Translation ComponentDepth Dependent

Rotation ComponentDepth Independent

x and y components of image velocity:

TY

TX

TZ

RZ

R X

RY

Y

X

Z

P O

y

xp

Longuet-Higginsand Prazdny, 1980

NearFar

Diff

6

Neurons vs. Pure Math

+

−+

−+

−

Middle Temporal Area Difference Vectors

AsymmetricSurround

Circularly-SymmetricSurround

1. Spatially extendedreceptive fields.

2. Response is tunedto speed and direction.

3. Center and surroundtend to have the samebest direction.

1. Vector subtractionat a single point.

2. Accurate velocitymeasurements assumed.

3. Vectors may differsubstantially in direction.

VectorSubtraction

7

Motion-subtraction by neurons

Odd Symmetric Even Symmetric

8

Computing Heading

Visual Field

Operator GroupReceptive Field Spacing

9

Layer 2 is a Template

Maximally RespondingOperators

Translational Heading Template

Template & MST cells both:1. Have large receptive fields.2. Respond to expansion/contraction.3. Are tuned to center of expansion.

10

Motion toward a 3D cloud

Observer

11

Translation + Rotation

Flow Field

12

Operator Responses

13

Model Heading Estimates

Mod

el R

espo

nse

14

Moving Objects

Demo

15

Hea

ding

Bia

s (d

eg)

16

Model vs. Experiment.

-0.25

0

0.25

0.5

0.75

-10 -5 0 5 10

Center Position (deg)

Right Object Motion

-1

-0.5

0

0.5

-5 0 5 10 15

Left Object Motion

Model

Psychophysics

Response Bias, Model vs. Psychophysical Data

17

Radial Optic Flow Field

Scene Focus of Expansion (FOE)

18

Lateral Flow Field

19

Illusory Center of Expansion(Duffy and Wurtz, 1993)

Scene Focus of Expansion (FOE)

Perceived Focus of ExpansionDemo

20

Difference Vectors for Illusion

Center of Difference Vectors

21

Model Response to Illusion

-15

-10

-5

0

5

10

15E

stim

ated

Cen

ter

-15 -10 -5 0 5 10 15

Lateral Dot Speed

Model

Calculated

22

Model vs. Human Response

-15

-10

-5

0

5

10

15

Est

imat

ed C

ente

r

-15 -10 -5 0 5 10 15

Lateral Dot Speed

Model

Calculated

Average

23

Conclusions1. A model based on motion subtraction done by neurons in

MT can accurately compute observer heading in the presence of rotations.

2. The model shows biases in the presence of moving objects that are similar to the biases shown by humans.

3. The model responds to an illusory stimulus in the same way that people do.

4. The fact that the model responds in the same way as humans with stimuli for which it was not designed provides evidence that the human brain uses a mechanism similar to that of the model to compute heading.

24

How does the brain process heading?

•It is not known how the brain computes observer heading, but there are numerous models and hypotheses.

•One of the simplest ideas is based on template models: Neurons in the brain are tuned to patterns of velocity input that would result from certain observer motions.

•Support for this idea:•Tanaka, Saito and others found cells in the dorsal part of the Medial Superior Temporal area (MSTd) that respond well to radial, circular or planar motion patterns.

•Since then, people have assumed that MSTd is involved in heading computation.

25

Visual Pathway

26

Types of Responses in MSTd

(from Duffy & Wurtz, 1991)

27

Combinations of Patterns•Duffy and Wurtz (1991) tested cell responses to planar, circular and radial patterns. •They found some cells responded only to one type of pattern (e.g. only to circular). Others responded to two or three types of patterns (e.g. both planar and circular).

Single Component Double Component Triple ComponentRadial Plano-Radial Plano-Circulo-RadialCircular Plano-CircularPlanar

•They did not suggest a model of how these might be involved in heading detection.

•They also showed there is not a simple way that MST receptive fields are made from inputs from MT cell receptive fields.

28

Spiral Patterns

Graziano et al. (1994) showed that MSTd cells respond to spiral patterns of motion:

29

Does MST compute heading?

Prediction: If MST is involved in heading computation, one would expect to find cells tuned to a particular position for the center of expansion.Duffy and Wurtz (1995) tested this prediction.

30

Do MSTd cells use eye-movement information?

•Psychophysical experiments showed that humans can make use of eye movement information to compute heading.

•Some MSTd cells have responses that are modulated by eye movements.

•Do eye movements affect the responses of MSTd cells to compensate for rotation?

•This was tested in an experiment by Bradley et al (1996).•They recorded from MSTd cells while showing flow fields that consisted of an expansion plus a rotation. The rotation was generated by real or simulated eye movements.

31

Real eye movement condition

32

Simulated eye movement condition

33

Results

No eye movement

Eye movement in preferred direction.

Eye movement in anti-preferred direction.

This cell seems to take into account eye movements.

The effect was not consistent among all cells tested.