1

Computational Vision

CSCI 363, Fall 2012Lecture 14

Spatial Frequency in Striate Cortex

2

Another viewpoint: V1 cells are spatial frequency filters

•DeValois (and others) proposed that V1 cells are tuned to spatial frequency.

•V1 simple cells can be modeled as Gabor filters.

•A Gabor filter is the product of a sinewave and a gaussian.

sin(10x)

G(x)

sin(10x)G(x)

3

Fourier Transform of a Gabor Filter

The Fourier Transform of a Gabor filter is a localized set of spatial frequencies.

Gabor filters are band-pass filters. They are tuned to spatial frequency.

If Striate Cortex cells are like Gabor filters, then they are also acting as band-pass filters.

FourierTransform

4

2D Gabor filter

5

Simple Cell Receptive Field vs. Gabor Function

Solid line: Simple Cell Receptive Field.

Dashed line: Best fitting Gabor function.

From: DeValois and DeValois, "Spatial Vision", 1988.

6

Contrast Sensitivity of V1 Cells

CSF for individual V1 cellsDistribution of tuning bandwidth

7

Spatial Profile vs. CSF

The spatial profile of the simple cell receptive field is predicted by taking the inverse Fourier transform of the contrast sensitivity function for that cell.

8

Spatial Frequency Columns

As with orientation and ocular dominance, spatial frequency shows columnar organization in the cortex.

9

Simple vs. Complex Cells

The response of simple cells to drifting gratings shows a big oscillation over time.

The complex cell response does not oscillate much.

10

Classifying simple and complex cells

•Simple cells have a larger AC (Alternating current) response. •Complex cells have a larger DC (Direct current) response.•The ratio of AC/DC allows classification.AC/DC >1 => simple cells. AC/DC < 1 => complex cells

11

Temporal Frequency

•Temporal frequency is measured in cycles/sec = Hertz (Hz)

•Temp. Freq. (cycles/sec) = Speed (deg/sec) x Spatial Freq (cycles/deg)

•Faster drift => Higher temporal frequency•Higher spatial frequency => Higher temporal frequency

•Temporal frequency is the frequency of oscillation of light intensity over time.•For a drifting sinewave grating, the luminance at a single point in space oscillates over time.•http://web.mit.edu/~jgolomb/www/drifting.gif

(Prof. Julie Golomb, MIT)

12

Speed vs. Temporal Frequency•To test whether V1 cells are tuned to speed or temporal frequency, Tolhurst and Movshon (and others) examined speed tuning for sinewave gratings with different spatial frequencies.

•Example: Test spatial frequencies of 1.0, 2.0 and 3.0 cycles/deg.

•If neurons are tuned for speed, the tuning curves will peak at the same speed , independent of spatial frequency. (For example, if the neuron prefers 5 deg/sec, all the graphs will have a peak there).

•If neurons are tuned for temporal frequency (e.g. 10 cycles/sec), then the peaks for speed tuning will depend on the spatial frequency (The peaks will be at 10, 5 and 3.33 deg/sec).

•Cells in striate cortex appear to be tuned to temporal frequency. (Cells at higher levels may be tuned to speed).

13

V1 cells and Temporal Frequency

Response of neuron tuned for speed.

MT cells match this pattern.

Response of neuron tuned for temporal frequency.

V1 cells match this pattern

Simoncelli and Heeger, Nature Neuroscience, 2001

14

Linear Systems•Linear functions:

F(x1 + x2) = F(x1) + F(x2)F(ax) = aF(x)

•Linear systems are nice to work with because you can predict (or compute) the responses of the system relatively easily.

•For example, if you double the input, the output doubles.

•Fourier Transforms are linear operations. (The Fourier transform of the sum of two images is the sum of the Fourier transforms of each image).

•Gabor filters are linear filters.

•Neurons are not linear.

15

Threshold and Saturation

Threshold non-linearity: Neurons do not respond until the input reaches a minimum level (threshold).

Saturation non-linearity: Neurons have a maximum firing rate. The response saturates after they reach this maximum.

Res

pons

e

Input strength

Threshold Saturation

Linear response region

16

Phase and Half-wave Rectification

Phase non-linearity: Complex cells are insensitive to the phase (position) of a grating within the receptive field.Complex cells do not sum inputs within the receptive field.

Half-wave Rectification: Cortical cells have a low spontaneous firing rate. There cannot be as large a negative response as a positive response. The bottom half of the waveform is clipped off.

Res

pons

e

time

This can be alleviated with pairs of matched cells that are 180 deg out of phase with one another. The difference in responses acts like a linear response.

17

Lateral Inhibition•There is evidence that a spatial frequency channel is inhibited by other channels tuned to nearby frequencies. (Also true for orientation tuning).

•This is accomplished by lateral inhibitory connections within the cortex, known as lateral inhibition.

•This can cause interesting effects, such as repulsion of perceived orientation when 2 lines of similar orientation are shown close together.

•If you adapt 1 spatial frequency, there is an increased sensitivity at other nearby frequencies.

•Inhibitory interactions can help to make tuning curves narrower.