Gabor Filter: A model of visual processing in primary

visual cortex (V1)

Presented by: CHEN Wei (Rosary)

Supervisor: Dr. Richard So

AgendaVisual Physiological MechanismPrimary Visual Cortex (V1) PropertiesGabor Filter ModelImplicationsQ & A

Anatomy of the Early Visual Pathways

lightretina

optic nerve

LGN

optic radiation

primaryvisual cortex

Visual PathwayLight fall into the eye, form two images on the left and right retinaEach retina transforms the incoming light intensity distribution into spike patterns, transmitted by optic nerves to the Central nervous system (CNS)CNS target: Lateral Geniculate Nuclei(LGN) of the thalamusProceeding from the LGN, optic radiation contracts the primary visual cortex (area 17 or V1, or Striate Cortex)

Primary Visual Cortex (V1)A thin sheet of neurons The largest cortical area in primates, containing some 200 million cellsComplex visual processing occurs hereWell known area in brain

What are cells in V1 doing?

Hubel and Wiesel’s outstanding research and other studies revealed: cells in the primary visual cortex have receptive fields which are restricted to small regions of space and highly structured.

They are working as linear filters to decompose the visual image

These cells are highly orientation and spatial frequency selectively

Spatial Frequency Theory

Atomistic assumption: the representation of any image, no matter how complex, is an assemblage of many primitive spatial “atoms”------ spatially extended patterns called sinusoidal gratings

Sine Grating

•Luminance profile is described as sine wave•Important parameters:

•Amplitude contrast•Spatial frequency (cycle/degree)•Orientation•phase

Image decomposeAny 2D image can be decomposed into a unique set of sine gratings

According to Fourier’s Theorem, a complex pattern (like a face) is just the sum of a particular set of sine gratings

A Cell is a filterSachs, Nachmias (1971), H.R.Wilson (1979,1993) etc. Each cell is trying to decompose the incoming information to particular componentStudies show the stimulus is processed by mechanisms selective for both spatial frequency and orientation. The response of each mechanism is produced by linear filtering by its receptive field

Spatial frequency selective

V1 cells respond to narrowranges of stimulusspatial frequency

Orientation Selective• Similarly, V1 cells respond to narrow

ranges of stimulus Orientation

Preferred orientation: only excitation=big response

Orthogonal orientation: excitation and inhibition cancel out= no response

V1 functional architecture Receptive field: linear, weighted sum of stimulus intensities.

This model is attractive because it allows for a complete description. Measure the linear weighting function for a given cell, you can predict that cell’s response to any visual stimulus

In cats, the spatial frequency dimensionis hypothesized tobe orthogonal to the orientation dimension in aCartesian structure,whereas in monkeysit is hypothesized tobe radially organizedin a polar structure

How do we model both spatial frequency and orientationSelectivity Simultaneously??

Gabor Filter ModelA mathematical model used to describe receptive field properties of the cells in V1The receptive field structure is called a Gabor filter (or wavelet) in constructed by multiplying a global sinusoidal grating by a bell-shaped Gaussian envelope. Describe the 2D information

Elliptic Gaussian A 2D elliptic Gaussian centered on the

origin of a Cartesian coordinate system:

)}//(2/1exp{),( 2222 byaxyxw

Where, 2a is variance in the

x direction2bis variance in the y direction

Elliptic Gaussian cont. The elliptic Gaussian can be centered at any

desired spatial location traslating offset x0 and y0 and aligned in an arbitrary orientation by a rotation of angle A

)}//(2/1exp{),( 2222 byaxyxw ss

CosAyySinAxxy

SinAyyCosAxxx

s

s

)()(

)()(

00

00

Where,

Sine Plane Wave A 2D sinusoidal plane wave can be

written

)](2[)](2[),( 0000 yVxUiSinyVxUCosyxm

Where, is spatial frequencies

in the x direction

is spatial frequencies in the y direction

0U

0V

Sine Wave Plane cont.Since only the real signals (cosine term) can be observed in a experiment and we can choose some other origin for the modulation term,

)(

],)(2cos[),(

00

00

mm yVxUPWhere

PyVxUyxm

Observable form of the 2D Gabor filter in the space domain

The 2D Gabor filter can now be written as the product of an elliptic Gaussian and a sinusoidal plane wave

])(2cos[*)//(2/1exp[),( 002222 PyVxUbyaxKyxg ss

K --- scale factor (amplitude)

sample output of Gabor filters with different scales and different orientations

00, scale 1 360, scale 2 720, scale 3 1080, scale 4 1440, scale 5

The evaluation of Gabor filter

Jones and Palmer (1987) concluded that the Gabor function provides a useful and reasonably accurate description of cells in cat’s V1

In Jones & Palmer´s experiment, simple cells responses were measured with a micro electrode.

refer to Judson and Palmer, 1987, The two-Dimensional spatial structure of simple receptive fields in cat striate cortex, Journal of Neurophysiology

An example of the resulting responses

                                                                

left: experimental data in the space domain

middle: adapted Gabor filter

right: difference of the adapted filter values to the experimental data

Conclusion of the evaluation

The differences between the data and the fits were statistically indistinguishable from random error

Simple receptive fields in cat primary visual cortex are linear filters having the function form of 2D Gabor filters

00 22.50 450 67.50

900 112.50 1350 157.50

An Example: Output after Gabor Filter

ImplicationsEarly visual processing (V1) is commonly modeled with a population of visual filters selective for different orientations and spatial frequencies.Besides V1 model,we are studying more models related to computational modeling to gain insight into biological brain visual function

ImplicationsThus, we study biologically-plausible brain models, and compare the predictions resulting from model simulations to empirical measurements from living systems.The target: We are trying to build a quantitative computational model to predict the severity of visually-induced motion sickness

Thank You!

Your comments and suggestions are appreciated

Linear Filters

It is a computation which takes one sequence of numbers (the input signal) and produces a new sequence of numbers (the filtered output signal). A filter is linear means simply that

Scaling: the amplitude of the output is proportional to the amplitude of the input Superposition:when two signals are added together and fed to the filter, the filter output is the same as if one had put each signal through the filter separately and then added the outputs

Visual cortical pathways after V1

V1 forms an “internal representation”Neurons in higher cortical areas make use of the internal representation to infer what is out there in the world.30 or more secondary visual areas after V1 in the occipital lobe and in parts of the parietal and temporal lobes, for example,

MT: visual motion processing areaHandle where/what, motion processing involved in motion analysis

Data Fit algorithmIn the space domain, it was found 2D Gabor filters that fit the 2D spatial response profile of each simple cell in the least squared error sense (with a simplex algorithm), and it was shown that the residual error is devoid of spatial structure and statistically indistinguishable from random error.

Primary Visual CortexHubel and Wiesel (1959) revealed:

Simple cells: their response to complex stimuli can be predicted from their response to individual spots of light. A simple cell’s receptive field can therefore be mapped just by determining it’s response to a small spot of light at each position on the retinaComplex cells:integrating the response of many simple cellsHypercomplex cells:end-stopped simple or complex cells