fMRI Methods

Lecture 12 – Adaptation & classification

Neurons

Neural computation

Neural selectivity

Hierarchy of neural processing

Integration of information

Retinal ganglion cell receptive fields

V1 neuron receptive field(Hubel & Wiesel)

Integrate

Grandma cell vs. distributed populationSparse codingNarrow selectivity

Grandma cell vs. distributed populationDistributed codingBroad selectivity

VisionComponents of visual computation:

Locations in visual field

Contrast

Orientation

Spatial frequency

Direction of motion

Categories – objects, faces, houses

Neural selectivity!

Test selectivity

Change a stimulus attribute in a controlled manner and see how the neural response changes...

Neural tuning curves

Particular neurons prefer specific stimulus attributes:

“right” neuron “left” neuron

0° 45° 90° 135° 180° 225° 270°

Similar tuning within a column

Organization of orientation selective neurons in primary visual cortex.

200-300 µm wide

Cortex is 2-4 mm thick

Each voxel = 3 mm3

40-50,000 neurons per mm3

~ 1,000,000 per voxel

Resolution

Fine for distinguishing between gross neural systems in separate brain areas (“modality selectivity”).

How can we tell what intermingled neural populations within a particular area are selective for?

Adaptation & Classification

Adaptation

Sight Touch

Sound

Smell

Short term memory? Allowing attention to novelty?

Habituation in Aplysia

Location of adaptation

Different forms of adaptation take place at different synaptic and cellular locations and have different time-scales…

There are multiple sensory neurons on the siphon, if adaptation is pre-synaptic will it generalize across locations?

Location of adaptation

Many possibilities:

Amount of neurotransmitter released.

Amount of receptors available (receptor traffic) at the post synaptic cleft.

Activity dependant ion channel changes (conductance): either pre-synaptic or post-synaptic.

Protein dependant structural synaptic changes (longer term).

Membrane conductance change

Contrast adaptation in V1 neurons – 100’s of milliseconds

Carandini et. al. Science 1997

Tester strength (% contrast)

1.5% adapt

47% adapt

1.5% adapt

47% adapt

Synapse specific adaptation

Object adaptation in IT neurons – secondsMiller et. al. Science 1994

fMRI AdaptationFirst presentation Repeat

Models of fMRI adaptation

Grill-Spector et. al. TICS 2006

Adaptation carry over to higher cortical areas?

Models of fMRI adaptation

Grill-Spector et. al. 2001

Different number of stimuli types within a block of 32 stimuli

In an event related design

Repeats Non-repeats

Selectivity, adaptation, & invariance

Grill-Spector et. al. 2001

A true test of “high level” selectivity would be invariance across “low level” changes…

Selectivity, adaptation, & invariance

Every neuron from retina up will adapt…

None will adapt…

Generic “face” neurons will adapt…

Only Individual “face” neurons will adapt…

fMRI adaptation study

Yellow: overlap of motor and visual adaptation

Mirror system areas

Dinstein et. al. 2007

Repeat Non-repeat

Classification

Classification

Typical visual area = 300 voxels

fMR

I re

spon

se

Classification

Direction selective pattern

Strong Weak

Strong Weak

How consistent are the patterns?

Trial #1

….

….

….

Trial #2

Trial #3

Trial #4

Trial #1

Trial #2

Trial #3

Trial #4

Trial #1

Trial #2

Trial #3

Trial #4

Response per trial…Y (bold) = X (model) * b (response amplitudes) + error

Y

=

Tim

e po

ints

X (column for each trial)

010000000000

000010000000

000000000100

100000000000

000100000000

000000010000

001000000000

000000001000

000000000001

… … …

b for each trial

*

b1

b2

b3

b4

000....

000b1

b2

b3

b4

00..

0000000b1

b2

b3

b4

Y

=

Tim

e po

ints

X (column for each trial)

… … …

b for each trial

*

b1

b2

b3

b4

000....

000b1

b2

b3

b4

00..

0000000b1

b2

b3

b4

Response per trial…Y (bold) = X (model) * b (response amplitudes) + error

Multivariate pattern classificationResponses of a single voxel to different direction trials

Resp

onse

Am

plitu

de

Multivariate pattern classificationResponses of two voxels to different direction trials

Res

pons

e A

mpl

itude

(vo

xel 1

)

Response Amplitude (voxel 2)

Left

Up

Right

Multivariate pattern classificationR

espo

nse

Am

plitu

de (

voxe

l 1)

Response Amplitude (voxel 2)

Left

Up

Right

Responses of two voxels to different direction trials

Multivariate pattern classificationDecode direction using brain pattern

Res

pons

e A

mpl

itude

(vo

xel 1

)

Response Amplitude (voxel 2)

Left

Up

Right

Left?

Multivariate pattern classificationDecode direction using brain pattern

Res

pons

e A

mpl

itude

(vo

xel 1

)

Response Amplitude (voxel 2)

Left

Up

Right

Multivariate pattern classificationR

espo

nse

of v

oxel

#1

Response of voxel 2

Decode trials (leave one out)

Trial #1

….

Trial #2

Trial #3

Trial #4

Trial #1

….

Trial #2

Trial #3

Trial #4

Trial #1

….

Trial #2

Trial #3

Trial #4

Decode direction trialsD

ecod

ing

accu

racy

Area MT Motor cortex

Chance

Decoding orientation in visual areas

Kamitani et. al. Nat. Neurosci 2005

Decoding orientation in visual areas

Decoding orientation in visual areas

Decoding attended stimulus

The orientation that the subject attends is evident in the distributed response pattern of V1 neurons!

Classify movement identity

Dinstein et. al. 2008

Classify movement identityD

ecod

ing

accu

racy

Central sulcus

Executed Observed

Classify movement identityD

ecod

ing

accu

racy

Early visual areas

Executed Observed

Classify movement identityD

ecod

ing

accu

racy

Anterior IPS

Executed Observed

Left Right

To the lab!