functional specialization · match each cortical area to its corresponding function: v1 v2 v3 v3a...

Functional Specialization

Functional specialization:shape perception

LO

Malach et al.

V1 V2 V3V3A

/B V5 V7hV

4VO1

LO1

LO2

!0.6

!0.4

!0.2

0

0.2

0.4

0.6

0.8

Motion vs static

Kinetic boundaries vs transparent motion

Objects & faces vs scrambled

fM

RI r

espo

nse

(per

cent

mod

ulat

ion)

vs.

vs.

vs.

Faces & objects Scrambled

Motionboundaries

Transparent motion

Motion Static

fMR

I re

spon

se(%

chan

ge im

age

inte

nsit

y)

Functional properties of LO1 & LO2

Functional specialization:motion perception

MT+ (V5)

Subdividing MT+ (retinotopy)

small RF(MT)

large RF(MST)

small RF(MT)

large RF(MST)

Subdividing MT+ (ipsilateral stimulation)

Subdividing MT+

Huk, Dougherty, & Heeger, J Neurosci, 22:7195-7205, 2002

V1

V2

V3

MT

V3a

MST

Human MT and MST

Functional specialization

Match each cortical area to its corresponding function:

V1V2V3V3AV3BV4V5V7V9IPS1IPS2Etc.

MotionStereoColorTextureSegmentation, groupingRecognitionAttentionWorking memoryMental imageryDecision-makingSensorimotor integrationEtc.

Selectivity

Primary visualcortex

Columnar architecture

Columnar architecture

1 cm

1.5-2 mm

1 cm

Hubel & Wiesel

Horton & Hedley-Whyte, Philos Trans R Soc Lond B (1984)

fMRI spatial and temporal resolution

Time series of fMRI images

Time

Typical pixel size: 3mm x 3mm x 3mm

Typical frame time: 2 sec

Convention spatial resolutionVoxel size: 3 x 3 x 3 mm3

FOV: 192 x 192 x 72 mm3

24 slices in 1.5 sec

Response to annulus

Response to center

Response to periphery

100 200 300 400 500 600 700 800

100

200

300

400

500

600

vs.

Stimulus:

100 200 300 400 500 600 700 800

100

200

300

400

500

600

1.5º

2.5º

High spatial resolution fMRI(3D zoomed EPI with OVS)

Zoomed field of view

Voxel size: 0.7 x 0.7 x 0.7 mm3

FOV: 157 x 39 x 21 mm3

30 slices in 3 sec

100 200 300 400 500 600 700 800

100

200

300

400

500

600

1.5º

2.5º

Reliability

100 200 300 400 500 600 700 800

100

200

300

400

500

600

Annulus

Center & periphery

Neurovascular coupling limits spatial

specificity

I

II

III

IV

V

VI

Reina de la Torre et al Anatomical Record (1998)

3.5 mm full-width at half-max (Engel el et al., 1997)

Spatial specificity (spread and mislocalization) depends on vein size.

Sluggishness of hemodynamics limits

temporal specificity

fMR

I re

spon

se(%

chan

ge im

age

inte

nsit

y)

Time (s)

0 2 4 6 8 10 12 14 16

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

Brief pulse of neural activity

Columns with fMRI: proof of principle

Cheng, Waggoner, & Tanaka, Neuron (2001)

Voxel size: 0.5 x 0.5 x 3 mm3

FOV: 240 x 240 x 9 mm3

3 slices in 9.6 sec

Same session

Different sessions

Adaptation

Parallel trials

Orthogonal trials

Blank trials

Adapter orientation Probe orientation

blank screen

Adapter 4 s

Probe 1 s

ISI 1 s

Response 1.2 s

how many X’s?

Unattendedperifoveal stimulus

(1.5-5 deg)

Attendedfoveal RSVP task

(<1 deg)

A

Parallel trials

Orthogonal trials

Blank trials

Adapter orientation Probe orientation

blank screen

B

Targets presented

11

2

2

3

3

4

4

Targ

ets

dete

cted

Subject S1Subject S2Subject S3

C

Trial types

Trial structure

Orientation-selective adaptation protocol

Adaptationmagnitude

Time (s)

Orthogonal trialsParallel trials

FM

RI r

espo

nse

(% m

odul

atio

n)

Adapter Probe

0 5 10 15

0

-0.2

0.2

0.4

0.6

0.8Parallel

Orthogonal

Adapter ProbeTrial type

Orientation-selectivity in human V1

Larsson, Landy, & Heeger,

J Neurophysiol (2006)

Time (sec)

fMR

I re

spon

se(%

chan

ge int

ensi

ty)

Adaptation index:

fMR

I re

spon

se a

mpl

itud

e

Visual area

Orientation-selective adaptation

A

A



Adaptation index

Visual area

Ada

ptat

ion

inde

x

V1 V2 V3 hV4 VO1 LO1 V3A/B0

0.2

0.4

0.6

0.8 LM:LM• Adaptation indices

constant across visual areas

• No significant differences between V1 and extrastriate visual areas

• Adaptation in V1 can account for adaptation in extrastriate visual areas



Visual area

MT+ V1 V2 V3 V4v V3A0

0.25

0.5

Huk, Ress, & Heeger (2001) Neuron, 32:161-

Dir

ecti

on-s

elec

tive

ad

apta

tion

ind

ex

Direction-selective adaptation

+ =

Pattern motion

component-motion cell

=

pattern-motion cell

pattern moving up-right strong response

=

grating component moving up-right => strong response

Component vs. pattern motion selectivity

Component gratings

Adapted

direction

plaids

Mixed

direction

plaids

Component gratings

Pattern motion adaptation protocol

Huk & Heeger, Nature Neurosci, 5:72-75, 2002


Pattern motion adaptation

74 nature neuroscience • volume 5 no 1 • january 2002

articles

tiples of 90° from trial to trial to minimize adaptation. Theresponse modulations in MT+ were not significantly differentfrom zero (A.C.H., p = 0.35; D.J.H., p = 0.74, two-tailed t-test),demonstrating that the two blocks of plaids elicited similarresponse levels when the effects of both component- and pat-tern-motion adaptation were absent. This result provides furtherevidence that adaptation, due to the repetition of pattern direc-tion, was the key factor in the original experiment.

DISCUSSIONOur findings demonstrate that human MT+ contains a popula-tion of pattern-motion cells and that the activity of those neu-rons is linked to the perception of coherent pattern motion. Thepattern-motion responsivity of human MT+ adds to the case fora homology to macaque MT, which includes a relatively largeproportion of pattern-motion cells1. We also observed lesserdegrees of pattern-motion adaptation in V2, V3, V3A and V4v.Macaque V3 is known to have a minority of pattern-motioncells13, but there are no published investigations of pattern-motion cells in macaque V2, V3A or V4. Although our datademonstrate pattern-motion responses in each of these visualareas, we cannot determine if pattern motion is computed sepa-rately in each visual area or if the responses in V2–V4 are affect-ed by the adaptation that is taking place in MT+. We emphasizethat fMRI adaptation studies14–17 can reveal the selectivities ofsubpopulations of neurons in the human brain, even when thoseneurons are intermingled at a spatial scale that is finer than thespatial sampling resolution (voxel size) of the fMRI measure-ments.

METHODSWe collected fMRI data in 3 subjects, males, 25–39 years old, all withnormal or corrected-to-normal vision. Experiments were undertakenwith the written consent of each subject, and in compliance with the safe-ty guidelines for MR research. Each subject participated in several scan-ning sessions: one to obtain a high-resolution anatomical volume, oneto identify MT+, one to identify the retinotopically organized corticalvisual areas, 2–3 to measure motion adaptation, one to measure base-line responses and 1–3 to perform control measurements. In each subject,we collected 8–20 repeats of the pattern-motion adaptation experimentand 8–16 repeats of the various control experiments.

Stimulus and protocol. Stimuli were presented on a flat-panel display (NEC,multisynch LCD 2000, Itasca, Illinois) placed within a Faraday box with aconducting glass front, positioned near the subjects’ feet. Subjects lay ontheir backs in the MR scanner and viewed the display through binoculars.

Subjects viewed a pair of circular patches 12° in diameter centered 7.5°to the left and right of a central fixation point. Patches were filled witha plaid stimulus comprised of two superimposed sinusoidal gratings.Individual component gratings had 20% contrast, and spatial and tem-poral frequencies were selected to yield a variety of pattern directionswhen superimposed in various combinations (Fig. 1).

Each scan consisted of 6 (32-s) cycles; each cycle consisted of alter-nating adapted-direction and mixed-direction blocks. Adapted-direc-tion blocks consisted of 8 consecutive trials in which the plaid stimulusalways appeared to move in the same direction (horizontally, at 12.9 or1.9°/s; Fig. 1a); mixed-direction blocks consisted of 8 trials in which thedirection of the plaids varied from trial-to-trial (possible plaid direc-tions, computed from the intersection-of-constraints of the componentgratings, were ±31°, ±123° from horizontal at 5.3°/s and 6.3°/s; Fig. 1b).The component gratings with orientations of ±72° had spatial frequen-cies of 0.5 cycles/degree and temporal frequencies of 2 cycles/second (Fig. 1a, components above first plaid); the component gratings withorientations ±45° had spatial frequencies of 0.5 cycles/degree and tem-poral frequencies of 0.67 cycles/second (Fig. 1a, components above sec-ond plaid). In the component-motion experiment, perceptualtransparency was achieved by scaling one component’s spatial frequencyup to 1 cycle/degree and the other down to 0.125 cycle/degree, producinga 3-octave separation. (Temporal frequencies were also scaled accord-ingly to leave component velocities unchanged.)

To control attention, subjects performed a speed discrimination judg-ment on each stimulus presentation16. Each 2-s trial consisted of 1300 ms of plaid motion followed by a 700-ms luminance-matched blankperiod during which subjects pressed a button to indicate which plaid(left or right of fixation) moved faster. The speed differences were deter-mined by an adaptive staircase procedure, adjusting the speeds from trialto trial so that subjects would be approximately 80% correct.

Across different blocks and experiments, we chose to equate percent-correct performance, instead of the exact stimulus speed (although speedsdid remain within a few percent), because in previous work, we and oth-ers have noted large attentional effects on MT+ responses, but no effectsof slight differences in speed18–20. Although the speed discriminationthresholds were larger for non-coherent (transparent gratings) than forcoherent plaids (but not for mixed- versus adapted-direction blocks),the differences were not very large (percent speed-increment thresholdswere !15% versus !10% for non-coherent versus coherent, respective-ly). These small speed differences might affect the responses of someindividual neurons (although speed tuning curves of all direction-selec-tive cells are rather broad), but these speed differences would not beexpected to evoke measurable changes in the pooled activity (as mea-sured with fMRI) of large populations of neurons.

In the adaptation experiments, equal numbers of scans were collectedwith the plaids moving in opposite directions (for example, inward

0 16 32–0.5

0

0.5

fMR

I res

pons

e(p

erce

nt B

OLD

sig

nal)

Pattern motionComponent motion

Adapteddirection

Mixeddirection

Visual area

0

0.1

0.2

V1 V2 V3 V4v V3A MT+

Pat

tern

ada

ptat

ion

inde

x(a

dapt

atio

n re

spon

se/b

asel

ine

resp

onse

)Pattern motion

Component motion

Time (s)

Fig. 2. Pattern-motion adaptation in human visual cortex. (a) Averagetime series in MT+. Pattern-motion adaptation produced strong modu-lations in MT+ activity. Transparent component motion evoked muchless adaptation. Each trace represents the average MT+ response, aver-aged across subjects and scanning sessions. (b) Adaptation index acrossall visual areas. Pattern-motion adaptation was largest in MT+, but alsoevident in other extrastriate visual areas. Adaptation was weak androughly equal across visual areas in the transparent component-motionexperiment. Height of bars, geometric mean across subjects (arithmeticmean yielded similar results). Error bars, bootstrap estimates of the 68%confidence intervals.

a

b

©20

01 N

atur

e P

ublis

hing

Gro

up h

ttp:

//neu

rosc

i.nat

ure.

com

© 2001 Nature Publishing Group http://neurosci.nature.com

Visual area

Dir

ecti

on-s

elec

tive

ad

apta

tion

ind

exCoherent (pattern) motion

Transparent (component) motion

+ =

strongpattern-motion

percept

weakpattern-motion

percept

+ =

Coherent vs. transparent percepts


fMRI

response (

%)

Time (sec)

7 14

.4

0

150°

30°

60°

90°

120°

180°

0°

No Adaptor

Direction difference (°)

|Test - Adaptor|

Lee, Lee, & Lee, SFN, 2005

Adaptation vs motion direction

Test direction (°)0 180-180

Motion direction (°)0 180-180

Before

After

Response gain reduction

Response Gain


fMRI

responseNeural response

Response Gain

Bandwidth



Before

After

Bandwidth reduction


fMRI


Response Gain

Bandwidth

Direction Tuning



Before

After

Attractive shift

Attractive shift


fMRI


Response Gain

Bandwidth

Direction Tuning

fMRI

response



Neural response

Before

After

Repulsive shift

Repulsive shift


V1

V2

V3

MT

MST

0 37.5 75.0 112.5 150.0

Vis

ual ar

ea

Est. tuning bandwidth (deg)

35.0°

52.1°

57.4°

63.7°

117.0°

Estimated tuning bandwidth


Response gainreduction

Response gainreduction and attractive shift

Classification

Kamitani & Tong, Nat Neurosci (2005)

Classifying stimulus orientation with

conventional resolution fMRI

Classifying orientation

Clockwise tilt

Counter-clockwise tilt

Stimulus

Classifier weights

Justin Gardner

Both

Haynes & Rees, Curr Biol (2005)

Classifying

percepts

during

binocular

rivalry

Time

Vox

el

Percept

Haxby et al, Science (2001)

Cor

rela

tion

Object-category classification

How the human brain interacts with the

world in real life

Simple sensory stimuli:

The full complexity of real life:

Do we share the same conscious

experience from the same sensory

stimulus?

Hasson et al., Science (2004)

“Mind reading” competition at Human Brain Mapping 2006(http://www.ebc.pitt.edu/competition.html)

fMRI responses to video from 3 segments of the Home Improvement TV series, rated for a variety of features (e.g., faces, emotions).

Utilize data from segments 1 and 2 to train classifier, then generate predicted behavior ratings for segment 3.

Brain activity interpretation competition

http://www.ebc.pitt.edu/competition.html

http://www.ebc.pitt.edu/competition.html

Classifying spatial patterns of brain activity with machine learningmethods: Application to lie detection

C. Davatzikos,a,* K. Ruparel,b Y. Fan,a D.G. Shen,a M. Acharyya,a J.W. Loughead,b

R.C. Gur,b and D.D. Langlebenb,c

aDepartment of Radiology, University of Pennsylvania, 3600 Market Street, Suite 380, Philadelphia, PA 19104, USAbDepartment of Psychiatry, University of Pennsylvania, 3400 Spruce Street, Philadelphia, PA 19104, USAcTreatment Research Center, University of Pennsylvania, 3900 Chestnut Street, Philadelphia, PA 19104, USA

Received 1 March 2005; revised 15 June 2005; accepted 4 August 2005

Available online 5 October 2005

Patterns of brain activity during deception have recently been

characterized with fMRI on the multi-subject average group level.

The clinical value of fMRI in lie detection will be determined by the

ability to detect deception in individual subjects, rather than group

averages. High-dimensional non-linear pattern classification methods

applied to functional magnetic resonance (fMRI) images were used to

discriminate between the spatial patterns of brain activity associated

with lie and truth. In 22 participants performing a forced-choice

deception task, 99% of the true and false responses were discriminated

correctly. Predictive accuracy, assessed by cross-validation in partic-

ipants not included in training, was 88%. The results demonstrate the

potential of non-linear machine learning techniques in lie detection and

other possible clinical applications of fMRI in individual subjects, and

indicate that accurate clinical tests could be based on measurements of

brain function with fMRI.

D 2005 Elsevier Inc. All rights reserved.

Introduction

A large body of functional neuroimaging literature has

elucidated relationships between structure and function, as wellas functional activity patterns during a variety of functionalactivation paradigms. Statistical parametric mapping (SPM)

(Friston et al., 1995) has played a fundamental role in thesestudies, by departing from the conventional biased ROI- andhypothesis-based methods of data analysis and enabling unbiased

voxel-by-voxel examination of all brain regions. While a great dealof knowledge has been gained during the past decade regardingbrain regions that are activated during various tasks using voxel-

based SPM analysis, the quantitative characterization of entirespatio-temporal patterns of brain activity, as opposed to voxel byvoxel examination, has received much less attention, especially asa means for deducing ‘‘the state of the mind’’ from functional

imaging data. The important distinction between a voxel-basedanalysis and the analysis of a spatio-temporal pattern is the same asthe distinction between (mass) uni-variate and multi-variate

analysis (Davatzikos, 2004). Specifically, a pattern of brain activityis not only a collection of active voxels, but carries with itcorrelations among different voxels. Notable efforts towards thefunctional activity pattern analysis have been made (Strother et al.,

1995; McIntosh et al., 1996), some of which, attempt to use thesemethods to classify complex activation patterns using machinelearning methods (Cox and Savoy, 2003; LaConte et al., 2005).

In this paper, we present an approach to the problem ofidentifying patterns of functional activity, by using a high-dimensional non-linear pattern classification method. We apply

this approach to one of the long-standing challenges in appliedpsychophysiology, namely lie detection. Deception is a sociallyand legally important behavior. The limitations of the specificityof the currently available physiological methods of lie detection

prompted the exploration of alternative methods based on thecorrelates of the central nervous system activity, such as EEG andfMRI (Rosenfeld, 2001; Spence et al., 2001; Langleben et al.,

2002). Using SPM-based analyses of multi-subject average groupdata, several recent fMRI studies demonstrated differences inbrain activation between truthful and non-truthful responses in

various experimental paradigms (Langleben et al., 2002; Langle-ben et al., in press; Ganis et al., 2003; Kozel et al., 2004a,b; Leeet al., 2002). In order to translate these data into a clinically

relevant application, discrimination between lie and truth has tobe achieved at the level of single participants and single trials(Kozel et al., 2004b), not just via group analysis. The potential ofthe SPM-based approach to achieve this goal is limited due to the

between-subject variability of regional brain activity. In thecurrent work, we have overcome this limitation using a multi-variate non-linear high-dimensionality pattern classification tech-

nique (Lao et al., 2004) applied to spatial patterns of brainactivation recorded via fMRI. Using data acquired with apreviously reported formal deception paradigm (Langleben et

al., in press), we have tested the hypothesis that truthful and non-

1053-8119/$ - see front matter D 2005 Elsevier Inc. All rights reserved.

doi:10.1016/j.neuroimage.2005.08.009

* Corresponding author. Fax: +1 215 614 0266.

E-mail address: [email protected] (C. Davatzikos).

Available online on ScienceDirect (www.sciencedirect.com).

www.elsevier.com/locate/ynimg

NeuroImage 28 (2005) 663 – 668

•Claim 88% accuracy

•Technology underlying commercial venture (http://www.noliemri.com/)

Neuroscience-based lie detector

http://www.noliemri.com

http://www.noliemri.com

functional specialization · match each cortical area to its corresponding function: v1 v2 v3 v3a...

Documents