1

Data Analysis for fMRI

Computational Analyses of Brain Imaging

CALD 10-731 and Psychology 85-735

Tom M. Mitchell and Marcel Just

January 15, 2003

2

Ten Minutes of Activity for One VoxelIndicates

experimental condition

3

4

…

5

fMRI Data Visualization[from W. Schneider]

Slice View

Time Series

3 D view

Rendered View

Inflated View

6

Many Types of Analysis

• Transformation from fourier space into spatial images, adjusting for head motion, noise, drift,... (FIASCO, SVM)

• Warping individual brains to canonical structure (Talairach, AIR, SPM)

• Identifying voxels activated during task (t-test, F-test,…)• Finding temporally correlated voxels (clustering)• Factoring signal into few components (PCA, ICA)• Modeling temporal evolution of activity (diffeqs, HMMs)• Learning classifiers to detect cognitive states (Bayes, SVM)• Modeling higher cognitive processes (4CAPS, ACT-R) • Combining fMRI with ERP, behavioral data, …

7

Identifying Voxels Activated During Task

For each voxel, vi, calculate t statistic comparing activity of vi during task versus rest condition.

Retain voxels with t-statistic above some threshold

8

Mental Rotation of Imagined Objects

Clock rotation

Shephard-Metz rotation

both

[Just, et al., 2001]

9

“Men listen with only one side of their brains, while women use both”

(IU School of Medicine Department of Radiology)

Men listening

Women listening

Study of Men and Women Listening

10

Identifying Voxels with Similar Time Courses

(functional connectivity)

11

The activation in two cortical areas (parietal/dorsal and inferior temporal/ventral) becomes more synchronized as the object recognition task becomes more difficult.

Easier

Harder

Increase in functional connectivity between parietal and inferior temporal areas with workload

(from Diwadkar, Carpenter, & Just, 2001)

12

Factoring fMRI Signals into Fewer Components

PCA, ICA, SVD, Hidden Units

13

Independent Component Analysis of fMRI time-series

• ICA discovers statistically independent components that combine to form the observed fMRI signal

• ICA is a data-driven approach, complementary to hypothesis-driven methods (e.g. GLM) for analyzing fMRI data

• Finds reduced dimensionality descriptions of poorly understood, high dimensional spaces

• Requires no a-priori knowledge about hemodynamics, noise models, time-courses of subject stimuli,…

14

(McKeown et al., 1998)

Independent Component Analysisof fMRI time-series: data-model

15

fMRI time-series

ICA

algorithm

.

.

. .......

IC #1

IC #2

IC #T

Independent Component Analysisof fMRI Time-series

[from W. Schneider]

16

ICA Solution

IC1

IC2

Independent Component Analysis of fMRI time-series

GLM Solution

Images Elia Formisano & Rainer Goebel 2001

17

Advantages of ICA

• Interpretation of non-explicit condition manipulation – Not just AB type designs– Applications driving, reading, problem solving

• Identify dimensions of poorly understood spaces– Reduce high dimension data to few components– Applications: structure of semantic memory, processes

underlying visual scene analysis in visual cortex

18

Learning Classifiers to Decode Cognitive States from fMRI

Bayes classifiers, SVM’s, kNN, …

19

Study 1: Word Categories

• Family members

• Occupations

• Tools

• Kitchen items

• Dwellings

• Building parts

• 4 legged animals• Fish• Trees• Flowers• Fruits• Vegetables

[Francisco Pereira et al.]

20

Training Classifier for Word CategoriesLearn fMRI(t) word-category(t)

– fMRI(t) = 8470 to 11,136 voxels, depending on subject

Feature selection: Select n voxels– Best single-voxel classifiers

– Strongest contrast between fixation and some word category

– Strongest contrast, spread equally over ROI’s

– Randomly

Training method:– train ten single-subect classifiers

– Gaussian Naïve Bayes P(fMRI(t) | word-category)

21

Results

Classifier outputs ranked list of classesEvaluate by the fraction of classes ranked ahead of true class

0=perfect, 0.5=random, 1.0 unbelievably poor

22

Impact of Feature Selection

23

24

Summary

• Able to classify instantaneous cognitive state– in contrast to describing average activity over time

• Significance– Virtual sensors for mental states– Step toward modeling sequential cognitive processes?– Potential clinical applications: diagnosis = classification

25

Modeling temporal evolution of activity

HMMs, Diffeqs, …

26

Challenge: learn process model -- HMM’s?

a=6,… 3x+a=2

recall correct

recall error

answer

transform correct

transform error

read problem

time

start

…

27

V1

V4

BA37

STG

BA39Perturbing inputs

Stimuli-bound u1(t){e.g. visual words}

DCM [Friston 2002]Aim: Functional integration and the modulation of specific pathways

y

y y

y

y

Contextual inputsStimulus-free -

u2(t){e.g. cognitive set/time}

28

yy

y

Hemodynamic model

},,{ CBA

The DCM and its bilinear approximation [Friston 02]

Inputu(t)

activityx1(t)

activityx3(t)

activityx2(t)

23b

12a

1c

The bilinear model

nnnnn

n

nnn

n

n c

c

u

x

x

bb

bb

u

aa

aa

x

x

11

1

111

1

1111

neuronalchanges

intrinsicconnectivity

inducedresponse

inducedconnectivity

CuxBuAxj

jj )(

))(()( txhty

29

Constraints on•Connections

•Hemodynamic parameters

Applications•Simulations

•Plasticity in single word processing•Attentional modulation of coupling

Models of•Hemodynamics in a single region

•Neuronal interactions

Overview[Friston, 2002]

Bayesian estimation

)(p

)()|()|( pypyp

)|( yp

30

Cognitive Models Grounded in fMRI Data

31

4CAPS Model of Language Processing[Just, et al., 2002]

32

The player was followed by the parent.

[Just, et al., 2002]

334

4CAPS Prediction of fMRI Activity

Figure 10

Model CU

transform

CU in 4CAPS comprehension model components

fMRI

data

Modelprediction

34

[Anderson, Qin,& Sohn, 2002]

35

Cognitive model:

Observed image

sequence:

See word

Recognize word

Understand statement

Answer question

Hypothesized intermediate

states, representations,

processes:

time

Understand question

What We’d Like

36

Machine Learning Problems

• Learn f: image(t) cognitiveState(t)

• Discover useful intermediate abstractions

• Learn process models

6

Cognitive model:

Observed image

sequence:

See word

Recognize word

Understand statement

Answer question

Hypothesized intermediate

states, representations,

processes:

time

Understand question

What We’d Like