Krishna Prasad MiyapuramCognitive Science & Computer Science

Indian Institute of Technology Gandhinagar

Computational Methods in Neuroimaging

kprasad@iitgn.ac.in

19982000

20022004

20082011

2012

ElectronicsArtificial Intelligence

Cognitive Neuroscience

Neuroeconomics

Predictive coding

M.Tech.

M.Sc.

Journey of a thousand miles begins with a single step

Outline

• Imaging the Human Brain

• Physics of Functional MRI

• Classical analysis: Statistical Parametric Mapping

• Data Visualization

• Beyond Blobs: Functional Connectivity

• Machine Learning Methods

• Data Mining Techniques

The BIG Question

Cognitive Science

Psychology

Artificial Intelligence

Neuroscience

EducationLinguistics

Anthropology

Philosophy

What is the nature of human MIND?

The small Answer

Study the human BRAIN!

Neuroimaging Techniques

Frontal LobeParietal Lobe

Occipital Lobe

Temporal Lobe

Cerebellum

Basal ganglia

The Human Brain

Trees do not have an organ called “brain”.

AnteriorPosterior

Superior(Dorsal)

Inferior(Ventral)

Parts of the Brain

Temporal LobeOccipital Lobe

Frontal LobeParietal Lobe

Terminology – Planes and Sections

Coronal

Saggital

Axial

Axial / Horizontal Plane

Saggital Plane

Coronal Plane

3D imaging

The Nobel Prize in Physiology or Medicine for2003 jointly to

Paul C. Lauterbur Sir Peter Mansfield

"for their discoveries concerning magnetic resonance imaging“

http://www.nobel.se/medicine/laureates/2003/press.html

Sectional view of an MRI Scanner

Scanner room

Console room

Patient Table

RF (Head) coil

Gradient coil

Static magnetic field

Physics of MRI

Brain activity

Oxyhaemoglobin

Deoxyhaemoglobin

MRI signal intensity

Oxygen consumption

Cerebral blood flow

Rest (Normal blood flow)

Activation (High blood flow)

(A) (B)

(C)

Experimental Design

A B A B B A B A A B A

B BA A

BA

BA

B BA

BA

B

Input

Output

Input

Output

Process

BaselineTask condition

Data Analysis

fMRI time series

Statistical Parametric Map

Within-subject registrationslice-timing correction

RealignmentCoregistration

(structural to functional)

Between-subject registrationspatial normalization

Spatial smoothing

General Linear ModelDesign matrix

Parameter estimation

Statistical InferenceLinear Contrasts

Thresholding

Random Effects Analysis(Group analysis only)

Preprocessing

Statistical Analysis

Softwares for fMRI Analysis

Statistical Parametric Mapping

• SPM is a form of data reduction, condensing information (in a statistically meaningful way) from a number of individual scans into a single image volume that can be more easily viewed and interpreted.

SPM has an extensive web site at:http://www.fil.ion.ucl.ac.uk/spm

Image Processing

Within-subject registrationslice-timing correction

RealignmentCoregistration

(structural to functional)

Between-subject registrationspatial normalization

Spatial smoothing

Need for motion correction

• People move, even if they don’t realize!

(A) (B)

(D)(C)roll

pitch

yaw

x

y

z

Rigid body movement: 3 translation parameters

3 rotation parameters

Same location in the grid

Same location in the brain

Statistical Analysis

General Linear ModelDesign matrix

Parameter estimation

Statistical InferenceLinear Contrasts

Thresholding

Random Effects Analysis(Group analysis only)

Statistics: How?fMRI model setup

• A General Linear Model (GLM) is setup modellingthe control and test conditions as effects of interest.

y = Xb + e• The GLM is used to specify

the conditions in the form of a design matrix, which defines the experimental design and the nature of hypothesis testing to be implemented.

Specifying Contrasts– A contrast can be used to compare different

conditions in the study.

– The conditions that are of interest are given a positive value, such as 1, and conditions that are subtracted from the conditions of interest are given a negative value, such as -1.

Thresholding:During the assessment of

Results, height and extent thresholds are applied to determine significant activations.

Visualization

Glass Brain for Active-Rest Brain Slice picture for Active-Rest

Rendering onto Subject’s Anatomical Brain

• A High resolution anatomical image (dimensions: 128x128x160 , resolution 1.95 x 1.95 x 1 mm) is acquired.

• This image is Segmented into Grey, White and CSF images.

• The subjects brain is extracted from the Grey and White matter images.

• The activations can now be rendered onto 3D anatomical image of the subject

Psychophysiological Interactions

• Slides from Roland Benoit, MfD 2007/8• Data from

– C. Buchel and K. Friston. Modulation of connectivity in visual pathways by attention: Cortical interactions evaluated with structural equation modelling and fMRI, Cerebral Cortex, 7: 768-778, 1997

• Figures from – K.J. Friston, C. Buchel, G.R. Fink, J. Morris, E. Rolls, and

R. Dolan. Psychophysiological and modulatory interactions in Neuroimaging. NeuroImage, 6:218-229, 1997

– Christian Ruff’s ppt “Experimental Design”

• Tutorial: http://www.fil.ion.ucl.ac.uk/spm/data/

Functional Connectivity

Functional IntegrationFunctional Segregation

Effective ConnectivityFunctional Connectivity

Attention

V1

V5

An Example

Set

source

target

stimuli

source

target

Two Interpretations

Context-sensitive connectivity Modulation of stimulus-specific responses

How it works: Interactions

V1 X Attention

How it is done: PPI & SPM5

• Estimate GLM

• Extract time series at Region of Interest

How it is done: PPI & SPM5

3. Deconvolve, Calculate Interaction, Reconvolve

How it is done: PPI & SPM5

3. Estimate new GLM

How it works: GLM

0 0 1

V1 Att V1XAtt

z = -9 mm

Multi Voxel Pattern Analysis

Problem Statement

• Over the past decade functional Magnetic Resonance Imaging (fMRI) has emerged as a powerful technique to locate activity of human brain while engaged in a particular task or cognitive state.

• We consider the inverse problem of detecting the cognitive state of a human subject based on the fMRI data.

• f : fMRI-sequence(t1,tN) CognitiveState

Dominant Cognitive StatefMRI SCAN 1 fMRI SCAN 2 fMRI SCAN N

fMRI time series (N >= 1)

f

Definition and Motivation

• What is Cognitive State?

• It is the state of a process/operation within the human brain that affects it’s mental contents.

• Motivation :

– Such functions could provide the basis for a new approach– to study human reasoning processes.

– Also deeper understanding of the functioning of human – brain could help us build more advanced AI systems.

Visuo Motor Sequence Learning

Visuo-Motor Mapping:• Association of various visual

instructions to appropriate actions.

• -Stopping at red traffic signal• -Driving slowly at speed• breaker

Sequence Learning:• Learning a task that requires

sequencing a number of actions to achieve a goal.

• -Driving a car• -Lacing a shoe

Visual

Instruction 1Motor Action 1

Visual

Instruction 2

Visual

Instruction 3

Visual

Instruction N

Motor Action 2

Motor Action 3

Motor Action N

Visuo-Motor Mappings

S

e

q

u

e

n

c

e

L

e

a

r

n

i

n

g

Position-to-Position Mapping

P

o

s

i

t

i

o

n

S

e

q

u

e

n

c

e

1

2

1

2

1

2

1 2

1

2

1

2

Visual Display Keypad Response

Position-to-Color Mapping

P

o

s

i

t

i

o

n

S

e

q

u

e

n

c

e

2

1

1

2

2

1

1

2

2 1

2

1

Visual Display Keypad Response

Visuo-Motor Tasks

VisualStimuli

ArbitraryMapping

(Position-to-Color)

Response

P2C

P2P

VisualStimuli

Trial & Error

Response

EarlyLearningLate

Learning

Classification Problem

P2P Vs P2C: Detect the following cognitive states

– “subject is paying attention only towards the position of the visual stimuli”

– “subject is paying attention towards the position and color of the visual stimuli”

Early Vs Late Learning: Detect– the following cognitive

states– “subject has learnt the V-M

sequence”– “subject is in the early

process of learning the V-M sequence”

Machine Learning Approach

• To estimate the function f: fMRI-sequence(t1, tN) -> CognitiveState we have explored the following machine learning techniques:

Gaussian Naïve Bayes (GNB) Classifier k-Nearest Neighbor (kNN) Algorithm Support Vector Machines (SVM)

• Single-Subject Classifier: Classifiers that are trained and tested with a single subject’s fMRI data.

• Multiple-Subject Classifier: Classifiers that are trained with fMRI data of multiple subjects and tested with data of a new subject,

1. Very high dimensional data (184707 voxels/features).

2. Variation in shapes and sizes of brain across human subjects.

3. Variation in the level of fMRI activity across subjects.

Major Challenges

Feature Selection

• Select the n most discriminating voxels (Discrim) : Voxels are selected based on their ability to distinguish one target class (Cognitive States) from the other.

• – Select the n most active voxels (Active) : Voxels are selected based on their ability to distinguish either target class (Cognitive States) from the baseline condition.

• – Select the n feature pairs whose correlation discriminates the target classes (CorrPair) : Voxel pairs are selected based on the ability of their correlation to discriminate the target classes.

• We observed Poor performance of Discrim and Active features and relatively better performance of CorrPair features for multiple-subject classifiers.

20

40

60

80

t=12 t=36 t=72 t=12 t=36 t=72 t=12 t=36 t=72 t=12 t=36 t=72

GNB KNN (k=5) KNN (k=9) SVM

200

474

88

92

96

100

50 100 200 50 100 200 50 100 200 50 100 200

GNB KNN (k=5) KNN (k=9) SVM

Discrim

Active

Sin

gle

Su

bje

ctM

ult

iple

Su

bje

ct

No. of features

Classification Accuracy (%)

No. of features

Time Interval

CorrPair Feature Selection

Feature Selection

P2P Vs P2C Classification Study

97

100

40

55

70

85

50 100 200 50 100 200 50 100 200 50 100 200

GNB KNN (k=5) KNN (k=9) SVM

Discrim

Active

20

40

60

80

t=12 t=36 t=72 t=12 t=36 t=72 t=12 t=36 t=72 t=12 t=36 t=72

GNB KNN (k=5) KNN (k=9) SVM

200

445

Sin

gle

Su

bje

ctM

ult

iple

Su

bje

ct

No. of features

Classification Accuracy (%)

No. of features

Time Interval

CorrPair Feature Selection

Feature Selection

Early Vs Late Learning Study

Interim Conclusions

• The problem of detection of cognitive states in such a high dimensional feature space is feasible when right choice of features is made along with suitable methods for representation of data.

• Overall much better performance of single-subject classifiers over the multiple-subject classifiers.

• We were unsuccessful in learning a classifier function for “Four-Way Classification Study", the question that we can detect all the cognitive states is yet to be answered.

P2P

P2C

Early Late

P2P Early P2P Late

P2C Early P2C Late

Imagery Conditioning

• Imagery: Mental States like those that arise during perception but occur in the absence of immediate sensory input.

• What occurs in your mind when you see the following word

Imagery Conditioning

• Were your mental contents like this

– Has shape (round)

– Has colour (brown)

– Is type (cake)

•

Neither of the above …

Umm! It’s yummy!

Pylyshyn

Kosslyn

OR

OR

•

Experimental Paradigm

Perception Imagination

Reward

No Reward

Poisson ITI (mean 4 sec)

++

2 sec1 sec

3 sec

NothingScrambled

PictureMoney

BillNothing

Scrambled Picture

Money Bill

What did you See?

Reward Predicting Responses

Activation in Midbrain is greater for stimuli predicting reward than control stimuli irrespective of the reward being Perceived or Imagined

Predicting the imagined contents

• Support Vector Machine classifier trained on midbrain activation from visually presented trials successfully predicts whether the participant is imagining a reward or control picture

Participants

Mid

bra

in a

ctiv

atio

n (

clu

ster

ave

rage

)

Visual presentation CS+ CS-Imagination CS+ CS-

Data mining: Meta Analysis

Summary

• Computational Tools are indispensible for neuroimaging

• Classical Analysis uses a standard framework for functional localization

• We can ask questions about functional Integration (a.k.a. effective connectivity)

• Machine Learning Methods have made the reverse inference of cognitive states possible

• Further advances in Computational data mining techniques are to bring in a revolution in Neuroinformatics

Thank you

http://cogs.iitgn.ac.in

The Nobel Prize in Physiology or Medicine for2003 jointly to

Paul C. Lauterbur Sir Peter Mansfield

"for their discoveries concerning magnetic resonance imaging“

http://www.nobel.se/medicine/laureates/2003/press.html

The small Answer

Study the human BRAIN!