Multimodal Brain Imaging

Will D. Penny FIL, London

Guillaume Flandin, CEA, ParisNelson Trujillo-Barreto, CNC, Havana

ExperimentalManipulation

Neuronal Activity

MEG,EEGOpticalImaging

PETfMRI

Single/multi-unitrecordings

Spatialconvolution via Maxwell’sequations

Temporal convolutionvia Hemodynamic/Balloon models

FORWARD MODELS

Sensorimotor MemoryLanguageEmotionSocial cognition

ExperimentalManipulation

Neuronal Activity

MEG,EEG

fMRI

Spatialdeconvolution via beamformers

Temporal deconvolutionvia model fitting/inversion

INVERSION

1. Spatio-temporal deconvolution

2. Probabilistic treatment

OverviewOverview

Spatio-temporal deconvolution for M/EEG

Spatio-temporal deconvolution for fMRI

Towards models for multimodal imaging

Spatio-temporal deconvolution for M/EEG

Add temporal constraints in the form of a General Linear Model to describe the temporal evolution of the signal.

Puts M/EEG analysis into same framework as PET/fMRIanalysis.

Work with Nelson. Described in chapter of new SPMbook.

1

1

1

:, ,

:, ,

1,: ,

tM T M G G T M T

TtG T K T G K G T

TT TkK G K G

k

t N

t N

k N

Y K J E e E 0 Ω

J X W Z z Z 0 Λ

W β β Β 0 D D

, 1, ,

, 1, ,

m

g

diag m M

diag g G

Ω

Λ

,g gg Ga b c ,m Ga b c ,k Ga b c

Generative Model:

Hyperpriors:

Variational Bayes: Mean-Field Variational Bayes: Mean-Field ApproximationApproximation

Repeat

• Update source estimates, q(j)• Update regression coefficients, q(w)• Update spatial precisions, q()• Update temporal precisions, q()• Update sensor precisions, q()

Until change in F is small

L

F

KL

1 2

, 0

Pp q q q

L M F KL q p KL

θ v v v v

v θ v θ v

1 11

, , , ,G

T g Kg

q q q q q q

θ j j w Ω Λ

11

, ,T

T tt

q q

j j j ˆ ˆ,tt tq N jj j Σ

ˆˆ ,gg gq N ww w Σ 1

1

, ,G

G gg

q q

w w w

1

G

gg

q q

Λ

1

M

mm

q q

Ω ˆ ˆ,m post postq Ga b c

ˆ ˆ,g post postq Ga b c

11

, ,K

K kk

q q

ˆ ˆ,kq Ga b c

Mean-Field Approximation:

Approximated posteriors:

1ˆ ˆ ˆ ˆ ˆ ˆT T T Tt t t

j K ΩK Λ K Ωy ΛW x

ˆTtyK Ω

ˆˆ T TtxW Λ

Corr(R3,R4)=0.47

1.00 0.16 0.09 0.11

0.16 1.00 0.29

0.09 0.86 1.00

0.11 0.29 0.47 1.

0.86

0.4

00

7Corr

o

+

500ms

LowSymmetry

LowAsymmetry

HighSymmetry

HighAsymmetry

Phase 1

Time

600ms

+700ms

+

o

2456ms

+

Fa

+

Sb

Ub

+

Sa

Henson R. et al., Cerebral Cortex, 2005

B8

A1 Faces minus Scrambled Faces

170ms post-stimulus

B8 A1

Faces

Scrambled Faces

Daubechies Cubic Splines

Wavelets

28 Basis Functions 30 Basis Functions

Daubechies-4

ERP Faces

ERPScrambled

t = 170 ms

t = 170 ms

Faces – Scrambled faces: Difference of absolute values

Spatio-temporal deconvolution for fMRI

Temporal evolution is described by GLM in the usual way.

Add spatial constraints on regression coefficients in the form of a spatial basis set eg. spatial wavelets.

Automatically select the appropriate basis subset using a mixture prior which switches off irrelevant bases.

Embed this in a probabilistic model.

Work with Guillaume. To appear in Neuroimage very soon.

Spatial Model eg. Wavelets

Mixture prior on wavelet coefficients

(1) Wavelet switches: d=1 if coefficient is ON. Occurs with probability (2) If switch is on, draw z from the fat Gaussian.

Probabilistic Generative Model

fMRI data

General LinearModel

Waveletcoefficients

TemporalModel

Spatial Model

Waveletswitches

Switchpriors

Compare to (i) GMRF prior used in M/EEG and (ii) no prior

Inversion using wavelet priors is faster than using standard EEG priors

Results on face fMRI data

Towards multimodal imaging

Use simultaneous EEG- fMRI to identify relationship Between EEG and BOLD (MMN and Flicker paradigms)

EEG is compromised -> artifact removal

Testing the `heuristic’

Start work on specifying generative models

Ongoing work with Felix Blankenburg and James Kilner

fMRI results

fMRI results

We have “synchronized sEEG-fMRI” – MR clock triggers both fMRI and EEG acquisition; after each trigger we get 1 slice of fMRI and 65ms worth of EEG. Synchronisation makes removal of GA artefact easier

MRI Gradient artefact removal from EEG

Ballistocardiogram removal

Could identify QRS complex from ECG to set up a ‘BCG window’ for subsequent processing

Ballistocardiogram removal

Ballistocardiogram removal

The EEG-BOLD heuristic (Kilner, Mattout, Henson & Friston) contends that increases in average EEG frequency predict BOLD activation.

g(w) = spectral density

Testing the heuristic

RMSF for Marta’s data at Cz

Log of Bayes factor for Heuristic versus Null

Log of Bayes factor for Heuristic versus Alpha

Tentative probabilistic generative model

THANK-YOU FOR

YOUR ATTENTION !