symmetrical eeg–fmri imaging by sparse...

Symmetrical EEG–fMRI imaging by sparse regularization

Thomas OberlinChristian Barillot, Rémi Gribonval, Pierre Maurel

HEMISFER ProjectPANAMA and VISAGES teams

Inria Rennes

EUSIPCO’15 – Nice, France

T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 1 / 21

Introduction

Introduction : EEG–fMRI

EEG� Measure of the synchronous electrical activity of groups of neurons (pyramidal cells)� Instantaneous measures, perfect temporal resolution but poor spatial information� Easy-to-use, cheap technique

fMRI (through BOLD signal)� BOLD (Blood-Oxygen Level Dependent) measures the hemodynamic activity� High spatial resolution� More involved and expensive technique (non-invasive, though)

Aims :� Coupling both modalities to achieve a high spatio-temporal resolution� Perspectives : used it for Neurofeedback and rehabilitation (Hemisfer project)


Introduction

EEG–fMRI coupling : quick review

Acquisition� Sequential : no artefacts, reproducibility issues� Simultaneous : need a strong pre-processing step a

a. Laufs et al., "Recent advances in recording electrophysiological data simultaneously with mag-netic resonance imaging", Neuroimage, 2008

Types of coupling� Asymmetrical (most of the studies) : one modality is used as a prior to inform theother. Examples : fMRI-constrained EEG a, EEG-augmented fMRI b.

� Symmetrical (some recent attempts) : uses equally both modalities. Tools : bayesianfusion c, Kalman-like filtering d. Main difficulty : complexity of the neurovascularcoupling

a. Liu et al., "fMRI–EEG integrated cortical source imaging by use of time-variant spatial con-straints", Neuroimage, 2008

b. De Munck et al., "Interactions between different EEG frequency bands and their effect onalpha–fMRI correlations", Neuroimage, 2009

c. Daunizeau et al., "Symmetrical event-related EEG/fMRI information fusion in a variationalBayesian framework", Neuroimage, 2007

d. Deneux et al., "EEG-fMRI fusion of paradigm-free activity using Kalman filtering", Neuralcomputation, 2010


Introduction

Aims and objectives

Solving the inverse problem� Simultaneous and symmetrical coupling of EEG and fMRI� Formulation : sparsity-penalized regression already used in EEG/MEG a b c

� Optimization through proximal alorithms d

a. Gaudes et al., "Structured sparse deconvolution for paradigm free mapping of functional MRIdata", ISBI’2012

b. Ou et al., "A distributed spatio-temporal EEG/MEG inverse solver", Neuroimage, 2009c. Gramfort et al., "Mixed-norm estimates for the M/EEG inverse problem using accelerated

gradient methods", IPMI’2011d. Combettes et al., "Proximal splitting methods in signal processing", Springer, 2011

Next step : (dictionnary) learning, application to Neurofeedback� Using this high resolution imaging modality for learning models, dictionnaries,patterns, etc

� Apply it to EEG–only neurofeedback with a pre-training with EEG–fMRI.Applications : brain rehabilitation for depression, strokes, etc


Outline

Outline

1 Introduction

2 The neurovascular couplingDistributed source model in EEGModeling the BOLD signal in fMRISolving the inverse problem

3 OptimizationForward-backward algorithmChoice of penalty

4 Numerical resultsModel and simulationsContribution of the couplingChoice of parametersChoice of the penalizationRobustness to BOLD false positives/negatives

5 Conclusion


The neurovascular coupling

Outline

1 Introduction

2 The neurovascular couplingDistributed source model in EEGModeling the BOLD signal in fMRISolving the inverse problem

3 Optimization

4 Numerical results

5 Conclusion


The neurovascular coupling Distributed source model in EEG

Distributed source model in EEG

Neuronal activity X ≡ S dipoles(sources) distributed on the corticalsurface, in the normal direction

Measures E : a linear mixing of the sources

E = GX + NE , (1)

where� X ∈ RS×T is the activity of the sources (unknown)

� E ∈ RN×T is the electrical activity measured on the scalp (known)� G is the leadfield mixing matrix N × S (known)

� The noise term NE ∈ RN×T is Gaussian (with estimated covariance matrix)

S ≈ 10000 sources, N ≈ 100 electrodes, T ≈ 1000 time instants


The neurovascular coupling Modeling the BOLD signal in fMRI

Modeling the BOLD signal in fMRI

� BOLD signal F : linked to the neuronal activity X by a cascade of highly complexphysiological processes� Modeling of the coupling : "balloon model" 1

� The linearization of this model 2 leads to a convolutive model

Yi(t) = (Xi ? h)(t), (2)

where h is the standard hemodynamical response function (HRF)� Global formulation (for all voxels) :

F = Q(XH) + NF ,

whereF ∈ RS×U (U ≈ 20 number of time instants)Hij = h(ti − uj ) is the linear operator associated to the convolution in timeQ is an interpolation operatorNF is the noise term, which is still supposed to be Gaussian, with known covariance

1. Friston et al., "Nonlinear responses in fMRI : the Balloon model, Volterra kernels, and otherhemodynamics", Neuroimage, 2000

2. Robinson et al., "Bold responses to stimuli : dependence on frequency, stimulus form, ampli-tude, and repetition rate", Neuroimage, 2006


The neurovascular coupling Solving the inverse problem

Solving the inverse problem

ModelingWe suppose that the BOLD signal is measured at the same position of the sources, whichgives : {

E = GX + NEF = XH + NF .

(3)

Solving the inverse problemMinimizing a data-fit term and a regularization term :

X∗ = argmin(α

2 ‖E − GX‖2F +1− α2 ‖F − XH‖2F + λφ(X)

), (4)

� α tunes the tradeoff between each modality� λ controls the regularization term

Numerical solution through proximal iterative algorithms.


Optimization

Plan

1 Introduction

2 The neurovascular coupling

3 OptimizationForward-backward algorithmChoice of penalty

4 Numerical results

5 Conclusion


Optimization Forward-backward algorithm

Implementing the optimization

Proximal gradient algorithm (aka "forward-backward") a b

a. Daubechies et al., "An iterative thresholding algorithm for linear inverse problems with asparsity constraint", Com. Pure App. Math., 2004

b. Combettes et al., "Signal recovery by proximal forward-backward splitting", MMS, 2005

� Minimize f (x) + φ(x) with f diff., L-Lipschitz gradient, φ proper and convex� Here f is quadratic with L ≤ α ‖G∗G‖+ (1− α) ‖HH∗‖ (‖.‖spectral norm)

� At iteration k, replacing f (X) by a quadratic majorant ∇f (Xk)T (X − Xk) + L2 ‖X − Xk‖2F :

Xk+1 = argmin12

∥∥∥X −(

Xk −1L∇f (Xk)

)∥∥∥2F+λ

Lφ(X). (5)

� A unique solution in terms of proximal operator proxφ(X) = arg minY∈Rn 12 ‖X − Y‖2F + φ(Y ) :

Xk+1 = prox λL φ

(Xk −1L∇f (Xk)), (6)

� Special kind of Majorization/Minimization algorithms� Asymptotic linear convergence� The only difficult step : compute the proximal operator.� At each step, only a few matrix/matrix multiplications


Optimization Choice of penalty

Choice of penalty

`1-norm (Lasso) φ(X) = ‖X‖1 =∑

i,j |Xij | : soft thresholding (ST)

[proxµ‖.‖1(x)]j = sign(xj)(|xj | − µ)+ =

(1− µ

|xj |

)+

xj .

`12 norm (group Lasso) a ‖X‖12 =∑

i ‖Xi‖2 =∑

i

√∑j X 2

ij : group ST

a. A. Gramfort et al., "Mixed-norm estimates for the M/EEG inverse problem using acceleratedgradient methods", Phys. in med. and biol., 2012

[proxµ‖.‖12(X)]i =

(1− µ

‖Xi‖2

)+

Xi .

Other types of penalties� Total variation (2D on the cortex)� Sparsity in a dictionnary : φ(X) = ‖DX‖1


Optimization Choice of penalty

Generalized shrinkages, nonconvex penalties

Approximating the `0 quasi-norm φ(x) = #{xi 6= 0}...

Empirical Wiener shrinkage a, log penalty b...

−10 −8 −6 −4 −2 0 2 4 6 8 100

2

4

6

8

10

l1

log a=0.1

log a=0.3

Wiener

log a=1

−3 −2 −1 0 1 2 3

−2.5

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

Identity

Soft

log a=0.1

log a=0.3

Wiener

log a=1

a. Siedenburg et al., "Audio declipping with social sparsity", ICASSP’14b. Selesnick et al., "Sparse signal estimation by maximally sparse convex optimization", IEEE

TSP, 2014

... or the `02 quasi-norm φ(X) = #{‖Xi‖ 6= 0}Replacing group soft-thresholding by structured empirical Wiener shrinkage (SEW)

[Sµ(X)]i =

(1−

(µ

‖Xi‖2

)2)

+

Xi . (7)


Numerical results

Plan

1 Introduction

2 The neurovascular coupling

3 Optimization

4 Numerical resultsModel and simulationsContribution of the couplingChoice of parametersChoice of the penalizationRobustness to BOLD false positives/negatives

5 Conclusion


Numerical results Model and simulations

Model and simulations

� 3-layer spherical head model, 272 dipoles distributed on the cortex� Simulation of 3 punctual active sources (2 damped waves, 1 LF)� EEG : sampling frequency 500 Hz, 31 electrodes� fMRI : sampling frequency 1 Hz� White Gaussian noise : SNR of 2 dB for EEG and -18 dB for fMRI� Evaluation metrics : SNR and Spatial Error (SE, cm)

0 0.1 0.2 0.3 0.4−0.01

0

0.01

0.02

0.03

time (s)

Am

plit

ude (

µ V

)

True sources X





electrodes

dipoles

active sources

3-layer spherical head model





0 0.1 0.2 0.3 0.4−0.1

0

0.1

0.2

time (s)

Am

plit

ud

e (

µ V

)

0 5 10 15 20

0

5

10

x 10−3

time (s)B

OL

D c

ha

ng

e (

a.u

.)

EEG measures (noise-free) fMRI measures (noise-free)





0 0.1 0.2 0.3 0.4−0.1

0

0.1

0.2

time (s)

Am

plit

ud

e (

µ V

)

0 5 10 15 20

−0.01

0

0.01

time (s)B

OL

D c

ha

ng

e (

a.u

.)

EEG measures (noisy) fMRI measures (noisy)


Numerical results Contribution of the coupling

Benefit of the coupling

0 0.1 0.2 0.3 0.4 0.5−0.01

−0.005

0

0.005

0.01

0.015

0.02

0.025

time (s)

Am

plitu

de (

µ V

)

EEG only, SNR = 6, SE = 6.3 cm

0 0.1 0.2 0.3 0.4 0.50

0.002

0.004

0.006

0.008

0.01

time (s)

Am

plitu

de (

µ V

)

fMRI only, SNR = 0 dB, SE = 0 cm

0 0.1 0.2 0.3 0.4 0.5−0.005

0

0.005

0.01

0.015

0.02

0.025

time (s)

Am

plitu

de (µ

V)

EEG–fMRI, SNR = 17 dB, SE = 0 cm

0 0.1 0.2 0.3 0.4 0.5−0.01

−0.005

0

0.005

0.01

0.015

0.02

0.025

0.03

time (s)

Am

plitu

de (µ

V)

Reconstruction with known sources (least-squares),SNR = 26 dB, SE = 0 cm


Numerical results Choice of parameters

Choice of parameters

� Heuristics for choosing parameters :

α∗ =Uσ2

FTσ2

E + Uσ2F,

λ < λM :=α2 ‖E‖

2F + 1−α

2 ‖F‖2F

φ(X∗) ,

� Numerical experiments

λ (log10)

α (

log

10

)

−5 −4 −3 −2 −1

−5

−4

−3

−2

−1

λ (log10)

α (

log

10

)

−5 −4 −3 −2 −1

−5

−4

−3

−2

−1

λ (log10)

α (

log

10

)

−5 −4 −3 −2 −1

−5

−4

−3

−2

−1

σE = 0.001σF = 0.001

σE = 0.001σF = 0.005

σE = 0.01σF = 0.001

� Conclusion :Large range of suitable parametersHeuristic not very precise, but satisfactoryMust be completed by cross-validation


Numerical results Choice of the penalization

Influence of the penalty φ

`1SNR-out = 16 dB

SE = 0 cm

`12SNR-out = 8 dB

SE = 5 cm

SEWSNR-out = 24 dB

SE = 0 cm

λ (log10)

-4 -3 -2 -1 0

α (

log10)

-4

-3

-2

-1

`12

λ (log10)

-4 -3 -2 -1 0

α (

log10)

-4

-3

-2

-1

log, a = 1λ (log10)

-4 -3 -2 -1 0

α (

log10)

-4

-3

-2

-1

SEW


Numerical results Robustness to BOLD false positives/negatives

Robustness to BOLD false positives/negatives

noise-free BOLD signal asymmetrical reconstruction symmetrical reconstruction

SNR = 25 dB, SE = 0 cm SNR = 24 dB, SE = 0 cm

SNR = 4 dB, SE = 6.4 cm SNR = 18 dB, SE = 0 cm

SNR = 3 dB, SE = 3 cm SNR = 19 dB, SE = 0 cm


Conclusion

Conclusion

On the coupling� A (too ?) simple model, linear in time and space� Adapting the HRF to indivuals and brain locations

A simple linear and structured inverse problem� Importance of the penalty, the parameters� Nice behavior of non-convex relaxations (but more sensitive to the choice ofparameters)

Perspectives� Investigation with real data (need for nonlinear neurovascular coupling)� Image fusion in remote sensing a

� Nonconvex structured penalties in EEG b

a. Q. Wei et al., “Hyperspectral and multispectral image fusion based on a sparse representation”,IEEE Trans. on Geoscience and Remote Sensing, 2015

b. F. Costa et al., “EEG Source Localization Based on a Structured Sparsity Prior and a PartiallyCollapsed Gibbs Sampler”, submitted, 2015


Conclusion

Thanks for your attention

Any question ?


symmetrical eeg–fmri imaging by sparse...

Documents