bayesian inference lee harrison york neuroimaging centre 01 / 05 / 2009
TRANSCRIPT
![Page 1: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/1.jpg)
Bayesian inference
Lee Harrison
York Neuroimaging Centre
01 / 05 / 2009
![Page 2: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/2.jpg)
Overview
• Probabilistic modeling and representation of uncertainty– Introduction
– Curve fitting without Bayes
– Bayesian paradigm
– Hierarchical models
• Variational methods (EM, VB)
• SPM applications– fMRI time series analysis with spatial priors
– EEG source reconstruction
– Dynamic causal modelling
![Page 3: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/3.jpg)
Overview
• Probabilistic modeling and representation of uncertainty– Introduction
– Curve fitting without Bayes
– Bayesian paradigm
– Hierarchical models
• Variational methods (EM, VB)
• SPM applications– fMRI time series analysis with spatial priors
– EEG source reconstruction
– Dynamic causal modelling
![Page 4: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/4.jpg)
Generation
Recognition
Introduction
time
![Page 5: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/5.jpg)
Overview
• Probabilistic modeling and representation of uncertainty– Introduction
– Curve fitting without Bayes
– Bayesian paradigm
– Hierarchical models
• Variational methods (EM, VB)
• SPM applications– fMRI time series analysis with spatial priors
– EEG source reconstruction
– Dynamic causal modelling
![Page 6: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/6.jpg)
Curve fitting without BayesData Ordinary least squares
yZZZE
ZyZyE
Zxfy
TTols
D
TD
1)(ˆ0
)()(
)(
y
![Page 7: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/7.jpg)
Curve fitting without BayesData Ordinary least squares
y Z
![Page 8: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/8.jpg)
Curve fitting without BayesOrdinary least squares
Z
Bases (explanatory variables) Sum of squared errors
y
Data and model fit
Bases (explanatory variables) Sum of squared errors
![Page 9: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/9.jpg)
Curve fitting without BayesData and model fit Ordinary least squares
Bases (explanatory variables) Sum of squared errors
y Z
![Page 10: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/10.jpg)
Curve fitting without BayesData and model fit Ordinary least squares
Bases (explanatory variables) Sum of squared errors
y Z
![Page 11: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/11.jpg)
Curve fitting without BayesData and model fit
Bases (explanatory variables) Sum of squared errors
y
Over-fitting: model fits noise
Inadequate cost function: blind to overly complex models
Solution: incorporate uncertainty in model parameters
Ordinary least squares
![Page 12: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/12.jpg)
Overview
• Probabilistic modeling and representation of uncertainty– Introduction
– Curve fitting without Bayes
– Bayesian paradigm
– Hierarchical models
• Variational methods (EM, VB)
• SPM applications– fMRI time series analysis with spatial priors
– EEG source reconstruction
– Dynamic causal modelling
![Page 13: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/13.jpg)
Bayesian Paradigm:priors and likelihood
eZy Model:
Z
![Page 14: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/14.jpg)
Bayesian Paradigm:priors and likelihood
1
2
eZy Model:
Prior:
)2/exp(
),0()(
2
122
T
kk INp
![Page 15: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/15.jpg)
Sample curves from prior (before observing any data)
Mean curve
x
Z
Bayesian Paradigm:priors and likelihood
1
2
eZy Model:
Prior:
)2/exp(
),0()(
2
122
T
kk INp
![Page 16: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/16.jpg)
1
2
Bayesian Paradigm:priors and likelihood
1
2
)2/)(exp(
),(),(
),|(),(
21
111
1
111
ii
ii
N
ii
Zy
ZNyp
ypyp
eZy Model:
Prior:
)2/exp(
),0()(
2
122
T
kk INp
Likelihood:
x
Z
![Page 17: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/17.jpg)
Bayesian Paradigm:priors and likelihood
1
2
eZy Model:
Prior:
)2/exp(
),0()(
2
122
T
kk INp
Likelihood:
)2/)(exp(
),(),(
),|(),(
21
111
1
111
ii
ii
N
ii
Zy
ZNyp
ypyp
x
Z
![Page 18: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/18.jpg)
Bayesian Paradigm:priors and likelihood
1
2
eZy Model:
Prior:
)2/exp(
),0()(
2
122
T
kk INp
Likelihood:
)2/)(exp(
),(),(
),|(),(
21
111
1
111
ii
ii
N
ii
Zy
ZNyp
ypyp
x
Z
![Page 19: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/19.jpg)
Bayesian Paradigm: posterior
yCZ
IZZC
CNyp
T
kT
1
1
21
, ,|
x
Z
eZy Model:
Prior:
)2/exp(
),0()(
2
122
T
kk INp
Likelihood:
Bayes Rule:
)|(),|(),( pypyp
Posterior:
N
iiypyp
111 ),|(),(
1
2
![Page 20: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/20.jpg)
Bayesian Paradigm: posterior
1
2
x
Z
yCZ
IZZC
CNyp
T
kT
1
1
21
, ,|
eZy Model:
Prior:
)2/exp(
),0()(
2
122
T
kk INp
Likelihood:
Bayes Rule:
)|(),|(),( pypyp
Posterior:
N
iiypyp
111 ),|(),(
![Page 21: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/21.jpg)
Bayesian Paradigm: posterior
1
2
x
Z
yCZ
IZZC
CNyp
T
kT
1
1
21
, ,|
eZy Model:
Prior:
)2/exp(
),0()(
2
122
T
kk INp
Likelihood:
Bayes Rule:
)|(),|(),( pypyp
Posterior:
N
iiypyp
111 ),|(),(
![Page 22: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/22.jpg)
Bayesian Paradigm:model selection
|log myp
2
olsZy
Cos
t fu
nct
ion
Bayes Rule:
)(
)|(),|(),(
myp
mpmypmyp
normalizing constant
dmpmypmyp )|(),|()(
)()(
)|(log
mcomplexitymaccuracy
myp
Model evidence:
constkmcomplexity
constZymaccuracy
2
21
2
2
1
log)(
)(
![Page 23: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/23.jpg)
Overview
• Probabilistic modeling and representation of uncertainty– Introduction
– Curve fitting without Bayes
– Bayesian paradigm
– Hierarchical models
• Variational methods (EM, VB)
• SPM applications– fMRI time series analysis with spatial priors
– EEG source reconstruction
– Dynamic causal modelling
![Page 24: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/24.jpg)
y
1
2
1
2
3
)|(),|(),,|(),,|(
)|,(
3222111 mpmpmpmyp
myp
Hierarchical models
111
2221
1332 ),0(~
eZy
eZ
Ne
recognition
y
time
space
1
space
2
generation
?)(q
![Page 25: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/25.jpg)
Overview
• Probabilistic modeling and representation of uncertainty– Introduction
– Curve fitting without Bayes
– Bayesian paradigm
– Hierarchical models
• Variational methods (EM, VB)
• SPM applications– fMRI time series analysis with spatial priors
– EEG source reconstruction
– Dynamic causal modelling
![Page 26: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/26.jpg)
Variational methods:approximate inference
)(
)|,(),|(
myp
mypmyp
True posterior
Initial guess and iteratively improve to approximate true posterior
KLFL
myp
q
mypq
q
q
myp
mypq
myp
mypmyp
),|(
)(log)(
)(
)|,(log)(
)(
)(
),|(
)|,(log)(
),|(
)|,(log)(log L
F
KL
fixed
Can computeMaximize minimize KL
Difference btw approx. and true posteriorBut cannot compute as do not know
But how?
)()( ii
![Page 27: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/27.jpg)
))|(|)((),|(log
)(
)|,(log)(
)(mpqKLmyp
q
mypqF
q
Variational methods:approximate inference
accuracy complexity
iii dmypqI \\ )|,(log)(
ii
ii
dI
Iq
)(exp
)(exp
If you assume posterior factorises
then F can be maximised by letting
where
)()( ii
![Page 28: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/28.jpg)
Overview
• Probabilistic modeling and representation of uncertainty– Introduction
– Curve fitting without Bayes
– Bayesian paradigm
– Hierarchical models
• Variational methods (EM, VB)
• SPM applications– fMRI time series analysis with spatial priors
– EEG source reconstruction
– Dynamic causal modelling
![Page 29: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/29.jpg)
AR coeff(correlated noise)
prior precisionof AR coeff
A
VB estimate of WML estimate of W
aMRI smoothed W (RFT)
fMRI time series analysis with spatial priors
observations
GLM coeff
prior precisionof GLM coeff
prior precisionof data noise
Y
W
Penny et al 2005
11,0 LNWp
degree of smoothness Spatial precision matrix
![Page 30: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/30.jpg)
AR coeff(correlated noise)
prior precisionof AR coeff
A
VB estimate of WML estimate of W
aMRI smoothed W (RFT)
)( nwq )( mwq
nwmw
fMRI time series analysis with spatial priors
observations
GLM coeff
prior precisionof GLM coeff
prior precisionof data noise
Y
W
Penny et al 2005
![Page 31: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/31.jpg)
Mean (Cbeta_*.img)
Std dev (SDbeta_*.img)
PPM (spmP_*.img)
)( thsWqp
activation threshold
ths
Posterior density q(wn)
Probability mass pn
fMRI time series analysis with spatial priors:posterior probability maps
probability of getting an effect, given the dataprobability of getting an effect, given the data
),()( nnn Nwq mean: size of effectcovariance: uncertainty
thpp
Display only voxels that exceed e.g. 95%Display only voxels that exceed e.g. 95%
![Page 32: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/32.jpg)
fMRI time series analysis with spatial priors:single subject - auditory dataset
0
2
4
6
8
0
50
100
150
200
250
Active > Rest Active != Rest
Overlay of effect sizes at voxels where SPM is 99% sure that the
effect size is greater than 2% of the global mean
Overlay of 2 statistics: This shows voxels where the activation is different
between active and rest conditions, whether positive or negative
![Page 33: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/33.jpg)
|log myp
2
olsZy
fMRI time series analysis with spatial priors:group data – Bayesian model selection
)()(log qFmyp
Compute log-evidence for each model/subjectCompute log-evidence for each model/subject
model 1model 1
model Kmodel K
subject 1subject 1
subject Nsubject N
Log-evidence mapsLog-evidence maps
![Page 34: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/34.jpg)
fMRI time series analysis with spatial priors:group data – Bayesian model selection
kr
k
BMS mapsBMS maps
PPMPPM
EPMEPM
model kmodel kJoao et al, 2009 (submitted)
)()(log qFmyp
Compute log-evidence for each model/subjectCompute log-evidence for each model/subject
model 1model 1
model Kmodel K
subject 1subject 1
subject Nsubject N
Log-evidence mapsLog-evidence maps
)( krq
kr
941.0)5.0( krq
Probability that model k generated data
Probability that model k generated data
![Page 35: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/35.jpg)
Overview
• Probabilistic modeling and representation of uncertainty– Introduction
– Curve fitting without Bayes
– Bayesian paradigm
– Hierarchical models
• Variational methods (EM, VB)
• SPM applications– fMRI time series analysis with spatial priors
– EEG source reconstruction
– Dynamic causal modelling
![Page 36: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/36.jpg)
DistributedSource model
K
J
MEG/EEG Source Reconstruction
KJy
Forward model(generation)
Inversion(recognition)
- under-determined system- priors required
Mattout et al, 2006
[nxt] [nxp] [nxt][pxt]
n : number of sensorsp : number of dipolest : number of time samples
![Page 37: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/37.jpg)
Overview
• Probabilistic modeling and representation of uncertainty– Introduction
– Curve fitting without Bayes
– Bayesian paradigm
– Hierarchical models
• Variational methods (EM, VB)
• SPM applications– fMRI time series analysis with spatial priors
– EEG source reconstruction
– Dynamic causal modelling
![Page 38: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/38.jpg)
),,( uxFx Neural state equation:
Electric/magneticforward model:
neural activityEEGMEGLFP
Neural model:1 state variable per regionbilinear state equationno propagation delays
Neural model:8 state variables per region
nonlinear state equationpropagation delays
fMRI ERPs
inputs
Hemodynamicforward model:neural activityBOLD
Dynamic Causal Modelling:generative model for fMRI and ERPs
![Page 39: Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009](https://reader036.vdocuments.site/reader036/viewer/2022062619/5515daf8550346dd6f8b4a7d/html5/thumbnails/39.jpg)
Thank-you