Bayesian Model Comparison

Will Penny

London-Marseille Joint Meeting,Institut de Neurosciences Cognitive de la Mediterranee,

Marseille, September 28-29, 2009

V1

V5

SPC

V1

V5

SPC

Wellcome Centre for Neuroimaging, UCL, UK.

Overview

• Priors, likelihoods and posteriors

• Model selection using evidence

• Model selection for groups

• Comparing model families

Overview

• Priors, likelihoods and posteriors

• Model selection using evidence

• Model selection for groups

• Comparing model families

Bayesian Paradigm:priors and likelihood

eZy Model:

Z

Bayesian Paradigm:priors and likelihood

1

2

eZy Model:

Prior:

)2/exp(

),0()(

2

122

T

kk INp

Sample curves from prior (before observing any data)

Mean curve

x

Z

Bayesian Paradigm:priors and likelihood

1

2

eZy Model:

Prior:

)2/exp(

),0()(

2

122

T

kk INp

1

2

Bayesian Paradigm:priors and likelihood

1

2

)2/)(exp(

),(),(

),|(),(

21

111

1

111

ii

ii

N

ii

Zy

ZNyp

ypyp

eZy Model:

Prior:

)2/exp(

),0()(

2

122

T

kk INp

Likelihood:

x

Z

Bayesian Paradigm:priors and likelihood

1

2

eZy Model:

Prior:

)2/exp(

),0()(

2

122

T

kk INp

Likelihood:

)2/)(exp(

),(),(

),|(),(

21

111

1

111

ii

ii

N

ii

Zy

ZNyp

ypyp

x

Z

Bayesian Paradigm:priors and likelihood

1

2

eZy Model:

Prior:

)2/exp(

),0()(

2

122

T

kk INp

Likelihood:

)2/)(exp(

),(),(

),|(),(

21

111

1

111

ii

ii

N

ii

Zy

ZNyp

ypyp

x

Z

Bayesian Paradigm: posterior

yCZ

IZZC

CNyp

T

kT

1

1

21

, ,|

x

Z

eZy Model:

Prior:

)2/exp(

),0()(

2

122

T

kk INp

Likelihood:

Bayes Rule:

)|(),|(),( pypyp

Posterior:

N

iiypyp

111 ),|(),(

1

2

Bayesian Paradigm: posterior

1

2

x

Z

yCZ

IZZC

CNyp

T

kT

1

1

21

, ,|

eZy Model:

Prior:

)2/exp(

),0()(

2

122

T

kk INp

Likelihood:

Bayes Rule:

)|(),|(),( pypyp

Posterior:

N

iiypyp

111 ),|(),(

Bayesian Paradigm: posterior

1

2

x

Z

yCZ

IZZC

CNyp

T

kT

1

1

21

, ,|

eZy Model:

Prior:

)2/exp(

),0()(

2

122

T

kk INp

Likelihood:

Bayes Rule:

)|(),|(),( pypyp

Posterior:

N

iiypyp

111 ),|(),(

Overview

• Priors, likelihoods and posteriors

• Model selection using evidence

• Model selection for groups

• Comparing model families

Model Selection

|log myp

2

olsZy

Cos

t fu

nct

ion

Bayes Rule:

)(

)|(),|(),(

myp

mpmypmyp

normalizing constant

dmpmypmyp )|(),|()(

)()(

)|(log

mcomplexitymaccuracy

myp

Model evidence:

constkmcomplexity

constZymaccuracy

2

21

2

2

1

log)(

)(

{ , , , }θ A B C h

( | , ) ( | )( | , )

( | )

p m p mp m

p m

y θ θθ y

y

V1

V5

SPC

Model, mParameters:

PriorPosterior Likelihood

( | ) ( )( | )

( )

p m p mp m

p

yy

y

PriorPosterior Evidence

Parameter Parameter

Model Model

Second level of Bayesian Inference

Bayes Factors

V1

V5

SPC

( | ) ( | , ) ( | )p m i p m i p m i d y y θ θ θ

( | ) ( | , ) ( | )p m j p m j p m j d y y θ θ θ

Model, m=i

V1

V5

SPC

Model, m=j

Model Evidences:

Bayes factor:( | )

( | )ij

p m iB

p m j

y

y

1 to 3: Weak3 to 20: Positive20 to 100: Strong>100: Very Strong

Overview

• Priors, likelihoods and posteriors

• Model selection using evidence

• Dynamic Causal Models

• Model selection for groups

• Comparing model families

Single region

1 11 1 1z a z cu

u2

u1

z1

z2

z1

u1

a11c

Multiple regions

1 11 1 1

2 21 22 2 2

0

0

z a z uc

z a a z u

u2

u1

z1

z2

z1

z2

u1

a11

a22

c

a21

Modulatory inputs

1 11 1 1 12

2 21 22 2 21 2 2

0 0 0

0 0

z a z z ucu

z a a z b z u

u2

u1

z1

z2

u2

z1

z2

u1

a11

a22

c

a21

b21

Reciprocal connections

1 11 12 1 1 12

2 21 22 2 21 2 2

0 0

0 0

z a a z z ucu

z a a z b z u

u2

u1

z1

z2

u2

z1

z2

u1

a11

a22

c

a1

2

a21

b21

Overview

• Priors, likelihoods and posteriors

• Model selection using evidence

• Dynamic Causal Models

• Model selection for groups

• Comparing model families

-5 -4 -3 -2 -1 0 1 2 3 4 5

Sim

ulat

ed d

ata

sets

Log model evidence differences

x1 x2u1

x3

u2

x1 x2u1

x3

u2

incorrect model (m2) correct model (m1)

Figure 2

m2 m1

-35 -30 -25 -20 -15 -10 -5 0 5

Sub

ject

s

Log model evidence differences

MOG

LG LG

RVFstim.

LVFstim.

FGFG

LD|RVF

LD|LVF

LD LD

MOGMOG

LG LG

RVFstim.

LVFstim.

FGFG

LD

LD

LD|RVF LD|LVF

MOG

m2 m1

Models from Klaas Stephan

)|(~ 111 mypy)|(~ 111 mypy

)|(~ 222 mypy)|(~ 111 mypy

)|(~ pmpm kk

);(~ rDirr

)|(~ pmpm kk2 2~ ( | )m p m p

),1;(~1 rmMultm

Random Effects Inference

Different subjects can use different models.

is the probability that model m is usedin the population at large.

We wish to make an inference aboutthis.

mr

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

r1

p(r 1

|y)

p(r1>0.5 | y) = 0.997

157.0,843.0

194.2,806.11

21

21

rr

Overview

• Priors, likelihoods and posteriors

• Model selection using evidence

• Model selection for groups

• Comparing model families

F

A

P

DCM of Auditory Word Processing: Data from an fMRI study by Alex Leff and Tom Schofield

P: Posterior STSA: Anterior STSF: Inferior Frontal Gyrus

How does processing change for speech versus reversed speech input ?

2^6=64 possible patterns of ‘modulation’.2^3=8-1=7 possible patterns of input connectivity7*64=448 possible networks26*448=11,648 models in group of 26 subjects

0 0.5 1

A

sk

p(s k|y

0 0.5 1

F

sk

0 0.5 1

P

sk

0 0.5 1

AF

sk

0 0.5 1

PA

sk

0 0.5 1

PF

sk

0 0.5 1

PAF

sk

Input families: Where does the input go ?

A F P AF PA PF PAF

0 50 100 150 200 250 300 350 400 4500

0.02

0.04

0.06

0.08

0.1p(

m|Y

)

m

0 50 100 150 200 250 300 350 400 4500

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

E[r

m|Y

]

m

F

A

P

F

A

P

F

A

P

F

A

P

(a)

(d)(c)

(b)

Four of the top 16 models:

-1 -0.5 0 0.5 1

A to P

-1 -0.5 0 0.5 1

F to P

-1 -0.5 0 0.5 1

P to A

-1 -0.5 0 0.5 1

F to A

-1 -0.5 0 0.5 1

P to F

-1 -0.5 0 0.5 1

A to F

)|(),|()|( ympmycpycp

Bayesian

Model

Averaging

-0.2 0 0.2 0.4

A to P

-0.2 0 0.2 0.4

F to P

-0.2 0 0.2 0.4

P to A

-0.2 0 0.2 0.4

F to A

-0.2 0 0.2 0.4

P to F

-0.2 0 0.2 0.4

A to F

Same but now for RFX model probs p(m|Y)

F

A

P

DCM of Auditory Word Processing: Data from an fMRI study by Alex Leff and Tom Schofield

P: Posterior STSA: Anterior STSF: Inferior Frontal Gyrus

How does processing change for speech versus reversed speech input ?

(1) Input goes to P.

(2) Connections from P to F, and P to A, are increased for speech versus reversed speech

Summary

• First and second levels of Bayesian inference

• Model selection for groups

• Comparing model families

• DCM for EEG-fMRI

• Thank-you !