probabilistic inference and the brain section 4: critical ......joining the dots in theoretical...

ABSTRACT: My treatment of critical gaps in models of probabilistic inference will focus on the potential of unified

theories to “close the gaps” between probabilistic models of perception and evidence accumulation - and how these

models can be understood in terms of embodied inference and action. Formally speaking, models of motor control

and choice behaviour can be cast in terms of (active) probabilistic inference; however, there are two key outstanding

issues. Both pertain to the implementation of active inference in the brain. The first speaks to the distinction between

discrete and continuous state-space models – and the requisite message passing schemes (e.g., variational Bayes

versus predictive coding). The second key distinction is between mean field descriptions in terms of population

dynamics (i.e., sufficient statistics) and their microscopic implementation in terms of spiking neurons (i.e., sampling

approaches to probabilistic inference). These are potentially important issues that constrain the interpretation of

empirical data – and how these data can be used to adjudicate among different models of the Bayesian brain.

.

Probabilistic Inference and the Brain

SECTION 4: CRITICAL GAPS IN MODELS

JOINING THE DOTS IN THEORETICAL NEUROBIOLOGY

Karl Friston, University College London

http://pibrainconf.sciencesconf.org/

http://www.google.co.uk/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCMzP4emU3ccCFYfSGgodfi4Exw&url=http://structurae.net/photos/24340-college-de-france-paris&psig=AFQjCNH3PyRG8xlHGA-CTl33YCEwUWHs2A&ust=1441448629891081

Does the brain use continuous or discrete

state space models?

Does the brain encode beliefs with ensemble

densities or sufficient statistics?


state-space models?

( | ) ( | , )q x p x s m

{1, , }

[ 0 , 1 ]

x N

x

States and their

Sufficient statistics

1x

2x

3x

x

https://www.google.co.uk/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCPGIt8bW4McCFcZZGgod5vUPIA&url=https://www.reddit.com/r/dataisbeautiful/comments/1d16k2/directional_tuning_of_an_upper_motor_neuron_in/&psig=AFQjCNEC088PUSdTLVcVF6DwzHft2HroXw&ust=1441569279091599

http://www.google.co.uk/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCIfkhMav4McCFcUEGgodYMUGKg&url=http://www.cell.com/trends/neurosciences/fulltext/S0166-2236(14)00060-5&bvm=bv.102022582,d.ZGU&psig=AFQjCNG4LKNtJwOjk5ox8IdJH9BpX2tgLQ&ust=1441558796235588

Approximate Bayesian inference for

continuous states: Bayesian filtering

a r g m i n ( , )

( , ) l n ( | )

F s

F s p s m

x

States and their

Sufficient statistics

( | ) ( | , )q x p x s m

Free energy Model evidence


“Objects are always imagined as being present in the field of

vision as would have to be there in order to produce the same

impression on the nervous mechanism” - von Helmholtz

Richard Gregory Hermann von Helmholtz

Impressions on the Markov blanket…

s S

Bayesian filtering and predictive coding

( ) ( , )Q F s

D

prediction update

( )s g

prediction error

Making our own sensations

Changing

sensations

sensations – predictions

Prediction error

Changing

predictions

Action Perception

the

(1 )

x

(1 )

v

( 2 )

x

( 2 )

v

( 2 )

x

( 3 )

v

( 2 )

v

(1 )

x

(1 )

v

( 0 )

v

Descending

predictions

Ascending

prediction errors

A simple hierarchy what where

Sensory

fluctuations

Hierarchical generative models

(1 )

x

(1 )

v

( 2 )

x

( 2 )

v

( 2 )

x

( 3 )

v

( 2 )

v

( 1 )

x

( 1 )

v

( 0 )

v

D

What does Bayesian filtering explain?


Cross frequency coupling

Perceptual categorisation

Violation (P300) responses

Oddball (MMN) responses

Omission responses

Attentional cueing (Posner paradigm)

Biased competition

Visual illusions

Dissociative symptoms

Sensory attenuation

Sensorimotor integration

Optimal motor control

Motor gain control

Oculomotor control and smooth pursuit

Visual searches and saccades

Action observation and mirror neuron responses

Dopamine and affordance

Action sequences (reversal learning)

Interoceptive inference

Communication and hermeneutics






Omission responses


Biased competition

Visual illusions


Sensory attenuation



Motor gain control








Specify a deep (hierarchical) generative model

Simulate approximate (active) Bayesian inference by solving:

( , )D F s

prediction update

An example: Visual searches and saccades

,x p Frontal eye fields

ps

,v p

qs

x

v

v

,x q

x

,v q

px

Pulvinar salience map Fusiform (what)

Superior colliculus

Visual cortex

oculomotor reflex arc

( )u

px

Parietal (where)

a

Deep (hierarchical) generative model

200 400 600 800 1000 1200 1400-2

0

2

Action (EOG)

time (ms)

200 400 600 800 1000 1200 1400

-5

0

5

Posterior belief

time (ms)

Saccadic fixation and salience maps

Visual samples

Conditional expectations

about hidden (visual) states

And corresponding percept

Saccadic eye movements

Hidden (oculomotor) states

vs. vs.

Is Bayesian filtering the only process theory for

approximate Bayesian inference in the brain?

What about state space models?

( | ) ( | , )q x p x s m

{1, , }

[ 0 , 1 ]

x N

1x

2x

3x

x

https://www.google.co.uk/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCPGIt8bW4McCFcZZGgod5vUPIA&url=https://www.reddit.com/r/dataisbeautiful/comments/1d16k2/directional_tuning_of_an_upper_motor_neuron_in/&psig=AFQjCNEC088PUSdTLVcVF6DwzHft2HroXw&ust=1441569279091599


Full priors

– control states

Empirical priors – hidden states

Likelihood

A (Markovian) generative model

Hidden states ts

to

, ,t T

u u

Control states

C

,

B

A

1ts

1ts

1to

0 0 1 1

1 1 0 1 0

1

( , | )

0

, , , | , | | | |

| ( | ) ( | ) ( | )

|

| ( | , ) ( | , ) ( | )

| , ( )

| ( )

[ l n ( , | ) l n ( | ) ]

|

t t

t t

t t t

t t t t

Q o s

P o s u a m P o s P s a P u P m

P o s P o s P o s P o s

P o s

P s a P s s a P s s a P s m

P s s u u

P u

E P o s Q s

P o m

P s

A

B

F

F

C

|

| ( , )

m

P m

D

http://www.clker.com/cliparts/H/8/R/H/H/5/lab-mouse-template-md.png

Generative models

Hidden states ts

to

, ,t T

u u

Control states

C

,

B

A

1ts

1ts

1to

(1 )

x

(1 )

v

( 2 )

x

( 2 )

v

( 2 )

x

( 3 )

v

( 2 )

v

(1 )

x

(1 )

v

( 0 )

v

( ) ( ) ( ) ( ) ( )i i i i i

D ( )i

1 1 1( )

t t t t t to

s A B s B s

ts

Continuous states Discrete states

Bayesian filtering

(predictive coding)

Variational Bayes

(belief updating)

( )0

i


http://www.flickr.com/photos/jamuudsen/145813811/

Variational updates

Perception

Action selection

Incentive salience

Functional anatomy

0 0.2 0.4 0.6 0.8 1

0

20

40

60

80

Performance

Prior preference

succ

ess

rate

(%

)

FE

KL

RL

DA

Simulated behaviour

1 1 1( )

( )

l n

t t t t t to

s A B s B s

π γ F

β π π

1a r g m in ( , , , )

tF o o

β

ta

VTA/SN

motor Cortex

occipital Cortex striatum

to

π

ts

s

prefrontal Cortex

F

hippocampus


Variational updating

Perception

Action selection

Incentive salience

Functional anatomy

Simulated neuronal behaviour

1 1 1( l n )

( )

l n

t t t t t t to

s A B s B s s

π γ F

β π π β

1( , , , )

tF o o

0.2 0.4 0.6 0.8 1 1.2 1.4

5

10

15

20

25

30

0.25 0.5 0.75 1 1.25 1.5

-0.02

0

0.02

0.04

0.06

0.08

10 20 30 40 50 60 70 80 90

-0.05

0

0.05

0.1

0.15

0.2

β

ta

VTA/SN

motor Cortex

occipital Cortex striatum

to

π

ts

s

prefrontal Cortex

F

hippocampus


What does believe updating explain?





Omission responses


Biased competition

Visual illusions


Sensory attenuation



Motor gain control








Specify a deep (hierarchical) generative model

Simulate approximate (active) Bayesian inference by solving:

1( , , , )

tF o o


Unit responses

time (seconds) 0.25 0.5 0.75

8

16

24

0.25 0.5 0.75

-0.02

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

Local field potentials

time (seconds)

Res

pons

e

Cross frequency coupling (phase precession)




Omission responses


Biased competition

Visual illusions


Sensory attenuation



Motor gain control








1

2

3

t

t

t

s

s

s


Time-frequency (and phase) response

time (seconds)

freq

uenc

y

1 2 3 4 5 6

5

10

15

20

25

30

0.25 0.5 0.75 1 1.25 1.5 1.75 2 2.25 2.5 2.75 3 3.25 3.5 3.75 4 4.25 4.5 4.75 5 5.25 5.5 5.75 6

-0.2

0

0.2

0.4


time (seconds)

Res

pons

e

50 100 150 200 250 300 350

0

0.05

0.1

0.15

Phasic dopamine responses

time (updates)

chan

ge in

pre

cisi

on

Cross frequency coupling (phase synchronisation)




Omission responses


Biased competition

Visual illusions


Sensory attenuation



Motor gain control









Time-frequency (and phase) response

time (seconds)

freq

uenc

y

0.2 0.4 0.6 0.8 1 1.2 1.4

5

10

15

20

25

30

0.25 0.5 0.75 1 1.25 1.5

-0.02

0

0.02

0.04

0.06

0.08


time (seconds)

Res

pons

e

10 20 30 40 50 60 70 80 90

-0.05

0

0.05

0.1

0.15

0.2


time (updates)

chan

ge in

pre

cisi

on





Omission responses


Biased competition

Visual illusions


Sensory attenuation



Motor gain control









0.25 0.5 0.75 1 1.25 1.5

-0.1

0

0.1

0.2

0.3

0.4


time (seconds)

Res

pons

e

10 20 30 40 50 60 70 80 90

0

0.05

0.1

0.15

0.2


time (updates)

chan

ge in

pre

cisi

on

50 100 150 200 250 300 350 -0.06

-0.04

-0.02

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

Time (ms)

LFP

Difference waveform (MMN)

oddball

standard

MMN





Omission responses


Biased competition

Visual illusions


Sensory attenuation



Motor gain control













Omission responses


Biased competition

Visual illusions


Sensory attenuation



Motor gain control




Dopamine and affordance (transfer of responses)









Omission responses


Biased competition

Visual illusions


Sensory attenuation



Motor gain control













Omission responses


Biased competition

Visual illusions


Sensory attenuation



Motor gain control





Action sequences (devaluation)




state space models or both?





Hidden states ts

to

, ,t T

u u

Control states

C

,

B

A

1ts

1ts

1to

(1 )

x

(1 )

v

( 2 )

x

( 2 )

v

( 2 )

x

( 3 )

v

( 2 )

v

(1 )

x

(1 )

v

( 0 )

v





u

u

u

Ensemble code

1 1

( | ) , ( ) ( )T

i i in ni iq x u N u u u

Laplace code

( | ) , ( )q x N

Ensemble density Parametric density

or

Variational filtering with

ensembles

( , )

( , , , , , , )

u D U s u

u v v v x x x

( )x t

( )v t

5 10 15 20 25 30 -1

-0.5

0

0.5

1

1.5

hidden states

5 10 15 20 25 30 -0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

cause

time {bins}

time

cause

Variational filtering

l n ( , ) ( )

ln ( )

q q

q

F G H

G p s u U u

H q u

u

u

u

Taking the expectation of the ensemble dynamics, under the Laplace

assumption,we get:

This can be regarded as a gradient ascent in a frame of reference that

moves along the trajectory encoded in generalised coordinates. The

stationary solution, in this moving frame of reference, maximises

variational action by the Fundamental lemma.

c.f., Hamilton's principle of stationary action.

u D U

D F D F

0 0 0u

D F F

From ensemble coding to predictive coding (Bayesian filtering)

5 10 15 20 25 30-0.2

-0.1

0

0.1

0.2

0.3predicted response and error

time {bins}

sta

tes

(a.u

.)

5 10 15 20 25 30-1

-0.5

0

0.5

1hidden states

time {bins}

5 10 15 20 25 30-0.4

-0.2

0

0.2

0.4

0.6

0.8

1causal states - level 2

time {bins}

sta

tes

(a.u

.)

Deconvolution with variational filtering

(SDE) – free-form

5 10 15 20 25 30-0.2

-0.1

0

0.1

0.2

predicted response and error

time {bins}

sta

tes

(a.u

.)

5 10 15 20 25 30-1

-0.5

0

0.5

1hidden states

time {bins}

5 10 15 20 25 30-0.4

-0.2

0

0.2

0.4

0.6

0.8

1causal states - level 2

time {bins}

sta

tes

(a.u

.)

Deconvolution with Bayesian filtering

(ODE) – fixed-form

u D U D F




densities or sufficient statistics or both?

u

u

u




densities or sufficient statistics or both?

Ensemble code

1

( | ) , ( )in i

q x u N u

Laplace code

Parametric description of ensemble density

u

u

u

u

u

u

Ensemble code

1

( | ) , ( )in i

q x u N u

Laplace code

Parametric description of ensemble density

u

u

u

An approximate description of

(nearly) exact Bayesian inference

A (nearly) exact description of

approximate Bayesian inference or

In other words, do we have:

And thanks to collaborators:

Rick Adams

Ryszard Auksztulewicz

Andre Bastos

Sven Bestmann

Harriet Brown

Jean Daunizeau

Mark Edwards

Chris Frith

Thomas FitzGerald

Xiaosi Gu

Stefan Kiebel

James Kilner

Christoph Mathys

Jérémie Mattout

Rosalyn Moran

Dimitri Ognibene

Sasha Ondobaka

Will Penny

Giovanni Pezzulo

Lisa Quattrocki Knight

Francesco Rigoli

Klaas Stephan

Philipp Schwartenbeck

And colleagues:

Micah Allen

Felix Blankenburg

Andy Clark

Peter Dayan

Ray Dolan

Allan Hobson

Paul Fletcher

Pascal Fries

Geoffrey Hinton

James Hopkins

Jakob Hohwy

Mateus Joffily

Henry Kennedy

Simon McGregor

Read Montague

Tobias Nolte

Anil Seth

Mark Solms

Paul Verschure

And many others

Thank you

probabilistic inference and the brain section 4: critical ......joining the dots in theoretical...

Documents