bayesian perception
Post on 02-Jan-2016
31 Views
Preview:
DESCRIPTION
TRANSCRIPT
Bayesian Perception
General Idea
Ernst and Banks, Nature, 2002
General Idea
• Bayesian formulation:
, || ,
,
| |
,
| |
P t v w P wP w t v
P t v
P t w P v w P w
P t v
P t w P v w P w
ConditionalIndependence assumption
ˆ arg max | ,w
w P w t v
General Idea
Ernst and Banks, Nature, 2002
w
v=w+n
t=w+n
+
+
noise
noise
w?
Generative model: , |P t v w
General Idea
Width
Prob
abil
ity
VisualP(v|w)
TouchP(t|w)
BimodalP(w|t,v)= P(v|w) P(t|w)
ˆ arg max | ,w
w P w t v
General Idea
2
2
2
2
2 2
2 2
2 22 2
2 2
2 2 2 2 2
2 2
2 22
2 2
2 2 2 2
2 2
2 2
| exp2
log |2
log | |2 2
2
2
2
2
2 /
v
v
v t
t v
v t
t v t v
v t
t v
t v
v t t v
t v
t v
v wP v w
v wP v w
v w t wP v w P t w
v w t w
w v t w C
v tw w C
v tw
2
2 2 2 22 /v t t v
C
General Idea
Mean and variance
2 2
2 2 2 2t v
t v t v
w v t
22 2
2 2
2 2 2 2log | ,
2 /
t v
t v
v t t v
v tw
P w v t
General Idea
Width
Prob
abil
ity
VisualP(v|w)
TouchP(t|w)
tv
2 2
2 2 2 2t v
t v t v
w v t
General Idea
Mean and variance
2 2
2 2 2 2t v
t v t v
w v t
22 2
2 2
2 2 2 2log | ,
2 /
t v
t v
v t t v
v tw
P w v t
2 2
2
2 2v t
w
t v
Optimal Variance
Variance
2 2 2
1 1 1
w v t
w v tI I I
22 2
2 2
2 2 2 2log | ,
2 /
t v
t v
v t t v
v tw
P w v t
Fisher information sums for independent signals
General Idea
0 67 133 2000
0.05
0.1
0.15
0.2
Th
resh
old
(S
TD
)
Visual noise level (%)
Measured bimodal STD
Predicted by the Bayesian model
Unimodal visual STD
Unimodal Tactile STD
Ernst and Banks, Nature, 2002 Note: unimodal estimates may not be optimal but the multimodal estimate is optimal
Adaptive Cue Integration
• Note: the reliability of the cue change on every trial
• This implies that the weights of the linear combination have to be changed on every trial!
• Or do they?
2 2
2 2 2 2t v
t v t v
w v t
General Idea
• Perception is a statistical inference
• The brain stores knowledge about P(I,V) where I is the set of natural images, and V are the perceptual variables (color, motion, object identity)
• Given an image, the brain computes P(V|I)
, ||
P P PP
P P
I V I V VV I
I I
General Idea
• Decisions are made by collapsing the distribution onto a single value:
• or
ˆ |P dV V I V V
ˆ arg max |PV
V V I
Key Ideas
• The nervous systems represents probability distributions. i.e., it represents the uncertainty inherent to all stimuli.
• The nervous system stores generative models, or forward models, of the world (P(I|V)), and prior knowlege about the state of the world (P(V))
• Biological neural networks can perform complex statistical inferences.
Motion Perception
The Aperture Problem
The Aperture Problem
The Aperture Problem
The Aperture Problem
Horizontal velocity (deg/s)V
erti
cal v
eloc
ity
(deg
/s)
The Aperture Problem
Horizontal velocity (deg/s)V
erti
cal v
eloc
ity
(deg
/s)
The Aperture Problem
The Aperture Problem
Horizontal velocity (deg/s)V
erti
cal v
eloc
ity
(deg
/s)
The Aperture Problem
Horizontal velocity (deg/s)V
erti
cal v
eloc
ity
(deg
/s)
The Aperture Problem
Horizontal velocity (deg/s)V
erti
cal v
eloc
ity
(deg
/s)
Standard Models of Motion Perception
• IOC: interception of constraints
• VA: Vector average
• Feature tracking
Standard Models of Motion Perception
Horizontal velocity (deg/s)V
erti
cal v
eloc
ity
(deg
/s)
IOCVA
Standard Models of Motion Perception
Horizontal velocity (deg/s)V
erti
cal v
eloc
ity
(deg
/s)
IOCVA
Standard Models of Motion Perception
Horizontal velocity (deg/s)V
erti
cal v
eloc
ity
(deg
/s)
VA
IOC
Standard Models of Motion Perception
Horizontal velocity (deg/s)V
erti
cal v
eloc
ity
(deg
/s)
IOCVA
Standard Models of Motion Perception
• Problem: perceived motion is close to either IOC or VA depending on stimulus duration, eccentricity, contrast and other factors.
Standard Models of Motion Perception
• Example: Rhombus
Horizontal velocity (deg/s)
Ver
tica
l vel
ocit
y (d
eg/s
) IOCVA
Horizontal velocity (deg/s)
Ver
tica
l vel
ocit
y (d
eg/s
) IOCVA
Percept: VAPercept: IOC
Moving Rhombus
Bayesian Model of Motion Perception
• Perceived motion correspond to the MAP estimate
* arg max |
|| |
, |i ii
P I
P I PP I P I P
P I
P P I x y
vv v
v vv v v
v v
Prior
• Human observers favor slow motions
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
2 2exp / 2 pP v v
Likelihood
• Weiss and Adelson
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
, , |i i iP I x y t v
Likelihood
, , , ,
, , , ,
x y
x y
I x y t I x t y t t t
I x y t I x t y t t t
v v
v v
, , , ,
, , , ,
i x i y i i x x y y t
i i i x i y x x y y t
x x y y t
I x t y t t t I x y t I t I t I t
I x y t I x t y t t t I t I t I t
I I I t
v v v v
v v v v
v v
2
2
, , , ,, , | exp
2
i i x y
i i
I x y t I x t y t t tP I x y t
v vv
2
2, , | exp
2x x y y t
i i
I I IP I x y t
v vv
Likelihood
2
2
2
2,
, , | , , |
1exp
2
1exp ,
2
i ii
x x y y ti
x x y y t
x y
P I x y t P I x y t
I I I
w x y I I I dxdy
v v
v v
v v
Binary maskPresumably, this is set by segmentation cues
Posterior
2 2 22 2
,
log | , , log , , | log
1 1
2 2x x y y t x yx y p
P I x y t P I x y t P
I I I
v v v
v v v v
Bayesian Model of Motion Perception
• Perceived motion corresponds to the MAP estimate
*
22
2
*
22
2
arg max | , | , |ii
i
x x yp x t
y tx y y
p
P I P I P P I x y
I I II I
I II I I
vv v v v v
v
Only one free parameter
Likelihood
2 2 22 2
, ,
, ,
2 2
,,
22
2, ,
2 22
2, ,
1 1( )
2 2
2 2( ) 1 1
2 2 22
1
x x y y t x yx y x yp
x x y y t x xx y x y
p yx x y y t yx yx y
x y x tx y x yp
y x yx y x y p
L I I I
I I I IL
I I I I
I I I I I
I I I
v v v v v
v v vv
v vv v
v,
,
0
xx y
t yx y
I I
Motion through an Aperture
• Humans perceive the slowest motion.
• More generally: we tend to perceive the most likely interpretation of an image
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
Motion through an Aperture
ML
MAP
Prior Posterior
Likelihood
Motion and Constrast
• Humans tend to underestimate velocity in low contrast situations
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
Motion and Contrast
ML
MAP
Prior Posterior
HighContrast
Likelihood
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
Motion and Contrast
ML
MAP
Prior Posterior
LowContrast
Likelihood
Motion and Contrast
• Driving in the fog: in low contrast situations, the prior dominates
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
Moving Rhombus
IOC
MAP
Prior Posterior
HighContrast
Likelihood
Moving Rhombus
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
-50 0 50
-50
0
50
Horizontal Velocity
Ver
tica
l Ve
loci
ty
IOC
MAP
Prior Posterior
LowContrast
Likelihood
Moving Rhombus
Moving Rhombus
• Example: Rhombus
Horizontal velocity (deg/s)
Ver
tica
l vel
ocit
y (d
eg/s
) IOCVA
Horizontal velocity (deg/s)
Ver
tica
l vel
ocit
y (d
eg/s
) IOCVA
Percept: VAPercept: IOC
Barberpole Illusion
Plaid Motion: Type I and II
Plaids and Contrast
Lower contrast
Plaids and Time
• Viewing time reduces uncertainty
Ellipses
• Fat vs narrow ellipses
Ellipses
• Fat vs narrow ellipses
• All motions agree
Ellipses
Ellipses
• Adding unambiguous motion
Ellipses
• Adding unambiguous motion
Other Prior
• Prior on direction of lightning
Generalization
• All computation are subject to uncertainty (ill-posed)
• This includes syntax processing, language acquisition… etc.
• Solution: compute with probability distributions
Binary Decision Making
Shadlen et al.
Race Model
• Standard theory: some signal is accumulated (or integrated) to a bound. Also known as race models.
• The signal to be integrated could be the response of sensory neurons.
Bayesian Strategy
• The ‘diffusion to bound’ model of Shadlen et al.
High motion strength
High m
otion stre
ngth
Low motion strength
Time
~1 secStimulus
onStimulus
off
Spikes/s
Time
~1 secStimulus
onStimulus
off
Spikes/s
Low motion strength
A Neural Integrator for Decisions?
MT: Sensory Evidence
Motion energy
“step”
LIP: Decision Formation
Accumulation of evidence
“ramp”
Threshold
Diffusion to bound model
Positive bound
Negative bound
Proposed by Wald, 1947 and Turing (WW II, classified); Stone, 1960; then Laming, Link, Ratcliff, Smith, . . .
Diffusion to bound model
Positive bound or Criterion to answer “1”
Negative bound or Criterion to answer “2”
Momentary evidencee.g.,
∆Spike rate:MTRight– MTLeft
Accumulated evidencefor Rightward
andagainst Leftward
Criterion to answer “Right”
Criterion to answer “Left”
Diffusion to bound model
Shadlen & Gold (2004)Palmer et al (2005)
kC
C is motion strength (coherence)Seems arbitrary but why not?
MT responses
60
40
20
0
Firi
ng r
ate
Direction (deg)
Height scales with coherence
MT MTRight Leftr r
Right Left
Diffusion to bound model
• Performance reaction time trade-off
Best fitting chronometric function“Diffusion to bound”
t(C) B
kCtanh(BkC) tnd
Predicted psychometric function “Diffusion to bound”
P 1
1 e 2k C B
Average LIP activity in RT motion task
Roitman & Shadlen, 2002 J. Neurosci.
choose Tin
choose Tout
Note the clear asymmetry
Bayesian Strategy
• The Bayesian strategy in this case consists in computing the posterior distribution given all activity patterns from MT up to the current time,
MT:1MT
:1 MT:1
MT
MT MT MT1:1 1:1
MT MT1:1
MT MT1:1
||
|
| , |
| |
| |
t
t
t
t
t t t
t t
t t
p s p sp s
p
p s p s
p s p s p s
p s p s p s
p s p s
rr
r
r
r r r
r r
r r
Bayesian Strategy
• Race models and Bayesian approach
MT MT MT:1 1:1
MT MT MT:1 1:1
MT
1
| | |
log | log | log |
log |
t t t
t t t
t
p s p s p s
p s p s p s
p s
r r r
r r r
r
Temporal sum
Unless is related to …
But not over rMT, or MT MTRight Leftr r
MTlog |p srMT MTRight Leftr r
Bayesian Strategy
• Are neurons computing log likelihood?
• The difference of activity between two neurons with preferred directions 180 deg away is proportional to a log likelihood ratio.
Bayesian Strategy
• Log likelihood ratio:
MT MT:1
MT MT1:1
| |log log
| |
tt
t
p s R p s R
p s L p s L
r r
r r
LR MT MTR, L,L r r
MT:1 MT MT
R, L,MT1:1
|log
|
tt
t
p s R
p s L
r
r rr
MT
LR
MT
|log
|
p s RL
p s L
r
r
Bayesian Strategy
• Is the log likelihood ration proportional to ?
2MT MTR L R LMT MT
R L 2
2MT MTR L R LMT MT
R L 2
R R| exp
2
L L| exp
2
p R
p L
r rr r
r rr r
MT MTR, L, r r
Coherence level
MT MT2 2R L MT MT MT MT
R L R L R L R LMT MTR L
|log R R L L
|
p R
p L
r rr r r r
r r
MT MTR L R LR R r r
MT MTR LC r r
Bayesian Strategy
• Note that if you know , you still don’t know the log likelihood ration unless you’re given the coherence level.
• Therefore, the animal can’t know its confidence level (the log likelihood ratio) unless it estimates C…
• Another important point: if we stop the race at a fixed level of we stop at different levels of log likelihood ratio depending on the coherence. This is why performance gets better when coherence increases, even though we always stop at the same activity threshold.
MT MTR, L, r r
MT MTR, L, r r
Decision Making
• Does that mean the animal does not know how much to trust its own decision?
• Does that mean the brain does not encode uncertainty or probability distribution?
• Seems unlikely…
• To be continued…
top related