Interaction of Sensory and Value Information in Decision-Making

Institute for Theoretical Physics and MathematicsTehran

January 16, 2006

Alan Rorie

SENSORY INPUT

DECISION MECHANISMS

ADAPTIVE BEHAVIOR

low level sensory analyzers

motor output structures

REWARD HISTORY

representationof stimulus/action value

SENSORY INPUT

DECISION MECHANISMS

ADAPTIVE BEHAVIOR

low level sensory analyzers

motor output structures

REWARD HISTORY

representationof stimulus/action value

QuickTime™ and aVideo decompressor

are needed to see this picture.

QuickTime™ and aVideo decompressor

are needed to see this picture.

QuickTime™ and aVideo decompressor

are needed to see this picture.

Motion discrimination task with multiple reward conditions.

• Monkey must discriminate the direction of the motion.

• Only correct choices are rewarded

• Variable coherences span psychophysical threshold, creating a range of difficulties

• Differs from the matching task because target values are fixed

• Creates conflict between sensory and reward information

QuickTime™ and aVideo decompressor

are needed to see this picture.

QuickTime™ and aVideo decompressor

are needed to see this picture.

“Absolute” and “relative” reward magnitude

Differ in absolute reward magnitude

Differ in relative reward magnitude

T1T2

T1 T2

T1T2

T1T2

Effect of absolute and relative reward magnitude on behavior

Absolute magnitude No effect on choice

Relative magnitude Biases choices

T1T2

n=51

• Motion coherence influences choices

• Relative magnitude influences choices

We know from behavior:

We ask:

• Whether, and how, absolute magnitude, relative magnitude, and motion coherence are represented in LIP as the decision unfolds in time?

• Absolute magnitude does not influence choices

Area LIP in the Macaque Brain

http://www.loni.ucla.edu/data/monkey

LIP

• Sensory-based decisions (Shadlen & Newsome, ‘96, ‘01)

• Value-based decisions (Sugrue, Corrado & Newsome, ‘04)

LIP neurons are spatially selective

GO!

LIP RESPONSE FIELD

GO!

LIP neurons are spatially selective

LIP RESPONSE FIELD

Representation of Absolute Reward Magnitude in LIP

n=51

Representation of Absolute Reward Magnitude in LIP

n=51

Representation of Absolute Reward Magnitude in LIP

n=51

Chose In

Representation of Absolute Reward Magnitude in LIP

n=51

Chose In

Chose Out

Representation of Relative Reward Magnitude in LIP

n=51

Representation of Relative Reward Magnitude in LIP

n=51

Representation of Relative Reward Magnitude in LIP

n=51

Chose In

Representation of Relative Reward Magnitude in LIP

n=51

Chose In

Chose Out

Representation of Relative Reward Magnitude in LIP

n=51

Chose In

Chose Out

Summary of population activity

Absolute Relative

Choice

Relative Absolute Absolute

Choice

How can we quantify these dynamics?

Modeling Dynamics

€

FiringRate = β 0 + β1Choice + β 2Coherence + β 3RelativeMag + β 4AbsoluteMag

Modeling Dynamics

€

FiringRate = β 0 + β1Choice + β 2Coherence + β 3RelativeMag + β 4AbsoluteMag

Modeling Dynamics

€

FiringRate = β 0 + β1Choice + β 2Coherence + β 3RelativeMag + β 4AbsoluteMag

Modeling Dynamics

€

FiringRate = β 0 + β1Choice + β 2Coherence + β 3RelativeMag + β 4AbsoluteMag

Modeling Dynamics

€

FiringRate = β 0 + β1Choice + β 2Coherence + β 3RelativeMag + β 4AbsoluteMag

Modeling Dynamics

€

FiringRate = β 0 + β1Choice + β 2Coherence + β 3RelativeMag + β 4AbsoluteMag

Modeling Dynamics

€

FiringRate = β 0 + β1Choice + β 2Coherence + β 3RelativeMag + β 4AbsoluteMag

Modeling Dynamics

€

FiringRate = β 0 + β1Choice + β 2Coherence + β 3RelativeMag + β 4AbsoluteMag

Conclusions: behavior

• Relative reward magnitude biases choice

• Absolute reward magnitude does not affect choice

• Motion coherence biases choice

• The biasing effects of relative magnitude and coherence are additive: reward information does not change psychophysical sensitivity to motion coherence (or vice versa).

Conclusions: physiology

• The critical decision variables—relative reward magnitude and motion coherence—are present in LIP at the precise time when the decision is being formed.

• The representation of sensory and reward information is dynamic; the profile changes dramatically during the course of a trial.

• Absolute reward magnitude is represented in LIP even though it does not influence choice behavior.

• Most single LIP neurons show effects of multiple variables; the representation is multiplexed.

Future Directions:

• How are sensory and reward signals cast into a common additive currency for guiding decisions?

• Origins of sensory and reward signals

• Why is the profile of signals in LIP changing so dramatically throughout the trial? What does this imply for the computational strategy embodied in cortical circuitry?

Indeed there are now no logical (and I believe no insurmountable technical) barriers to the direct study of the entire chain of neural events that lead from the

initial central representation of sensory stimuli…to the detection and discrimination processes themselves,

and to the formation of general commands for behavioral responses and detailed instructions

for their motor execution.

V . B . Mountcastle, Handbook of Physiology, 1985

€

HE(B,T1,T 2) =

Pcoh,BT1coh>0

∑ ⎛

⎝ ⎜

⎞

⎠ ⎟+ (1− Pcoh,B )T 2

coh<0

∑ ⎛

⎝ ⎜

⎞

⎠ ⎟+ 0.5* P0T1+(1− P0 )T 2( )

N coh>0T1+ N coh<0T 2+ 0.5(T1+T 2)

The Optimal Bias

T1T2

T1T2

78% 47%

55%