computational models of cognitive control (i) matthew botvinick princeton neuroscience institute and...

Computational models of cognitive control (I)

Matthew BotvinickPrinceton Neuroscience Institute andDepartment of Psychology, Princeton University

Atkinson & Shiffrin, 1968

Atkinson & Shiffrin, 1968

Structural elements

Short-term store

Sensory register

Long-term store

Atkinson & Shiffrin, 1968

Structural elements

Short-term store

Sensory register

Long-term store

Control elements

Search / RetrievalTransfer to/from STS

Which register?Forward into sts?

SearchRehearsal

Baddeley, 1986/2007

Baddeley, 1986/2007

“Slave systems”

Shiffrin & Schneider, 1977

Norman & Shallice, 1986

Contention scheduling system

Norman & Shallice, 1986

Supervisory attentional system (SAS)

Contention scheduling system

GREEN

< < < > < < <

-- Controlled (task-guided) attention: “attention for action” (Stroop)

-- Ignoring or inhibiting task-irrelevant stims/responses (Go/No-Go)

-- Manipulating information in working memory (N-Back)

-- Switching between tasks (Wisconsin Card Sort)

-- Planning / scheduling (Tower of London)

-- Navigating through extended, hierarchically structured tasks

“Executive/Cognitive Control”

AR

ZN

KZ

Q

AR

ZN

KZ

Q

AR

ZN

KZ

Q

AR

ZN

KZ

Q

-- Controlled (task-guided) attention: “attention for action” (Stroop)

-- Ignoring or inhibiting task-irrelevant stims/responses (Go/No-Go)

-- Manipulating information in working memory (N-Back)

-- Switching between tasks (Wisconsin Card Sort)

-- Planning / scheduling (Tower of London)

-- Navigating through extended, hierarchically structured tasks

“Executive/Cognitive Control”

-- Controlled (task-guided) attention: “attention for action” (Stroop)

-- Ignoring or inhibiting task-irrelevant stims/responses (Go/No-Go)

-- Manipulating information in working memory (N-Back)

-- Switching between tasks (Wisconsin Card Sort)

-- Planning / scheduling (Tower of London)

-- Navigating through extended, hierarchically structured tasks

“Executive/Cognitive Control”

-- Controlled (task-guided) attention: “attention for action” (Stroop)

-- Ignoring or inhibiting task-irrelevant stims/responses (Go/No-Go)

-- Manipulating information in working memory (N-Back)

-- Switching between tasks (Wisconsin Card Sort)

-- Planning / scheduling (Tower of London)

-- Navigating through extended, hierarchically structured tasks

“Executive/Cognitive Control”

-- Controlled (task-guided) attention: “attention for action” (Stroop)

-- Ignoring or inhibiting task-irrelevant stims/responses (Go/No-Go)

-- Manipulating information in working memory (N-Back)

-- Switching between tasks (Wisconsin Card Sort)

-- Planning / scheduling (Tower of London)

-- Navigating through extended, hierarchically structured tasks

“Executive/Cognitive Control”

-- Controlled (task-guided) attention: “attention for action” (Stroop)

-- Ignoring or inhibiting task-irrelevant stims/responses (Go/No-Go)

-- Manipulating information in working memory (N-Back)

-- Switching between tasks (Wisconsin Card Sort)

-- Planning / scheduling (Tower of London)

-- Navigating through extended, hierarchically structured tasks

“Executive/Cognitive Control”

-- Controlled (task-guided) attention: “attention for action” (Stroop)

-- Ignoring or inhibiting task-irrelevant stims/responses (Go/No-Go)

-- Manipulating information in working memory (N-Back)

-- Switching between tasks (Wisconsin Card Sort)

-- Planning / scheduling (Tower of London)

-- Navigating through extended, hierarchically structured tasks

GETTING WITH (AND STAYING WITH) THE PROGRAM

“Executive/Cognitive Control”

GETTING WITH (AND STAYING WITH) THE PROGRAM

Encoding / Formulation Maintenance Projection Updating

Task Context

GREEN

Cohen, McClelland & Dunbar, 1990

< < < > < < <

Cohen, Servan-Schreiber & McClelland, AJP, 1992.

Cohen, Servan-Schreiber & McClelland, AJP, 1992.

White & Wise, Exp Br Res, 1999

(See also: Assad, Rainer & Miller, 2000; Bunge, 2004; Hoshi, Shima & Tanji, 1998; Johnston & Everling, 2006; Wallis, Anderson & Miller, 2001; White, 1999…)

From Curtis & D’Esposito, TICS, 2003, after Funahashi et al., J. Neurophysiol,1989.

Questions…

-- What about manipulation in WM, etc? -- dynamics (switching, sequences)-- what controls control? (Homunculus)

Intermission

“A controlled process is a temporary sequenceof nodes activated under control of, andthrough attention by, the subject. Becauseactive attention by the subject is required,only one such sequence at a time may becontrolled without interference.”

Shiffrin & Schneider, 1977

Ridderinkhof et al., Science, 2004 (Based on Picard & Strick, Curr. Op. Biol., 2001)

Response override

Response override Underdetermined responding

Response override Underdetermined responding

Error commission

Response override Underdetermined responding

Error commission

Conflict

GREEN

GREEN

< < < > < < <

> > > > > > >

Botvinick, et al. (1999) Nature.

< < < > < < <

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Thompson-Schill et al., PNAS, 1997

High constraint: APPLE

Low constraint: BALL

Carter, Braver, Barch, Botvinick, Noll & Cohen, Science, 1998

Barch, et al. Cerebral Cortex, 2001 / Botvinick, Carter & Cohen, TICS, 2004

Ridderinkhoff et al., Science, 2004

Botvinick, et al. (2001) Psychological Review.

Botvinick, et al. (2001) Psychological Review.

Botvinick, et al. (2001) Psychological Review.

Carter, Braver, Barch, Botvinick, Noll & Cohen, Science, 1998

Gratton et al., JEPG, 1992

Low controlHigh conflict

High controlLow conflict

-0.05

0

0.05

0.1

0.15

0.2

scan 1 scan 2 scan 3 scan 4 scan 5

Scan within trial

MR

sig

na

l c

ha

ng

e (

%)

CcI

iI

Botvinick, et al. (1999) Nature.

Low controlHigh conflict

Tzelgov, et al. (1992) Memory & Cognition.

High controlLow conflict

Low controlHigh conflict

Carter, MacDonald, Botvinick et al. (2000) PNAS.

Low controlHigh conflict

Barch, Braver, Sabb & Noll, JCN, 2000

Underdeterminedresponding

ResponseOverride

Yeung, Botvinick & Cohen, Psychological Review, 2004

Stuphorn, Taylor & Schall, Nature, 2000

Ito et al., Science, 2000ACC

SEF

Nakamura, Roesch & Olson, J. Neurophys. 2005

Curtis et al., Cereb. Ctx., 2005

Davis et al., J. Neurosci. 2005

Neutral Incongruent

44%

WHY?

Control

Conflict

Gratton et al., JEPG, 1992

Botvinick, et al. (2001) Psychological Review.

Botvinick, et al. (2001) Psychological Review.

Tzelgov, et al. (1992) Memory & Cognition.

Botvinick, et al. (2001) Psychological Review.

Botvinick, et al. (2001) Psychological Review.

Mayr, Awh & Laurey, Nature Neuroscience, 2003

Mayr, Awh & Laurey, Nature Neuroscience, 2003

Ullsperger & Botvinick, PB&R, 2005 Kerns, et al. (2004) Science.

See also:Freitas, Bahar, Yang, and Banai, Psychological Science, 2007  Notebaert, Gevers, Verbruggen, & Liefooghe, Psychonomic Bulletin & Review, 2006  

Freitas, Bahar, Yang, and Banai, Psychological Science, 2007 

Kerns, et al. (2004) Science.

DiPellegrino, Ciaramelli & Ladavas, J. Cog. Neuro., 2007

Monitoring of action outcomes -- especially outcomes considered aversive or signaling reduction in reward

Gehring & Willoughby, Science, 2002Luu et al., Psychol. Sci., 2004 Niewenhuis et al., Cerebral Cortex, 2004Bush et al., PNAS, 2002Holroyd & Coles, Psychol. Rev., 2002

Use of outcome information to guide action selection

Matsumoto, et al. Science, 2003Bush, et al., PNAS, 2002Holroyd & Coles, Psychol. Rev. 2002Hadland, et al., J. Neurophysiol., 2003Kennerley, et al., Nature Neurosci., 2006

Action selection based on cost-benefit analysis

Rushworth, et al., TICS, 2004

Cost-benefit analysis might take effort into account

Walton, et al., J. Neurosci., 2003

Johansen & Fields, Nature Neuroscience, 2004

Glu antagonist Glu agonist(kynurenic acid) (homocysteic acid)

Jackson, Frost & Moghaddam, J. Neurochem., 2001

Toward an integrative account

Conflict is an outcome of action / strategy selection

Toward an integrative account

Conflict is an outcome of action / strategy selection

Conflict is aversive (registers as a cost)

Toward an integrative account

Conflict is an outcome of action / strategy selection

Conflict is aversive (registers as a cost)

Conflict informs subsequent decision making

Toward an integrative account

Conflict is an outcome of action / strategy selection

Conflict is aversive (registers as a cost)

Conflict avoidance

Conflict informs subsequent decision making

Toward an integrative account

task cue

strategy

stimulus task cue

strategyresponse

stimulus task cue

strategyresponse

conflict

stimulus task cue

strategyresponse

conflict

Law of least mental effort:

All else being equal, actions will be chosen so as to minimize the demand for cognitive control (indexed by processing conflict).

Law of least effort (Hull): All else being equal, actions will be chosen so as to minimize the amount of work performed.

Macleod, Hunt & Mathews, Journal of Verbal Learning and Verbal Behavior, 1978

STAR ABOVE CROSS

“One basis for strategy selection: minimization of cognitive workload.”

-- Reichle, Carpenter & Just, Cog. Psychol., 2003.

4

3

90% switch 10% switch

Botvinick, CABN, 2007

Card

Card

Botvinick, CABN, 2007

Card

Botvinick, CABN, 2007

Card

Botvinick, CABN, 2007

Anticipatory skin conductance responses

Bechara, Damasio, Damasio, & Lee, Journal of Neuroscience, 1999.

0

0.02

0.04

0.06

0.08

0.1

0.12

Performance Decision

SCR (area)

High demand

Low demand

Botvinick & Rosen, Psych Res, in press

Botvinick & Rosen, Psych Res, in press

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

Performance Decision

SCR (area)

High demand

Low demand

Botvinick & Rosen, Psych Res, in press

SCR ACC (Nagai, Critchley, Featherstone, Trimble, & Dolan, 2004)

ACC damage loss of effort- and IGT-related SCR (Naccache et al., 2005))

8

9

3

Deciding your pay…

X

6

1

9

3

4

Deciding your pay…

$

2000 ms

2000 ms

2000-8000 ms

2000 ms

2000-8000 ms

2

4

8

9

3

Deciding your pay…

X

6

1

9

3

4

Deciding your pay…

$

2000 ms

2000 ms

2000-8000 ms

2000 ms

2000-8000 ms

X $

Botvinick, Huffstetler & McGuire, in press

Botvinick, Huffstetler & McGuire, in press

-0.4

-0.2

0.2

0.4

-0.2 0.2

ACC EffectD

LP

FC

Eff

ect

-0.4

-0.2

0.2

0.4

-0.2 0.2

ACC EffectD

LP

FC

Eff

ect

-0.4

-0.2

0.2

0.4

-0.2 0.2

ACC EffectD

LP

FC

Eff

ect

-0.4

-0.2

0.2

0.4

-0.2 0.2

ACC EffectD

LP

FC

Eff

ect

-0.4

-0.2

0.2

0.4

-0.2 0.2

ACC EffectD

LP

FC

Eff

ect

-0.4

-0.2

0.2

0.4

-0.2 0.2

ACC EffectD

LP

FC

Eff

ect

-0.4

-0.2

0.2

0.4

-0.2 0.2

ACC EffectD

LP

FC

Eff

ect

-0.4

-0.2

0.2

0.4

-0.2 0.2

ACC EffectD

LP

FC

Eff

ect

-0.4

-0.2

0.2

0.4

-0.2 0.2

ACC EffectD

LP

FC

Eff

ect

-0.4

-0.2

0.2

0.4

-0.2 0.2

ACC EffectD

LP

FC

Eff

ect

-0.4

-0.2

0.2

0.4

-0.2 0.2

ACC EffectD

LP

FC

Eff

ect

-0.4

-0.2

0.2

0.4

-0.2 0.2

ACC EffectD

LP

FC

Eff

ect

Conclusions

Conflict can be viewed as an index of the demand for control

The occurrence of conflict appears to be detected in the brain

Conflict detection appears to impact cognitive control

Conflict may also register as a cost

Tasks / strategies may be chosen so as to minimize conflict

Where does this put us?

Where does this put us?

Chipping away at the homunculus

Where does this put us?

Chipping away at the homunculus

But this is just about modulating task representations

Where does this put us?

Chipping away at the homunculus

But this is just about modulating task representations

How are task representations selected in the first place?

Where does this put us?

Chipping away at the homunculus

But this is just about modulating task representations

How are task representations selected in the first place?

How are they sequenced?

Where does this put us?

Chipping away at the homunculus

But this is just about modulating task representations

How are task representations selected in the first place?

How are they sequenced?

Dynamics (decision-making) and Learning