Available online at www.sciencedirect.com

Behavioural and cognitive neuroscienceEditorial overviewAnn M Graybiel and Richard Morris

Current Opinion in Neurobiology 2011, 21:365–367

Available online 2nd July 2011

0959-4388/$ – see front matter

Published by Elsevier Ltd.

DOI 10.1016/j.conb.2011.06.005

Ann M Graybiel1 and RichardMorris2

1 Department of Brain and CognitiveSciences and the McGovern Institute forBrain Research, Massachusetts Institute ofTechnology, United States2 Centre for Cognitive and Neural Systems,

University of Edinburgh, United Kingdom

e-mail: graybiel@mit.edu

Ann Graybiel is an Institute Professor

at the Massachusetts Institute of

Technology, where she directs a

laboratory in the Department of Brain

and Cognitive Sciences and the

McGovern Institute for Brain

Research. Her work focuses on the

functions of brain systems related to

the basal ganglia, in the normal state

and in basal ganglia-based disorders.

Richard Morris is a Professor of

Neuroscience at the University of

Edinburgh. His primary research

interests are the neurobiology of

learning and memory, and the

applications of concepts and

techniques developed in this work to

help develop new therapeutics for

Alzheimer’s disease. He serves as an

advisor for a number of international

research institutes, and is a member of

the Scientific Advisory Board of the

Alzheimer’s Research Trust in the UK,

and the Scottish Science Advisory

Council. Latterly he served as Life

Sciences Coordinator of a UK

Government initiative on Cognitive

Systems. He is also an active member

of the Council of the European Dana

Alliance for the Brain (EDAB) whose

The brain has long been held to be the organ of the body responsible for

‘action’ while also mediating ‘perception, memory and thought’. This issue

of Current Opinion in Neurobiology considers the realm of what we may think

of as ‘thoughtful action’. Our invited contributors focus on the neurobio-

logical mechanisms responsible for the cognitive processes that underlie

both ‘action’ and ‘memory’ in circuits related to the striatum and hippo-

campus, and to regions of the neocortex and amygdala with which these

structures are directly connected. Importantly, the authors recognise, on the

one hand, the distinction between deliberative and goal-directed processes

of action-selection and processes that allow habitual behaviours, once

acquired, to unfold without continuous cognitive representation of their

consequences. On the other hand, the contributions reflect current interest

in distinctions among forms of memory categorized as recognition, episodic

and spatial memory, and the neurophysiologic mechanisms mediating

memory consolidation. Specifically, the reviews collected here range from

consideration of updated forms of reinforcement learning (RL) algorithms,

and how dopamine and local-circuits impact learning and cognition, to the

role of oscillatory rhythms in coordinating hippocampal and prefrontal

cortical circuits, the potential role of the striatum in language function,

and the neurobiology interrelating stress and memory. So broad a set of

topics reflects an increasing awareness that the functional categories that we

have traditionally built up in cognitive neuroscience do not so much fit single

brain regions as they do entire sets of interacting brain circuits.

A central focus of current work on basal ganglia-based circuits is how these

contribute to learning and memory functions in addition to executive func-

tions in the motor and cognitive realm [1]. A refreshing set of reviews directly

address this question, and together offer a rich survey of the reinforcement

learning (RL). Ito and Doya suggest that different cortico-basal ganglia loops

implement different forms of RL learning, and introduce hierarchical archi-

tectures in addition to model-free and model-based algorithms to account for

RL-related functions of different cortico-striatal systems. Bornstein and Dawdirectly address proposed subdivisions of striatum-based circuits and suggest

that blends of model-free and model-based formulations and the introduction

of Bayesian approaches may be important for understanding the sweep of

basal ganglia functions. Michael Frank highlights the direct-indirect pathway

dichotomy of trans-striatal circuits as a basis for action selection functions of

the basal ganglia, and emphasizes cross-talk across circuits related to cogni-

tion, movement and motivation. A forth review by van der Meer and Redishcontrasts the view of the ventral striatum as part of the motivational circuitry of

the brain, driving on-going behavioural decisions, with the view that it is the

critic RL architectures, proposing that these roles coexist and

odated by a dual ‘critic’ role.

mission is public awareness of

neuroscience.‘critic’ in actor-

can be accomm

www.sciencedirect.com Current Opinion in Neurobiology 2011, 21:365–367

366 Behavioural and cognitive neuroscience

Central to current views about the biological instantiation

of RL algorithms by the basal ganglia is evidence that

dopamine could signal positive and negative reward-pre-

diction errors and salience. Koos and Tepper review the

increasingly convincing evidence that the dopamine-con-

taining neurons of the ventral tegmental area (VTA)

projecting to the nucleus accumbens also release the fast

excitatory transmitter, glutamate. This set of findings

could presage a major reconsideration of VTA-accumbens

signaling.

Cools highlights evidence that dopamine’s actions in the

neocortex, hippocampus and striatum can have major

effects in biasing both cognitive functions and functions

related to the acquisition of habits by model-free RL. The

complementary review by Shohamy reviews evidence

from human brain imaging that further emphasizes the

broad range of functions of the striatum, driven by social

signals and a variety of other positive and negative feed-

back-based signals, including those generated internally.

The social importance of striatal function is further

emphasized by Enard, who engagingly reviews work

on the human ‘language gene’, FoxP2, and summarizes

evidence that overexpression of human FoxP2 in mice

has remarkably strong effects on the striatum, including

augmented plasticity, dendritic growth and decreases in

dopamine levels. Understanding the cognitive functions

of the basal ganglia will be critical for translational work

related to neuropsychiatric disorders.

If motor and cognitive action selection depend on circuits

leading through the striatum, what is it about the striatum

that makes this so? Two reviews approach this issue by

considering the intrinsic circuitry of the striatum. Old-enberg and Ding take us through the multiple ways that

the cholinergic interneurons interact with both input

fibers and output neurons of the striatum, themselves

influenced by thalamic and other inputs. Kravitz andKreitzer review the latest use of optogenetics to probe

neural functions, including those of striatal circuits. The

landscape of information about the striatum is broadening

at great speed, thanks to these powerful methods.

A perennial issue in the analysis of memory concerns the

place of ‘automatic processing’ [2,3] during what may

appear to be complex, intentional forms of cognitive

learning. This is particularly relevant to both recognition

memory — such as the visual paired-comparison (VPC)

task for humans and non-human primates and the analo-

gous novel object recognition (NOR) task for rodents (rats

and mice) — and spatial memory that many suppose also

consists of incidental processing components. In the

NOR task for example, there is no explicit reward for

any particular pattern of behaviour; rather, exploratory

behaviour occurs, habituates, and then changes in

response to the presentation of novel stimuli or changes

in the location or temporal context of a stimulus object.

Current Opinion in Neurobiology 2011, 21:365–367

Vann and Albasser provide a useful update of recent

studies pointing to a shift toward a deeper understanding

of the associative basis of some aspects of recognition

memory processing and the need to examine structures

beyond the hippocampus and perirhinal cortex with

which this form of memory has hitherto been associated.

One important theme of this issue is memory consolida-

tion. When a new event occurs and a memory trace is

encoded in the distributed networks of hippocampus,

amygdala and cortex, there is no guarantee that it will

last. It is widely believed that, if activated, the intersect-

ing processes of cellular- and systems-consolidation

are necessary for the initial and then eventual long-

term stabilisation of memory traces. Sutherland andLehmann’s review challenges this orthodoxy, questioning

whether systems consolidation actually occurs. They

propose a multiple storage site hypothesis with a single

consolidation mechanism operating in each. On this view,

and following multiple event reiterations, different mem-

ory representations are independently established in

multiple networks. Some detailed memories always

depend on the hippocampus, while others may be estab-

lished and maintained independently. In contrast,

Girardeau and Zugaro adopt a more conventional view

of systems consolidation, and outline their innovative

new experiments that deploy a neurophysiological ‘ripple

killing’ technique to establish a causal role for hippo-

campal sharp waves in the cortical stabilization of memory

traces. Interestingly, they suggest that this process may

happen during both sleep and waking restfulness. Finally,

Goosens reviews evidence that the hippocampus, both its

ventral and its dorsal parts, is important for acquisition,

consolidation, extinction, and renewal of aversive mem-

ories, and relates these hippocampal functions to control

of stress-induced corticosterone secretion as well as to

circuit functions relating the hippocampus to other

regions including the amygdala.

Two other reviews also consider the role of other oscil-

lations, notably theta and gamma rhythms, in hippo-

campal memory processing. Colgin notes how spatial

memory processing by the hippocampal formation is

gradually being broken down into the dissociable process

of location, metric and direction. She reviews her own

new research on how gamma rhythms gate discrete

aspects of hippocampal processing. Further, coherent

oscillations are observed across the hippocampus and

medial prefrontal cortex during various states of wakeful-

ness and during sleep. Theta oscillations may facilitate

interactions between the two regions during goal-directed

behaviour and working memory, and may recruit gamma-

mediated cell assemblies during these tasks. Echoing

Girardeau and Zugaro, she suggests that slow oscillations

(such as hippocampal sharp waves and medial prefrontal

cortex spindles) may promote off-line memory transfer

during slow-wave sleep, but in addition, hints that oscil-

www.sciencedirect.com

Editorial overview Graybiel and Morris 367

latory phase-locking between hippocampus and medial

prefrontal cortex may be disrupted during schizophrenia.

Benchenane, Tiesinga and Battaglia take this analysis a

step further to the level of mechanism. Agreeing that

theta and gamma oscillations are instrumental for inter-

actions between the prefrontal cortex with other brain

structures (including hippocampus), they suggest that

interneurons control oscillations and coherence across

structures. They also argue that dopamine is important

for engaging coherence and neural synchrony. Finally,

Gordon provides a thoroughgoing and critical view of the

steps needed to relate securely the occurrence of particu-

lar patterns of oscillatory activity and cross-region coher-

ence patterns, focusing on hippocampal-prefrontal

synchrony.

Three reviews consider aspects of basic neuroscience of

translational relevance. Bast’s analysis of hippocampal

function zeroes in on the often forgotten step of how

new memory traces come to control behaviour, a process

that he believes is mediated by an intermediary zone

along the longitudinal axis between the septal and

temporal poles. On the basis of anatomical, lesion and

single-unit recording studies, this intermediate hippo-

campal region is well-placed to transfer information to

striatal action circuits. Further, he considers the relevance

of overactivity of neurons in the hippocampus as occurs in

schizophrenia and suggests that, by virtue of the circuitry

mediating the translation from learning to behaviour,

could be a contributor to the neural network disruptions

underlying aspects of psychosis. A second domain con-

cerns stress. Research on stress and cognition has recently

been addressing intriguing neurobiological challenges

associated with non-genomic as well as genomic con-

sequences of corticosteroids and their impact on gluta-

www.sciencedirect.com

mate receptors and synaptic proteins. The complexity of

these has attracted the attention of computational scien-

tists interested in mathematical models of learning, and

the question of how stress may impact on specific sub-

processes. Luksys and Sandi consider diverse aspects of

this interdisciplinary analysis, and suggest that in the

computational terms of temporal-difference learning,

stress results in higher exploitation and steeper future

discounting. Finally, Chatterji reviews recent findings

emerging from mouse models of Fragile-X syndrome

including the recent excitement of pharmacological res-

cue. Aberrant synaptic transmission seems to be caused in

hippocampus by both pre- and post-synaptic deficits, one

prominent theory focusing on the role of metabotropic

glutamate receptors in mediating these deficits and point-

ing to drugs that might rescue the phenotype. Chatterji

notes that long-term potentiation in the amygdala is

impaired, indicating that deficits in neurotransmission

there may be different to those seen in hippocampus.

Despite these subtle differences, there is evidence for

reversal and rescue in amygdala also.

References and recommended readingPapers of particular interest, published within the period of review,have been highlighted as:

� of special interest�� of outstanding interest

1. Graybiel AM: Habits, rituals and the evaluative brain, Ann. Rev.Neurosci. 2008, 31:359-387.

2. Moscovitch M: Memory and working-with-memory: Acomponent process model based on modules and centralsystems. J. Cognit. Neurosci. 1992, 4:257-267.

3. Morris RG, Frey U: Hippocampal synaptic plasticity: role inspatial learning or the automatic recording of attendedexperience? Philos. Trans. R. Soc. Lond. B. Biol. Sci. 1997,352:1489-1503.

Current Opinion in Neurobiology 2011, 21:365–367