Research Notes in Neural Computing

Managing Editor Bart Kosko

Editorial Board S. Amari M.A. Arbib R. Eckmiller C. von der Malsburg

Advisory Board Y. Abu-Mostafa A.G. Barto E. Bienenstock l. Cowan

M. Cynader W. Freeman G. Gross U. an der Heiden M. Hirsch T. Kohonen l.W. Moore

L. Optican A.I. Selverston R. Shapley B. Soffer P. Treleaven W. von Seelen B. Widrow S. Zucker

Michael A. Arbib Shun-ichi Amari Editors

Dynamic Interactions in Neural Networks:

Models and Data

With 87 Illustrations

Springer-Verlag New York Berlin Heidelberg

London Paris Tokyo

Michael A. Arbib Center for Neural Engineering University of Southern California Los Angeles, CA 90089-0782 USA

Managing Editor Bart Kosko Engineering Image Processing Institute University of Southern California University Park Los Angeles, CA 90089-0782 USA

Shun-ichi Amari Department of Mathematical Engineering

and Instrumentation Physics University of Tokyo Tokyo 113 Japan

Library of Congress Cataloging-in-Publication Data Dynamic interactions in neural networks: models and data / Michael A.

Arbib and Shun-ichi Amari, eds. p. cm.-(Research notes in neural computing; I)

Bibliography: p.

ISBN-13: 978-0-387-96893-3 e-ISBN-13: 978-1-4612-4536-0 DOl: 10.1007/978-1-4612-4536-0

I. Neural circuitry. 2. Neural computers. I. Arbib, Michael A. II. Amari, Shun'ichi. III. Series. QP363.3.D96 1988 006.3~cI9 88-29500

Printed on acid-free paper © 1989 by Springer-Verlag New York Inc. All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer-Verlag, 175 Fifth Avenue, New York, NY 10010, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use of general descriptive names, trade names, trademarks, etc. in this publication, even if the former are not especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone.

Camera-ready copy provided by the authors.

9 8 7 654 3 2

PREFACE

This is an exciting time. The study of neural networks is enjoying a great renaissance, both

in computational neuroscience - the development of information processing models of living

brains - and in neural computing - the use of neurally inspired concepts in the construction of

"intelligent" machines. Thus the title of this volume, Dynamic Interactions in Neural Networks: Models and Data can be given two interpretations. We present models and data on the dynamic

interactions occurring in the brain, and we also exhibit the dynamic interactions between research

in computational neuroscience and in neural computing, as scientists seek to find common

principles that may guide us in the understanding of our own brains and in the design of artificial

neural networks. In fact, the book title has yet a third interpretation. It is based on the

U.S.-Japan Seminar on "Competition and Cooperation in Neural Nets" which we organized at

the University of Southern California, Los Angeles, May 18-22, 1987, and is thus the record of

interaction of scientists on both sides of the Pacific in advancing the frontiers of this dynamic,

re-born field.

The book focuses on three major aspects of neural network function: learning, perception,

and action. More specifically, the chapters are grouped under three headings: "Development and

Learning in Adaptive Networks," "Visual Function", and "Motor Control and the Cerebellum."

In Chapter 1, we have provided a brief outline of the contents of each chapter in this book,

placing it in the perspective of current developments in the field of neural networks. Here we

simply offer a quick glimpse of how the contributions in each of the three parts of this volume

hang together.

Part I, Development and Learning in Adaptive Networks, begins with a mathematical

perspective on "Dynamical Stability of Formation of Cortical Maps" by Amari. We then turn to

some amazing empirical data which encourages the search for general principles of neural

development as Sur reports on the functional properties of visual inputs that he has induced into

auditory thalamus and cortex. Schmajuk reports on his modeling of "The Hippocampus and the

Control of Information Storage in the Brain" to give us fresh insight into the role of this region in

the formation of long term memories. We then turn to three contributions to neural computing.

In "A Memory with Cognitive Ability," Shinomoto studies learning rules which satisfy the

physiological constraint that excitatory synapses must remain excitatory and inhibitory synapses

inhibitory. In "Feature Handling in Learning Algorithms," Hampson and Volper add an

important new chapter in the study of computational complexity of neural networks. Finally,

vi

Miyake and Fukushima build on earlier work on the Neocognitron to present "Self-OrganIzing

Neural Networks with the Mechanism of Feedback Information Processing."

Part II, Visual Function, starts with Arbib's "Interacting Subsystems for Depth Perception

and Detour Behavior," another chapter in the evolution of Rana computatrix, the computational

frog. The rest of Part II presents important neurophysiological data ripe for modeling. Hikosaka

analyzes the "Role of Basal Ganglia in Initiation of Voluntary Movement," while the final two

papers analyze the visual mechanisms in monkey cortex that lie beyond the primary visual areas.

Desimone, Moran and Spitzer probe "Neural Mechanisms of Attention in Extrastriate Cortex of

Monkeys," while Miyashita discusses "Neuronal Representation of Pictorial Working Memory

in the Primate Temporal Cortex."

The last Part of the volume deals with Motor Control and the Cerebellum. Kawato, Isobe,

and Suzuki apply models of "Hierarchical Learning of Voluntary Movement by Cerebellum and

Sensory Association Cortex" to learning trajectory control of an industrial robotic manipulator.

The role of the cerebellum in adapting the control of eye movements is taken up by Fujita in his

paper on "A Model for Oblique Saccade Generation and Adaptation," and by Miyashita and Mori

in their study of "Cerebellar Mechanisms in the Adaptation of the Vestibulo-Ocular Reflex."

Paulin offers a more abstract mathematical perspective by arguing for "A Kalman Filter Theory

of the Cerebellum." To close the volume, Moore and Blazis again integrate a theory of

adaptation in neural networks with data on animal conditioning experiments to provide new

insights into "Conditioning and the Cerebellum."

We close this preface with a number of acknowledgements. The U.S.-Japan Seminar held

at USC in May of 1987 was the sequel to an earlier one held in Kyoto in February of 1982. The

Proceedings of the 1982 meeting were published as Competition and Cooperation in Neural Nets

(S. Amari and M. A. Arbib, Eds.) in the Springer Lecture Notes in Biomathematics. We wish to

record our thanks to the National Science Foundation (USA) and the Japan Science Foundation

for their support of both meetings, and to Dean Wagner and the USC program in Neural,

Informational and Behavioral Sciences (NIBS) for supplementary funding of the USC Meeting.

Finally, special thanks to Lori Grove and Debbie O'Rear for all they did to make our meeting

such a success.

Los Angeles and Tokyo,

June 1988

Michael A. Arbib

Shun-ichi Amari

Table of Contents

Preface ........................................................ v

Dynamic Interactions in Neural Networks: An Introductory Perspective

Michael A. Arbib .............................................. 1

I. Development and Learning in Adaptive Networks .................... 13

Dynamical Stability of Fonnation of Cortical Maps

Shun-ichi Amari . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Visual Plasticity in the Auditory Pathway: Visual Inputs Induced into Auditory Thalamus

and Cortex illustrate Principles of Adaptive Organization in Sensory Systems

Mriganka Sur. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

The Hippocampus and the Control of Information Storage in the Brain

Nestor A. Schmajuk ............................................ 53

A Memory with Cognitive Ability

Shigeru Shinomoto . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

Feature Handling in Learning Algorithms

S.E. Hampson and D.J. Volper. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Self-Organizing Neural Networks with the Mechanism of Feedback

Infonnation Processing

Sei Miyake and Kunihiko Fukushima . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 107

II. Visual Function 121

Interacting Subsystems for Depth Perception and Detour Behavior

Michael A. Arbib .............................................. 123

Role of Basal Ganglia in Initiation of Voluntary Movements

Okihide Hikosaka ............................................. 153

Neural Mechanisms of Attention in Extrastriate Cortex of Monkeys

Robert Desimone, Jeffrey Moran and Hedva Spitzer. . . . . . . . . . . . . . . . . . . . . . . 169

Neuronal Representation of Pictorial Working Memory in the Primate Temporal Cortex

Yasushi Miyashita ............................................. 183

III. Motor Control and the Cerebellum ............................. 193

Hierarchical Learning of Voluntary Movement by Cerebellum and Sensory

Association Cortex

Mitsuo Kawato, Michiaki Isobe and Ryoji Suzuki ........................ 195

viii

A Model for Oblique Saccade Generation and Adaptation

Masahiko Fujita. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

Cerebellar Mechanisms in the Adaptation of Vestibuloocular Reflex

Yasushi Miyashita and Koichi Mori . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

A Kalman Filter Theory of the Cerebellum

Michael Paulin. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

Conditioning and the Cerebellum

John W. Moore and Diana EJ. Blazis ................................ 261

Dynamic Interaction in Neural Networks:

An Introductory Perspective

Michael A. Arbib

Center for Neural Engineering

University of Southern California, Los Angeles

It is the purpose of this introduction to briefly review the papers in each of the three parts of

the volume, and then conclude with a brief unifying perspective.

1. Development and Learning in Adaptive Networks

In "Dynamical Stability of Formation of Cortical Maps," Shun-ichi Amari emphasises

mathematical models. These are important because they point to intrinsic mechanisms - simple

models provide a basis for understanding details. In particular, Amari stresses a field theory of

self-organizing neural nets. An important case is a layered network, with field equations set up

for each layer. The aim is to solve these equations andlor provide a stability analysis. Studies in

the dynamics of learning and self-organization include the learning of input-output relations.

Turning to neural representation of signals, Amari notes that local self-organization of a field

may yield the representation of a localized pattern in the external world by a localized

"grandmother cell" representation (e.g. orientation- selective cells; retinotopy), or by a distributed

representation (i.e., a pattern distributed across all or part of the network). He thinks that the

brain uses both types of representation, and offers a mathematical demonstration that neural

networks can form both types. Key questions here are: How can topological arrangements be

reflected in a localized representation - especially when the topology is of higher dimension

than the two dimensions of the neural layer? And how does the resolution of such a

representation reflect the varying interest of different signals?

Amari studies three properties of a cortical map formed by self-organization: the topology of

a signal space is preserved in some sense in the map; frequently applied signals occupy wide

areas in the map with fine resolution; and, even when both the signal space and the neural field

are continuous, a mosaic or block structure emerges in both spaces, and similar signals are

categorized and represented by one mosaic. The last part of the paper treats associative memory.

Earlier work emphasized the statistical neurodynamics of similarity. His results treated the stable

states, but now he stresses that the real interest is in the transients, and these are addressed in

recent computer simulations.

2

In evaluating these results, it should be noted that they are based on very general field

equations. However, circuits in different parts of the brain have different characteristics. As we

develop different models for hippocampus (Chapter 4), cerebellum (Part III of this volume),

cerebal cortex, brain stem, etc., it will be interesting to see to what extent Amari's statistical

neurodynamics can be adapted to these more specific circuitries.

Turning from general mathematical theory to experimental data on the development of neural

wiring systems, Mriganka Sur argues that "Visual Inputs induced into Auditory Thalamus and

Cortex Illustrate Principles of Adaptive Organization in Sensory Systems." By adapting surgical

procedures developed in hamster by Schneider, he has been able to induce retinal projections in

newborn ferrets to enter the medial geniculate nucleus (MGN), the principal auditory thalamic

nucleus. Electrophysiological recordings reveal that the "visual cells" in the MGN have large,

diffuse receptive fields and receive input from retinal ganglion cells with slow conduction

velocities. Visual cells with long conduction latencies and large receptive fields can also be

recorded in the primary auditory cortex! Importantly, these receptive fields are confined to the

contralateral visual field, indicating theat the cortical visual responses arise from input through

the MGN and not from the intact hemisphere via the corpus callosum. Finally, some visual cells

in auditory cortex have oriented receptive fields that resemble those of complex cells in the

primary visual cortex. These findings prompted John Moore to ask at the USC meeting "What do

the lights sound like?"

These results suggest that some of the operations considered "wired into" the visual

thalamus and cortex may in fact be the result of general developmental mechanisms responding to

structured visual inputs. In other words, what is intrinsically visual about central visual circuitry

may not be so much intrinsic to the circuitry itself but rather the result of receiving visual input.

Such results are encouraging for general models which explain the brain's wiring in terms of

general mechanisms (e.g., those that involve excitation from below and recurrent inhibition as

suggested by von der Malsburg and Amari) without making any assumption about the visual

nature of the input.

Sur speculates that at least some aspects of intrinsic connectivity may be quite similar across

thalamic and cortical areas, and one function of sensory cortex, for example, may be to perform

stereotypical transformations of input akin to the simple and complex transformations done by

visual cortex. What might the function of complex cells be in the auditory system? There are

cells in the auditory system which respond either to sweeps from low to high or high to low,

and this may be the auditory analog of orientation-tuning; while Merzenich has found auditory

cells with broad tuning curves, but which respond to any input within the range.

Nestor Schmajuk provides an important bridge between the study of neural networks and

the experimental data on animal conditioning as he analyzes "The Hippocampus and the Control

3

of Information Storage in the Brain." The general idea is that different information is stored in

different areas of the brain and that the hippocampus acts as coordinator. In the 60's, Sokolov

proposed that we model the world, generating an orienting response when input differs from

model. The q-rhythm in hippocampus seems proportional to the size of the orienting response.

Schmajuk models hippocampus in terms of attentional-associative networks for higher-order

conditioning and sensory preconditioning - he call these S-P-H networks, the Schmajuk

version of Pearce and Hall networks. A given CSi can yield both a direct flrst-order prediction

of some US, and a second-order prediction via some intervening CSr. This involves prediction

of which CSks occur and which do not occur. The model makes assumptions as to how

attentional terms change with hippocampal lesions. Also, it assumes that CS-CS associations

vanish with hippocampal lesions. Schmajuk develops a top-down approach: he tunes the model

to yield the effects of hippocampal lesions, L TP, etc. He then tries bottom-up to relate the model

to plausible hippocampus circuitry. The model seems to flt well about 80% of the phenomena of

conditioning.

With the next three papers, we tum to studies in the theory of learning networks in general;

not of speciflc brain regions or of realistic neural circuitry. The role of learning is to build

"memory structures" that fill the gap between information supplied by the environment and that

required to get the job done. In his study of "A Memory with a Cognitive Ability," Shigeru

Shinomoto joins that growing group of physicists who provide mathematical analyses of

auto-correlation matrix memories. Where many such memories are constructed with the

possibility of arbitrary coupling between the neurons, Shinomoto introduces the physiological

constraint that the sign of the synaptic coupling is uniquely determined by presynaptic neuron -

i.e., neurons must be either excitatory or inhibitory. For a novel pattern, the Hopfleld model

gets trapped in spurious attractors. However, Shinomoto's networks have the property that they

can signal non-retrieval - if an input signal has little correlation with any of the stored

memories, it gives a clear response by going into a special mode which may be stationary or

periodic, depending on a population of excitatory or inhibitory neurons. Thus the system

acquires a new computational ability to determine whether an input pattern is identiflable or not,

and thus tends to get rid of spurious memories. In concluding his talk at the USC meeting,

Shinomoto quoted Confucius: "To recognize a thing, you should flrst identify whether or not it is

one with which you are acquainted."

Hampson and Volper, in their paper on "Feature Handling in Learning Algorithms," extend

the theoretical analysis of learning networks by offering a complexity analysis of connectionistic

representation and learning schemes. They note that the Perceptron convergence theorem gives a

bound on the number of trials required to achieve convergence, but that the bound M Iw12/a2

has the unfortunate property that it depends on the choice of a solution w, and it is such a w we

seek. However, one can use the upper bound to evaluate various learning schemes. Empirical

results generally reflect the time complexity based on the upper bound. They note that one aspect

4

that slows Perceptron convergence is that learning does not occur for features absent on a given

trial. An alternative model codes absence by -1 instead of O. Thus a feature gets adjusted

whether present or absent. Another model is the two-vector model using two vectors of length

d. It is representationally equivalent, but has a different learning character.

They have also analysed neuron learning schemes based on conditional probabilities,

relating them to laws in the animal learning literature. Adding nodes to express conditional

probabilities speeds learning dramatically: they call the result an OT (operator training)

algorithm. Neurons which respond most strongly to a new input also learn the most from that

new input. For the multiplexer, Barto's 4 node network takes 130,000 presentations; whereas

an OT net with 5 nodes needs only 524 presentations. It thus seems possible to speed up

learning by use of salience.

In their analysis of "Self-OrganIzing Neural Networks with the Mechanism of Feedback

Information Processing," Sei Miyake and Kunihiko Fukushima study several neural network

models in which a feedback signals are used to emphasize novel features. (1) A multilayered

network which has both feedforward connections and feedback connections from the

deepest-layer cells to the front-layer cells, with both types of connection being self-organized. (2)

n another algorithm, the growth of connections is controlled by feedback information from

postsynaptic cells. Even if a new pattern is presented, resembling one of the learning patterns

with which the network has been organized, the network is capable of being self-organized

again, and a cell in the deepest layer comes to acquire a selective responsiveness to the new

pattern. 3) A third model has modifiable inhibitory feedback connections between the cells of

adjoining layers. If a feature-extracting cell is excited by a familiar pattern, the cell immediately

feeds back inhibitory signals to its presynaptic cells. On the other hand, since the

feature-extracting cell does not respond to an unfamiliar pattern, and so circuits detecting novel

features develop. (4) Finally, a self-organizing neural network which has an ability of symbol

information processing has been proposed to, in some sense, take "context" into account. Even

if an imperfect word is given to the network after completion of the self-organization, the

network should be able to estimate its omitted letter by contextual information.

2. Visual Function

Providing a novel perspective on the notion of "sensory fusion," Michael Arbib starts his

study of "Interacting Subsystems for Depth Perception and Detour Behavior" with an analysis of

neural networks for depth perception. The problem for many models of binocular perception is

to suppress ghost targets. The Cue Interaction Model uses two systems, each based on a

cooperative computation stereopsis model, to build a depth map. One is driven by disparity

cues, the other by accomodation cues, but corresponding points in the two maps have excitatory

5

cross-coupling. The model is so tuned that binocular depth cues predominate where available,

but monocular accomodative cues remain sufficient to determine depth in the absence of

binocular cues. The Prey Localization Model incorporates a triangulation hypothesis. Each side

of the brain selects a prey target based on output of the contralateral retina, and computes a depth

estimate by triangulation to adjust lens focus. If the selected retinal points correspond to the

same prey-object, then the depth estimate will be accurate and the object will be brought into

clearer focus, "locking on" to the target. If the points do not correspond, the resulting lens

adjustment will tend to bring one of the external objects into clearer focus, and the two halves of

the brain will tend to choose that object over the other.

Arbib then introduces the notion of a schema as a unit of analysis intermediate between

overall behavior and the details of neural networks. Various models of detour behavior are

presented in which the above depth models may be seen as subschemas which help a toad locate

worms and barriers as it determines the path to its prey. Such schema/neural considerations are

relevant to the desigu of "perceptual robots. "

Okihide Hikosaka's paper on "Role of Basal Ganglia in Initiation of Voluntary Movements"

returns us to the experimental analysis of neural mechanisms, this time in the study of eye

movements. Wurtz trained a monkey to fIxate a light while holding a lever and then release the

lever when the light dims to get water reward. The substantia nigra is a very busy region, with

incessant activity at 100 Hz even when the animal is sleeping. But one sees cessation of activity

there after the target dims, there is then a saccade, after which substantia nigra background

resumes. The cessation of substantia nigra activity occurs at the same time as a burst of activity

of superior colliculus neurons. Since substantia nigra projects to superior colliculus this strongly

suggests an inhibitory connection. But how do the nigral neurons stop discharging? The nigra

receives input from other regions of the basal ganglia, including the caudate nucleus. So

Hikosaka worked on the caudate with a similar paradigm. Caudate is a quiet area; but he found

cells which discharge just before saccadic movements. This suggests that the caudate acts by

disinhibiting the phasic inhibition of superior colliculus by substantia nigra. It is the phasic

inhibition from the nigra that stops superior colliculus from yielding constant eye movements in

response to its bombardment of excitation.

Many units in the basal ganglia are related to movement, but are not purely motor. Hikosaka

found a cell which did not fIre for a visually - directed saccade, but did fIre for a

"memory-guided saccade," i.e., a saccade to a recalled target position. Another unit started

discharging on receiving an instruction to saccade to a yet-to- be-presented, and continued fIring

until the target appeared at the anticipated location. There is reciprocal interaction between basal

ganglia and cortex; as well as mutually excitatory connections between thalamus and cortex.

Caudate activity can release the cortical-thalamic system from substantia nigra inhibition. These

substantia nigra cells seem to act as a short-term memory. They fIre when a target is flashed

6

before it can be used, but not if a saccade can follow immediately. A sequence of substantia

nigra disinhibitions may be involved in complex movements. The loop from cortex-thalamus to

caudate to substantia nigra could act as a flip-flop circuit to hold short-term information in the

brain.

The experimental analysis of vision continues with the study of "Neural Mechanisms for

Preattentive and Attentive Vision in Extrastraite Cortex of the Primate" by Robert Desimone,

Jeffrey Moran, and Hedva Spitzer. Since we are aware of only a small portion of the

information on our retinas at anyone moment, most of this information must be filtered out

centrally. Yet, this filtering cannot easily be explained by the known physiological properties of

visual cortical neurons. At each successive stage along the pathway from the primary visual

cortex into the temporal lobe, the pathway known to be crucial for pattern perception, there is an

increase in receptive field size. Moran and Desimone recorded from this pathway, studying

covert attention: without eye movements, it is possible to attend to one or other of two stimuli.

The locus of attention has dramatic effect on activity of V 4 cells. They found V 4 cells which, for

a fixed stimulus, respond when the focus of attention is in one subfield, but not the other. The

animal effectively shrinks its receptive field, so that the cell gives a response only inside the RF

to a "good" stimulus. However, when the animal attends outside the overall RF of a cell, the

cell seem to fire its normal response to any and all stimuli within that RF.

In plotting a histogram for the response of a V 4 cell to attended and ignored stimuli,

Desimone et al. see a difference with latency of 90 msec., which may thus be thought of as the

time required for selective attention to turn off the cell's response. They posit that the V 4 cell

receives input both from the unattended stimulus and the attended stimulus, but with the

unattended stimulus gated out by the attentional filter. n the chronic filter model, the attentional

filter is chronically active; while in the triggered filter model, it requires a signal from the

unattended stimulus to activate the filter. The data favor the triggered filter model. In discussion

at the USC meeting, Desimone asserted that he does not think selective attention works at the

geniculate level. The "ignored" data can yield interference effects even at quite late stages.

Thinks attention may only operate after segmentation of the input into objects. Suppressive

surrounds for cortical cells seem to come from other cells in the same area for figure-ground,

color constancy, etc. - one does not want to tum off visual input before it enters into inter-areal

processing. He thinks suppression acts at the module level: gating cells within the module

containing the locus of attention, but not affecting other modules.

Our tour of "modeling-rich" experimental data on the visual system concludes with Yasushi

Miyashita's account of "Neuronal Representation of Pictorial Working Memory in the Primate

Temporal Cortex." it has been proposed that visual memory traces are located in the temporal

lobes of cerebral cortex, but in the past neuronal responses to specific complex objects such as

hands and faces have been found to cease soon after the offset of stimulus presentation.

7

Miyashita has recently found a group of shape-selective neurons in an anterior ventral part of

monkey temporal cortex which exhibit maintained activity during the delay period of a visual

working memory task. He thus argues that working visual memory is encoded in temporary

activation of an ensemble of neurons in visual association cortex, rather than in a brain area

specialized for working memory per se .

3. Motor Control and the Cerebellum

The analysis of motor control and the cerebellum commences with the presentation of "A

Hierarchical Neural-Network Model for Control and Learning of Voluntary Movement and its

Application to Robotics" by Mitsuo Kawato, Michiaki Isobe and Ryoji Suzuki. The approach is

to study trajectory formation and control of human and robotic arms as an optimization problem

with constraints in nonlinear dynamics. Abend, Bizzi, and Morasso studied human planar

horizontal arm movements between a pair of targets, and observed a straight line trajectory with

bell-shaped velocity curve. Flash and Hogan proposed that the trajectory minimizes "jerk;" but

Uno, Kawato and Suzuki offered an alternative, the minimum-torque change model, and

developed an iterative learning algorithm to determine the optimal trajectory, constrained by the

dynamics of the arm. Their model is better than the minimum jerk model in predicting the bowed

trajectory humans use to bring the arm from side to front.

The two steps of coordinate transformation and generation of motor command can be solved

simultaneously by trial and error learning. Kawato et al. postulate that learning control by

iteration in body space might be achieved in Area 2; while learning control by iteration in visual space might occur in Areas 5 and 7.

At the Kyoto meeting in 1982, Tsukuhara and Kawato gave a model of rubro-cerebellar

learning, a neural identification algorithm structured as a hierarchical neural network model.

Association cortex sends the desired trajectory in body coordinates to motor cortex, which in tum

sends muscle commands. The cerebro-cerebellum and parvocellular red nucleus generate an

internal model of the inverse dynamics. The spino-cerebellum and magnocellular red nucleus

build an internal model of dynamics. The learning equation involves heterosynaptic plasticity.

More recently, Kawato and his colleagues have applied such a controller to a robot manipulator.

Control performance improved, and the feedback mode changed to feedforward as learning

proceeded. Moreover, the learning generalized from a few taught trajectories to yield smooth

control of new trajectories. With a 6 degree of freedom manipulator, they used 925 synaptic

weights. They predict that, with parallel implementation, the method is IOO-fold-faster than the

computed torque method.

8

Before outlining the other papers in this section, we fIrst recall a presentation that Jarnes

Bloedel made at the USC meeting, summarizing a variety of viewpoints on the question "The

Operation of the Climbing Fibers: Establishment of an Engram for Motor Learning or an

Interaction Critical for Real Time Processing?" The basic organizational point is that cerebellar

cortex receives two types of input fIber: MFs (mossy fibers) originate in brainstem, spinal cord,

cerebellar nuclei, etc. They synapse on granule cells, which have axons forming parallel fibres

which contact Purkinje cells along their course. By contrast, CFs (climbing fIbers) originate from

only one site, the inferior olive. Each Purkinje cell has its dendrites entwined with the branches

of one climbing fIber - a unique synaptic relationship. Current debate over the involvement of

cerebellar cortex in motor learning focuses on three general issues:

1. Is the cerebellum involved in motor learning?

2. Is the cerebellum the site of learning?

3. What is the role of the climbing fiber system?

Bloedel summarized the viewpoints on these issues as follows:

(1): Involvement: All investigators accept that cerebellum is involved in some way in some

aspects of motor learning.

(2) Site: There are three camps: camp A strongly advocates cerebellum as the site of motor

learning; carnp B sees such a role for cerebellum as strongly supportive of their data; while carnp

C argues that cerebellum is not the site of learning.

(3) Role ofCF system: Here the debate is between storage theories which hold that he action of

the CF on the dendrites of the PC (Purkinje cell) produces long-lasting changes in the parallel

fibre synapses, especially with concurrent activation with them; and the notion that the CF

system is involved in real-time processing operations.

Masahiko Fujita proposes "A Model for Oblique Saccade Generation and Adaptation." It

explains a possible neural mechanism which successfully decomposes a vectorial eye movement

velocity into horizontal and vertical eye velocities. Functional roles and neural mechanisms of

the cerebellar vermis in saccadic eye movements in adaptation will also be discussed on the basis

of such model. He stresses the importance of the idea of population coding in motor systems, in

contrast with the idea of feature extracting in sensory systems. A saccade of 1 -400 takes 20-100

msc. What is the spatial-to-temporal transformation? Fujita studies the logical structure for the

generation of oblique saccades.

Building on a model of Robinson, Scudder incorporated the superior colliculus into his

model of saccade generation. The position of the target together with the direction of gaze

determine the pattern of retinal activity which is transmitted to the cerebrum for visual

processing, yeielding activity in the frontal eye fIelds which combines in the superior colliculus

with direct retinal input. Outflow thence to midbrain and pons projects to the horizontal

andvertical motor systems, thus in tum affecting gaze direction. Robinson's model was a

high-gain position servo, discretely sampling visual cortex. The estimated difference between

9

target and foveal angle drives rapid eye movement via a burst discharge. The saccade loses

velocity rapidly where the error falls to zero. A neural integrator holds the eye at its new position

despite the elasticity of eye muscles. van Gisbergen et al gave 2 models extending this to 2D

saccades. In model A, the error vector is decomposed intoto vertical and horizontal dimensions,

each driving a pulse generator for its respective direction. In model B, the error vector drives a

pulse generator whose vector is then decomposed. The problem with A is that if both systems

saturate, one gets saccade starting in fixed direction - and this doesn't happen. B yields correct

saccades. Fujita thus incorporates superior colliculus in a B-type model.

Eric Schwartz showed that the projection to visual cortex is described by the complex log z.

Fujita assumes the same differential distribution of nerve fibres in deep layers of superior

colliculus. He codes saccade size by the number of excited nerve fibres projecting from given

spot in superior colliculus to long-lead burst neurons in brainstem - which form what he calls

the "LLBN plane." He assumes a rectangular dendritic tree for LLBN neurons, and assumes a

uniform distribution of size and location of the rectangles. He then gives a circuit diagram for

the reticular formation. The model has gain variables for the projection of superior colliculus to

the various populations of LLBN s. Fujita has a model for how the olivocerebellar system might

adjust the gains, in line with Ito's ideas on the corticonuclear microcomplex. The model

simulates them well, including the long-term change in the gain.

A more abstract, mathematical view of the role of the cerebellum is provided by Michael G.

Paulin's Kalman Filter Theory of the Cerebellum. Under certain conditions a Kalman filter is the

optimal estimator for the state variables of a dynamical system. The hypothesis put forward here

is that the cerebellum is a neural analog of a Kalman filter, providing optimal state variable

estimates for reflex control. In particular it is hypothesized that interactions between the

cerebellar cortex and the intracerebellar nuclei are described by the matrix Ricatti equations of the

Kalman filter. The Vestibulo-Ocular Reflex (VOR) affords a unique opportunity for developing

and testing a model of cerebellar function based on the Kalman filter hypothesis. The

vestibulocerebellum and vestibular nuclei can be regarded as prototypes of the cerebellar

neocortex and intracerebellar nuclei. The VOR can be modelled as a Kalman filter which

estimates head rotational velocity from sense data. This model parsimoniously describes known

VOR dynamics and predicts new observable phenomena. Its specific predictions include

time-varying VOR dynamic parameters during head movements, frequency selectivity,

autocorrelogram storage and predictive feedforward in the VOR.

According to the model, the vestibulocerebellum provides optimizing time-variation (context

sensitivity) of VOR dynamic parameters. Therefore the model predicts that the VOR will become

time-invariant when the relevant regions of the vestibulocerebellum are disabled. It predicts

dysmetria due to the inability to regulate movements along trajectories. Segmentation of

maneuvers ("loss of coordination") is a consequence of dysmetria. Decreased ability to perform

10

simultaneous tasks is a consequence of segmentation. The model predicts that cerebellar patients

will adopt maneuvering and locomotory strategies which limit the need for time-variation in

reflex dynamics.

The model predicts that reflex learning is accompanied by modifications in both the

cerebellar cortex and the cerebellar nuclei. In particular, it predicts that VOR learning involves

modifications in the vestibular nuclei. Damage to the cerebellar cortex will have a small effect on

reflex learning, damage to the nuclei will have a larger effect. In each case this is accounted for

as a side-effect of the normal role of the cerebellum in movement control rather than as a specific

"repair" function of the cerebellum.

Yasushi Miyashita and Koichi Mori study "Cerebellar Mechanisms in the Adaptation of the

Vestibulo-Ocular Reflex." They use electrophysiology on a cerebellar slice to study

heterosynaptic interaction between climbing fibre and parallel fibre synapses. They use

"threshold straddling" for CF activation - setting the potential at a level where minute changes

can trigger firing without changing input to other elements. Antidromic activation of PC was

observed, but it did not invade the dendrites. For conditioning experiments, they pair CF and PF

activation. Lesion of flocculus abolished adaptability of the VOR; interruption of the CF

pathway also impairs VOR adaptibility. A flocculus lesion also affects the dynamics ofVOR and

OKR, whereas a preolivary lesion does not affect the dynamics.

Our final chapter returns us to the animal conditioning literature introduced by Schmajuk in

his chapter. John Moore and Diana Blazis' study "Conditioning and the Cerebellum." They

consider simulation of the classically conditioned nictitating membrane response by a neuron-like

adaptive element: a real-time variant of the Sutton-Barto model. They show

(a) how the model replicates most real-time conditioning data;

(b) a possible implementation in cerebellar cortex; and

(c) a mix of anatomical and physiological data.

Sutton and Barto consider an element with n inputs x I , ... ,xn corresponding to CS b ... ,CSn

plus an input Xo corresponding to VCS. These inputs have weights vCS 1, ... ,vCSn and I

respectively. The output s(t) serves as CR and VCR. The experimenter provokes a move of

rabbit's nictitating membrane by a brief mild electric shock. The eye retracts; the membrane

movement signals this. Where the CS as presented is an on-off step, the CS as it arrives

centrally is both smoothed and delayed. Sutton-Barto add to usual conditioning models of the

neuron the notion of an eligibility period - a trace of the input x, but shifted in time and

decaying with a different rate. Moore and Blazis give a complex diagram of anatomical

connections, including HVr in cerebellum (the only area of cerebellar cortex found by Glickstein

and Yeo to be involved in conditioning of NMR, the nictitating membrane response); IP (the

only nucleus involved, according to Thompson); red nucleus (the path for NMR motor output,

11

according to Moore) and DAO, the part of 10 which is involved. Cortical, hypothalamic, and

hippocampal lesions have no longtenu effect.

In their model, the only CF input is to Golgi cells. Basket cells inhibit PCs on another

beam. PC's inhibit IP cells, while PC axon collaterals inhibits Golgi cells. (This anatomy is

based on Ito's book.) They considers a number of Hebbian-like environments. Their model

doesn't give a role for CFs, yet olivary lesion disrupts conditioning. Others see precise pieces of

the olive as critically necessary. However, olivary lesion experiments do not implicate CF

synapses as the site of learning. Olivary lesions disrupt the function of the part of cerebellar

cortex to which it projects - a disruption irrespective of learning.

4. Conclusion

The papers reviewed above contain a number of crisp mathematical analyses of abstract

neural networks; they also offer intriguing recent biological findings on development, learning,

vision and movement. More importantly, they give a vivid view of the excitement to be gained

by confronting the study of biological neural networks with the design and analysis of artificial

neural networks. What the experimental papers make dazzlingly clear is that the brain is a

"house with many mansions," and that we have much to learn yet from how the distinctive

neuronal structure of each region fits it for its role in the overall structural architecture of the brain

and the overall functional architecture of behavior. This leads me to argue that the future of

neural computing (the technological spin-off of all this) lies not in the development of huge

homogeneous networks evolving according to some grand unified learning rule, but rather in

tenus of an understanding of a mode of cooperative computation which integrates the activity of

diverse subsystems, many of which are quite specialized in structure. I see the design of Sixth

Generation Computers as employing adaptive network methods to tune specific subsystems, but

still requiring us to understand how complex tasks are most effectively decomposed, and to

analyze the important questions of interfacing with a complex and dynamic environment. And

thus the book has not only presented studies of learning and memory, but has also charted neural

mechanisms underlying vision and action.

Part 1 Development and Learning in Adaptive Networks

Dynamical Stability of Formation of Cortical Maps

Shun-ichi Amari

Faculty of Engineering, University of Tokyo,

Tokyo, Japan

Abstract. A cortical map is a localized neural representation of the signals in the outer world. A rough map is formed under the guidance of genetic information at the initial stage of development, but it is modified and refined further by self­organization. The persent paper gives a mathematical theory of formation of a cortical map by self-organization. The theory treats both dynamic of excitation patterns and dynamics of self-organization in a neural field. This not only explains the resolution and amplification properties of a cortical map, but elucidates the dynamical stability of such a map. This explains the emergence of a microcolumnar or mosaic structure in the cerebrum.

1. Introduction

There have been proposed a vast number of neural network models and

parallel information processing systems inspired by brain mechanisms. In most cases, computer simulated experiments are used to demonstrate their characteristics. In order to establish a unifying theory of neuro-computing or computational neuroscience, we need to develop mathematical theories systematically together with mathematical methods of analysis. We can then

elucidate intrinsic neural mechanisms which are common to many of such

models. Such an approach makes it possible to understand abilities and

limitations of neural mechanisms.

The present paper demonstrates an example of such mathematical

approaches toward constructing a unified theory. We show a mathematical

method for analyzing the dynamical stability of cortical maps formed by self­

organization in neural fields. This requires to connect dynamics of excitation

patterns in a neural field with dynamics of self-organization of synaptic efficacies of neural connections.

Many researchers have so far proposed models offormation of cortical maps

16

by self-organization. When a set of signals are applied to a neural field in the cortex, under certain conditions, the field is expected to be self-organized in such a manner that each location of the field corresponds to each signal. This defines a mapping from the signal space X to the neural field F. This map is called a

cortical map of the signal space X. When a signal is applied, the positions

corresponding to that signal in the map are excited. Von der Malsburg [1973] proposed a model, where the signal space consists

of bars of various orientations, so that orientation detective cells are formed in a

cortical map. See also Spinelli [1970], Fukushima [1975], Amari and Takeuchi

[1978], Cooper et al. [1979], etc. When the signal space is two-dimensional, e.g.,

each signal representing a location of the presynaptic neural field, a topographic mapping is formed between the two fields by self-organization. Such types of

formation of cortical maps have been studied by Willshaw and von der Malsburg [1976] , and then by Amari [1980], Takeuchi and Amari [ 1979 ], Overton and Arbib [1982], Kohonen [1984], Bienenstock et al. [1982], etc. Most of them are

based on computer simulated studies. It is important to construct a mathematical

theory which elucidates such mechanisms of formation of cortical maps by self­

organization. We point out three properties which a mathematical theory should

elucidate; 1) resolution of a map, 2) an amplification property of a map, and 3)

dynamical stability of a continuous map. The resolution of a map shows the size

of the receptive field or the range of the signal space which excites the same cortical neuron. We need to know the intrinsic mechanism which controls such resolution. A cortical map is not fixed but modifiable ( Merzenich [1986]). It is plausible that those signals which appear more frequently are projected on a

larger portion of the cortical field with finer resolution. This is called the amplification property. It is surprising that, even when the signal space is

continuous, a continuous cortical map becomes dynamically unstable under a

certain condition. In this case, both the signal space and the cortical map are

quantized to have a mosaic or columnar structure. We can also prove this by

using a mathenatical model, where modifiability of inhibitory synapses plays a

role of competitive learning.

We present a mathematical theory capable of treating these properties

along the lines studied by Amari [1980 ; 1983], Amari and Takeuchi [1978],

Takeuchi and Amari [1979]. There are other types of interesting mathematical

theories shown in Amari [1972 ; 1974 ; 1977a, b], Amari et al. [1977] and Amari

and Maginu [1988] .

17

2. Dynamics of excitation patterns in neural fields

Let us consider a two-dimensional cortex or neural field F, on which a

cortical map of signals is formed. Neurons are continuously distributed on the

field F, and let ~ = ( ~1, ~ ) be a coordinate system in F. The average membrane

potential of the neurons at location ~ at time t is denoted by u( ~, t). We denote by

z ( ~, t ) the average action potential or output pulse frequency emitted from the

neurons at location~. It is determined from u( ~, t ) by

z ( ~, t) = f[ u( ~, t) 1 • (2.1)

f{u)

u

Fig. 1 Output function

where f{u) is a monotone increasing function of u. Its typical shape is given in Fig 1. Sometimes, we approximate f{u) by a unit step function 1(u), which is equal to

1 when u > 0, and is otherwise equal to O. The average membrane potential u(~, t ) increases in proportion to a weighted sum of stimuli applied to the neurons at ~, and decreases at the level of the resting potential - h when no stimuli are

applied. Let s( ~ ) be the total sum of stimuli, which the neurons at ~ receive. The

dynamical equation of the membrane potential is then written as

ilu(f" t) .-- = -u(f"t)+S(~,t)-h ilt '

(2.2)

where"'(; is the time constant of neural excitation.

Neurons in the field receive two kinds of stimuli. One is those directly

18

applied from the signal space X. Let x = (Xl, ... ,xn) be a vector signal applied to the neural field F. Every neuron in F receives a common signal x belonging to the

signal space X. Let s(~) = ( Sl (~), "', sn(~) )be the synaptoc efficacies of the neurons

at ~ for a bundle of stimuli x = ( Xl, ... , xn). The weighted sum of stimuli which

neurons at ~ receives is then written as the inner product of s and x,

n

s(~)'x= L si(~)xi' i = 1

Fig. 2 Neural Field

z (~', t)

z (~, t)

when signal x is applied ( Fig. 2). It is assumed that F receives one more signal xo which is inhibitory in nature. Let so(~) be the synaptic efficacy of the neurons at ~

receiving the inhibitory xo. The weighted sum of stimuli given from the signal

space X is then written as

V( ~,x) = s(~)· x - So (~) xo , (2.3)

when x is applied.

The field F is assumed to have recurrent connections such that the output

19

z( ~', t) = f[ u( ~', t) ] at ~ 'is fed back and is connected to neurons at every ~ with synaptic efficacy w( ~ , ~'). The total weighted sum of recurrent excitations is

denoted symbolically by

w 0 feu) = J w (I:" ~') f[ u (~' , t) ] d~' •

Therefore, the dynamical equation of neural excitations is

(2.4)

au (~, t) (2 5) "(; -- = - u (~, t) + w 0 f(u)+ v (~, x) - h • at ,

When the recurrent connection function w (~, ~') is symmetric, we have a

global Lyapunov function L[u] such that the value of L[u] monotonically

decreases as an excitation pattern u( ~, t) evolves. See, for example, Cohen and

Grossberg [1983]. In the present case, we have

L[u]= J A[u(~,t)]d~- ~ J w(~,~')f[u(l:"t)]f[u(~',t)]d~d~' ,(2.6)

where

A [u] = {u (~, t) + h - V (~, x) } f[ u (~, t)] - F [u (I:" t)] , (2.7)

F (u) = I: f( v ) d v • (2.8)

Theorem 1. The value of L[u] decreases monotonically in the course of

dynamics and the neural excitation converges to one which minimizesL[u].

Proof. We easily have

d J d J a - L [ u] = - A [u ] d ~ - w (I:" ~ ') f[ u ( ~ " t)] f' [ u ( ~: t) ] - u (~ , t )d ~ d ~' dt dt at

f au = {A - (wonf '-}d~ ,

u at

where

aA A =-=(u-V+h)f'

u au

20

{'= df du

Therfore, becauseoff'(u) ~ 0,

is obtained.

When the recurrent connection function w(~, ~') is homogenious, depending

only on I ~ - ~'I and when it satifies

w (~, ~') > 0 when I ~ - ~'I is small ,

w (~, ~') < 0 when I ~ - ~'I is large ,

Fig. 3 Lateral inhibitory connection w

the recurrent connections are said to be lateral inhibitory ( Fig. 3). Dynamics of lateral inhibitory neural fields is studied by Wilson and Cowan [1973], and a detailed mathematical analysis is given in Amari [1977b] and Kishimoto and Amari [1979]. This analysis is useful for studying dynamical behaviors of

21

disparity fusion models in stereopsis perception ( Amari and Arbib [1977], Arbib [1988 D. Let U( ~ ; x ) be the equilibrium solution of the field equation (2.5). When x is input from the signal space X, it satisfies the integral equation

U(~ ; xl = w 0 fW] + V(~; xl - h . (2.9)

This U( ~ ; x ) represents the excitation pattern aroused in the field F when x is applied. Neurons at those positions ~for which U( ~; x» 0 are excited.

The set

E(x) = W U(~;x) > O} (2.10)

U(x; ~ )

X

x

Fig. 4 Cortical map

denotes the positions of excited neurons when x is applied. When lateral inhibition is strong, it is expected that the equilibrium U(~ ; x) is unique and that E(x) is concentrated on a small region. This implies that signal x is represented by an excitation at a small region E(x). Let m(x) be the center of E(x). See Fig. 4. Then, m(x) represents the cortical map of the signal space on the neural field F,

22

m:x /-- m(x)

In other words, ~= m(x) represents the position which is eventually excited when

signal x is applied.

The map m(x) or equivalently the equilibrium solution U(~ ; x) depends on the synaptic efficacies s(~) and so(~) with which neurons at position ~ receive inputs. Therefore, when synaptic efficacies s(~) and so(~) are modifiable, the cortical map m(x) is also modifiable. When the cortical map m(x) is formed by self-organization, it is expected that the map m(x) represents characteristic features of the set X of signals applied from the outside.

3. Dynamics of self-organization of neural fields

It is assumed that the synaptic efficacies s(~) and so(~) are modifiable due to the generalized Hebbian rule, while the lateral inhibitory connections w(~, ~') are fixed. The dynamics of synaptic modification is then written as

ils (~t) "t;'-- = -s(~t)+cxf[u(~t)] ,

ilt

elsO (~ t)

"t;' -a-t - = - so(~ t) + c'xof[u(~.t)l

(3.1)

(3.2)

where "t' is the time constant of synaptic change, c and c' are constants, and %0 is assumed to be constant for the sake of simplicity.

Now we need to specify the structure of the input signals. It is assumed that input signals x are emitted from an information source I with prescribed probability distribution p(x). More precisely, a signal x is chosen independently from X at each trial with probability p(x), and is applied to the field F for a short time. Another signal is then chosen independently and is applied. This implies

that the sequence {x (t)} of input signals forms an ergodic stochastic process, and signal x appears in the sequence with relative frequency p(x).

It is difficult to solve the equations (2.5), (3.1) and (3.2) simultaneously.

However, when the time duration for which a signal x is applied is larger than the

time constant"t, u( ~ .. t ) quickly converges to the equilibrium solution U(~ ; x ).

Hence, we may replace u( ~ .. t) in (3.1) and (3.2) by the equilibium U(~ ; x ), where "t'is much larger than 'to This type of approximation is called the adiabatic hypotheseis ( Caianiello [1960] ) or the slaving principle ( Haken [1979]). Since

x(t) is a stochastic process, the equations (3.1) and (3.2) are a set of stochastic

23

differential equations. When -c 'is very large, we may take the ensemble average

over possible input signals x, of the equation to obtain

a <s(~ t) > .' = - <s(~t» +c<xf[u(~,x)l> at ' etc., where < > denotes the ensemble average over possible input sequences x(t). It should be noted that U depends on s(~), so that the last term of the above

equation is very complicated. If we replace s(~) in U by < s(~) > , the equation is simplified. In order to avoid unnecessary complications, we hereafter use s(~) for

< s(~) > and take the above approximation. We then have

where

as (~, t) • '-- = - s(~ t)+ c < xf[ U(~, t)] > at

aso (~t) • ' -a-t - = - So (~, t) + c' < xof[ U (~, t)] >

< xf[ U(~; x)] > = J P (x) xf[ U(~; x) ldx .

(3.3)

(3.4)

This type of approximation was proposed by Amari [1977a] and its validity

was discussed by using the stochastic approximation technique. Geman [1979] extended this result and gave a more elaborated mathematical result.

The weighted sum V(~ ; x) of stimuli which neurons at ~ receive when x is applied, changes as the synaptic efficacies change. By differentiating

V(C x) = s(~) . x - So (Q Xo

with respect to t, we have

av as aso "(;'- =l:;' -·x--x

at at at 0

By substituting (3.3) and (3.4), this reduces to

av • ' - = - V + k * f[ Ul at (3.5)

24

where

k(x,x') = ex·x' - e'x~ (3.6)

k *([Ul= <k(x,x/){[U(~;x/)l>x'

= f p(x')k(x,x'){[u(~;x)l dx' . (3.7)

This, together with (2.9), gives a fundamental equation of self-organization of the cortical map. An equilibrium cortical map is given therefrom by putting av I at= O.

Theorem 2. An equilibrium solution U(~, x) of self-organization is given by solving the following equation

U(~;x)=wo{[Ul+k*{[Ul-h (3.8)

An equilibrium solution is not necessarily stable. In order to show the dynamic stability of an equilibrium U(~, x), we need to study the linearized variational equation around the equilibrium,

ao V "t'-- = -ov + k*'Of[ Ul at ,

(3.9)

oV=oU-woo{[Ul (3.10)

We will show later that a continuous map becomes unstable under a certain condition, so that some columnar microstructure emerges automatically.

4. Resolution and amplification properties of discrete maps

We consider a simple case where only a finite number of signals X = {Xl,

"', Xn } are applied to the neural field as a training set under learning phase.

Corresponding to these Xi, n discrete regions at ~i = m(xi), i = 1,2, "', n, will be formed on the cortical map, i. e., neurons around ~i are excited by receiving Xi.

There are two problems to be solved. One is resolution, which is

respresented by the size of the recptive field R(~i) of neurons at ~i , i.e., the set of

signals X which excite neurons at ~i. If the size of R(~i) is small, then the

25

resolution is fine and the neurons at ~i are excited only by signals very close to Xi.

The other is the size of E(Xi), i.e., the size of the cortical region excited by signal

Xi·

In order to show the resolution, we consider a very simple situation where

no recurrent connections exist in the field F. The equation (3.8) giving the

equilibrium cortical map is reduced to

U(xl=k*f[Ul-h' (4.1)

because of w(~, ~') = o. We here omitted the argument~, because every position ~

behaves similarly but independently of each other in the present simplified

situation. This can be written as

- L 2 -U(xl= p.(cx·x.-c'xolf[ U(x.ll-h, J J J

(4.2) j

where Pj is the probability (relatibve frequency) of Xj in the training signal

sequence.

Let us further assume that all the signals are normalized,

x·x = 1,

and that f{u) is approximated by the unit step funcion l(u). When Xi is mapped on

the position ~i = m(xi), its resolution is measured by the inner product or the

direction cosine A such that, those signals which satisfy

X'xi< A

excite the neurons at position ~i.

Theorem 3. A map of Xi is formed when, and only when,

h c' A. = - +- x2 < 1

, cPi co, (4.3)

and its resolution is given by Ai, i.e., the neurons are excited by those signals X

which satisfy

X·X,>A, . (4.4)

26

Proof. Let us assume that the neurons at ~i are excited by Xi but by no other xj s. We then have from (4.2)

- 2 U (x) = Pi ( ex' xi - C ' Xo ) - h ,

as an equilibrium solution. This solution satisfies U(Xi) > 0, when and only when

Ai < 1. Moreover, U(x) > ° for those X satisfying (4.4).

When h= 0, Ai depends only on CX02. Therefore, as xo becomes larger, the

resolution becomes finer, and the neurons are excited only those very close to Xi.

The resolution in general depends on the frequency Pi of signal Xi.

We next study the size !E(Xi)! of the cortical region corresponding to Xi.

Recurrent connections are the fundamental factor to determine the size. When recurrent connections exist, the equilibrium equation (3.8) is written as

U(~;x)=wof(Uj+ 2: p.(cx.x.-c'x02 )f[U(~;x.)]-h. (4.5) . J J J

J

When cortical regions Ei = E(Xi ) and Ej = E(xj) do not overlap, the equation is

simplified to

U (~;x) = wof[ Uj + PikJ [ U (~;xi)] - h

at a neighborhood of Xi, where

(4.6)

ki(X)=CX'Xi-C'X~ (4.7)

We search for the region Ei which is excited by Xi. In the region, we have

where

This U(~ ; Xi) vanishes on the boundary of Ei, so that we have

27

Since W(~) depends on I~ -~i 1 , we have

1~_~i1=W-l{h-Pi(C-C\2)} ,

where W-l(U) is the inverse function of u. A typical shape of WCI~ - ~i I) is shown in

Fig. 5. Since we know that the monotonically decreasing branch indicates the

stable solution of neural excitations (Amari [1977b)), we use this branch for

defining the inverse function W-l.

W(~)

I~I

Fig. 5 Shape ofW

Theorem 4. The size of Ei is determined by

1~_~i1=W-l{h-Pi(C-C'X:)} , (4.8)

and is monotonically increasing with respect to the frequency Pi of Xi.

5. Dynamical stability of continuous cortical maps

Let us finally study characteristics of a continuous cortical map. Since X is

n-dimensional, there is no topological maps between X and F, unless n = 2.

When all the training signals belong to a one- or two-dimiensional submanifold of

X, it is possible to have a continuous map between this submanifold and the

cortical field F. We can easily obtain such a continuous map as an equilibrium

solution of the equation of self-organization. However, if the solution is unstable,

such a continuous map is never realized. Computer simulated experiments show

that a mosaic or columnar structure automatically emerges in such a case. This

28

is a very interesting map, because the topological structure of X is preserved in the map approximately by quantizing the both spaces. This explains the neural

mechanism of keeping the columnar structure in cortices on one hand, and also

explains the mechanism of categorizing a signal space.

Let xeS) be a curve in X parametrized by a scalar S. We assume a simple

case where the training signal set includes only signals on the curve. Moreover,

we assume that the cortical neural field F is one-dimensional, i.e., ~ is a scalar parameter. We then search for the map of this one-dimensional training set

{xeS)}, m = mixeS)} = ~, or ~ = m(S) which shows that signal xes) excites position

~ = m(S). Since both S and ~ are one-dimentional, there exists a very natural

continuous map m(S). We analyze its dynamical stability.

To this end, we put

U( ~ ; 9) = U{ ~; x(9l} ,

k( 9, 9 ') = k{ x(9), x(9'l}

The equilibrium equation is then rewritten as

U(~,9) = J w(~- Vld~' + J k(9 ,9' )d9' - h , (5.1) E(S) R(O

/ ~=m(S)

I--------+.-+-"!IF---- U(~, S) > 0

E(S)

e 8 R(~)

Fig. 6 Cortical map m(S) and excited region U(~, 8)

29

where E(9) is the region in F which is excited by x(9), and R(~) is the receptive field, i. e., the set of signals which excite ~ ( Fig. 6). In order to show the stability discussions clearer, we assume that the training subset and the lateral inhibition are homogeneous and isotropic

k( 8, 8') = k( 8 - 8'),

w(~,~') = w(~- ~'),

where k(9) and w(Q are symmetric functions. Moreover, we assume that p(9) is constant. Then, choosing the origins and the scales of9 and ~ adequately, we have a natural map

~ = m(8) = 8 .

The corresponding equilibrium solution is written as

U(~, 8) = g(~ - 8) •

where g is a unimodal function satisfying

g (x) > 0

g (x) = 0

a Ixl < - •

2

a Ixl= - ,

2

a g (x) < 0 Ixl> -

2

(5.2)

(5.3)

This shows that the width of an excitation is a in F. The function g and the width a can be obtained as follows. Let us define two functions

W(x) = I: w(x)dx , (5.4)

K(x) = I: k(x)dx . (5.5)

Theorem 5. The width a of an excited region is given by a larger solution of

W(a) + K(a) - h = 0 . (5.6)

30

The wavefonn function g of U is given by

g (x) = W( ~ + x) + W( ~ - x) + K( ~ + x) + K( ~ - x) _ h (5.7) 2 222

Proof. Since E(e) andR(~) are, respectively, intervals oflength a,

a a E(a):~=a---a+- ,

2 2

a a R(~):a=~---~+- ,

2 2 it is easy to see that U(~, e) is a function of~ - e. By integrating it, we have

a a a a U (~,O)=W(~+ - )-W(~- -)+K(~+ -)-K(~- -)-h

2 2 2 2'

which gives (5.7). The length a is detennined from U(a / 2,0) = g(a / 2) = 0, which is (5.6). This equation usually has two solutions. The larger one is shown to be stable (Amari [1977b]) from the stability analysis of an excitation pattern in a neural field.

In order to analyze the stability of the natural continuous cortical map ~ = m(e) = e , we add a small perturbation tenn 8U(~ ; e, t) to the equilibrium solution, and see the dynamical behavior. Let us consider two boundary lines on which

U(~,a)=o

This is obviously given by

a a ~=a+ 2"' ~=a- 2"

By perturbing U(~; e) to U(~; e) + 8U(~; e, t), the boudary lines are perturbed to

We search for the dynamical behavior of the pertubations 81r(e, t), 82r(e, t). From the variational equation (3.10), we have

31

where

a= -g'(aI2) .

by putting

we have from (3.9)

aQ2 "{;' at = w (r){82 r m - 81 r (~+ r)} + k (r){82 r(~) - 81 r (~)}

By substituting (5.8) in the above equations, we have the following variational

equation

aQl

[ iit 1 [Q1 1 "{;' aQ2 =A Q2

(5.9)

at where A is a 2 X 2 matrix. In order to see the stability of(5.9), we expand Qi(~,t)

in following Fourier serieses,

Qi(~.t) = 1: b.' (t) exp{ - 2nn I r} . (5.10)

Then, the variational equations split into those of Fourier components. We can

determine the stability by using the method similar to that given in Takeuchi and Amari [1979].

32

Theorem 6. The continuous cortical map is stable when k(a) < 0, and is unstable when k(a) > O.

When the continuous map is stable, it is formed by self-organization. There are many of such examples (Willshaw and von der Malsburg [1976], Biennenstock et al. [1982], Kohonen [1984], Amari [1980]). However, when it is

unstable, both the neural field and signal space are quantized into block structures, and a topological correspondence between such blocks are formed ( Fig. 7). This aspect of self-organizing neural maps seems to be much more interesting than continuous maps.

References

Amari,S.

Amari,S.

Amari,S.

a) Continuous map

D D

D 1....--.1 _______ e

b) Block structure

Fig. 7 Two types of cortical maps

[1972] : "Learning patterns and pattern sequences by self­

organizing nets of threshold elements", IEEE Trans., C-21, 1197-

1206.

[1974] : "A method of statistical neurodynamics", Kybernetik, 14,

201-215.

[1977a] : "Neural theory of association and concept-formation",

Biol. Cybernetics, 26,175-185.

Amari,S.

Amari,S.

Amari,S.

33

[1977b] : "Dynamiics of pattern formation in lateral-inhibition

type neural fields", BioI. Cybernetics, 27, 77-87.

[1980] : ''Topographic organization of nerve fields", Bull. Math.

BioI., 42, 339-364.

[1983] : "Field theory of self-organizing neural nets", IEEE Trans.

SMC-13,741-748.

Amari, S. and Arbib, M. A. [1977a]: "Competition and cooperation in neural

nets", Systems Neuroscience, (J. Metzler ed.)Academic Press, 119-

165.

Amari, S. , Yoshida, K. and Kanatani, K. [1977]: "A mathematical foundation

for statistical neurodynamics", SIAM J. App. Math., 33, 95-126.

Amari, S, and Maginu, K. [1988]: "Statistical neurodynamics of associative

metrwry", 1,63-73.

Amari, S.and Takeuchi, A. [1978]: "Mathematical theory on formation of

category detecting nerve cells", BioI. Cybernetics, 29, 127-136.

Arbib, M. A. [1988]: "Interacting subsystems for depth perception in the frog",

this volume.

Caianiello, E. R. (1961]: "Outline of a theory of thought processes and thinking

machines", J. Theor. Biol., 1,204 - 235.

Cohen, M. and Grossberg, S. [1983] : "Absolute stability of global pattern

formation and parallel memory storage by competitive neural

networks" IEEE Trans., SMC - 13, 815 - 826.

Cooper, L. N., Lieberman, F. and Oja, E. [1979]: "A theory for the acquisition and

loss of neuron specificity in visual cortex. BioI. Cybern., 33, 9 - 28.

Fukushima, K. [1975]: "Cognition: a self-organizing multilayered network",

BioI. Cybern., 20, 121 - 136.

Haken, H. [1979] : "Synergetics, An Introduction", 2nd enlarged ed., Berlin,

Germany: Springer.

34

Kishimoto, K. and Amari, S. [1979]: "Existence and Stability of Local

Excitations in Homogeneous Neural Fields", J. Math. Biology, 7,

303-318.

Kohonen, T. [1982]: "Self-organized formation oftopologically correct feature

maps", BioI. Cybern., 43, 59 - 69.

Merzenich, M. M. [1987]: "Dynamic neocortical processes and the origins of

higher brain functions", in The Neural and Molecular Bases of

Learning, eds. Changeux, J.-P. and Konishi, M., Dahlem Conference

38 : Wiley.

Overton, K. J. and Arbib, M. A. [1982]: "Systems matching and topographic

maps: The branch-arrow model (BAM)", in Competition and

Cooperation in Neural Nets, Amari, S. and Arbib, M. A. eds.,

Lecture Notes in Bilmathematics, 45. Berlin,Germany : Springer.

Spinelli, D. N. [1970]: "OCCAM: a computer model for a content addressable

memory in the central neurons system", in Biology of Memory,

Pribram, K. M. and Broadbent, D. E. eds., Academic Press, 293-

306.

Takeuchi, A. and Amari, S. [1979]: "Formation of topographic maps and

columnar microstructures", BioI. Cybern., 35, 63-72.

von der Malsburg, C. [1973]: "Self-organization of orientation sensitive cells in

the striate cortex", Kybernetik, 14,85 - 100.

Willshaw, D. J. nad von der Malsburg, C. [1976]: "How patterned neural

connections can be set up by silf-organization", Proc. Roy. Soc., B-

194,431- 445

Wilson, H. R. and Cowan, J. D. [1973] : "A mathematical theory for the functional

dynamics of cortical and thalamic nervous tissues", Kybernetik, 12,

1- 24.

Abstract

Visual Plasticity in the Auditory Pathway: Visual Inputs Induced into Auditory

Thalamus and Cortex Illustrate Principles of Adaptive Organization in Sensory Systems

Mriganka Sur Department of Brain and Cognitive Sciences

M.I.T., Cambridge

We have induced, by appropriate surgery in newborn ferrets, retinal

projections into the medial geniculate nucleus, the principal

auditory thalamic nucleus. In operated animals studied as adults,

retinal ganglion cells that give rise to the projection have small

and medium sized somata and heterogeneous dendrite morphologies.

Each retina projects to the auditory thalamus in patchy fashion.

Various nuclei in auditory thalamus project normally to auditory

cortex. Visual cells in auditory thalamus have circular receptive

fields and receive input from slowly conducting afferents

characteristic of retinal W cells. Many visual cells in primary

auditory cortex have oriented receptive fields that resemble those

of complex cells in striate cortex. Primary auditory cortex also

contains a two dimensional visual field map. Our results carry

several implications for sensory cortical function. A parsimonious

explanation for the visual receptive field properties in auditory

cortex is that sensory cortex carries out certain stereotypical

transformations on input regardless of modality. The response

features of visual cells and the two dimensional visual field map

in primary auditory cortex appear to be products of adaptive

organization arising from a highly divergent thalamocortical

projection characteristic of the auditory system.

36

Introduction

An enduring question in understanding the development and

plasticity of the brain is: does function in a target structure

derive from specific inputs during development, or from

intrinsic microcircuitry in the structure independent of

afferents? In this brief review, I shall describe experiments

that my coworkers and I have done to address this question. We

suggest that these experiments not only demonstrate the capacity

of sensory systems in the developing brain for extensive cross­

modal plasticity, but also indicate general principles of

operation of central sensory structures - in particular, sensory

neocortex - that derive specificity only from specific sensory

inputs.

Our experimental strategy has been to route retinal

projections during development to nonvisual targets in the

thalamus, and then study the physiological consequences of

visual input to cells in nonvisual thalamus and cortex.

Schneider ('73) first demonstrated that by ablating the superior

colliculus and deafferenting auditory thalamus in developing

hamsters, retinal proj ections can be induced to grow into the

medial geniculate nucleus (MGN), the principal auditory thalamic

nucleus. Since then, several investigators have shown that

ablating normal axon targets in rodents during development and

creating alternative target space in other structures causes

plasticity of ingrowing afferents in the visual (Cunningham '76;

Finlay et al. '79; Lund and Lund '76), or olfactory (Devor '75;

Graziadei et al. '79) systems. Recently, Frost and his

coworkers have shown that retinal projections that exuberantly

invade the outer margin of the ventrobasal nucleus, the

principal somatosensory thalamic nucleus, in neonatal hamsters

can be stabilized by deafferenting the somatosensory thalamus at

birth (Frost '81, '86). The visual projections to somatosensory

thalamus exhibit some retinotopic order (Frost '81), take part

in synaptic arrangements that resemble those in the normal

ventrobasal complex (Campbell and Frost '87), and can impart

visual driving to neurons in somatosensory thalamus and cortex

(Frost and Metin '85).

In our experiments, we have applied to ferrets Schneider's

procedure for inducing cross-modal plasticity during

37

development. Ferrets and mink belonq to the mustelid family of carnivores. Of the carnivores, the visual system of cats has been studied intensively both anatomically and physioloqically, and the orqanization of the visual pathway in mustelids is very similar (McConnell and LeVay '86). Importantly for our purpose, ferrets are born at a very early staqe of development, at embryonic day 41 (instead of embryonic day 65 for cats); at birth, retinofuqal fibers in ferrets have entered the lateral qeniculate nucleus (LGN) of the thalamus but have not seqreqated into eye-specific laminae (Cucchiaro and Guillery '84). This is very similar to the status of retinofuqal projections in cats at embryonic day 41 (Shatz '83), and subsequent development in ferrets closely matches that in cats. We reasoned that the more immature the retinofuqal projection, the more miqht be the ability of retinal fibers to innervate novel tarqets. Indeed, the same procedures that lead retinal fibers to innervate the auditory thalamus in ferrets do not cause aberrant projections in kittens.

The procedure for inducinq retinal projections to the auditory system is shown in Fiqure 1. The two major tarqets of the retina are the LGN in the thalamus and the superior colliculus (SC) in the midbrain. In the auditory pathway,

Retina

Normal Ferrets

A1 Other

Auditory Areas

\/ T

Ie

Fiqure 1

Extrastrlate Cortical Areas

1

Retina

Other

A1 A:~!~:,

\/ MGN

Operated Ferrets

38

information ascends from the inferior colliculus through various nuclei of the MGN to primary auditory cortex as well as other cortical areas (Aitkin et al. '84). When we ablate the SC and visual cortical areas 17 and 18 in newborn ferrets (thereby causing the LGN to atrophy severely by retrograde degeneration), and concurrently create alternative terminal space for retinal afferents in the MGN by sectioning ascending auditory fibers in the brachium of the inferior colliculus, the retina develops projections to the MGN (Sur and Garraghty '86~ Sur et al. '88). The MGN retains its projections to auditory cortex (Figure 1).

Visual projections to auditory thalamus By inj ecting an eye wi th an anterograde tracer, we can

demonstrate that each eye proj ects in patchy fashion to the dorsal, ventral and medial divisions of the MGN. There are retinal projections as well to the lateral part of the posterior nucleus adjacent to the MGN. Retrograde labeling of retinal ganglion cells by injections of horseradish peroxidase (HRP) in the MGN indicates that the cells that project to the MGN have small and medium sized somata (Roe et al. '87). These cells also have rather heterogeneous morphologies and constitute the diverse group of retinal "w cells". The ferret retina contains, in addition to W cells, Y cells that morphologically resemble alpha cells of the cat retina and X cells that resemble beta cells in cats (Vitek et al. '85). Alpha and beta cells do not project to the MGN in operated ferrets.

Physiological recordings from the MGN in operated animals indicate that cells that receive retinal input have long latencies to optic chiasm stimulation~ the conduction velocity of retinal afferents to the MGN is similar to that of W cells that innervate the LGN or SC (Hoffman '73~ Sur and Sherman '82). While many visual cells in the MGN respond rather weakly to visual stimuli, cells have concentric receptive fields similar to those of retinal ganglion cells. Receptive fields are large, indicative of retinal W cell input or perhaps increased convergence of retinal afferents on single MGN neurons.

Cells in the surviving LGN in operated animals receive input from retinal Y cells or W cells, and, unlike normal animals, very few cells with X cell properties are found (Figure

39

1) . This result is consistent with a loss of X cells in the retina and LGN that has been reported in cats following neonatal ablation of visual cortex (Tong et al. 182) • The likely explanation is that LGN X cells appear to project essentially to area 17, and these cells atrophy after neonatal ablation of area 17 and adjacent visual cortical areas. Retinal X cells in cats atrophy transneuronally after such ablation, and we have found as well a loss of medium-sized beta cells in the retinas of our operated ferrets (Sur et al. 187).

The retina maps systematically on to the MGN in operated ferrets: central visual field is represented medially and peripheral field laterally, while upper field is represented dorsally and lower field ventrally within each eyels projection zone in the MGN.

Visual projections to auditory cortex In operated ferrets, most cells in primary auditory cortex

are driven by electrical stimulation of the optic chiasm, indidating visual input to these neurons. In contrast, no neurons in primary auditory cortex of normal animals can be driven electrically from the optic chiasm. Three-fourths of the cells driven by optic chiasm stimulation in primary auditory cortex of operated animals are responsive to visual stimulation, though many respond rather weakly. The optic chiasm and visual latencies of neurons in primary auditory cortex are consistent with these cells receiving their major visual driving through the ipsilateral MGN. Anatomically, the major thalamic input to primary auditory cortex arises from the ventral and dorsal divisions of the MGN. This is similar to thalamocortical auditory projections in normal animals, except that the MGN in operated animals receives retinal input. (In operated animals, there are also weak retinal projections to parts of the lateral posterior nucleus and pulvinar, and projections from these thalamic structures to auditory cortex).

Visual cells in primary auditory cortex have larger receptive fields than cells in striate cortex (primary visual cortex) of normal animals, and prefer flashing or slowly moving spots or bars of light. Nearly a third of the visual cells in primary auditory cortex are direction selective. Visual cells

40

have either nonoriented receptive fields, or orientation­selective fields that resemble complex cells in striate corte.x (one-third of the visual cells in primary auditory cortex have such receptive fields). The orientation tuning of these cells is broader than those of cells in normal striate cortex. However, the appropriate comparison of receptive fields in primary auditory cortex is with cells in a purely W cell pathway in normal cortex. striate cortex in normal animals is dominated by the X and Y cell pathways through the LGN (Sherman and Spear '82), and little of the published literature on visual response properties of neurons in striate or extrastriate cortex derives from cells with pure W cell input (see, however, Dreher et al, , 80) •

Perhaps the central issue with regard to receptive fields is whether some of the transformations carried out in the normal visual pathway through the LGN to striate cortex are carried out as well in the aberrant visual pathway through the MGN to primary auditory cortex. oriented complex-like receptive fields in primary auditory cortex created from nonoriented thalamic input argue that certain key transformations of input indeed occur in auditory cortex just as they do in visual cortex.

The contralateral visual field is represented in two­dimensional fashion in primary auditory cortex. Central visual field is represented medially in cortex while peripheral field is represented laterally. The antero-posterior axis in cortex shows more variability in representation. In most animals, upper visual field is represented anteriorly and lower field posteriorly in cortex, with a systematic progression in receptive field sequence as one moves across the map. In other animals, receptive fields show discontinuous progressions along this axis, and may even show an inverse of the normal progression over some expanses of cortex. That is, over short distances in the antero-posterior direction in some animals, lower visual field regions occupy more anterior representations in cortex compared to upper visual field regions.

Thalamocortical orqanization and mapping in auditory cortex The induced map of visual space in primary auditory cortex

relates to an important aspect of thalamocortical organization

41

and cortical function. In normal animals, the retina, which is a

two-dimensional epithelium, projects in point-to-point fashion

to two dimensions of the LGN and to the plane of layer 4 in

primary visual cortex. This is illustrated schematically in

Figure 2 (top), which shows a one-dimensional transverse slice,

i.e. a line, through the retina and corresponding lines through

LGN and striate cortex demonstrating the representation in

point-to-point fashion of discrete regions of visual space

(points A, B and C). In the auditory pathway, the cochlea, which

is a one-dimensional epithelium, projects as a line to only one

dimension of the MGN and primary auditory cortex (Aitkin et al.

'84). That is, only one axis in these structures carries a

tonotopic or frequency-specific representation. The orthogonal

dimension in MGN or in the plane of layer 4 in primary auditory

cortex re-represents the same frequency, and is termed the

isofrequency axis. Figure 2 (middle) illustrates a transverse

slice through the cochlea, which represents a point on the

cochlea (or a single frequency A), and its representation along

an isofrequency line in MGN and in layer 4 of primary auditory

cortex. The dimension orthogonal to that shown in Figure 2

(middle), that enters the plane of the paper, represents

cochlear points of changing frequency and thalamic and cortical

lines of different isofrequencies. Consistent with the

orthogonal tonotopic and isofrequency representations in central

auditory structures, the anatomical organization of projections

from the cochlea (through intermediate stations) to the MGN, and

from the MGN to primary auditory cortex suggests marked

anisotropy: each point on the cochlea projects in highly

divergent fashion to a slab of cells in the MGN, and a slab of

cells along an isofrequency line in the MGN projects in an all­

to-all fashion to an isofrequency line in primary auditory

cortex (Merzenich et al. '84). At the same time, the anatomical

spread of input is much more restricted along the frequency

representational or tonotopic axis in MGN or cortex.

In normal ferrets, we have confirmed such a thalamocortical

projection system. In primary auditory cortex, the tonotopic

axis is oriented mediolaterally while isofrequency lines lie

anteroposteriorly in cortex (Kelly et al. '86). Small injections

of retrograde tracers {HRP or fluorescent tracers such as Fast

42

Blue, Fluoro Gold and Rhodamine microspheres) into discrete

locations in primary auditory cortex label slabs of cells in the

dorsal and ventral divisions of the MGN. These slabs occupy thin

Retina LGN V1 rp:: .r;\ .. 7:\

B B B

C C -"" C '-- - '--

Cochlea MGN A1

AIE-------~

A

Retina MGN A1

AI--_____ ~A~-----~

BI-------~BIE_--"'*--~

Cl-------~c~----.-;;;.c

Figure 2

43

curved laminae that, in coronal sections, are oriented

dorsoventrally along the MGN (the cells also extend as a

projection column or sheet in the rostrocaudal dimension through

most of the nucleus); the dorsoventral lines of cells are

consistent with isofrequency lines in the MGN (Middlebrooks and

Zook 183). Injections of multiple tracers into cortical loci

representing different frequencies label dorsoventral slabs in

the MGN that are shifted medially or laterally from each other.

Inj ections into discrete cortical loci along an isofrequency

line label highly overlapped populations of cells in the MGN.

When we induce retinal projections into the auditory system,

we overlay a two dimensional sensory epithelium on a one

dimensional thalamocortical projection (Figure 2, bottom). The

anatomical organization of thalamocortical proj ections between

the MGN and primary auditory cortex in operated animals is

similar in essential aspects to that in normal animals. As in

normal animals, the tonotopic or frequency-representational axis

represents an axis of point-to-point anatomical mapping between

MGN and primary auditory cortex (this axis is orthogonal to the

axis illustrated in Figure 2, bottom), while the isofrequency

axis represents a highly overlapped all-to-all projection system

(as illustrated in Figure 2, bottom). The expectation from the

anatomy then is that the map of visual space in primary auditory

cortex would also be anisotropic, with the "tonotopic" or

mediolateral axis in cortex carrying a discrete, orderly,

representation of visual space and the "isofrequency" or

anteroposterior axis exhibiting perhaps a high degree of overlap

among elongated receptive fields, or even disorder.

We indeed find a systematic and consistent map of visual

space in the mediolateral dimension of primary auditory cortex,

with receptive fields progressing from the vertical meridian

medially to peripheral locations laterally. However, we also

find a systematic map in the anteroposterior dimension of

cortex, with receptive fields in most cases progressing from

upper field anteriorly to lower field posteriorly in cortex. As

noted earlier, there is variability in this axis of

representation, with receptive fields sometimes showing reverse

progressions. Individual receptive fields are not excessively

elongated dorsoventrally in visual space, nor do they exhibit

44

excessive overlap along the dorsoventral dimension.

We interpret these observations to mean that while the map

of visual space along the mediolateral axis of primary auditory

cortex arises as a result of discrete thalamocortical

projections, along its anteroposterior axis auditory cortex

creates a map of visual space from a highly divergent

thalamocortical projection that only implicitly provides spatial

information.

Novel inputs and intrinsic organization in thalamus and cortex

The two major physiological consequences of routing visual

projections to the auditory system that we have described above

relate to (1) receptive field properties in the MGN and primary

auditory cortex (visual cells in the MGN have circular receptive

fields while cells in primary auditory cortex have either

circular or oriented, complex-like, receptive fields), and (2)

maps in the MGN and primary auditory cortex (both structures

have orderly two-dimensional maps of visual space). In these

respects, both the MGN and primary auditory cortex function

essentially similar to the LGN and primary visual cortex. What

mechanisms might operate in development that allow auditory

thalamus and cortex to process visual information?

In general, intrinsic organization and microcircuitry in a

central sensory structure can develop either by afferent-induced

differentiation or target-induced differentiation. If specific

afferents provided key signals for the development of

microcircuitry, we would expect that the intrinsic (and perhaps

extrinsic) connectivity of auditory thalamus and cortex would be

altered by visual input and resemble that in visual structures.

If central structures carry internal programs for generating

specific intrinsic connections, the fact that auditory thalamus

and cortex can transmit and transform visual information much

like visual thalamus and cortex must imply that key aspects of

intrinsic connections are essentially similar in sensory

thalamic nuclei or cortical areas. On this view, one function of

sensory thalamus or cortex is to perform certain stereotypical

operations on input regardless of modality; the specific type of

sensory input of course provides the substrate information that

is transmitted and processed.

45

The latter possibility is structurally parsimonious, and there is both direct and indirect evidence to support it. Our own anatomical experiments demonstrate that an important feature of thalamocortical connections in the auditory system, an all­to-all projection between isofrequency bands in the MGN and primary auditory cortex, remains unaltered by the induction of retinal input to the MGN. The notion that different parts of sensory thalamus or neocortex share basic commonalities is not new (Lorente de No '38: Mountcastle '78: Shepherd '79). Indeed, there is an impressive similarity of cell types in different laminae and of interlaminar connections in different areas of sensory neocortex (eg., Jones '84). In particular, intrinsic projections of cells in different laminae are remarkably similar in primary visual cortex (Gilbert and Wiesel '79: Ferster and Lindstrom '83) and primary auditory cortex (Mitani et al. '85). In the thalamus, Jones ('85) has emphasized the similarity of cell types and ultrastructural organization in different sensory nuclei. Examples of target-controlled differentiation of ultrastructure during development include the fact that retinal axons proj ecting to the ventrobasal nucleus in hamsters form synapses similar to those in the normal ventrobasal nucleus (Campbell and Frost '87), retinal axons projecting to the LGN and superior colliculus participate in different synaptic arrangements in the two structures (Lund '69; So et al. '84), and mossy fibers from different sources form similar synapses on granule cells in the cerebellum (Palay and Chan-Palay '74).

Adaptive organization in cortex If the normal organization of central auditory structures is

not altered, or at least not altered significantly, by visual input, then we might expect that some operations similar to those we observe on visual inputs in operated ferrets be carried out as well in the auditory pathway in normal ferrets. In other words, the animals with visual inputs induced into the auditory pathway provide a different window on some of the same operations that should occur normally in auditory thalamus and cortex.

Physiologically, auditory thalamus in operated animals appears to relay the receptive fields of its retinal afferents

46

(though the thalamus may control the gain of transmission), similar to the fact that auditory response profiles of neurons in the dorsal and ventral divisions of the MGN in normal animals largely resemble those of their afferents from the inferior colliculus (Aitkin et al. '84). New properties such as orientation selectivity and direction selectivity emerge for at least some visual neurons in auditory cortex of operated animals. In the normal auditory pathway, direction selectivity for a one-dimensional sensorium such as the cochlea is a selective response to an upward or downward frequency sweep, and selectivity for the direction and rate of frequency modulation exists or arises for many neurons in primary auditory cortex (Mendelson and cynader '85~ Whitfield and Evans '65~ see also Suga '84).

We suggest that, in auditory cortex of operated animals, the response features that depend specifically on the two­dimensional nature of visual input indicate a form of adaptive self-organization in cortex. These response features include orientation selectivity,complex receptive fields, and in particular the two-dimensional map of visual space with spatially restricted receptive fields. Organization of this sort implicates physiological selection of relevant subsets from an extensive input set available anatomically (Edelman '78). The mechanism behind such organization or adaptation might generally involve spatiotemporal coactivation in subsets of the visual input along with lateral inhibition, enabling modification of synaptic efficacy between presynaptic elements and restricted groups of postsynaptic neurons or sets of postsynaptic elements (Edelman and Finkel '84~ Finkel and Edelman '87).

A similar process may operate in normal auditory cortex in general, and along the isofrequency axis in particular. Neurons along this axis integrate physiological features of inputs from a highly overlapped anatomical projection system. Cells in the isofrequency axis are sensitive to interaural excitation and inhibition (Imig and Adrian '77), and may, as one example, be involved in binaural mechanisms of sound localization that require the convergence of interaural time and intensity information along with the resolution of frequency components in complex inputs (Merzenich et al. '84). There is good evidence

47

that complex acoustic signals are processed by neurons sensitive

to specific parameters of the signal (Suga '84). Quite specific

neuronal response features can be shown to arise adaptively from

highly interconnected networks; network modification rules that

can lead to the extraction of specific features (such as

orientation selectivity) from generalized inputs need not make

explicit assumptions about the visual nature of input (eg.

Linsker '86). Finally, in our maps of visual space in operated

animals, the isofrequency axis in cortex shows pronounced

variability, suggestive of dynamic organization. The

isofrequency dimension in primary auditory cortex of normal

animals shows considerable variability as well (Merzenich et al

'84), and dynamic organization dependent on input activity has

been shown to be an essential feature of maps in somatosensory

cortex (Merzenich et al. '83).

Acknowledgements

I thank Anna Roe for making the figures and for her comments

on the manuscript, and Linda Beers for her help in preparing the

manuscript. Supported by grants from the NIH, the Whitaker Fund,

the March of Dimes and the MCKnight Foundation.

48

References Aitkin, L.M., Irvine, n.R.F. & Webster, W.R. (1984) Central

neural mechanisms of hearing. In: Handbook ofPhysiologv: The Nervous System III, Amer. Physiol. soc., Bethesda, MD, pp. 675-737.

Campbell, G. & Frost, D.O. (1987) Target-controlled differentiation of axon terminals and synaptic organization. Proc. Natl. Acad. Sci. USA, 84: 6929-6933.

Cucchiaro, J. & Guillery, R.W. (1984) The development of the retinogeniculate pathways in normal and albino ferrets. froc. R. Soc. Lond. B. 223: 141-164.

Cunningham, T.J. (1976) Early eye removal produces extensive bilateral branching in the rat. science, 194: 857-859. Devor, M. (1975) Neuroplasticity in the sparing or deterioration of function after early olfactory tract lesions. Science, 190: 998-1000.

Dreher, B., Leventhal A.G., & Hale, P.T. (1980) Geniculate input to cat visual cortex: a comparison of area 19 with areas 17 and 18. J. Neurophysiol., 44: 804-826.

Edelman, G.M. (1978) GrOUp selection and phasic reentrant signaling: a theory of higher brain function. In: ~ Mindful Brain, M.I.T. Press.

Edelman, G.M. and Finkel, L.H. (1984) Neuronal group selection in the cerebral cortex. In: Dynamic Aspects of Neocortical Function, ed. by G.M. Ed4!l1man, W.E. Gall, W.M. cowan. Wiley­Interscience.

'erster, D. & Lindstrom, S. (1983) An intracellular analysis of geniculocortical connectivity in area 17 of the cat. !L.. Physiol •. Lond., 342: 181-215.

Finkel, L.H. and Edelman, G.M. (1987) Population rules for synapses in networks. In: Synaptic Function, ed. by G.M. Edelman, W.E. Gall and W.M. Cowan. wiley-Interscience.

Finlay, B.L., Wilson K.G., & Schneider, G.E. (1979) Anomalous ipsilateral retinotectal projections in Syrian hamsters with early lesions: Topography and functional capacity. J. Compo Neurol., 183: 721-740.

Frost, D.O. (1981) ordered anomalous retinal ·projections to the medial geniculate, ventrobasal and lateral posterior nuclei. J. Compo Neurol., 203: 227-256.

49

Frost, D.O. (1986) Development of anomalous retinal projections to nonvisual thalamic nuclei in syrian hamsters: a quantitative study. J. Compo Neurol., 252: 95-105.

Frost, D.O. & Metin, C. (1985) Induction of functional retinal projections to the somatosensory system. Nature, 317: 162-164.

Gilbert, C.D. & Wiesel, T.N. (1979) Morphology and intracortical proj ections of functionally characterized neurones in cat visual cortex. Nature, 3: 120-125.

Graziadei, P.P.C., Levine, R.R. & Monti-Graziadei, G.A. (1979) Plasticity of connections of the olfactory sensory neuron: regeneration into the forebrain following bulbectomy in the neonatal mouse. Neuroscience, 4: 713-727.

Hoffmann, K.-P. (1973) Conduction velocity in pathways from retina to superior colliculus in the cat: a correlation with receptive field properties. J. Neurophysiol., 36: 409-424.

Imig, T.J. and Adrian, H.O. (1977) Binaural columns in the primary field (AI) of cat auditory cortex. Brain Res. 138: 241-257.

Jones, E.G. (1984) Identification and classification of intrinsic circuit elements in the neocortex. In: Dynamic Aspects of Neocortical Function, ed. by G.M. Edelman, W.E. Gall, W.M. Cowan. Wiley-Interscience.

Jones, E.G. (1985) The Thalamus. Plenum. Kelly, J.G., Judge, P.W. & Phillips, D.P. (1986) Representation

of the cochlea in primary auditory cortex of the ferret (Mustela putorius). Hearing Res. 24: 111-115.

Linsker, R. (1986) From basic network principles to neural architecture: Emergence of orientation-selective cells. Proc. Natl. Acad. Sci. USA. 83: 8390-8394.

Lorente de No, R. (1938) Physiology of the Nervous System. Oxford Univ. Press.

Lund, R.D. (1969) synaptic patterns of the superficial layers of the superior colliculus of the rat. J. Compo Neurol. 135: 179-208.

Lund, R.D. & Lund, J.S. (1976) Plasticity in the developing visual system: the effects of retinal lesions made in young rats. J. Compo Neurol., 169: 133-154.

McConnell, S.K. & LeVay, S. (1986) Anatomical organization of

50

the visual system of the mink, Mustela vison. J. Compo

Neurol. 250: 109-132.

Mendelson, J.R. & cynader, M.S. (1985) Sensitivity of cat

primary auditory cortex (AI) neurons to the direction and

rate of frequency modulation. Brain Res. 327: 331-335.

Merzenich, M.M., Jenkins, W.M. and Middlebrooks, J.C. (1984)

Observations and hypotheses on special organizational

features of the central auditory nervous system. In: Dynamic

Aspects of Neocortical Function, ed. by G.M. Edelman, W.E.

Gall, W.M. Cowan. wiley-Interscience.

Merzenich, M.M., Kaas, J.H., Wall, J., Nelson, R.J., Sur, M., &

Felleman, D. (1983) Topographic reorganization of

somatosensory cortical areas 3B and 1 in adult monkeys

following restricted deafferentation. Neuroscience. 8: 33-

55.

Middlebrooks, J.C. and Zook, J.M. (1983) Intrinsic organization

of the cat's medial geniculate body identified by

projections to binaural response-specific bands in the

primary auditory cortex. J. Neurosci. 3: 203-224.

Mitani, A., Shimokouchi, M., Itoh, K., Nomura, S., Kudo, M. &

Mizuno, N. (1985) Morphology and laminar organization of

electrophysiologically identified neurons in the primary

auditory cortex in the cat. J. Compo Neurol., 235: 430-447.

Mountcastle, V.B. (1978) An organizing principle for cerebral

function: the unit module and the distributed system. In:

The Mindful Brain, M.I.T. Press.

Palay, S.L. and Chan-Palay, V. (1974) Cerebellar Cortex:

Cytology and organization. springer.

Roe, A.W., Garraghty, P.E. & Sur, M. (1987) Retinotectal W cell

plasticity: experimentally induced retinal projections to

auditory thalamus in ferrets. Soc. Neurosci. Abst., 13:

1023.

Schneider, G.E. (1973) Early lesions of the superior colliculus:

factors affecting the formation of abnormal retinal

projections. Brain Behav. Evol., 8: 73-109.

Shatz, C.J. (1983) The prenatal development of the cat's

retinogeniculate pathway. J. Neurosci., 3: 482-499.

Shepherd, G. (1979) The Synaptic Organization of the Brain.

Oxford Univ. Press, 2nd ed.

51

Sherman, S.M. & Spear, P.O. (1982) organization of visual pathways in normal and visually deprived cats. PhysioL Rev., 62: 738-855.

So, K.-F., Campbell, G., & Lieberman, A.R. (1985) synaptic organization of the dorsal lateral geniculate nucleus in the adult hamster. Anat. Embryol., 171: 223-234.

Suga, N. (1984) The extent to which biosonar information is represented in the bat auditory cortex. In: Dynamic Aspects of Neocortical Function. ed. by G.M. Edelman, W.E. Gall, W.M. Cowan. Wiley-Interscience.

Sur, M. & Garraghty, P.E. (1986) Experimentally induced visual responses from auditory thalamus and cortex. Soc. Neuroscience Abst., 12: 592.

Sur, M., Garraghty, P.E., & Roe, A.W. (1988) Experimentally induced visual projections into auditory thalamus and cortex. Submitted for publication.

Sur, M., Roe, A.W. & Garraghty, P.E. (1987) Evidence for early specificity of the retinogeniculate X cell pathway. Soc. Neurosci. Abstr., 13: 590.

Sur, M. & Sherman, S.M. (1982) Linear C-1aminae of the cat's lateral Neurophysiol., 47: 869-884.

and nonlinear W cells in geniculate nucleus. ~

Tong, L, Spear, P.O., Kalil, R.E., & Callahan, E.C. (1982) Loss of retinal X-cells in cats with neonatal or adult visual cortex damage. Science, 217: 72-75.

Vitek, D.J., Schall, J.D., & Leventhal, A.G. (1985) Morphology, central projections, and dendritic field orientation of retinal ganglion cells in the ferret. J. Compo Neuro1., 241: 1-11.

Whitfield, I.C. & Evans, E.F. (1965) cortical neurons to stimuli of Neurophysiol., 28: 655-672.

Responses of auditory changing frequency. ~

The Hippocampus and the Control of Information Storage in the Brain

Nestor A. Schmajuk Center for Adaptive Systems

Boston University

Abstract. The present chapter assumes that the amount of information stored in the brain at a given moment is proportional to the mismatch between internally predicted events and the actual events ocurring in the external world. We have proposed ({29J, [30/) that the hippocampus is involved in the computation of the "aggregate predictJon" of ongoing events. This prediction is compared with infor­mation from the external world in order to determine the amount of information to be stored in the brain.

According to the "aggregate prediction" hypothesis (a) the effect of hippocam­pal lesions (HL) is an impairment in the integration of the aggregate prediction used to compute attentional variables (b) the effect of the induction of hippocampal long­term potentiation (LTP) is an increase in the value of the aggregate prediction by way of increasing the value of CS-CS associations and (c) that neural activity in hip­pocampus is proportional to the instantaneous value of the aggregate prediction. In addition, the present chapter introduces the hypothesis that medial septum activity is proportional to the sum of the values of different attentional variables.

The present chapter presents computer simulations for delay conditioning, con­ditioned inhibition, extinction, latent inhibition, and blocking for normal and HL cases. The "aggregate prediction" hypothesis proved capable of simulating most, but not all, experimental data regarding hippocampal manipulations in the rabbit nictitating membrane response preparation.

Introduction

It has been suggested that the brain is organized into separate modules capa­ble of storing a limited amount of different types of information, such as sensory, temporal, contextual, spatial, motor, etc. [12], [71. Gazzaniga [81 proposed that the hippocampus regulates the storage of new information into the memory modules.

The process of information storage in the brain has been related to atten­tional mechanisms that determine the level of processing assigned to each stimulus. Sokolov [321 proposed that the intensity of stimulus processing is proportional to the novelty of the stimulus. Sokolov suggested that the brain constructs a neural model of external events, and that an orienting response (OR) appears every time the sensory input does not coincide with a "neuronal model" previously established in the brain. In this condition, the novel aspects of the input are stored in the neural model. When there is coincidence between the stimulus and the model, the animal may respond without changing its neural model of the world.

Kaye and Pearce [111 supported Sokolov's view that the strength of the OR

54

might be an index of the amount of processing afforded to a given stimulus, sug­gesting that the strength of the OR elicited by a stimulus would be inversely pro­portional to its predictive accuracy. Pearce and Hall (P-H) [19] offered a model of Pavlovian conditioning in which new information is stored in memory when there is a mismatch between predicted and actual events. More specifically, associability 0:. of a given C S. on a given trial depends on the predictions of the US made by all the CS's acting on the previous trial, 0:. =1 A - EiVi I, where EiVi is the sum of the associative values of all CS's present on the preceding trial, and A is the US intensity on the previous trial. In the P-H model associability of a given CS decreases as its association with the US increases. The strength of the OR is therefore proportional to 1 A - EjVj I, that is, proportional to the level of total uncertainty about the US.

Both Sokolov's model and the OR have been related to hippocampal function. For instance, Douglas [6] suggested that once a neural model is obtained, the hip­pocampus reduces attention to stimuli associated with nonreinforcement. In the same vein, Vinogradova [39] suggested that the hippocampus is an active filter of information which stops the process of information storage when the environment is stable. Specifically, Vinogradova proposed that region CA3 of the hippocampus compares signals coming from the reticuloseptal input with signals coming from the cortical neural model. When novelty is detected in CA3, the reticular formation is desinhibited, and the activated state of the reticular formation facilitates the storage of new information in the cortex.

Grastyan, Lissak, Madarasz, and Donhoffer [9] and Anchel and Lindsley [1] found that hippocampal theta activity was positively correlated with the intensity of the OR. Furthermore, Vanderwolf, Kramis, Gillespie, and Bland [38] suggested that hippocampal theta was correlated with "voluntary" behaviors, and that hip­pocampal non-theta was correlated with "automatic" behaviors. Pearce and Hall [19] also pointed out the correlation between large values of 0:; and "controlled" (or "voluntary") behavior, and small values of 0:; and automatic behavior. There­fore, theta activity, voluntary behavior, orienting response, and information storage would be associated to large values of 0:; whereas, non-theta activity, automatic behavior, absence of orienting response, and no information storage would be asso­ciated to small values of 0:.

The relationship among hippocampal theta, orienting response, and the atten­tional term 0:, suggests that the hippocampus might be regulating the information storage in the brain. The amount of information acquired would be proportional to the degree of uncertainty about the ongoing events that the organism is experi­encing at a given time. When temporal and spatial arrangements of the external world are perfectly predicted, no additional information is stored.

Schmajuk [27] suggested that in the absence of the regulatory function of the hippocampus, each brain module stores an amount of information that is indepen­dent of the information stored in the other modules. Schmajuk [27] proposed that the effects of hippocampal lesions (HL) can be accurately described in terms of the P-H model: HL would cause the associability of a CS on trial n to be independent of its previous values and of predictions of the US made by other CS's.

55

THE S-P-H MODEL

Schmajuk and Moore [29) presented a version of the P-H model (designated the S-P-H model) that incorporates real- time expressions for the equations defining associative and attentional variables. Computer simulations of a revised version of the S-P-H model show that with the assumptions regarding HL proposed by Schmajuk [27) the model describes the behavior of HL animals in delay condition­ing, partial reinforcement, differential conditioning, extinction, latent inhibition, blocking, overshadowing, and discrimination reversal. More recently, Schmajuk [28) presented a version of the S-P-H model that provides explicit performance rules that permit real-time descriptions of the rabbit classically conditioned nictitating membrane (NM) response in complex conditioning paradigms.

In previous papers ([28), [30)) we contrasted experimental results regarding the hippocampal formation in the NM preparation with computer simulations con­cerned with the effect of HL and hippocampal long-term potentiation (LTP) in­duction on the topography of the classically conditioned NM response in different conditioning paradigms. In addition, the model provided a description of hippocam­pal and medial septal neuronal activity during classical conditioning.

Formal Description of the S-P-H Model

First-order associative values. Associative value, Vi', represents the prediction of event Ie by C Si. Whenever the intensity of event Ie, >.", is greater than B" as defined by Equation 5, the excitatory associative value vi' between CSi and event Ie increases by

(1)

where 9 is the rate of change of ¥S" , Si is the salience of C Si , a: represents the associability of CS with event Ie, >." represents the intensity of event Ie, and Ti

represents the trace ·of C Si.

Whenever >." S B", the inhibitory associative value between C Si and event Ie, Nf, increases by

(2)

where 9' is the rate of change of Nf , and X" = B" - >.". The net associative value of a C Si with event Ie is

Vi" = Vi" - Nf. (3)

When i = Ie,Vi" defines the net prediction of the event i by itself.

Second-order associative values. Bf ' the sum of first- and second- order pre­dictions of event Ie by C Si , is

(4)

Vr" is the net associative value of C Sr with event Ie. The sum over index r involves all the CS's with index r '::f; Ie. ~r is the net associative value of CSi with all CS's with index r '::f; Ie. Vr" is the net associative value of all CS with event k. Ti is the trace of C Si . The mathematical expression for Ti is given below. Coefficient

56

w~ serves to adjust the relative weights of first- and second- order predictions in paradigms such as conditioned inhibition. In order to avoid redundant C Si - US and CSi - CSi - US associations, w~ = 0 when i = r, and wf > 0 when i,# r.

B", the aggregate prediction of event k made upon all CS's (including the context) with Ti > 0 at a given moment, is given by

(5)

The sum over index i involves all the CS's acting at a given moment.

Associability. The associability of C Si with event k at time step t is given by

(6)

By Equation 6, vl does not increase when at(t) equals zero, that is when >."(t) equals B"(t - 1).

Pearce and Hall (1980) suggested that when afs tends to zero, the CS still evokes a response through an "automatic" process. The intensity of "automatic" attention and responding to C Si is proportional to BfS.

Salience. In the P-H model, salience Si is a constant. However, in the present model rendering of the S- P-H model, Si is defined by

(7)

where 0i is a constant and a: is the associability of C Si with itself.

Replacing a: by its value in Equation 6, it results

(8)

Equation 7 implies that when a: equals zero salience Si equals OJ . According to Equation 8, Si equals zero when the intensity of C Si is perfectly predicted by all acting CS's including itself at a given time step. Conceptually, this means that salience Si decreases as C Si becomes increasingly "familiar" to the animal. Larger increments in vl and Nt are obtained with novel rather than with familiar CS's.

In addition, Equation 7 implies that when C Si is predicted by a CS preceding it, C S, , the association between C Si and C S, retards the formation of the association between C Si and event k. This property is used to describe successive conditional responding.

Equation 7 is also used to yield latent inhibition, i.e., the effect of CS pre­exposure in the absence of the US on the subsequent acquisition of the CS-US association. Wagner [40] proposed a similar mechanism for latent inhibition in the context of the Rescorla-Wagner [24] model. Latent inhibition is predicted because CS pre-exposure reduces the value of Si, thereby retarding subsequent acquisition of the CS-US association.

Trace function. It is assumed that C Si generates a trace, Ti • This trace has a delay, increases over time to a maximum, stays at this level for a period of time independent of CS duration, and then gradually decays back to zero.

57

Formally, and specifically for the rabbit NM preparation, the trace is defined for t ~ 200 msec by

(9)

where C Si max is the maximum intensity of the trace recruited by C Si , and ki is a constant, 0 < ki ~ 1. Parameter kl is selected so that, when applying Equations 1, 2, 9, and 10, the lSI for NM optimal conditioning is 200 msec. For any CS duration the amplitude of the trace rises during the first 200 msec after C Si onset and remains equal to C Si max as long as C Si does not decay.

If CSi = 0 and t > 200 msec, Ti(t) decays by

(10)

Performance rules. Performance rules were selected to relate variable BUS to the topography of the NM response. Performance rules allow the computation of the instantaneous values of CR using the instantaneous values of BUs.

The NM response is characterized by (a) the latency to CR onset, (b) shape during the CS period, (c) shape during the US period, and (d) decay to baseline.

Latency to CR onset. Let tes denote the time step at which CS onset occurs. Then the time of CR onset, denoted teR, is the earliest time t' such that

t L L Bfs (t/) >= L1. (11)

t'=t, ;

The sum over the index i involves Bfs of aU CS's with T; > 0, excluding the context. The sum over index k involves all time steps on which T; > 0, starting at the time step when the amplitude of the NM response as defined by Equations 12 and 13 equals zero (see below). Time increments, ~t , are equal to one time step. L1 is a threshold greater than zero. Equation 11 implies that as BfS increases over trials, CR onset latency moves progressively toward an asymptote determined by L1.

CS period. For time steps t > teR , i.e., after the time of the CR onset, the amplitude of the NM response, NMR (t), is changed by

(12)

where kz is a constant (0 < k2 ~ 1). By Equation 5, BUS(t) increases with the time constant kl of trace Ti , k2 is selected kz > leI so that NMR (t) reaches BUS(t) during the CS period. For t < teR, the amplitude does not change.

US period. During the US period, while BUS(t) > ~US(t), NMR (t) still increases by Equation 12. However, when ~US(t) > BUS(t), NMR (t) increases by

~ NMR (t) = k2(~US (t) - NMR (t - 1». (13)

Decay to baseline. When BUS(t) and ~US(t) equal zero, NMR (t) decays to baseline by

~ NMR (t) = -kz NMR (t - 1). (14)

58

By Equations 12, 13, and 14, NMR (t) is bounded between zero and >..US.

The S-P-H Model as a Model for Classical Conditioning in Normal Ani­mals.

The S-P-H model is able to describe normal behavior in simultaneous, delay, and trace conditioning; partial reinforcement; conditioned inhibition; differential conditioning; extinction; latent inhibition; blocking; overshadowing and mutual overshadowing; discrimination acquisition and reversal; sensory preconditioning; and secondary reinforcement [28], [30].

The S-P-H model belongs to a class of models that allows second-order asso­ciations. The integration of different predictions into a larger and new prediction, is similar to the process Tolman [37] called inference. For Tolman, expectancies can be combined in order to form new expectancies and organized in a "cogni­tive map". Up to the present, models for classical conditioning did not have any mechanism to account for "inference" processes. The introduction of second-order associations allows to build "computational cognitive maps" [17] in which CS-CS predictions can be combined among them, and with CS-US associations. By the introduction of second-order associations the S-P-H model is capable of describing sensory preconditioning and secondary reinforcement.

The S-P-H model predicts sensory preconditioning by allowing CS(B) to be as­sociated to CS(A) in a first phase, and CS(A) to be associated to the US in a second phase. Rescorla [23] found that simultaneous presentations of CS(A) and CS(B), produced higher levels of sensory preconditioning than successive presentations. In conflict with this results, the S-P-H model predicts optimal CS-CS conditioning for successive rather than for simultaneous presentations.

The S-P-H model predicts secondary reinforcement by allowing CS(A) to be associated to the US in a first phase, and a CS(B) to be associated to CS(A) in a second phase. Because CS(B) is never associated with the US, the model predicts that extinction of the CS(A)-US association entails extinction ofresponding to CS(B). In agreement with this prediction Rashotte, Griffin, and Sisk [22] and Leyland [13] found that extinction of the CS(A)-US association led to substantial reduction in responding to CS(B). Opposite results however were obtained by Rizley and Rescorla [26] and Holland and Rescorla [10].

HYPOTHESES

The "aggregate prediction" hypothesis regarding HL, LTP induction, and hippocampal neuronal activity

This section introduces the "aggregate prediction" hypothesis regarding HL, LTP induction, and hippocampal neuronal activity; and the "associability" hypoth­esis regarding medial septum neuronal activity. These hypotheses are independent of the description of the S-P-H model as a model for classical conditioning.

Schmajuk [27], [29] suggested that the effect of HL can be described as an impairment in the computation of CS-US associability values. This impairment

59

results from the lack of integration of new and old predictions about an event arising from all CS's present at a given time. Because individual predictions replace the aggregate prediction in the computation of ai(t) for the HL case, this assumption is called the "aggregate prediction" hypothesis.

In the present version of the S-P-H model, BUs, the aggregate prediction of the US, can be derived from Equations 4 and 5

(15)

Since by Equation 15 the integration of predictions is achieved through the sum of the individual net predictions Li and through second-order associations (V[VPS) , lack of integration of predictions implies that both computations are absent in the HL case.

When Li is not computed and V[ becomes zero in Equation 15, associability for the HL case becomes

(16)

Equation 16 is equivalent to that used by Schmajuk and Moore [29] for the HL case. In the S-P-H model, blocking and overshadowing are achieved when the asso­ciability of the to-be-blocked CS is reduced due to the association of the blocker with the US. Since by Equation 16 associability of the to-be-blocked es is independent the association accrued by the blocker es, use Equation 16 implies impairments in blocking and overshadowing. In the S-P-H model, inhibitory association is achieved when the associability of CS- is increased due to the excitatory association of the CS+. Since by Equation 16 associability of es- is independent of the association accrued by CS+, associability of CS- remains zero and CS- cannot gain inhibitory association. Consequently, Equation 16 implies impairments in inhibitory condi­tioning paradigms, such as conditioned inhibition and differential conditioning.

Because Equation 16 allows the context to accrue more association with the US than it does when Equation 6 is applied, responding in HL animals depends relatively more on the association gained by the context and relatively less on the association gained by the es. Consequently, changes in the characteristics of the nominal es do not affect responding in HL animals as much as it does in normal animals. Therefore, generalization gradient is sharper in normal animals than in HL animals.

As mentioned above, Kaye and Pearce [11] suggested that the strength of the OR is proportional to ai. Under this assumption, since ays for the HL case (Equation 16) is greater than ays for the normal case (Equation 6), use of Equation 16 also implies that HL animals display stronger ORs than normal animals.

When V{ becomes zero in Equation 15, the intensity of the inhibitory reinforcer, US, becomes

(17)

Equation 17 is equivalent to that used by Schmajuk and Moore [29] for the HL case.

60

When ~i is not computed and V{ becomes zero in Equation 15, salience Si is given by

(18)

Equation 18 means that salience does not decrease over trials, or equivalently, that the CS does not become increasingly "familiar" over time. Use of Equation 18 implies impairments in latent inhibition.

Because all V{ equal zero, BYs , defined in Equation 4, is given by

(19)

In general, use of Equation 19 implies impairments in cognitive mapping. Specifi­cally in the case of classical conditioning, use of Equation 19 implies impairments in sensory preconditioning, secondary reinforcement, compound conditioning, and serial compound conditioning. In addition, performance rules translate BYs values into NM responses, thereby accounting for changes in NM response topography after HL.

Whereas the "aggregate prediction" hypothesis regards the effect of HL as an impairment in the integration of multiple predictions, it assumes that this integra­tion increases when LTP is induced. Since this integration is achieved through the sum of the individual net predictions (~i) and through second-order associations (V{V,US) in Equation IS, the "aggregate prediction" hypothesis proposes that all V{s increase whenever LTP is induced. Since the "aggregate prediction" hypothesis accounts for HL and LTP effects by affecting the computation of BUs, for consis­tency we assume that activity of some hippocampal neurons is proportional to the instantaneous value of the aggregate prediction, BUs.

The "assoeiability" hypothesis regarding medial septum neuronal activity

Berger and Thompson [41 suggested that neural activity in medial septum rep­resents an arousal signal that controls hippocampal theta. As discussed before, theta activity is correlated with at. Therefore, the present study assumes that the frequency of medial septum neuronal firing is proportional to the instantaneous magnitude of ~A: ~i at, Le., the sum of CS-CS and CS-US associabilities of all CS's present at a given time. As mentioned above, ~A: ~i at is proportional to the total degree of uncertainty about ongoing events in the external world and determines the total amount of information to be stored in the brain.

COMPUTER SIMULATIONS

This section presents some relevant experimental data and contrasts the data with the results of computer simulations describing the NM topography of nor­mal and HL animals. Details about the method employed in the simulations were presented in [281 and [301. Parameters values in the present simulations were a = .3,e' = .015,wf = 2,A = .5,kl = .I,k2 = .5, and Ll = .5.

Figure 1. Delay conditioning: L: HI. cue. N: normal cue. A: CS(A). X: Context. ALPHA: UIIOciability. S: Salience. Left Panels: NM response topography in 10 reinforced trials. Upper-right Panels: Net lI8IIociative values (VT) at the end of each trial, u a function of trials. Lower-right Panels: AlIOciability (ALPHA) at S50 maec, u a function of trials.

1. Acquisition of delay classical conditioning

Experimental data. Several studies describe the effect of HL on acquisition rates. Using a delayed conditioning paradigm, Schmaltz and Theios [31] found faster than normal acquisition of the conditioned NM response in HL rabbits with a 250-meec CS, a 50-meec shock US, and a 250-meec lSI. In contrast with these data, Solomon and Moore [34] and Solomon [33] found no difference in the rate of acquisition between normal and HL rabbits in forward delayed conditioning of the NM response using a 450-meec CS, a 50-meec shock US, and a 450-meec lSI. In summary, acquisition rates become accelerated or remain unaffected by HL.

Several studies describe the effect of HL on NM topography during acquisition. Solomon and Moore [34] and Solomon [33] found that conditioned response (CR) topography did not differ in normal and HL rabbits in forward delayed conditioning of the NM response using a 450-meec CS, a 50-meec shock US, and a 450-meec lSI. Port and Patterson [20] found that CR latency was shorter in rabbits with fimbrial lesions (Le., hippocampal output) than in rabbits with cortical or sham lesions, mainly during the first day of acquisition. Summarizing, CR onset latencies in a delay conditioning paradigm become shorter or remain unaffected after HL. Powell and Buchanan [21] reported increased OR (measured as an increased bradycardia)

62

L R!S10Nltl I

N RtSPONltI1

A -----X

IRIALl

IRIALl

Figure 2. Extinction. L: HL case. N: normal case. A: CS(A). X: Context. ALPHA: associability. S: Salience. Left Panels: NM response topography in 10 extinction trials. Upper-right Panels: Net associative values (VT) at the end of each trial, as a function of trials. Lower-right Panels: AS80ciability (ALPHA) at 350 msec, as a function of trials.

over conditioning trials in HL rabbits relative to controls.

Simulation results. Figure 1 shows simulations of 10 trials in a delay condition­ing paradigm.

In agreement, Schmaltz and Theios [31] simulation results show faster than normal acquisition rate in the HL case. Simulations for the normal case show that both context and CS associabilities decreased over trials, the CS overshadowing the context. In the HL case both the CS and the contextual associabilities were larger than in the normal case and therefore both CS and context were able to acquire larger associative values at a faster rate than in the normal case. Also in agreement with Port and Patterson [20], simulated CR latency is shorter for the HL than for the normal case.

As mentioned above, Kaye and Pearce [11] suggested that the strength of the OR is proportional to 0:,. Figure 1 shows greater o:fs for the HL than for the normal case, a result in agreement with Powell and Buchanan's data [21].

2. Extinction

Experimental data. Acquisition of the CR proceeds with an orderly sequence of changes: percentage of NM responses generated in each session increases, CR

l ItlIOItID

N IIDIOItID I I I I I

s- I

63

iV!:1

I. 411 ii81S1t

~ --IBL---ISL-

Figure 3. Conditioned inhibition: L: HL case. N: normal case. A: CS(A). B: CS(B). X: Context. ALPHA: associability. S: Salience. Left Panels: NM reaponae topography in A+, (A+B)-, A-, and B- trials. Upper-right Panels: Net associative values (VT) at the end of each trial, as a function of trials. Lower-right Panels: Associability (ALPHA) at 350 msec, as a function of trials.

latency decreases, and CR amplitude increases. This sequence is reversed in extinc­tion.

Two studies describe the effect of HL on extinction. After initial acquisition, extinction of conditioned NM response in rabbit appeared to be unaffected by HL [3], [31]. However, normal rabbits decreased the number oftrials to reach extinction criterion whereas HL rabbits increased the number of trials to criterion, following alternating acquisition-reacquisition sessions [31].

The effect of HL on NM topography has also been studied. Orr and Berger [18] found that HL did not affect CR topography during extinction using a 850-msec CS, a 100-msec air puff US, a 750-msec lSI, and a 6O-sec IT!.

Simulation results. Figure 2 shows the results of simulations of extinction using the S-P-H model. Ten extinction trials were simulated with initial values resulting from simulations of 10 reinforced trials in a delay conditioning paradigm.

Simulations show that during extinction CR latency increases and CR amplitude decreases over trials. Extinction proceeded at the same rate for both HL and normal cases. Simulation results are in agreement with Berger and Orr's [3] and Schmaltz and Theios's [31] studies showing no difference in the rate of extinction of normal

64

Figure 4. LlLtent inhibition. CR ILmplitude ILfter 5 tri&!a of CS preexposure (PRE) or preexposure

to the context &lone (SIT) ILnd 5 tri&!a of dellLY conditioning, for norm&l ILnd HL cue.

and HL animals.

3. Conditioned Inhibition

Experimental data. Only one study describes the effect of HL on conditioned inhibition. Solomon [331 presented HL and control rabbits with light CS+ trials interspersed with light-plus-tone CS- trials. He found that this procedure yields inhibitory conditioning of the tone in normal and HL animals.

Simulation results. Figure 3 shows the simulations of a conditioned inhibi­tion paradigm. During conditioned inhibition two types of trials were alternated: reinforced trials consisted of a single reinforced CS (A), and nonreinforced trials consisted of a compound CS (A and B). Stimulus B was the conditioned inhibitor. After 40 simulated trials, the CR elicited by A and B together was smaller than that elicited when A was presented alone because B has acquired inhibitory asso­ciative value. Simulations for the HL case show that the CR elicited by A and B together was not smaller than that elicited when A was presented alone because B had not acquired inhibitory associative value. Simulations for the HL case show larger associabilities than the normal case. Solomon [33] reports no impairment in conditioned inhibition for HL animals, in disagreement with the simulations.

4. Latent inhibition

Experimental data. The effect of HL on latent inhibition has been described in the rabbit NM preparation. Latent inhibition (LI) refers to the finding that repeat­edly presenting the CS alone, before pairing it with the US, produces retardation in the acquisition of the CR. Solomon and Moore [34] report that animals with HL showed impaired 11 after preexposure to a tone CS.

Simulation results. Figure 4 shows simulations of a LI paradigm. Simulations consisted of 5 trials of CS preexposure followed by 5 trials in which the CS is paired

L RISIOII$IS:

N

I 1/ I !

B- : I

IImIItID I I I I

65

B

111Il0l

~~~~ •• r-~~~~~~~~~~~ ~ ~ ~

Figure 5. Blocking. L: HL cue. N: normal cue. A: C5(A). B: C5(B). X: Context. ALPHA: auociability. 5: Salience. Left Panels: NM reapoIIH topolI'aphy in A- and B- teat trials, after 5 CS(A) reinforced trials and 5 CS(A) and CS(B) reinforced trials. Upper-right Panels: Net auociative values (VT) at the end of each trial, as a function of trials. Lower-right Panels: AlIOciability (ALPHA) at 350 maec, as a function of trials.

with the US (PRE groups). Control groups received 5 context-only trials followed by 5 CS-US trials (SIT groups).

Simulations revealed that CS and context saliences decre88ed in the normal PRE group but not in the normal SIT group. The decreaaed saliences caused the normal PRE group to acquire CS-US 88sociations at a slower rate than the normal SIT group did. Therefore, after 5 CS-US trials, the normal PRE group generated CRe of smaller amplitude than those generated by the normal SIT group. (Figure 4). Simulations revealed no differences in saliences between the HL PRE and SIT groups. Since both HL PRE and SIT groups acquired CS-US 88sociations at a similar rate, both groups generated CR's of equal amplitude after 5 CS-US trials (Figure 4). The model is consistent with Solomon and Moore's 134] data.

5. Blocking

Experimental data. In blocking, an animal is first conditioned to a CS(A), and this training is followed by conditioning to a compound consisting of A and a second stimulus B. This procedure results in a weaker conditioning to B. Solomon

66

[33] found that HL disrupted blocking of the rabbit NM response. Control groups in Solomon's [33] investigation provide evidence regarding the effect of HL on mutual overshadowing between two compounded CS's of differential salience. Unlike the case of blocking, HL rabbits showed no deficit in mutual overshadowing.

Simulation results. Figure 5 shows simulations of a blocking paradigm. Exper­imentals received 5 trials with one CS (blocker) paired with the US followed by 5 trials with the same CS and a second (blocked CS) paired with the US.

Figure 5 shows that the model simulated blocking in the normal case. Consistent with Solomon [33], simulations show that HL virtually eliminated blocking.

DISCUSSION

The present chapter assumes that the hippocampus is involved in the com­putation of the aggregate prediction of ongoing events. This internal prediction is compared with information from the external world in order to determine the amount of processing to be assigned to external stimuli. It is assumed that the ag­gregate prediction is disrupted by HL, enhanced by LTP induction, and represented in hippocampal neuronal activity. Activity in medial septum is assumed to be pro­portional to the degree of uncertainty about ongoing events and to the amount of processing and storage of external information. Results obtained through the application of these hypotheses to the S-P-H model are discussed below.

HL effects. The "aggregate prediction" hypothesis regards the effect of HL as an impairment in the integration of multiple predictions into the aggregate prediction BUs. Table 1 summarizes the results of the simulation experiments for the HL case obtained in the present paper together with results obtained in a previous studies [28]. Under the aggregate prediction hypothesis, the model successfully described HL effects on delay conditioning, conditioning with different ISIs and shock US, conditioning with long lSI and air puff US, extinction, latent inhibition, blocking, discrimination reversal, and sensory preconditioning. In addition, the model pre­dicts that paradigms involving cognitive mapping (such as secondary reinforcement, compound conditioning and serial compound conditioning), differential condition­ing, and overshadowing are impaired by HLj and that partial reinforcement and simultaneous conditioning are facilitated by HL. These predictions await experi­mental testing in the rabbit's NM response preparation.

The S-P-H model has problems describing conditioned inhibition, and mutual overshadowing for the HL case. The failure of the model to explain HL effects in an overshadowing paradigm is limited to the case when both CS's have similar saliences: the S-P-H model correctly describes HL effects on overshadowing when CS's of different saliences are used [25]. Also, the failure of the model to explain HL effects in an inhibitory conditioning paradigm is limited to the case of conditioned inhibition in the NM response preparation: the S-P-H model correctly describes HL effects on differential conditioning in rats [15].

LTP effects. The "aggregate prediction" hypothesis assumes that LTP induction increases the integration of multiple predictions into the aggregate prediction BUS by way of increasing CS-CS associations. Computer simulations show that under

67

TABLEl Simulations of the S-P-R Model Compared with the Experimental Results

of Classieal Conditioning of the NM Response

Paradigm

Delay Conditioning

lSI effects (shock US) zero lSI short lSI

optimal lSI

long lSI

lSI effects (air puff US) long lSI

Orienting response Conditioned Inhibition Extinction Latent Inhibition Generalization Blocking Mutual Overshadowing Discrimination Reversal

Sensory Preconditioning

Discrimination acquisition

Acquisition

Extinction

Acquisition

Observed Simulated

HIPPOCAMPAL LESION EFFECT

normal/ shorter latency normal/faster acquisition

? shorter latency faster acquisition normal/ shorter latency normal/faster acquisition normal/shorter latency normal/faster acquisition

shorter latency faster acquisition

+ o o

+

o

greater NM peak greater CS+ area greater CS- area

shorter latency faster acquisition

+ shorter latency faster acquisition normal latency faster acquisition shorter latency faster acquisition

shorter latency faster acquisition

+ *

o

+

*

greater NM peak greater CS+ area greater CS- area

LONG - TERM POTENTIATION EFFECT

+ HIPPOCAMPAL NEURAL ACTIVITY

increases precedes behavior models NM response decreases precedes behavior

increases

precedes behavior models NM response decreases simultaneous with behavior"

MEDIAL SEPTUM NEURAL ACTIVITY

decreases decreases

Note: - = deficit; + = facilitation; 0 = no effect; ? = no available data; * = the model fails to describe accurately the experimental result.

68

cs·-~ ·r I \/.

CSr-~'----r-I-",

cx~ I

. US \/.

I

~-CS· I

~-US

Figure 6. S-P-H attentional-associative mooel. C S" C S,: conditioned stimuli. V[: C Si -CS, associative value. VPS: CS, - US associative value. ays : CS, - US associability. a~: C Si - C S, associability. C-L: cortico-Iimbic system. S-T: striato-cerebellar system. H: hippocampus.

this hypothesis the 8-P-H model describes a deficit in discrimination during the first trials of training, rather than the facilitation shown by rabbits in the acquisition of a discrimination after LTP induction in the hippocampus [2J. The failure of the model to explain LTP effects in a discrimination acquisition paradigm is limited to the case of classical conditioning: McNaughton, Barnes, Rao, Baldwin, and Rasmussen [141 found that LTP induction in the hippocampus produced a deficit in acquisition in a spatial learning paradigm.

Hippocampal neuronal activity. The "aggregate prediction" hypothesis assumes that neural activity in hippocampus is proportional to the instantaneous value of the aggregate prediction, BUs. We have shown [301 that activity inpyramidal cells dorsal hippocampus is correctly described as proportional to BUS during acquisition but not during extinction of classical conditioning.

Medial septum neuronal activity. The "associability" hypothesis proposes that neuronal activity in the medial septum is proportional to 1:k 1:. a~ ,Le., to the de­gree of uncertainty about ongoing events in the external world and the amount of information to be stored in the brain. We have shown [301 the 8-P-H model adequately describes medial septal unit activity over acquisition trials. Simulation results suggest that neural activity in the medial septum is proportional to the sum of different a~ =1 )..k - Bk I, i.e., comparisons between actual and predicted events.

CONCLUSION

As pointed out in the Introduction, Gazzaniga (1984) proposed that the hip­pocampus regulates the storage of new information in all modules of the brain. Figure 6 shows a ft.ow diagram of information storage during classical condition­ing. Variables a~ and ays regulate the total amount of information stored in each module of the system. Both a~ and afs are proportional to the difference between actual and predicted values of ongoing events. In order to compute "aggregate

69

predictions", the hippocampus receives vFs information from the cerebellum [5). and Vl information from the association cortex (see [35)). Aggregate predictions, B", are used to compute a~ and ays, which are broadcast back to cerebellum and association cortex in order to modulate the associations accrued in each module. It is possible that ai has an excitatory influence over the association cortex, whereas ays has an inhibitory action over the cerebellum. After HL, since aggregate pre­dictions, ar, and ays are no longer computed, CS-US (striata-cerebellar) modules become independent of each other and able to store more information than before, and CS-US (neocortical) modules do not receive the necessary activation needed to store new information.

The "aggregate prediction" hypothesis assumes that HL impairs the formation of CS-CS associations, an effect that might be equivalent to HL effects as described by theories of hippocampal function in humans. For example, Squire [36) suggested that monkeys with combined hippocampal and amygdalar lesions were impaired in their ability to acquire new information about the world (declarative memory) but not in their ability to acquire new perceptual-motor skills (procedural memory). In the same vein, other authors proposed that the limbic-cortical regions of the brain would be involved in processes such as stimulus configuration [16), or vertical associative memory [41). Each of these processes may involve CS-CS associations. Striatal and cerebellar regions of the brain would be involved in processes such as habit formation [16], or horizontal associative memory [41], each of which appear to involve CS-US associations. In the context of the S-P-H model, impairments in CS-CS, but not in CS-US, associations might be regarded as parallel to deficits in declarative memory, stimulus configuration, vertical associative memory, or repre­sentational memory.

The present study shows that the S-P-H model is able to describe CR topogra­phy of the rabbit NM response in real-time in a large number of classical condition­ing paradigms involving cognitive mapping. Although the "aggregate prediction" hypothesis provides interesting insights about diverse aspects of hippocampal func­tion, it is unable to describe some effects of hippocampal manipulations. It is ap­parent, however, that the approach presented in this study provides a powerful tool to compare theoretical variables with brain activity in real time, contributing very precise information to the guidance of experimental research and the interpretation of experimental data.

ACKNOWLEDGEMENT

This research was supported in part by the National Science Foundation (NSF IST-84-17756). The author thanks Carol Yanakakis for her valuable assistance in the preparatIon of the manuscript.

BmLIOGRAPHY

11] H. Anchel and D.B. Lindsley. Differentiation of two reticula-hypothalamic sys­tems regulating hippocampal activity. Electroencephalography and Clinical Neuro­physiology, 32,209-226, 1975.

121 T.W. Berger. Long-term potentiation of hippocampal synaptic transmission 8.ftects rate of behavioral learning. Science, 224,627-630, 1984.

70

[3J T.W. Berger and W.B. Orr. Hippocampectomy selectively disrupts discrimina­tion reversal conditioning of the rabbit nictitating membrane response. Behavioral Brain Research, 8, 49-68, 1983.

[4J T.W. Berger and R.F. Thompson, Neuronal plasticity in the limbic system dur­ing classical conditioning of the rabbit nictitating membrane response. I. The hip­pocampus. Brain Research, 145, 323-346, 1978.

15J G.A. Clark, D.A. McCormick, D.G. Lavond, and R.F. Thompson. Effects of esions of cerebellar nuclei on conditioned behavioral and hippocampal neuronal

responses. Brain Research, 291, 125-136, 1984.

[61 R. Douglas. Pavlovian conditioning and the brain. In R.A. Boeakes and M.S. Halliday (Eds.). Inhibition and Learning. London: Academic Press, 1972.

[7J M.S. Gazzaniga. The biology of memory. In Neural Mechanism of Learning and Memory, M.Rosenzweig and E. Bennet (Eds.). Cambrige, MA: Mit Press, 1976.

[8J M.S. Gazzaniga. Advances in cognitive neurosciences: The problem of informa­tion storage in the human brain. In Neurobiology of Learning and Memory, G.Lynch, J.L.McGaugh, and N.M.Weinberger (Eds.). New York: Guilford Press, 1984.

[9J E. Grastyan, K. Lissak, 1. Madaraz, and H. Donhoffer. Hippocampal electrical activity during the development of conditioned reflexes. Electroencephalography and Clinical Neurophysiology, 11,409-430,1959.

[10J P.C. Holland and R.A. Rescorla. Second-order conditioning with food un­conditioned stimulus. Journal of Comparative and Physiological Psychology, 88, 459-467, 1975.

[l1J H. Kaye and J.M. Pearce. The strength of the orienting response during Pavlovian conditioning. Journal of Experimental Psychology: Animal Behavior Processes, 10, 90-109, 1984.

[12J R.P. Kesner. An attribute analysis of memory: The role of hippocampus. Physiological Psychology, 8,177-203,1980.

[13] C.M. Leyland. Higher-order autoshaping. Quarterly Journal of Experimental Psychology, 29, 607-619, 1977.

[14J B.L. McNaughton, C.A. Barnes, G. Rao, J. Baldwin, and M. Rassmussen. Long-term enhancement of hippocampal synaptic transmission and the acquisition of spatial information. The Journal of Neuroscience, 6, 563-571, 1986.

[15] D.J. Micco and M. Schwartz. Effects of hippocampal lesions upon the devel­opment of Pavlovian internal inhibition in rats. Journal of Comparative & Physio­logical Psychology, 16, 371-377, 1971.

[16] M. Mishkin and H.L. Petri. Memories and habits: Some implications for the analysis of learning and retention. In L.R. Squire and N. Butters (Eds.), Neu­ropsychology of Memory. New York: Guilford Press, 1984.

[17] J.W. Moore. Brain processes and conditioning. In A. Dickinson and R.A. 130akes (Eds.), Mechanisms of Learning and Behavior. Hillsdale: Lawrence Erlbaum, 1979.

[18] W.B. Orr and T.W. Berger. Hippocampectomy disrupts the topography of conditioned nictitating membrane responses during reversal learning. Journal of Comparative and Physiological Psychology, 99, 35-45, 1985.

[19] J.M. Pearce and G. Hall. A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 81,532-552, 1980.

71

[201 R.L. Port and M.M. Patterson. Fimbriallesions and sensory preconditioning. Bebavioral Neuroscience, 98, 584-589, 1984.

[21] D.A. Powell and S. Buchanan. Autonomic-somatic relationships in the rabbit (Oryctolagus cuniculus): Effects of hippocampalleions. Physiological Psychology, 8, 455-462, 1980.

[22] M.E. Rashotte, R.W. Griffin, and C.L. Sisko Second-order conditioning of the pigeon's keypeck. Animal Learning and Behavior, 5, 25-38, 1977.

[23) R.A. Rescorla. Simultaneous and succesive associations in sensory precon­ditioning. Journal of Experimental Psychology: Animal Behavior Processes, 6, 207-216, 1980.

[24] R.A. Rescorla and A.R. Wagner. A theory of Pavlovian conditioning: Varia­tions in the effectiveness of reinforcement and non-reinforcement. In A. H. Black and W.F. Prokasy (Eds.), Classical Conditioning n: Theory and Research. New York: Appleton-Century-Crofts.

[25] E.J. Rickert, J.F. Lorden, R. Dawson, E. Smyly, and M.F. Callahan. Stimulus processing and stimulus selection in rats with hippocampal lesions. Behavioral & Neural Biology, 27, 454-465, 1979.

[26] R.C. Rizley and R.A. Rescorla. Associations in second-order conditioning and sensory preconditioning, Journal of Comparative and Physiological Psychology, 81, 1-11,1972.

[27] N .A. Schmajuk. A model for the effects of the hippocampal lesions on Pavlovian conditioning. Abstracts of the 14th Annual Meeting of the Society for Neuroscience, 10, 124, 1984.

[28] N .A. Schmajuk. Real-time attentional models for classical conditioning and the hippocampus. Unpublished Doctoral Dissertation, University of Massachusetts, 1986.

[29] N .A. Schmajuk and J. W. Moore. Real-time attentional models for classical conditioning and the hippocampus. Physiological Psychology, 13, 278-290, 1985.

[30] N.A. Schmajuk and J.W. Moore. The hippocampus and the classically con­ditioned nictitating membrane response: A real-time attentional-associative model. In press, Psychobiology, 1988.

[31) L.W. Schmaltz and J. Theios. Acquisition and extinction of a classically con­ditioned response in hippocampectomized rabbits (Oryctolagus cuniculus). Journal of Comparative and Physiological Psychology, '19, 328-333, 1972.

[32] E.N. Sokolov. Neuronal models and the orienting reflex. In M.A.B. Brazier (ed.), The Central Nervous System and Behavior, New York: Macy Founda­tion, 1960.

[33J P.R. Solomon. Role of the hippocampus in blocking and conditioned inhibition of rabbit's nictitating membrane response. Journal of Comparative and Physiolog­ical Psychology, 91,407-417, 1977.

[34J P.R. Solomon and J.W. Moore. Latent inhibition and stimulus generalization of the classically conditioned nictitating membrane response in rabbits (Orytolagus cuniCUlus) following dorsal hippocampal ablation. Journal of Comparative and Physiological Psychology, 89, 1192-1203, 1975.

[35J R.F. Thompson and R.F. Kramer. Role of association cortex in sensory pre­conditioning. Journal of Comparative and Physiological Psychology, 60, 186-191, 1005.

[36J L. Squire. The neuropsychology of human memory. Annual Review of Neuro­science, 5, 241-273, 1982.

72

[37J E.C. Tolman. Cognitive maps in rats and men. Psychological Review, 55, 189-208, 1932.

[38J C.H. Vanderwolf, R. Kramis, L.A. Gillespie, and B.H. Bland. Hippocampal rhythmic slow activity and neocortical low- voltage fast activity: Relations to be­havior. In R.L. Isaacson and K.H. Pribram (Eds.), The Hippocampus. New York: Plenum Press, 1975.

[39] 0.8. Vinogradova. Functional organization of the limbic system in the process of registration of information: Facts and hypotheses. In R.L. Isaacson and K.H. Pribram (Eds.), The hippocampus. New York: Plenum Press, 1975.

[401 A.R. Wagner. Habituation and memory. In A. Dickinson and R.A. Boakes {Eds.), Mechanisms of learning and Motivation. Hillsdale, N.J.: Erlbaum, 1979.

[41] W.A. Wickelgren. Chunking and consolidation: A theoretical synthesis of se­mantic networks, configuring in conditioning, 8-R versus cognitive learnig, normal forgetting, the amnestic syndrome, and the hippocampal arousal system. Psycho­logical Review, 86, 44-60, 1979.

A Memory with Cognitive Ability

Shigeru Shinomoto

Department of Physics

Kyoto University. Japan

The cognition is based on classification processes. In every stage of computation

or information processing, a sub-unit in each site of the brain will also classify whether

or not the input signal to that unit is identifiable in its part.

Some possible neuronic connections in the real neural networks for creating as­

sociative ability are proposed. First, it is found that a simple modification of the

auto-correlation matrix memory based on a physiological constraint endows the sys­

tem with the above-mentioned cognitive ability. The constraint here is that the

attribute of a given synapse (excitatory or inhibitory) is uniquely determined by the

neuron to which it belongs. Thus the synaptic connection is generally not symmetric.

Even in the presence of this constraint, the system is able to retrieve its memory if

an input pattern is close to the pattern of one of the stored memories. If the input

pattern is too remote from all memory patterns, the system gets into a mode in which

almost all neurons are in the same state in each time. This distinctive mode is a clear

response of the processing unit by making explicit that the input is unidentifiable.

Secondly, a more realistic situation is examined assuming the hippocampal forma­

tion. In this site, excitatory neurons are supposed to work as direct transition units

from input to output. Inhibitory neurons surve as feed-back inhibition to the excita­

tory neurons. With the assumption that the inhibition works as effective inhibitory

couplings between excitatory neurons, the system becomes a variant of the above-

74

mentioned cognitive memory; In this case, the connection recovers the symmetry

while the system maintains the cognitive ability such as seen in the above-mentioned

model.

Rules of synaptic modification for the acquisition and maintenance of environ­

mental information are also of current interest. Some possible rules for the plasticity

of the networks are briefly discussed.

1. The auto-correlation matrix memory

The instantaneous state of a model network composed of N neurons is represented

by a vector s = (sl ... s N)' Each neuron is assumed to be a binary element, whose

symmetric representation is s. = +1 (firing) and -1 (resting). The firing neuron )

sends its signal to the others via its synapses, and post-synaptic potential of the

i-th neuron caused by the j-th neuron is K .. x (s. + 1), where K .. (or 2K .. ) is the ')) I) ')

synaptic coupling from j to i. A neuron fires if the sum of the post-synaptic potential

U. = L:. K. (s. + 1) exceeds its own threshold H.: , ) I) ) ,

(1)

where

v. = U. - H. = '" K.s. - (H. - '" K.) , , , L...J ')) 1 L...J I) (2)

j j

is the reduced input signal. Any representational scheme of the binary element can

be translated into the above scheme by adjusting threshold values. We shall come

back later to the discussion of the role of the reduced threshold in the present unit:

Li == Hi - LKij , j

(3)

but for the moment we consider the case that the reduced thresholds are negligible.

We shall first adopt the synchronous processing algorithm, where the rule (1) is

applied simultaneously to all neurons in each time step. Thus the state s(t) at a time

step t is transformed to s( t + 1) in a deterministic manner.

75

A standard scheme for embedding given patterns in the network is the auto­

correlation matrix memory. The network is designed so that the given patterns may

be stable and the rest unstable. Under some assumptions, a fuzzy input pattern

is expected to relax toward a neighbouring stable pattern chosen from the memory

patterns (Amari 1977; Hopfield 1982; Shinomoto 1987a; Meir and Domany 1987;

Amari and Maginu 1988). The synaptic coupling of the auto-correlation matrix

memory is chosen as

_l,,",mm K;j = T;j = N L.Js; Sj ,

m

(4)

where 8m = (s;n··· sN) is the firing pattern of the m-th memory (m = 1··· M). We

shall use the notation T.. for this standard connection. 'J

The input signal to the i-th neuron obtained by the application of one of the

memory patterns 8 = 8~ is estimated as

v. = ~ ""' s"!' ""' s"!'s~ • NL.J·L.JJJ m j (5)

rv sf ± O(JM/N),

where we have assumed mutual independence of the memory patterns, and the second

term in the last line represents a natural statistical deviation. Thus, in the case that

the numbers of the memories M is sufficiently small compared to the number of the

elements N, all memory patterns are expected to be stable. Under some conditions,

it was also proved that each memory pattern is a global attractor. The iteration in

eq.(l) then corresponds to the retrieval process which may be completed in a few

time steps. A number of interesting features of such a model which is usually called

the Hopfield model has been studied intensively by many researchers (Kohonen 1972,

1984; Kohonen et al.1976; Nakano 1972; Anderson 1972; Cooper 1973; Amari 1977;

Little and Shaw 1978; Hopfield 1982; Amit et al.1985, 1987).

76

2. Cognitive memory

In the above-mentioned standard model of associative memory, neuronic function

is generally heterogeneous. Namely, the same neuron may have different kinds of

synaptic endings, excitatory (T .. > 0) and inhibitory (T/. < 0). In the mammalian Y J

central nervous systems, however, neuronic function is supposed to be homogeneous,

i.e., the attributes of synaptic endings are uniquely determined by the neuron at the

origin (see for instance Eccles 1977; Kumer et al.1984).

We shall put (j = +1 if the j-th neuron is excitatory (Kij 2: 0 for any i) and -1

if the neuron is inhibitory (Kij ::; 0 for any i). The synaptic couplings of our model

are chosen as

K., = 2T.,8((.T.,), 'J 'J J 'J

(6)

where T. is the conventional auto-correlation matrix (4) and 8(x) is the Heaviside 'J

step function (= 1 if x 2: 0, and = 0 otherwise). Couplings {'f;j} which are not obe-

dient to the attribute of the j-th neuron are absent. The numerical factor 2 in eq.(6)

is introduced simply to normalize the input signal and is irrelevant to the present

deterministic rule (1). We shall assume that various memory patterns are chosen in

the way that each neuron is active (+1) or inactive (-1) with equal probability, and

the distribution of {(j} is chosen to be independent of {sm} or {'f;j}' The parameters

characterizing our system are the number of neurons N, the number of memories M

and the difference between the numbers of excitatory and inhibitory neurons, N E

and N] , or

(7)

We shall show some implications of the existence of two kinds of global attractors:

the first corresponding to the retrieval of memories and the second to the uniform

mode irrelevant to the stored memories.

(i) retrieval mode

77

Let us give an estimate of the input signal to each neuron obtained by the appli­

cation of a memory pattern s = s/J, i.e.,

Vi = ! Lsi L sj"j8«(i L"~s~). mil

In the absence of correlation between (. and T .. , the argument of the step func-J IJ

tion may be positive or negative with equal probability. The effective number of

summands is then reduced to the half of N. Thus the signal is estimated as

(8)

A natural statistical deviation is magnified only by the factor V2 compared with that

of the conventional model. Thus the retrieval of memories seems stilI possible.

(ii) uniform mode

It is known that a neural network becomes bistable or periodic in the presence

of imbalance between the populations of excitatory and inhibitory synapses (Amari

1971; Shinomoto 1986). Our model with the synchronous processing algorithm would

also have stable fixed points or a limit cycle orbit irrelevant to the stored memories.

To see this, let us apply a uniform pattern 1 = (1· . ·1). Then the input signal to the

i-th neuron is

Vi = L2~/1«(iTii) i

'" "T .. [(1 + q)8(T .. ) + (1- q)8(-T .. »). ~ Q Q Q i

Because ofthe equi-probability offiring and resting in memory patterns, {~) are ex­

pected to distribute around zero with variance (aT)2 = M/N2. The above quantity

is estimated as

(9)

Thus if J1/N <:: q < 1 (excitatory-dominant), s = 1 and -1 become fixed points.

On the contrary, if -Jl/N > q > -1 (inhibitory-dominant), the system has a

periodic orbit with period 2 such that 1 and -1 appear alternatively.

78

SYMMETRIC (THE HOPFIELD MODELl EX. & INH. (THE PRESENT MODELl N= 200 M= 15 N= 200 N= 15 C= -0.50

HAMM. 25 HAM. 50 HAMM. 75 HAMM. 100 HAMM. 25 HAMM. 50 HAMM. 75 HAMM. 100

I'~~ II~':I :~~~ .~:. .:.a-!l:' . : 1.'.~-:!oV" :J L!.!....:..-'..-'-!C!.!

~~ .~"Jl .~.m.~. ~ ':;:.":'; I~I~·-- ~";J, Em···· ... ·,· =ar . ""r-' •• ;'-::"7' ....... .,.

I~~I' .'~@" I •. L-:":'::--=:..J

~ I ~p II ~ II 'f!~:~'1 D ... ~.. -,... . ~

1_11_;.1'. Fig.1 Retrieval processes in the Hopfield model (left) and the present model (right).

Indistinctive patterns are going to be trapped by spurious memories in the former model,

while they are getting into a uniform mode in our model. The number following to 'HAMM.

, is the Hamming distance between the input and a memory pattern which is designed for a

heart shape.

The coexistence of global attractors (i) and (ii) should be investigated in further

detail. In the previous paper (Shinomoto 1987a), the stability of the above fixed

points and the periodic orbit was investigated analytically and numerically. Here,

we shall simply demonstrate that such attractors can actually coexist. So~e of the

simulation results of the Hopfield model and the present model are compared in

Fig.1. We prepared a specific memory pattern which is designed to be heart-shaped,

and other memory patterns chosen independently. We have shown how the pattern

changes from an initial pattern which is given randomly with the Hamming distance

to the memory pattern fixed. Both models can retrieve memory in response to a

fuzzy input, provided the initial pattern is close to the pattern of the memories. A

difference between the Hopfield model and the present model arises when the input

pattern largely deviates from any ofthe memories. In the Hopfield model, the pattern

is eventually trapped by a spurious memory or otherwise some uncorrelated memory

is forced to be taken out. On the other hand, the present model then assumes a

uniform mode. The latter can be interpreted as a system's clear statement that the

input pattern'is not identifiable.

79

3. Possible connections in the brain

Physiological constraint on the neuronic connections in the brain is more restric­

tive than the one introduced preliminarily in the previous section. In most sites of

the brain, respective parts of its function are assigned to neurons of the same kind.

Let us choose the hippocampal formation as an example. In the hippocampus, the

pyramidal cells which are excitatory work as direct transition units from input to

output. The basket cells serve as feed-back inhibition to the pyramidal cells (see

Fig.2a). The relationship between the granule cells and the basket cells in the den­

tate gyrus appears similar to the above. The hippocampal formation is supposed to

be an important site to the entrance of long-term memory to the neo-cortex (Eccles

1977; Mahut et al.1982: Yamamoto 1986). Note, however, that the hippocampal

formation is not the site where the long-term memory is stored. There might be al­

ternative assumptions on the function of the site. Namely, it might be either a filter

which simply translates input signals, or an intermediate-term memory (Eccles 1977;

Olton et al.1979) which supports the information until the contents are stored in the

neo-cortical regions. We take the latter hypothesis and point out the possibility of

the site being a kind of associative memory.

In order to maintain firing pattern, an extra feed-back excitation loop between

excitatory neurons would be required (see Fig.2b). In principle, a simple circuit with

feed-back inhibition such as shown in Fig.2a is sufficient to maintain firing patterns,

provided a specific structure in the inhibitory couplings is assumed and the threshold

values of excitatory neurons are adjusted to be sufficiently low. The population of

inhibitory neurons is considerably smaller than the population of excitatory neurons,

and the inhibitory neurons are not considered to be plastic. Thus it would be natural

to suppose that this feed- back inhibition is roughly uniform and the specific structure

relevant to the maintenance of firing patterns is attributed to the feed-back excitation.

The presence of the feed-back excitation between excitatory neurons is implied by

anatomical data, and this is called the recurrent facilitation by physiologists (Rolls,

private comunication; Miyashita, private communication).

The intra-couplings between excitatory neurons are denoted by E;j(?' 0). The

80

( b)

Fig.2 Schematic representation of possible neuronic connection in the hippocampal for-

mation, (a): the system with feed-back inhibition alone, (b): the system with feed-back

inhibition and excitation. Here, 'e ' represents an excitatory cell such as the pyramidal cell

or the granule cell, and 'i ' an inhibitory cell such as the basket cell. Small open and closed

circles represent the excitatory and inhibitory synaptic couplings, respectively.

effective inhibitory action between excitatory neurons via the inhibitory neurons is

assumed to be linearly dependent on the excitation. This could be translated as

effective inhibitory couplings between excitatory neurons, Ji/~ 0). We shall not go

into the problem of the asynchrony and the time-delay in the arrival of the signal,

and are mainly concerned with the stationary states of the system. The input signal

to the i-th excitatory neuron (i = 1· .. N) is thus written as

v. = ""' E .. (5. + 1) + ""' 1 .. (5. + 1) - H. , L...t I) ) L...t '1 ) 1

j j

= ""'(E .. + J .. )5. - L., L...t 'J 'J J ,

(10)

j

where

L. = H. - ""'(E .. + I .. ) • ,L...t'J .j (11)

j

IS the reduced threshold which can be controlled independently by adjusting the

81

original threshold value Hi' The inhibitory couplings will be supposed uniform, or

Iij = -I/N« 0). Let the excitatory couplings {Eij} be the same as the previously

studied couplings in the cognitive memory {K..} (eq.(6», with a constraint that all '3

neurons are excitatory, or q = 1 (NJ = 0 and NE = N). By making use of the results

in the previous section, we can estimate the input signal in a few extreme cases.

(i) retrieval mode

Input signal to the i-th neuron by the application of the I'-th memory pattern is

estimated as

v. '" s~ - 1. ± O(.j2M/N), , I , (12)

where we have used the relation (8). We have also assumed that each firing pattern

is chosen in the way that each neuron is active or inactive with equal probability.

Retrieval of each memory appears possible if I Li I~ 1.

(ii) Uniform mode

The input signal to each neuron by the application of the uniform pattern 1 = (1·· ·1) or -1 = (-1··· - 1) is estimated by making use of the relation (9). Then

we get

(13)

which is the result obtained by the input 1. The sign of the two terms in the bracket

is reversed if -1 is applied. Thus in the present model, we have another parameter

I in place of q in eq.(9).

Most results obtained in the previous section would also hold in the present

model. The present model differs from the previous one in the asynchrony of the

processing procedure. In the asynchronous processing, the presence of the limit cycle

orbit in the inhibitory dominant case, i.e., ..j2M < I, involves a subtle problem yet

to be clarified.

It should be noted that in the present model the excitatory connection is sym-

82

metric, i.e.', Eij = E jj . This is because q = 1 or

E .. = 2T. .O(T. .), 'J 'J 'J

(14)

where the original auto-correlation matrix T .. is symmetric. Since we have assumed 'J

the uniformity in the effective inhibitory couplings, i.e., I .. = -1/ N, the present 'J

connection recovers the symmetry, or E .. + I .. = E .. + I ... Thus, if we can neglect 'J 'J J' J'

the self-coupling, Eji + Iii' and assume the complete asynchrony of the processing

algorithm such that only one neuron is processed iteratively in each time step, the

system is then equivalent to a physical spin system.

4. Learning procedures

There are several proposals on learning procedures to get the associative ability

such as seen in the ad-hoc choices of the synaptic connection.

The conventional one is the Hebb rule:

~Kij ex: (sY' + l)(sj + 1), (15)

where ~K .. is the variation of the coupling strength and sJ"!' is the neuronic state 'J

(+1 or -1) in the m-th pattern to be acquired. This is a simple reinforcement rule

for the excitatory neurons. The rule produces the connection similar to (14), but the

efficiency of the resultant connection for the memory storage is not so good as for

(14).

The elementary rule to construct couplings similar to the Hopfield model is the

generalized Hebb rule:

(16)

The complete auto-correlation matrix (4) is obtained if the acquisition begins from

tabula rasa, {K .. } = 0 and it ceases at a reasonable stage 1 ~ m < M ~ N. 'J

83

The above-mentioned rules of synaptic modification, however, lead explosion of

synaptic strength and at the same time catastrophic deteriolation of memories (Hop­

field 1982; Amit et al.1985, 1987). In order to avoid the above-mentioned troubles

in memory process, synaptic decay was introduced in addition to the simple learning

(Nadal et al.1988; Parisi 1986; Mezard et al.1986). A representative form is

D..K.. ex -'YK.. + sr'sJ1!I. 'J 'J

(17)

The system has a kind of steady state in the continual presentation of patterns and

acquires a stationary capacity for I > IC' where IC is some constant determined by

the number of neurons. This model is discussed in relation to the behaviour of human

short-term memory.

The above-mentioned rule (17) cannot maintain the acquired memories for a long

time. Acquired memory fades in time due to the damping effect. A self-trapping

mechanism in the synaptic modification was proposed (Shinomoto 1987b) to support

the acquired memories. The rule is

D..K.. ex -IK..+ < s.s. >, 'J 'J , J

(18)

where the bracket represents a temporal average in the system characterized by

temporal connection {K .. } . . We have assumed that each neuronic state obeys the 'J

stochastic rule of Little (1974), and the time-scale of synaptic modification is suf-

ficiently large compared to that of neuronic modification. The system is supposed

to be subjected to infrequent renewal in which neuronic states are clamped at a

pattern of environmental information. It was found that the present rule does not

allow for the overloading of memories but stabilizes the synaptic connection similar

to the auto-correlation matrix. The rule (18) which may seem similar to the rule (17)

has a completely different meaning, because the second term in (18) is not a corre­

lation of the presented pattern but the temporal average over all autonomous states

organized by the stochastic rule implemented by the temporal connection itself. The

rule stabilizes the Hopfield coupling if 'Y ~ 1. Note that the inequality for the proper

storage is opposite to the one for the rule (17).

84

One can also define the learning procedure as the optimization problem of how

to maximize the efficiency of memory process. There are several interesting ap­

proaches in line with this assumption (Hinton and Sejnowski 1986j Sejnowski et

al.1986j Rumelhart et al.1986j Krauth and Mezard 1987j Hopfield 1987). In those

approaches, however, the maintenance of acquired memories as we have seen above

is not discussed.

It would be interesting to take the physiological constraint into account in the

learning and maintenance algorithms. Such a problem will be left to future studies.

5. Conclusion

We have shown some possible choices of synaptic connection in which the net­

work has an associative ability even in the presence of some typical physiological

constraints. It was found that the trouble caused by the physiological constraint is

not very serious. On the contrary, the physiological constraint naturally provides the

system with the ability in cognition, i.e., the ability to identify an input signal by its

proximity to any of stored memories.

We have also discussed a possible learning algorithm enabling acquision and main­

tainance of the environmental information. To the author's knowledge, the consis­

tency between the physiological constraint and learning algorithm is not resolved

in the studies of the memory with distributed representation. This point will be

discussed elsewhere.

ACKNOWLEDGEMENTS

The author would like to express his gratitude to Yoshiki Kuramoto for his con­

tinual encouragement and advices, and to Shun-ichi Amari for his informative com­

ments.

The idea of cognition in section 2 was refined by consulting the review article by

Daniel Amit (1988), where a different point of view on the cognition or "generation of

meaning" is presented with the comparison to the present model (Shinomoto 1987a)

and the model proposed by Parisi (1986).

85

The author's interest in the real neuronic connections in the hippocampal for­

mation in section 3 was aroused by Edmund Rolls. The author is also grateful to

Yasushi Miyashita for his advice in the physiological knowledge. They are, however,

not responsible for possible misunderstanding contained in section 3.

The author would like to thank Terrence Sejnowski, who increased the author's

interest in the learning procedures such as discussed in section 4.

REFERENCES

Amari S (1971) Proc. IEEE 59, 35

-- (1977) BioI. Cybern, 26, 115

Amari S, Maginu K (1988) Neural Networks (to be published)

Amit DJ (1988) in "The Physics of Structure Formation" (to be published)

Amit DJ, Gutfreund H. Sompolinsky H (1985) Phys. Rev. Lett. 55, 1530

-- (1981) Annals Phys. 173, 30

Anderson JA (1912) Math. Biosci. 14, 191

Cooper LN (1913) in "Proceeding of Nobel Symposium on Collective Properties of

Physical Systems" , eds. B. Lundquist and S. Lundquist (Academic Press, NY)

Eccles JC (1911) "The Understanding of the Brain" , 2nd ed. (McGraw-Hill, NY)

Hinton GE and Sejnowski TJ (1986) in "Parallel Distributed Processing " ,

eds. J.L. McClelland, D.E. Rumelhart, and PDP Research Group (MIT Press,

Cambridge)

Hopfield JJ (1982) Proc. Natl. Acad. Sci. USA, 79, 2554

-- (1981) Proc. NatI. Acad. Sci. USA, 84, 8429

Kohonen T (1912) IEEE Trans. C-21, 353

--- (1984) "Self-Organization and Associative Memory" (Springer-Verlag,

Berlin)

Kohonen T, Reuhkala E, Makisara K, Vainio L (1916), BioI. Cybern. 22, 159

86

Krauth W, Mezard M (1986) J. Phys. A20, L745

Kufller SW, Nicholls JG, Martin AR (1984) "From Neuron to Brain" 2nd ed.

(Sinauer, Massachusetts)

Little WA (1974) Math. Biosci. 19, 101

Little WA, Shaw GL (1974) Math. Biosci. 39,281

Mahut H, Zola-Morgan S, Moss M (1982) J. Neurosci. 2, 1213

Meir E, Domany E (1987) Phys. Rev. Lett. 59, 359

Mezard M, Nadal JP, Toulouse G (1986) J. Physique 47, 1457

Nadal JP, Toulouse G, Changeux JP, Dehaene S (1986), Europhys. Lett. 1, 535

Nakano K (1972) IEEE Trans. SMC-2, 380

Olton DS (1979) Behav. Brain. Sci. 2, 313

Parisi G (1986) J. Phys. A19, L675

Rumelhart DE, Hinton GE, Williams RJ (1986) in "Parallel Distributed

Processing" ,eds. J.L. McClelland, D.E. Rumelhart, and PDP Research

Group (MIT Press, Cambridge)

Sejnowski TJ, Kienker PK, HintonGE (1986) Physica 22D, 260

S.hinomoto S (1986) Prog. Theor. Phys. 75, 1313

-- (1987a) BioI. Cybern. 57, 197

-- (1987b) J. Phys. A20, L1305

Yamamoto C (1986) in "Physiology" (in Japanese) eds. M. Iriki and K. Toyama

(Bunkoudo, Tokyo)

Abstract

Feature Handling in Learning Algorithms

S. E. Hampson and D. J. Vol per Information and Computer Science Department

University of California, Irvine

TL U (Threshold Logic Unit) representation and training provides a simplified formal model of neuron-like computation. Based on this abstract model, various formal properties and acceleration techniques are considered, the results of which appear relevant to observed biological phenomina.

1 Introduction

One of the more striking characteristics of connectionistic learning models (e.g., Hinton et al., 1984; Ackley et al., 1985; Barto, 1985; Rumelhart et al., 1986), is their apparent slowness. Hundreds or sometimes thousands of presentations may be required to learn a simple 2-feature (4 input pattern) Boolean function. Given suffi­cient time, interesting network structures can be learned, but aside from occasional empirical comparison on selected functions, little is known about the time/space characteristics of such algorithms. This makes it difficult to realistically evaluate and compare the efficiency of alternative learning and representation schemes.

In this paper we provide empirical evidence that connectionistic learning can be significantly accelerated. We also develop a line of analysis that provides some time/space bounds on individu!il nodes. This analysis centers on thresholded linear equations as the basic representation structure and the percept ron training algorithm as a method of adaptive training. 'i'hresholded linear equations are a standard (sim­plified) model of neural computation, and many issues which can be formalized at this level appear to be relevant when considering more realistic models. Although perceptron training is not intended as a model of any particular biological learning process, it is sufficiently similar to classical conditioning to be of interest, and simple enough to permit some analysis. We find that many formal techniques for improving learning speed have natural analogs in biological systems. Further discussion and de­tails can be found in (Hampson and Kibler, 1983; Hampson and Volper, 1986, 1987; Volper and Hampson, 1986, 1987).

2 Node structure

The basic unit of computation (node/neuron model) is a Threshold Logic Unit (TLU). The standard TLU is a thresholded linear equation that is used for the binary categorization of feature patterns. Although it is possible to represent and train equations with more complex computational capabilities, only linear equations are considered here.

In the classical TLU model, the input patterns are represented as vectors of binary values, the absence of a feature being represented by 0 and its presence by l. One additional component is necessary to represent the constant or threshold term in the linear equation. Thus a feature vector with d features can be represented by Y = (Yo, Yl, ... , Yd) where Yi is the value of the i-th feature, and Yo, the additional component associated with the constant term of the linear equation, is always equal to 1. Similarly, the coefficients of the linear equation are represented as a weight vector W = (WO,Wl, ... ,Wd).

88

A feature vector is classified by forming its linear product with wand comparing with 0. Specifically, if

d

yw = LYiWi i=O

is greater than 0, the fea.ture vector is classified as a positive instance, and if less than 0, it is classified as a negative instance. Geometrically, a weight vector describes a hyperplane that partitions the input space (a d-dimensional hypercube) into positive and negative regions. More generally, the linear equation can be thought of as mea­suring similarity to a prototypic point on the surface of ad-dimensional hypersphere that circumscribes the hypercube. The threshold simply divides the similarity mea­sure into "true" and "false" regions. If the features are weighted equally, a TLU is specialized to the (at least z of m features) function. If m = d (that is, if all features are relevant), the prototypic points are restricted to the vertices of the hypercube. This function can be further specialized to (1 of m) = OR and (m of m) = AND (Muroga, 1971).

Thresholded summa.tion is the most common model of single neuron function. Its biological justification is that the decision to fire a neuron is made (ideally) in one place only, the base of the axon, where all dendritic inputs are summed. The actual process is more complex (Shepherd, 1983 ch. 8; KufRer et al., 1984; Kandel and Schwartz, 1985) and only partly understood, but at a minimum is presumably capable of (something like) simple summation.

One desirable characteristic of summation is that it can be implemented as a one-step, parallel operation. In particular, multiple current sources can add their contribution to a final summed current, independent of other current sources. Thus it is possible to integrate the contributions of any number of features in a constant amount of time. This may be an important property for neurons like the cerebellar Purkinje cell which can possess hundreds of thousands of inputs.

Many neuron models have a single threshold value which the summed inputs must exceed in order to fire the node at all, and above which it fires at full intensity. Besides any computational advantages this strategy may have, it is a reasonable approxima­tion of neural behavior. Standard neurons do respond in an all-or-nothing fashion. However, this does not mean that magnitude information cannot be conveyed. Input is also temporally summed, so that a strong input produces a higher frequency of firing than a weaker input. Firing frequency is commonly interpreted as a magnitude measure (Adrain, 1946; Barlow, 1972). This style of expressing magnitude informa­tion may have some theoretical advantages, but it may also simply reflect biological limitations inherent to neurons. There does not appear to be any immediate benefit in transmitting magnitude by frequency modulation, so in many models output mag­nitude is expressed directly in the amplitude, which is considerably easier to model. This continuous output can be bounded between lower and upper bounds (e.g., 0 .. 1) by thresholding, or by asymptotically approaching them, as in a sigmoid. Under some circumstances, the magnitude of neural response seems to vary with the logarithm (or as a power function) of input intensity (Shepherd, 1983 p. 198).

The limitations of a linear function may parallel biological limitations. Of the 16 possible Boolean functions of 2 features, only Exclusive Or and Equivalence cannot be expressed as a linear function, and those have been reported to be the most difficult to learn (Neisser and Weene, 1962; Hunt et al., 1966; Bourne, 1970; Bourne et al., 1979 ch. 7). On the other hand, other experiments have not found any consistent difference in difficulty in learning linearly and nonlinearly separable categories (Medin

89

and Schwanen:llugel, 1981; Medin, 1983; Wattenmaker et al., 1986), so that particular point is questionable. Of course any nonlinear Boolean function can be represented in disjunctive form as a two-level system using only the functions OR and AND. The relative contribution of one and multi-level learning would presuma.bly depend on the particular function and training circumstances.

Linear equations in general, and the (z of m) function in particular can be viewed as measuring prototypic similarity. They give their strongest response to some cen­tral prototype and decreasing output for decreasing similarity to it. The (z of m) function assumes that all features are of equal importance while a linear equation allows varying weight to be attached to the different features. With this representa­tion, a large weight means a feature is highly predictive of the category, not that it is (necessarily) frequently present when the category is. There are numerous formal models of similarity measurement, but some form of prototypic similarity, or family resemblance detection, is generally considered a useful and demonstrated capability for biological organisms (e.g., Mervis and Rosch, 1981; Smith and Medin, 1981)

3 Node training

There are many organizing processes that shape the nervous system. During early development, cell division, migration and differentiation take place, and specific pat­terns of interconnection develop. Later, selective cell death and synaptic degeneration occur, producing a highly structured system before it is exposed to the environment. Some aspects of neural growth are present throughout the life span of most organ­isms, and at the synaptic level, learning-related structural changes are often observed. Metabolic shifts may also contribute to adaptive plasticity. In addition, and perhaps most importantly, a considerable amount of neural, and therefore behavioral plasticity is possible due to the existence of variable strength synapses.

At a formal level, the ability to convergently train a TLU as a pattern classifier is well known as the perceptron convergence theorem or TL U training procedure (Nilsson, 1965; Minsky and Papert, 1972; Duda and Hart, 1973). It is appropriate with either binary or multi valued input features. In its simplest form it can be expressed as:

Wnew = Wold + y, if yw should have been positive W new = Wold - y, if yw should have been negative

where W is the weight vector and y is the input vector to be classified. One proof of the perceptron convergence theorem (Nilsson 1965, p. 82) provides

an upper bound on the number of adjustments needed for training a TLU:

Here M is the squared length of the longest input vector, W is any solution vector, and a is the minimum value of Iywi over all input vectors. Empirical results have generally re:llected the time complexity results based on this upper bound.

A similar algorithm has been proposed as an abstract model of learning during classical conditioning (Rescorla and Wagner, 1972), a process which appears to oc­cur in very simple organisms (Sahley et al., 1981). As a model of neural learning, perceptron training can be interpreted as: 1) If the neuron's output is too low, increase the synaptic weights ofthe active input

features. 2) If the neuron's output is too high, decrease the synaptic weights of the active

input features.

90

A similar learning process also occurs in the gill withdrawal reflex of Aplysia, a model system in which neural mechanisms have been extensively studied (Kandel, 1976, 1979; Hawkins and Kandel, 1984; Abrams, 1985). In gill withdrawal, a pain sensor can be thought of as "instructing" a motor neuron which receives input from a poke sensor. If pain follows being poked, the gill should have been withdrawn and its input synapses are (presynaptically) strengthened by associative sensitization. If no pain occurs, the gill should not have been withdrawn and its input synapses are weakened by habituation. The most significant difference between perceptron train­ing and Aplysia learning is that perceptron training adjusts only on mistakes, while gill withdrawal (as described) adjusts on all inputs. However, the resulting behavioral phenomena of "blocking" (a correct response to one feature blocks conditioning of a new feature paired with it) (Kamin, 1969; Mackintosh, 1978), has been demonstrated in other molluscs (Sahley et al., 1981; Sahley, 1984; Sahley et al., 1984; Gelperin et al., 1985), so it seems reasonably safe to assume that the appropriate mechanisms for perceptron training (and the corresponding characteristics of classical conditioning), do exist in simple organisms. Possible neural implementations have been suggested (Hawkins and Kandel, 1984; Gelperin et al., 1985; Tesauro, 1986; Gluck and Thomp­son, 1987), but a complete mechanism is still not known.

Although associative learning is thought to be a presynaptic process in Aplysia gill withdrawal, what appears to be (at least partly) postsynaptic associative learning has been demonstrated in mammalian neurons (McNaughton, 1983; Barrionuevo and Brown, 1983; Abraham and Goddard, 1984; Levy, 1985), suggesting the possibility of alternative neural mechanisms for certain functional properties. In the context of this relatively abstract model, any mechanism which can train a linear equation from input/output pairs is logically sufficient. Whether this is the functional "purpose" of classical conditioning is, of course, debatable, but it is certainly the intent of the perceptron training algorithm.

4 Input order

When measuring empirical behavior, a sufficient test for convergence is to make a complete cycle through all input vectors without requiring an adjustment. For test­ing convenience, it is therefore useful to use presentation orders in which such cycling occurs. Both the number of cycles and the number of adjustments to convergence provide useful measures of learning speed. Since all available information has been presented by the end of the first cycle, the total number of cycles to convergence is a reasonable measure of "memory efficiency". On the other hand, the perceptron convergence proof provides an upper bound on the number of adjustments. Conse­quently, most formal analysis is in terms of adjustments and not cycles. Since the order of input pattern presentation affects the number of cycles and adjustments to convergence, it is useful to consider a number of different presentation strategies.

The simplest ordering is numeric. With binary features, each input pattern can be viewed as a binary number. In a d-dimensional space there are 2d input vectors, which can be cycled through in the implied numeric order (i.e., 0 to 2d- 1 ). For multivalued features, the nd input vectors can be viewed as base n numbers, where n is the number of values that a feature can assume (for binary n = 2). The inputs can then be cycled through in the implied order.

Randomized cycle ordering randomizes the order of input patterns before each cycle. This would seem the least biased input ordering and a reasonable measure of "average" performance. However, it is of limited value in measuring best or worst case performance.

91

A third ordering technique has proved u.seful in measuring the extremes of single node performance, and provides some insight as to what constitutes "good" and "bad" training instances. Based on the upper bound on adjustments from the percept ron convergence proof, inputs can be ordered so as to maximize or minimize the term lyl2 /lywI2, given some known solution vector w. This leads to nearly worst case and best case performance respectively. Geometrically, this means that learning is slow for adjustments on boundary instances and rapid for adjustments on inputs close to the central prototype (or its negation). This is generally true in biological learning studies (Mervis and Rosch, 1981).

5 Alternative linear models

There are several alternative models for a linear TLU. For binary input they are computationally equivalent, though their training characteristics and their response to continuous input can differ considerably. In the classical model, feature absence and presence are represented as 0 and 1. In the symmetric model, feature absence is represented as -1 rather than O. Classification and training algorithms are the same with both representations. However, adjustment of the symmetric model always adjusts all weights, and allocates a fixed fraction, (1/(d + 1)}, of the adjustment to the threshold. Only features that are present are adjusted in the classical model, and the threshold fraction varies between 1 and 1/(d + 1}.

The two-vector model is the third representation. In this case a node associates two weights with each feature, one for feature presence and one for feature absence. For binary input, there is no need for an explicit threshold weight; it is implicit in the 2d weights of the weight vectors. For classification, present features use the present weights and absent features use the absent weights. Weight adjustment is done in a similar manner. For binary input, the two-vector model is equivalent to the symmetric model with a threshold fraction of 0.5; that is, with half the total weight change associated with the threshold.

At the conceptual level, there are different strategies for adjusting the threshold. At one extreme, the size of the category's positive region is not changed (i.e., the area of the positive region on the hypersphere does not change). The category is modified by shifting the direction of the central prototype to include or exclude the current input, but maintaining the same relative size of the threshold to the length of the vector. Equivalently, the weight vector (without the threshold) can be normalized to a fixed length (e.g., 1.0). Obviously this extreme case is not convergent unless the threshold is already fortuitously correct. However, anything short of a zero threshold fraction will eventually converge. At the other extreme, the prototype is not shifted at all, but the threshold is adjusted to include/exclude the current instance. This also will fail to converge if the central prototype is "misplaced", but, again, anything short of a threshold fraction of 1 will eventually converge. Since the threshold determines the size of the grouping, this provides a continuum of learning strategies with an adjustable "bias" in the amount of generalization. In the first case, approximately as many nodes are shifted out of the group recognized by the TLU as are shifted into it. In the second case, instances are only added or subtracted on a single adjust, but never both. An interesting intermediate point occurs if the threshold fraction is 0.5, as is the case with the two-vector model. Because at least half of the total adjustment is in the "correct" direction, this method of adjustment retains the characteristic that instances are only added or subtracted on a single adjustment while also accomplishing a shift in the central prototype. Empirically, a threshold fraction of about .2 to .3 appears to be optimum.

92

Geometrically, classical and symmetric representation correspond to coordinate systems in which the origin is at a corner, or at the center of the input hypercube, respectively. With binary input, only the corners of the cube are valid inputs, but with multivalued features the cube contains a grid of valid inputs. Besides the obvious choices of locating the origin in the center or at a corner, any number of coordinate systems are possible. These choices have equal representational power provided they are linear transformations of each other, although specific functions may be easier to learn in particular coordinate systems.

For binary features, the two-vector model is representationally equivalent to the classical and symmetric models, but for multivalued features the two-vector model is different. First, it lacks an explicit threshold; thus all solution surfaces must con­tain the origin. Second, the separator is not restricted to a single hyperplane, but may consist of different hyperplanes in each quadrant, providing that they meet at the quadrant boundaries. Consequently, if an explicit threshold (or constant input feature) is provided, the two-vector model is representationally more powerful.

The symmetric and two-vector models treat feature presence and absence in a symmetric fashion, thus avoiding the asymmetry inherent in the classical model. When feature presence and absence are equally informative, the explicit representa­tion of feature absence results in faster learning. Neurons can modulate their output above and below a resting frequency of firing, but the resting level is typically rather low (Crick and Asanuma, 1986), thus limiting the resolution of downward modula­tion. In addition, there is behavioral evidence that missing features are not utilized as readily as present ones (Jenkins and Sainbury, 1970; Hearst, 1978, 1984, 1987; Barsalov and Bower, 1984), so the mathematical option of treating feature presence and absence in a symmetric fashion may be of reduced biological relevance.

6 Output-specific feature associability

When positive and negative input patterns are linearly separable, use of the per­ceptron training procedure guarantees convergence on the correct output. Theoreti­cally this is sufficient, but in practice the process may converge unnecessarily slowly. One reason is that irrelevant input activity cannot be excluded from synaptic modifi­cation. Because of this, a significant improvement in learning speed can be achieved by using conditional probability in the adjustment of synaptic weights.

Conditional probability "traces" can be incrementally calculated as:

[zly.] :=[zly.] + (z - [zly.]) * Y' * rate

[zIY.] := [zIY.] + (z - [zIY.]) * Y. * rate

[z] := [z] + (z - [z]) * rate

where z is the correct output, Y' is the input value for feature(i), [:ely.] is the probabil­ity of:e given Y" [:e] is the probability of:e, and rate is a rate constant determining the "memory length" ofthe traces. For a linear function, if [:ely.] = [:e] (or [:ely.] = [:eIYi]), Yi is an irrelevant feature (with respect to :e) and can be ignored. Most contingency models compare [:ely,] to [:eIYi], though good results have also been reported by com­paring [:ely.] to [:e] (Gibbon, 1981; Jenkins et al., 1981; Miller and Schachtman, 1985). The latter approach has been used in this model. If [:eIYi] > [:e], Yi is predictive of :e's occurrence, and if [:eIYi] < [:e], Yi is predictive of :e's non-occurrence. Related proba­bilistically predictive processes have been utilized in numerous models (e.g., Rescorla and Wagner, 1972; Bindra, 1976; Lolordo, 1979a; Dickinson, 1980; Mackintosh, 1983 ch. 7; Schwartz, 1984).

93

In order to use this probabilistic information, when weights are adjusted, they are changed in proportion to each feature's predictive potential. Various formal mea­sures of contingency are possible (Gibbon et al., 1974; Allan, 1980; Hammond and Paynter, 1983; Scott and Platt, 1985), but empirically, a simple difference has proved satisfactory. In the simplest case, weights can be adjusted by the absolute value of ([xIYil- [x]) (which is zero for irrelevant features). Perceptron training time grows linearly with the number of irrelevant features, an effect which has been observed in human learning studies (Bourne and Restle, 1959; Bulgarella and Archer, 1962). This modification reduces the effects of irrelevant features, but has little effect when all features are relevant. More aggressive techniques are also possible which are ef­fective even when all features are relevant. For example, weights can be adjusted by ([xIYil - [x]) only when that term is of the proper sign (positive for increasing and negative for decreasing). Such an approach is no longer provably convergent, but empirically appears to work quite well. More specifically, the average number of adjustments to converge appears to grow as O( 4d ) for d relevant features using unmodified perceptron training, and O(2d) with the use of conditional probability.

The results of this modification are still generally consistent with observed charac­teristics of classical conditioning. In particular, Rescorla and Wagner (1972) showed that associative strength is positively correlated with the contingency between condi­tioned and unconditioned stimuli. Two general classes of theories have been proposed to explain this effect: the "molecular" Rescorla-Wagner learning model in which the contingency effect results indirectly from step by step contiguity during weight ad­justment, and theories in which the organism more directly computes contingencies (Rescorla, 1972). Both models have some support and some weaknesses.

The proposed two-stage model utilizes both approaches. Weights are adjusted by the basic Rescorla-Wagner mechanism, but contingency is also directly computed and used for the adjustment of each feature's salience. In the Rescorla-Wagner model, salience was modeled as a constant multiplying factor specific to each feature which determined the "associability" or plasticity (rate of change) ofthe feature's associative strength. The concept of variable salience helps explain a number of "latent" learning phenomena ("latent inhibition", "latent facilitation" and "learned irrelevance") that are not adequately captured by the Rescorla-Wagner model (e.g., Lolordo, 1979ab; Dickinson, 1980 ch. 4; Mackintosh, 1983 ch. 8; Pearce, 1987 ch. 5).

If the conditional probability traces are adjusted for every input presentation, the result is (vaguely) similar to Mackintosh's (1975) model in which a feature's salience is determined by its relative predictiveness for correct output. If the traces are adjusted only for inputs that result in an incorrect output, the result is more consistent with Pearce and Hall's (1980) suggestion that salience decreases for features which have reached their proper associative strength. The latent learning effects are somewhat different, but learning acceleration is roughly equivalent. There are other computational variations, each of which produces slightly different effects. The important commonality is that a two-stage model permits a significant acceleration of learning by explicitly calculating and utilizing feature salience.

In the proposed model, each feature's output-specific salience is computed and used at the level of the individual neuron. However, circuit-level systems involving the hippocampus have also been proposed to determine feature salience (Douglas, 1967, 1972; Solomon and Moore, 1975; Solomon, 1977, 1979, 1987; Moore and Stick­ney, 1980, 1982; Moore and Solomon, 1984; Schmajuk and Moore 1985; Gabriel et al., 1980, 1982; Nadel et al., 1985; Kaye and Pearce, 1987; Salafia, 1987). Behavioral evidence suggests that latent inhibition is not necessarily output-specific, so circuit-

94

level systems may be appropriate. Some results have cast doubt on the role of the hippocampus though (Garrud et al., 1984). The output-specific salience model de­veloped here does not require circuit-level systems, but a complementary model of input-specific salience does (section 11), and the hippocampus still seems a likely participant in that system.

7 Network results

The use of conditional probability can significantly accelerate single node training. Consequently, it might be expected that networks built of such CP-nodes would learn faster than networks built from standard nodes. This appears to be the case. In particular, the OT algorithm (Hampson and Volper, 1987) is much faster in training a 2-level network than the results published for other algorithms. The OT network is comprised of two levels of two-vector CP-nodes, and is trained to represent arbitrary Boolean functions in disjunctive form.

For example, the 2-feature parity function (Exclusive Or) was learned with 2 first-level nodes in 980 input presentations with back propagation (Rumelhart et al., 1986). The OT algorithm required 15. Barto (1985) reports that his A,.p algorithm learned the 6-feature multiplexer problem in 130,000 presentations using 4 nodes. The OT algorithm converged in 524 presentations using 5 nodes. Finally, the 6-feature symmetry problem was learned in 77,312 presentations using 2 nodes and back propagation (Rumelhart et al, 1986). The OT algorithm converged in 640 presentations using 3 nodes.

These results indicate that there is considerable room for improvement in network training speed. The particular characteristics of the OT algorithm are important, but much of the advantage simply comes from using smarter nodes. For example, by using the same CP-nodes as the OT algorithm, back propagation results are improved to almost match OT results.

8 Learning specific instances

Various tradeoffs between time and space efficiency are possible in connectionis­tic/neural models capable of learning arbitrary Boolean functions. One obvious effect is that while an ability to effectively generalize permits improved space efficiency, in­creased generalization leads to increased learning time when generalization is not appropriate. That is, by definition, a maximal generalization over observed positive instances includes as many unobserved (possibly negative) instances as possible. This is a significant drawback when specific instance learning is known to be appropriate. A second problem associated with learning generalizations is that, even when gen­eralization is appropriate, an incremental learning system that stores only a single generalization hypothesis can make repeated mistakes on the same input patterns, a situation which need not occur with specific instance learning.

The perceptron training process can be viewed as two distinct processes: a ro­tation of the weight vector and an adjustment of the threshold. On misclassified positive instances, the central prototype is rotated toward the current input pattern and the threshold is reduced (i.e., the size of the generalization is increased). For mis­classified negative instances, the prototype is rotated away from the current instance (toward its opposite) and the threshold is increased (the size of the generalization is decreased) .

The perceptron training algorithm has many desirable properties in that it is provably convergent, is a reasonable model of classical conditioning, and appropriate neural components have been worked out to a significant extent. However, while it is relatively good at learning generalizations (instance clusters of size greater than 1), it

95

is correspondingly poor at learning specific instances. In particular, with d relevant features it may require O( d3 ) adjustments to completely distinguish a single positive instance from all other negative instances. Besides over-generalizing and missing on negative instances, it can also repeatedly miss on the single positive instance. Empirically, about 50% of the total adjusts are on the single positive instance. The former is not unreasonable if it is not known beforehand to be an isolated instance, but the latter is a more serious objection.

Because a learning system that is biased toward generalization is inherently biased against learning specific instances, it might be advantageous to provide a specialized learning system with a bias toward specificity rather than generality. Interestingly, only a minor modification to the perceptron training algorithm is necessary. When presented with a positive instance, the weight vector is rotated toward it as before, but the threshold is increased rather than decreased. That is, the size of the generalization is decreased when presented with a positive instance rather than increased. Because it "focuses" the positive region of the TLU on the current input, this learning process is referred to as focusing. As the logical extreme, the weight vector can be rotated to point directly at the current instance and the threshold increased to the length of the vector. Thus a specific instance can be learned in one adjustment if desired, in effect forming an AND "snapshot" of the current input. Although this extreme case of one-shot learning can be advantageous, it runs the risk of being overly specific. By incrementally focusing, the relevant features and appropriate level of specificity can be identified. Since the rate of focusing is adjustable, it can be modified to suit the particular learning circumstances.

Humans sometimes display what is called "now print" (Livingston, 1967ab), "flashbulb" (Brown and Kulik, 1977), or one-shot learning. For example, after only a single presentation of a particular pattern (a picture for example), it can be reliably recognized for weeks. Lower animals demonstrate similar capabilities (Herrnstein, 1985). This is not to say that the memory is necessarily specific on every detail, just that there is no detectable generalization within the relevant domain of applica­tion. Percept ron training does not have this property as it tends to generalize quite aggressively.

Overall, generalization (perceptron training) and specific instance learning (fo­cusing) correspond well to the procedural / declarative distinction (e.g., Cohen and Squire, 1980; Squire, 1982, 1983; Squire et al., 1984; Squire and Cohen, 1984; Cohen, 1984). That is not to say that focusing is by itself necessarily "declarative", but rather that the bulk of declarative learning phenomena appear to have a common denominator in their reliance on rapidly acquired specific instances, while procedural learning is more compatible with slowly acquired generalized category detection.

A neural system displaying the appropriate components of focusing has been de­scribed in the hippocampus (Alger and Teyler, 1976; Dunwiddie and Lynch, 1978; Anderson et al., 1980; Abraham and Goddard, 1983, 1985). In that system, the currently active inputs to a neuron can become more effective in firing it (long-term potentiation), while the inactive inputs become less effective (heterosynaptic depres­sion). This is (apparently) achieved by simultaneously strengthening the synapses of the active inputs (rotate prototype) and (possibly) reducing the excitability of the cell as a whole (raise threshold). Large changes in synaptic strength can occur on a single stimulus presentation. The cell's firing function can thus be modified to respond more selectively to the current input. The actual neural process is con­siderably more complex and is incompletely understood (Bliss and Dolphin, 1984; Abraham and Goddard, 1985; Schwartzkroin and Taube, 1986; Teyler and DiScenna,

96

1987). However, it appears at least potentially capable of the desired characteristics of focusing. Long-term potentiation has been extensively studied only in the hippocam­pus (Seifert, 1983), but has also been reported in other brain structures (Teyler and DiScenna, 1984; Morris and Baker, 1984).

Changes in cellular excitability have been observed but at present there is consid­erably less physiological evidence for adaptive threshold adjustment than there is for the adaptive modification of synaptic strengths. However, as demonstrated by the two-vector model, a general linear equation can be represented without an explicit ad­justable threshold by using two weights per feature. In addition, the explicit threshold of a TLU can be represented as an additional input feature which is constantly on. Thus, any highly frequent input can be viewed as a threshold. A fixed threshold with an adjustable global multiplier (excitability) is also computationally equivalent. This indicates that while it may be conceptually useful to utilize an adjustable intrinsic threshold, there is some flexibility in the actual implementation.

9 Continuously valued features

It is often convenient to treat features as being either present or absent, but stim­ulus intensity is an important aspect of real-world perception. Animals demonstrate their sensitivity to this information in many circumstances including both classical and instrumental conditioning. Although all organisms must deal with stimuli of variable intensity, different strategies exist for representing and processing that infor­mation, each with its particular time/space characteristics.

One advantage of connectionistic representation and processing over more sym­bolic approaches is an ability to directly compute on continuous values between the binary extremes of full on and full off. Depending on the application, these intermedi­ate values can be viewed as an intensity measure or as a degree of certainty. At a more abstract level, the ability to represent intermediate values permits easy representation of attribute-value descriptions (e.g., color{red), size(3.2 inches), shape(square». Some attributes are generally multivalued (shape), while others are appropriately contin­uous (size). An ability to learn arbitrary Boolean functions of multivalued features consequently provides an ability to learn attribute-value concept descriptions.

If continuous output is viewed as a measure of certainty, continuous versions of the three binary TLU models provide distinct capabilities. If input and output values are limited to between 0 and 1 (classical), and interpreted as 0 to 100% probability of feature presence, there is no place on the scale which represents "unknown". Most logic applications simply confound false and unknown.

One possible approach to this problem is to use a second signal to represent confidence in the first. However, since this requires the tight coordination of two signals to represent each feature, it seems unlikely as a general biological principle. In addition, it invites an infinite regress of confidence values, value n + 1 indicating the confidence in value n.

A more tractable two-value approach is to use one signal to represent confidence in a feature's presence and another to represent confidence in its absence (two-vector). No special relationship between the signals is required; they can be treated as inde­pendent features. Unknown would be represented by zero evidence for both presence and absence. Positive values of both would indicate conflicting information, a com­mon state of affairs when dealing with real-world situations. A similar four-valued logic (true, false, unknown, conflicting) has been utilized in AI systems (Belnap, 1977; Doyle, 1979).

An alternative arrangement using only a single value is to establish a baseline

97

floating level of output and express presence or absence as variations above and below that level (symmetric). Input and output can be constrained to be between -1 (certainly not present) and 1 (certainly present). Unknown is represented as the "floating" middle value of o. This three-valued logic has also been employed in AI systems (Shortliife and Buchanan, 1975).

At present there are no completely successful formal logic models of intermedi­ate certainty, so a general description of the appropriate information to represent is impossible. Bayesian probability calculations provide a well founded approach, but the required conditions of mutually exclusive and exhaustive alternatives cannot be guaranteed in most real-world situations. Consequently, a strict application is not generally possible. If theoretical purity cannot be immediately achieved, ease of cal­culation has something to recommend it. Initial attempts to propagate a separate certainty signal were not rewarding. On the other hand, the continuous three or four-valued logic signals are easily propagated through thresholded summation. The resulting output magnitude is a useful value, but cannot be strictly interpreted as a certainty measure.

10 Origin placement

The upper bound on training time for a TL U increases by a multiplicative factor of 0(z4) with z, the distance from the solution hyperplane to the origin. Consequently, appropriate origin placement can significantly influence training time. For example, it is better for a feature to vary between 0 and 1 than between 9 and 10. In the absence of other information (e.g., some knowledge of the function(s) to be learned) a reasonable choice of origin is at the average value of each feature. That is, input feature values can be "centered" by the adjustment

where ave(Yi) is the average value of feature Yi. (Note that the constant threshold fea­ture must be excluded from this adjustment.) For fixed values of ave(Yi), percept ron training is still provably convergent. This can be achieved by averaging over some bounded sample of input patterns and fixing the ave(Yi)s at the resulting values.

Alternatively, the average value of a feature can be incrementally calculated as

where rate determines the memory length of the running average. Using this ap­proach, perceptron training is not provably convergent since a large value of rate can lead to significant wandering of the origin. The problem is that an origin shift can reduce the accuracy of existing weight settings. However, by choosing an appropri­ately small value of rate, the problem of origin wander can be reduced while still achieving a significant improvement in performance. At the extreme (rate = 1) only changed features are salient. Although not necessarily convergent, even this extreme case works reasonably well in practice.

This simple "one context" model can be modified to compensate for origin wan­der, or more generally, permit different origin placement in different contexts. This is possible because an origin shift can be exactly compensated for by an appropri­ate adjustment in the threshold weight. One approach is to provide a distinct con­stant/context feature for each distinct context. This would permit the node to learn its proper threshold setting for each context. This approach is developed further in section 11.

98

Adaptive origin placement is consistent with most behavioral models of habitua­tion (Domjan and Burkhard, 1982, p. 35; Mackintosh, 1983, p. 285; Mazur, 1986, p. 47). In general, these models assume that current sensory stimuli are compared to a memory of recent events, and that only those stimuli which differ from the expected value are available for further processing. More elaborate models are possible (and necessary), in which the "expected" value of a feature is the result of an arbitrarily complex, context-specific world model (e.g., Sokolov, 1960, 1963; Lynn, 1966; Lara and Arbib, 1985), but simple origin centering at the average values seems to be a desirable minimum capability.

11 Input-specific feature associability

In multi-level networks, a new set of computed features can be added to the initial input set. This is adequate for representational completeness, but may be far from optimal in the rate of higher level learning based on this combined feature set. In particular, the upper bound on perceptron training time increases linearly with the number of redundant features in the representation. Consequently, from the perspective of higher level learning, the number of adjustable features should, in general, be kept to a minimum. Output-specific salience is useful in reducing the effects of non-necessary features, but is not perfect.

One approach to the problem of redundant features is for first-level nodes (com­puted features) to reduce the salience of those lower level features that they adequately predict. This way the total number of adjustable features can be decreased rather than increased. For example, if categorization node Cn detected the category (Yl and Y2 and Y3) for input features Yl through Y6, the resulting feature set (from the viewpoint of higher level learning) would be (Cn, Y4, Y5, Y6) rather than (Cn, Yl, Y2, Y3, Y4, Y5, Y6) whenever en was present. H en corresponded to a recurring environmental context involving hundreds or thousands offeatures, the resulting reduction would be consid­erable.

This process can be formalized with the use of conditional probability. In addition to the forward conditional probability trace [enIYi] associated with each input (the probability of Cn given Yi), each categorization node can also maintain a reverse conditional probability trace [YiICn] (the probability of Yi given Cn) for each input. As before, the trace is incrementally computed as:

where Yi and en are the output values of those nodes, and rate is a rate constant determining the memory length of the trace.

Whenever both en and Yi are present, and [YiICn] is equal to 1, (that is, when Yi is adequately predicted), Yi can be safely deleted from the representation for the purpose of higher level learning. In practice, since [YiICn] can only asymptotically approach 1, an arbitrary threshold of .95 is used. Output calculation is the same as before, but a pass of feature deletion occurs before learning takes place. Any node whose current output is adequately predicted by higher level nodes has its salience set to zero; otherwise its salience remains at 1. A continuous version of this might make salience equal to IYi - [Yi len]l. The only things that are noticed as relevant for learning are high level categories and the remaining features that are not adequately predicted by those categories. Behaviorally, an orienting response is often directed toward unexpected stimuli, and its strength has been used as an index of stimulus associ ability (Kaye and Pearce, 1984ab, 1987; Collins and Pearce, 1985; Honey et al., 1987).

99

As an alternative to adjusting the associ ability of the predicted features, their current output values can be modified. In particular, an origin shift can be accom­plished by category node en by simply subtracting its reverse conditional probability vector from the current input vector. For a fixed set of c., the result is still provably convergent since any origin shift (produced by Yi - [Yi len]) can be exactly compensated for by a threshold shift (provided by the context feature cn ).

The separate storage and use of forward and backward predictiveness emphasizes the distinction between concept detection and description. The forward probabilities are appropriate for detection (categorization) while the reverse are appropriate for description. For convenience, the same set of concepts can be used for both processes. However, a better approach might be to train a separate set of categories that are optimized for input predictiveness. This more general class of "predictive" world models can be learned independently of proper behavior.

This method of adjusting feature associ ability is another form of latent learning. However, it is based on input regularities rather than predictiveness for any particular output (as with single node training). The concept of context-dependent salience is not new (Dexter and Merrill, 1969; Anderson et al., 1969ab; Lubow et al., 1976; Wagner, 1976, 1978, 1979; Baker and Mercier, 1982ab; Channell and Hall, 1983; Hall and Channel, 1985ab, 1986; Hall and Minor, 1984; Lovibond et al., 1984; Mackintosh, 1985ab; Kaye et al., 1987), and this particular implementation is similar to Nadel's model (Nadel and Willner, 1980; Nadel et al., 1985). The model proposed here is functionally simple, although an actual biological implementation would probably require rather complex circuit-level systems. Nadel identifies the hippocampus as a likely component of such a system. Other researchers (Gray 1982, 1984; Schmajuk and Moore 1985; Kaye and Pearce, 1987) also identify the hippocampus as a likely site for matching actual and expected conditions.

The proposed model addresses the general problem of redundant, co-occurring features, but does not directly address the problem of identifying context. Given the potentially huge number of redundant contextual features, it would make sense to specifically address that problem. More generally, context identification has been suggested to serve a number of beneficial functions (Balsam, 1985). Unfortunately there may be no precise criteria for defining context, (there may be a continuum between foreground and background features), but spatial cues would seem to be likely candidates for determining biological context. More complex "world models" would presumably permit more complex context models.

At its extreme, the problem of learning context is just the problem of rapid focusing. One extreme form of context identification would be to take a complete "snapshot" of the current state (completely focusing a node as an AND of all present features) and then use that node for higher level learning. In this extreme form, all features would always be covered by some uniquely specific context. This would learn rapidly but would be expensive in its space requirements. In addition, by representing every input pattern as a single, unique feature, there is no opportunity for generalization. Clearly, an improved understanding of the nature of context is needed.

12 Conclusions

This paper considers some properties of TLUs as a representation structure, and the percept ron training algorithm as a method of adaptive training. Although neither is intentionally biological, the results appear to be of biological interest. In particular, the perceptron convergence proof supplies some insight into the time complexity of

100

connectionistic learning. Several representation schemes are considered which differ in their treatment of absent features and threshold adjustment. Three learning strategies are described which have the potential for accelerating perceptron training. These are output-specific associ ability, input-specific associ ability, and origin shifting. In addition, focusing is proposed as an alternative to (or modification of) perceptron training when an ability to rapidly learn specific instances is important. Processes similar to these have been proposed to explain behavioral data.

A single, general learning rule such as perceptron training may be adequate, in principle, to learn all desired functions. In practice, however, it may be quite slow. If learning speed is of interest, numerous extensions may be desirable. These additional systems permit more specialized responses to different aspects of the learning task. This approach indicates that a "simple" function such as category learning may have a series of increasingly complex implementations, each improving on a particular aspect of the task. Quite possibly, the relative difficulty of implementing these capabilities in artificial neural systems will reflect their relative difficulty and extent of use in natural systems. Simple mechanisms (e.g., habituation) may occur in most organisms, while more complex computations (e.g., input-specific salience) may be limited to more complex organisms.

13 References

Abraham WC, Goddard GV (1983) Asymmetric relations between homosynaptic long-term potentiation and heterosynaptic long-term depression. Nature (Lon­don) 305:717-719

Abraham WC, Goddard GV (1985) Multiple traces of neural activity in the hip­pocampus. In: Weinberger NM, McGaugh JL, Lynch G (eds) Memory systems of the brain. The Guilford Press, New York

Abraham WC, Goddard GV (1984) Functions of afferent coactivity in long-term po­tentiation. In: Lynch, G, McGaugh JL, Weinberger NM (eds) Neurophysiology of learning and memory. Guilford Press, New York

Abrams TW (1985) Cellular studies of an associative mechanism for classical condi­tioning in Aplysia. In: Selverston AI (ed) Model neural networks and behavior. Plenum Press, New York

Ackley DH, Hinton GE, Sejnowski TJ (1985) A learning algorithm for Boltzmann machines. Cog Sci 9:147-169

Adrain ED (1946) The physical background of perception. Clarendon Press, Oxford, Eng.

Alger BE, Teyler TJ (1976) Long-term and short-term plasticity in CAl, CA3 and dentate regions of the rat hippocampal slice. Brain Res 110:463-480

Allan LG (1980) A note on measurement of contingency between two binary variables in judgement tasks. Bull Psychonomic Society 15:147-149

Anderson DC, O'Farrell T, Formica R, Caponegri V (1969a) Preconditioning CS exposure: variation in the place of conditioning and presentation. Psychon Sci 15:54-55

Anderson DC, Wolf D, Sullivan P (1969b) Preconditioning exposures to the CS: Variations in place of testing. Psychon Sci 14:233-235

Anderson P, Sunberg SH, Sveen 0, Swann JW, Wig strom H (1980) Possible mecha­nisms for long-lasting potentiation of synaptic transmission in hippocampal slices from guinea-pigs. J Physiol 302:463-482

Baker AG, Mercier P (1982a) Prior experience with the conditioning events: Evidence for a rich cognitive representation. In: Commons ML, Herrnstein RJ, Wagner AR (eds) Quantitative analyses of behavior (vol. III). Ballinger, Cambridge, MA

Baker AG, Mercier P (1982b) Extinction of the context and latent inhibition. Learning and Motivation 13:391-416

Balsam PD (1985) The functions of context in learning and performance. In: Balsam PD, Tomic A (eds) Context and learning. Lawrence Erlbaum Associates, Hillsdale, NJ

101

Barlow HB (1972) Single units and sensation: A neuron doctrine for perceptual psy­chology. Perception 1:371-394

Barrionuevo G, Brown TH (1983) Associative long-term synaptic potentiation in hip­pocampal slices. Proc Natl Acad Sci 80:7347-7351

Barsalov LW, Bower GH (1984) Discrimination nets as psychological models. Cog Sci 8:1-26

Barto AG (1985) Learning by statistical cooperation of self-interested neuron-like computing elements. Human Neurobiol 4:229-256

Belnap ND Jr (1977) A useful four-valued logic. In: Modern uses of multiple-valued logic. Dunn JM, Epstein, G (eds) Reidel Publishing, Boston, MA

Bindra D (1976) A theory of intelligent behavior. Wiley, New York Bliss TVP, Dolphin AC (1984) Where is the locus oflong-term potentiation? In: Lynch

G, McGaugh JL, Weinberger NM (eds) Neurobiology of learning and memory. Guilford Press, New York

Bourne LE (1970) Knowing and using concepts. Psychol Rev 77:546-556 Bourne LE, Restfe F (1959) Mathematical theory of concept identification. Psychol

Rev 66:278-296 Bourne LE, Dominowski RL, Loftus EF (1979) Cognitive Processes. Prentice Hall,

Englewood Cliffs, NJ Brown R, Kulik J (1977) Flashbulb memories. Cognition 5:73-99 Bulgarella RG, Archer EJ (1962) Concept identification of auditory stimuli as a func­

tion of amount of relevant and irrelevant information. J Exp Psychol 63:254-257 Channell S, Hall G (1983) Contextual effects in latent inhibition with an appetitive

conditioning procedure. Animal Learning and Behavior 11:67-74 Cohen NJ (1984) Preserved learning capacity in amnesia: Evidence for multiple mem­

ory systems. In: Squire LR, Butters N (eds) Neuropsychology of memory. Guilford Press, New York

Cohen NJ, Squire LR (1980) Preserved learning and retention of pattern-analyzing skill in amnesia: dissociation of knowing how and knowing that. Science 210:207-210

Collins L, Pearce JM (1985) Predictive accuracy and the effects of partial reinforce­ment on serial autoshaping. J Exp Psych, Animal Behav Processes 11:548-564

Crick FHC, Asanuma C (1986) Certain aspects of the anatomy and physiology of the cerebral cortex. In: McClelland JL, Rumelhart DE (eds) Parallel distributed processing v. 2. MIT Press, Cambridge, MA

Dexter WR, Merrill HK (1969) Role of contextual discrimination in fear conditioning. J Comp Physiol Psych 69:677-681

Dickinson A (1980) Contemporary animal learning theory. Cambridge University Press, New York

Domjan M, Burkhard B (1982) The principles of learning and behavior. Brooks/Cole Publishing Company, Monterey, CA

Douglas RJ (1967) The hippocampus and behavior. Psychol Bull 67:416-442 Douglas RJ (1972) Pavlovian conditioning and the brain. In: Boakes RA, Halliday

MS (eds) Inhibition and learning. Academic Press, London Doyle J (1979) A truth maintenance system. Artificial Intelligence 12:231-272 Duda R, Hart P (1973) Pattern classification and scene analysis. Wiley, New York Dunwiddie T, Lynch G (1978) Long-term potentiation and depression of synaptic re-

sponses in the rat hippocampus: Localization and frequency dependency. J Physiol 276:353-367

Gabriel M, Foster K, Orona E, Saltwick SE, Stanton M (1980) Neuronal activity of cingulate cortex, anteroventral thalamus, and hippocampal formation in discrim­inative conditioning: Encoding and extraction of the significance of conditional stimuli. In: Sprague JM, Epstein AN (eds) Progress in psychobiology and physi­ological psychology. vol. 9. Academic Press, New York

Gabriel M, Orona E, Foster K (1982) Mechanism and generality of stimulus signif­icance coding in a mammalian model system. In: Woody CD (ed) Advances in behavioral biology v. 26. Plenum Press, New York

Garrud P, Rawlins JNP, Mackintosh NJ, Goodall G, Cotton MM, Feldon J (1984) Successful overshadowing and blocking in hippocampectomized rats. Behav Brain Res 12:29-53

102

Gelperin A, Hopfield JJ, Tank D W (1985) The logic of Limax learning. In: Selverston AI (ed) Model neural networks and behavior. Plenum Press New York

Gibbon J, Berryman R, Thompson RL (1974) Contingency spaces and measure in classical and instrumental conditioning. J Exp Analysis Behav 21:585-605

Gibbon J (1981) The contingency problem in autoshaping. In: Locurto CM, Terrace HS, Gibbon J (eds) Autoshaping and conditioning theory. Academic Press, New York

Gluck MA, Thompson RF (1987) Modeling the neural substrates of associative learn­ing and memory: A computational approach. Psychol Rev 94:176-191

Gray JA (1982) The neuropsychology of anxiety: An enquiry into the function of the septo-hippocampal system. Oxford Univ Press, Oxford, Eng

Gray JA (1984) The hippocampus as an interface between cognition and emotion. In: Roitblat HL, Bever TG, Terrace HS (eds) Animal cognition. Lawrence Erlbaum Associates, Hillsdale, NJ

Hall G, Channell S (1985a) Differential effects of contextual change on latent inhibi­tion and on the habituation of an orienting response. J Exp Psych: Animal Behav Processes 11:470-481

Hall G, Channell S (1985b) Latent inhibition and conditioning after preexposure to the training context. Learning and Motivation 16:381-397

Hall G, Channell S (1986) Context specificity of latent inhibition in taste aversion learning. Quart J Expl Psych 35B:121-139

Hall G, Minor H (1984) A search for context-stimulus associations in latent inhibition. Quart J Exp Psych 36B:145-169

Hammond LJ, Paynter WE Jr (1983) Probabilistic contingency theories of animal conditioning: A critical analysis. Learning and Motivation 14:527-550

Hampson SE, Kibler DA (1983) A Boolean complete neural model of adaptive be­havior. BioI Cybern 49:9-19

Hampson SE, Volper DJ (1987) Disjunctive models of Boolean category learning. BioI Cybern 56, 121-137

Hampson SE, Volper D (1986) Linear function neurons: Structure and training. BioI Cybern 53:203-217

Hawkins RD, Kandel ER (1984) Is there a cell-biological alphabet for simple forms of learning? Psychol Rev 91:375-391

Hearst E (1984) Absence as information: some implications for learning, performance and representational processes. In: Roitblat HL, Bever TG, Terrace HS (eds) Animal cognition. Lawrence Erlbaum, Hillsdale NJ

Hearst E (1987) Extinction reveals stimulus control: Latent learning of feature nega­tive discriminations in pigeons. J Exp Psych: Animal Behav Processes 13:52-64

Hearst E (1978) Stimulus relationships and feature selection in learning and behavior. In: Hulse s1I, Fowler H, Honig WK (eds) Cognitive processes in animal behavior. Lawrence Erlbaum, Hillsdale NJ

Herrnstein RJ (1985) Riddles of natural categorization. Philos Trans R Soc Lond B 308:129-144

Hinton GE, Sejnowski TJ, Ackley DH (1984) Boltzmann machines: Constraint satis­faction networks that learn. Carnegie-Mellon Univ. tech. report CMU-CS-84-119

Honey RC, Schachtman TR, Hall G (1987) Partial reinforcement in serial autoshaping: The role of attentional and associative factors. Learning and Motivation 18:288-300

Hunt E, Martin J, Store P (1966) Experiments in induction. Academic Press, New York

Jenkins HM, Sainbury RS (1970) Discrimination learning with the distinctive feature on positive and negative trials. In: Mostofsky D (ed) Attention: contemporary theory and analysis. Appleton Century Crafts, New York

Jenkins HM, Barnes RA, Barrera FJ (1981) Why autoshaping depends on trial spac­ing. In: Locurto CM, Terrace HS, Gibbon J, (eds) Autoshaping and conditioning theory. Academic Press, New York

Kamin LJ (1969) Predictability, surprise, attention and conditioning. In: Campbell BA, Church RM (eds) Punishment and aversive behavior. Appleton-Century­Crofts, New York

103

Kandel ER (1979) Behavioral biology of Aplysia. W.H. Freeman and Company, San Francisco, CA.

Kandel ER (1976) Cellular basis of behavior. W.H. Freeman and Company, San Fran­cisco, CA.

Kandel ER, Schwartz J (eds) (1985) Principles of neural science. Elsevier North­Holland, New York

Kaye H, Pearce JM (1984a) The strength of the orienting response during Pavlovian conditioning. J Exp Psych: Animal Behav Processes 10:90-107

Kaye H, Pearce JM (1984b) The strength of the orienting response during blocking. Quart J Exp Psych 36B:131-144

Kaye H, Pearce JM (1987) Hippocampal lesions attenuate latent inhibition and the decline of the orienting response in rats. Quart J Exp Psych 39B:107-125

Kaye H, Preston GC, Szabo L, Druiff H, Mackintosh NJ (1987) Context specificity of conditioning and latent inhibition: Evidence for a dissociation of latent inhibition and associative interference. Quart J Exp Psych 39B:127-145

KufRer SW, Nicholls JG, Martin AR (1984) From neuron to brain: A cellular approach to the function of the nervous system. Sinaver Associates, Sunderland, MA.

Lara R, Arbib MA (1985) A model of the neural mechanisms responsible for pattern recognition and stimulus specific habituation in toads. BioI Cybern 51:223-237

Levy WB (1985) Associative changes at the synapse: LTP in the hippocampus. In: Levy WB, Anderson JA, Lehmkuhle S (eds) Synaptic modification, neuron selec­tivity, and nervous system organization. Lawrence Erlbaum Associates, Hillsdale, NJ

Livingston RB (1967b) Reinforcement. In: Quarton GC, Melnechuck T, Schmitt FO (eds) The neurosciences. Rockefeller Univ. Press, New York 568-576.

Livingston RB (1967a) Brain circuitry relating to complex behavior. In: The neuro­sciences. Quarton GC, Melnechuck T, Schmitt FO (eds) Rockefeller Univ. Press, New York 499-514

Lolordo VM (1979a) Selective associations. In: Dickinson A, Boakes RA (eds) Mech­anisms of learning and motivation. Lawrence Erlbaum Associates, Hillsdale, NJ

Lolordo VM (1979b) Classical conditioning: The Pavlovian perspective. In: Bitterman ME, Lolordo VM, Overmier JB, Rashotte ME (eds) Animal learning: Survey and analysis. Plenum Press, New York

Lovibond PF, Preston GS, Mackintosh NJ (1984) Context specificity of conditioning, extinction, and latent inhibition. J Exp Psych: Animal Behav Processes 10:360-375

Lubow RE, Rifkin B, Alek M (1976) The context effect: The relationship between stimulus preexposure and environmental preexposure determines subsequent learn­ing. J Exp Psych: Animal Behav Processes 2:38-47

Lynn R (1966) Attention, arousal and the orientation reaction. Pergamon Press, New York

Mackintosh NJ (1978) Cognitive or associative theories of conditioning: Implications of an analysis of blocking. In: Hulse SH, Fowler H, Honig WK (eds) Cognitive process in animal behavior. Lawrence Erlbaum Associates, Hillsdale, NJ

Mackintosh NJ (1983) Conditioning and associative learning. Oxford Univ. Press, New York

Mackintosh NJ (1985a) Contextual specificity or state dependency of human and ani­mal learning. In: Nilsson L, Archer T (eds) Perspectives on learning and memory. Lawrence Erlbaum Associates, Hillsdale, NJ

Mackintosh NJ (1985b) Varieties of conditioning. In: Weinberger NM, McGaugh JL, Lynch G (eds) Memory systems of the brain. The Guilford Press, New York

Mackintosh NJ (1975) A theory of attention: Variations in the associ ability of stimuli with reinforcement. Psychol Rev 82:276-298

Mazur JE (1986) Learning and behavior. Pretice-Hall, Englewood Cliffs McNaughton BL (1983) Activity dependent modulation of hippocampal synaptic ef­

ficacy: Some implications for memory processes. In: Seifert W (ed) Neurobiology of the hippocampus. Academic Press, New York

Medin DL (1983) Structural principles in categorization. In: Tighe TJ, Shepp BE (eds) Perception, cognition and development. Lawrence Erlbaum Associates, Hillsdale

104

Medin DL, Schwanenflugel PJ (1981) Linear separability in classification learning. J Exp Psychol Hum Learn Mem 7:353-368

Mervis CB, Rosch E (1981) Categorization of natural objects. Ann Rev PsychoI32:89-115

Miller RR, Schachtman TR (1985) The several roles of context at the time of re­trieval.In: Context and learning. Balsam PD, Tomie A (eds) Lawrence Erlbaum Associates, Hillsdale, NJ.

Minsky M, Papert S (1972) Perceptrons. MIT Press, Cambridge, MA. Moore JW, Solomon PR (1984) Forebrain - brain stem interaction: Conditioning and

the hippocampus. In: Squire LR, Butters N (eds) Neuropsychology of memory. Guilford Press, New York,

Moore JW, Stickney KJ (1980) Formation of attentional- associative networks in real time: Role of the hippocampus and implications for conditioning. Physiol Psych 8:207-217

Moore JW, Stickney KJ (1982) Goal tracking in attentional- associative networks: Spatial learning and the hippocampus. Physiol Psych 10:202-208

Morris R, Baker M (1984) Does long-term potentiation/synaptic enhancement have anything to do with learning or memory. In: Squire LR, Butters N (eds) Neu­ropsychology of memory. Guilford Press, New York

Muroga S (1971) Threshold logic and its applications. Wiley, New York Nadel L, Willner J (1980) Context and conditioning. A place for space. Physiol Psych

8:218-228 Nadel L, Willner J, Kurz EM (1985) Cognitive maps and environmental context. In:

Balsam PD, Tomie A (eds) Context and learning. Lawrence Erlbaum, Hillsdale, NJ.

Neisser U, Weene P (1962) Hierarchies in concept attainment. J Exp PsychoI64:64G-645

Nilsson NJ (1965) Learning machines. McGraw-Hill, New York Pearce JM (1987) An introduction to animal cognition. Lawrence Erlbaum Associates,

Hillsdale, Ni Pearce JM, Hall G A (1980) A model of Pavlovian learning: variations in the effec­

tiveness of conditioned but not of unconditioned stimuli. Psychol Rev 87:532-552 Rescorla RA (1972) Informational variables in Pavlovian conditioning. In: Bower GH

(ed) Psychology oflearning and motivation. vol. 6. Academic Press, New York Rescorla RA, Waj9ler AR (1972) A theory of Pavlovian conditioning. In: Black AH,

Prokasy WF leds} Classical conditioning II. Appleton-Century-Crofts, New York Rumelhart DE, Hinton GE, Williams RJ (1986) Learning internal representations by

error propagation. In: Rumelhart DE, McClelland JL (eds), Parallel distributed processing. MIT Press, Cambridge MA

Sahley CL (1984) Behavior theory and invertebrate learning. In: Marler P, Terrace HS (eds) The biology of learning. Springer-Verlag, New York

Sahley C, Rudy JW, Gelperin A (1981) An analysis of associative learning in a ter­restrial mollusc. J Comp Physlo1144:1-8.

Sahley CL, Rudy JW, Gelperin A (1984) Associative learning in a mollusc: A compar­ative analysis. In: Alkon DL, Farley J (eds) Primary neural substrates on learning and behavioral change. Cambridge Univ. Press, New York

Salafia WR (1987) Pavlovian conditioning, information processing, and the hippocam­pus. In: Gormezano I, Prokasy WF, Thompson RF (eds) Classical conditioning (3rd ed) Lawrence Erlbaum Associates, Hillsdale, NJ

Schmajuk NA, Moore JW (1985) Real-time attentional models for classical condi­tioning and the hippocampus. Physiol Psych 13:278-290

Schwartz B (1984) Psychology of learning and behavior. W. W. Norton and Company, New York

Schwartzkroin PA, Taube JS (1986) Mechanisms underlying long-term potentiation. In: Alkon DL, Woody CD (eds) Neural mechanisms of conditioning. Plenum Press, New York

Scott GK, Platt JR (1'985) Model of response-reinforcer contingency. J Exp Psychol: Animal Behav Processes 11:152-171

Seifert W (ed) (1983) Neurobiology of the hippocampus. Academic Press, New York Shepherd GM (1983) Neurobiology. Oxford Univ. Press, New York

105

Shortliffe EH, Buchanan BG (1975) A model of inexact reasoning in medicine. Math Biosci 23:351-379

Smith EE, Medin DL (1981) Categories and concepts. Harvard Univ. Press, Cam­bridge, MA

Sokolov EH (1960) Neuronal models and the orienting reflex. In: Bazier MAB (ed) The central nervous sy,stem and behavior. Josiah Macy Jr. Foundation, New York

Sokolov EN (1963) The orienting reflex. Ann Rev Psych 25:545-580 Solomon PR (1977) Role of the hippocampus in blocking and conditioned inhibition of

the rabbit's nictitating membrane response. J Comp Physiol Psychol 91:407-417 Solomon PR (1979) Temporal versus spatial information processing theories of hip­

pocampal function. Psych Bull 86:1272-1279 Solomon PR (1987) Neural and behavioral mechanism involved in learninl? to ignore

irrelevant stimuli. In: Gormezano I, Prokasy WF, Thompson RF (eds) Classical conditioning (3rd ed) Hillsdale, NJ: Lawrence Erlbaum Associates (1987)

Solomon PR, Moore JW (1975) Latent inhibition and stimulus generalization of the classically conditioned nictitating membrane response in rabbits following dorsal hippocampal ablation. J Comp Physiol Psychol 89:1192-1203

Squire LR (1982) The neuropsychology of human memory. Annu Rev Neurosci 5:241-273

Squire LR (1983) The hippocampus and the neuropsychology of memory. In: Seifert W (ed) Neurobiolo~y of the hippocampus. Academic Press New York

Squire LR, Cohen NJ 11984) Human memory and amnesia. In: Lynch G, McGaugh JL, Weinberger NM (eds) Neurobiology of learning and memory. Guilford Press, New York

Squire LR, Cohen NJ, Nadel L (1984) The medial temporal region and memory consolidation: A new hypothesis. In: Weingartner H, Parker E (eds) Memory consolidation. Lawrence Erlbaum, Hillsdale, NJ.

Tesauro G (1986) Simple neural models of classical conditioning. BioI Cybern 55:187-200

Teyler TJ, Discenna P (1984) Long-term potentiation as a candidate mnemonic de­vice. Brain Res Rev 7:15-28

Teyler TJ, Discenna P (1987) Long-term potentiation. Ann Rev Neurosci 10:131-161 Volper DJ, Hampson SE (1986) Connectionistic models of Boolean category repre­

sentation. BioI Cybern 54:393-406 Volper DJ, Hampson SE (1987) Learning and using specific instances. BioI Cybern

57:57-71 Wagner AR (1976) Priming in STM: An information processing mechanism for self­

generated. or retrieval-generated depression in performance. In: Tighe TJ, Leaton RN (eds) Habituation: perspectives from child development, animal behavior and neurophysiology. Lawrence Erlbaum Associates, Hillsdale, NJ

Wagner AR (1978) Expectancies and the priming of STM. In: Hulse SH, Fowler H, Honig WK (eds) Cognitive processes in animal behavior. Lawrence Erlbaum Associates, Hillsdale, NJ

Wagner AR (1979) Habituation and memory. In: Dickinson A, Boakes RA (eds) Mech­anisms of learning and motivation. Lawrence Erlbaum Associates, Hillsdale, NJ

Wattenmaker WD, Dewey GI, Murphy TD, Medin DL (1986) Linear separability and concept learning: context, relational properties, and concept naturalness. Cog Psych 18:158-194

Self-Organizing Neural Network with the Mechanism

of Feedback Information Processing

SeiMiyake ATR

Auditory and Visual Perception Research Laboratories Osaka,Japan

Kunihiko Fukushima NHK

Science and Technical Research Laboratories Tokyo,Japan

Abstract

Several neural network models, in which feedback signal effectively acts on their functions, are proposed.

1. A multilayered network which has not only feedforward connections from the deepest-layer cells to front-layer cells, is proposed. The feedback connections, as well as the conventional feedforward connections, are self-organized. After completion of the self-organization, even though an imperfect or an ambiguous pattern is presented, the response of the network usually converges to that for one of the learning patterns. It may be seen that the network has characteristics quite similar to the associative recall in the human memory.

2. A rule for the modification of connections is proposed, suggested by the hy­pothesis that the growth of connections is controlled by feed back information from postsynaptic cells. Even if a new pattern resembling to one of the learning patterns with which the network has been organized, is presented the network is capable to be self-organized again, and a cell in the deepest layer comes to acquire a selective responsiveness to the new pattern. This model shows a characteristic closely resembling to that of human being, such as the ability to adapt flexibly to different new environments.

3. A model which has modifiable inhibitory feedback connections between the cells of adjoining layers, is proposed. If a feature-extracting cell is excited by a familiar pattern, the cell immediately feeds back inhibitory signals to its presynaptic

108

cells. On the other hand, the feature-extracting cell does not respond to an unfamiliar pattern, and the responses from its its presynaptic cells are therefore not suppressed. In the network, connections from cells yielding a large sustained output are reinforced. Since familiar features do not elicit a sustained response from the cells of the network, only circuits detecting novel features develop. The network therefore quickly acquires favora.ble pattern-selectivity.

1 Introduction

In recent years, studies of techniques for capturing, transmitting and reproducing visual images with high fidelity have greatly advanced. Now, the demand for high-quality images is increasing in broadcasting, telecommunications and other fields. There is no doubt that, in the near future, greater importance than ever will be attached to the development of technology for intelligent processing and utilizing high-quality visual information. The most important problem which we have to solve in developping such visual information processing technology is pattern recognition. Pattern recognition is a very flexible function realized by the visual mechanism of humans. However, the mechanism by which this function works remains obscure.

In order to develop a system for intelligent processing of visual information, it is essen­tial to study, first of ali, the basic principle of human pattern recognition. In this sence, it is reasonable to investigate the information processing mechanisms of humans in depth, and to apply the algorithms of these mechanisms to the new artificial systems.

Until now, we have studied the pattern information processing mechanism using neural network models, inspired by new physiological findings. In this paper, the authors propose several neural network models which achieve new functions by the effective use of feedback information, and discuss the validity of these models by computer simulation.

2 Preparation for neural network modeling

2.1 Self-organization of neural network of the brain

Blakemore and Cooper [1) brought up cats from birth in an abnormal environment where the cats could see only vertical stripes. The visual cortex of the grown-up cats consisted of only such neurons which responded to vertical lines alone. The kittens rared in a normal environment consists not only of neurons responding to vertical lines but also of neurons which respond to horizontal or oblique lines.

The phenomenon that neural networks change in response to extrinsic stimuli is called self-organization. A recent physiological study has disclosed that such changes of neural networks according to extrinsic stimuli are seen not only during the stage of growing but also after maturation [2). The flexible nature of the brain, represented by this self­organization, seems to be responsible for the learning functions which are specific to living organisms.

109

Figure 1: Modifiable synaptic connections

2.2 A learning rule: maximum value detection type

So far, only very few findings have been obtained regarding the plastisity of synaptic connections. This problem should be solved by also theoretically studying which rule is required to enable meaningful self-organization of networks.

The hypothesis of maximum value detection type [3] resembles the Hebb's hypothesis [4], but the condition is more severe with the former than with the latter. In the network shown in Fig.l, firing of a postsynaptic cell Y upon arrival of a signal from the cell X is not sufficient to induce reinforcement of the connection from X to Yj rather, the connection is reinforced only when the firing frequency of the cell Y is larger than that of the neighboring cells (Y', Y", etc.).

Selection of the cell showing the largest output in a vicinity area is equivalent to se­lecting the cell whose input connection is most suitable for the input stimulus. This way of selection is comparable to special education for brilliant children in that the cell (child) showing the best response to a certain stimulus is selected from each vicinity, and the connection is reinforced so as to develop the ability of the selected cell (child). When this is done, formation of redundant networks is rare and the ability of the neural networks can be effectively exhibited [3].

From this point of view, we adopt, in our models, modification rule which is based on the hypothesis of maximum value detection.

2.3 Unsupervised self-organization

Self-organization of neural networks can be roughly divided into self-organization with a teacher (supervised) and self-organization without a teacher (unsupervised). In the case of self-organization with a teacher, the network is instructed by the teacher to which category each pattern (presented for learning) belongs, or the network is asked by the

110

teacher what the presented pattern is, and the teacher tells the network whether or not the answer is correct. In this way, the network advances self-organization utilizing the information about the categories of patterns given by the outside teacher. In the case of self-organization without a teacher, several kinds of patterns to be learned are repeatedly presented to the network without any accompanying instruction about which categories they should be classified into. The neural network organizes itself under the influence of its initial condition, the shape and the frequency of the presented patterns. In the course of self-organization without a teacher, the network comes to prepare the criteria for classification by itself.

This paper will deal with neural network models which learn without a teacher.

3 Neural network with a function of association

The network has not only the feedforward connections as in the conventional network, but also modifiable feedback connections from the deepest layer cells to the front layer cells. After the completion of the self-organization, several test patters are presented, and the response are observed. Even if an imperfect, noisy or ambiguous pattern is presented, the response converges to one of the learning patterns [5].

3.1 Associative memory

We humans have the ability to associate a new event with a past event in the memory, and to create a new concept by integrating the memorized events. For example, when an incomplete pattern with heavy noise or a distorted pattern is presented, we can easily judge what the given pattern is, and infer its original form. This ability is gradually improved through experience and learning.

In a conventional cognitron [3], the entire network is composed of cascade-like connec­tions of multiple layers, each of which has the same structure, and the input ~nformation is gradually integrated during its flow from the input layer to deeper layers.

However, with the cognitron in which the input information flows in only one direction (from the input layer to deeper layers), it is not easy to prepare a network which is capable of utilizing memories for information processing as the associative function in humans does.

For this reason, a new network was designed, which has a feedback loop connections to allow the information to return from the last layer to the preceding layer (See Fig.2 Arrow:1). In this model, the feedback loop connections plastically change according to the nature of stimulus patterns given from outside.

3.2 Self-organization of the network

The network has not only the feedforward connections as in the conventional cognitron [3], but also modifiable feedback connections from the deepest layer cells to the front layer cells. Detailed mathematical expression of the structure is described in Fukushima and Miyake [5]. If several stimulus patterns are repetedly presented to the network, the

111

Figure 2: Feedback information flows

interconnections between the cells are gradually organized. The feedback connections, as well as the feedforward ones, are self-organized depending on the characteristics of the externally presented patterns. After adequate number of stimulus presentations, each cell usually acquires the selective responsiveness to one of the stimulus patterns which have been frequently given. That is, every different pattern becomes to elicit an individual response to the network.

Computer simulation was done using an IBM/370 model 135. The number of layers was 5 (Uo - U4 ) . Each layer has 144 (12 x 12) excitatory cells and 144 (12 x 12) inhibitory cells. The receptive field of each cell is 7 x 7 on its preceding layer. The vicinity area in which reinforcement of the connections is suppressed is smaller; it forms a diamond with the dimensions 5 x 5 (area:25). In the process of self-organization, a certain pattern was continuously given to the input layer while the information was circulating within the loop several times (4 times in this experiment) . This procedure was done for each of the 5 input pat terns" 0", "1", "2", "3", "4". We call a series of repeating this procedure one cycle of pattern presentation.

3.3 Experiment of associative recall

After completion of the above-mentioned self-organization, a test stimulus pattern was presented once to the layer Uo and the input was cut off. In this experiment, the information continued to circulate in the loop even after the input was cut off, followed by gradual changes in the response patterns at each layer. We observe how the response pattern at layer U1 changes with time. The response pattern at U1 , however, is not always easy to understand visually and hence it is not suitable for simple comparison. For this reason, we employed in this observation the same method which was employed to assess the degree of completion of self-organization. That is, the response at Uo is reversely reproduceq from the response at U1 , as schematically shown in Fig.3.

Fig.3 indicates that the reversely reproduced pattern, following presentation of a test pattern, changes gradually with time, and it finally becomes identical to the pattern "3".

112

Figure 3: Experiment of associative recall

This means that the response of the layer U1 to the test pattern finally becomes almost equal to the response of layer U1 to one of the learning patterns" 3". This situation is interpreted that the pattern "3" was associatively recalled by the test pattern.

FigA shows the sequence of associatively-recalled outputs, which were observed in response to various test patterns.

In this figure, the pattern shown at left end of each row is the test pattern, and reversely reproduced patterns are shown in order of time sequence. Even though an imperfect or an ambiguous pattern is presented, the response usually converges to one of the patterns which have been frequently given during the process of self-organization.

The lowest row shows the response to the test pattern which is a logical sum the 5 memorized patterns. The function of associative recall does not continue and the response disappears completely, since the test pattern is composed of information related to all of the memorized patterns.

4 Neural network with an ability of frlexibility

In the conventional networks, only a one-way information flow at the synapse was allowed. Models with this structure were often unable to form cells which selectively respond to the new pattern, when the new pattern slightly modified from one of the memorized patterns was given for the purpose of re-Iearning. That is, once the learning process has been completed, these models lose their flexibility to new environments.

113

Figure 4: Examples of associative recall simulation

To explore a clue to solve this problem, we reviewed the findings of morphological studies of biological neural networks. This review led our attention to the phenomenon that there is some feedback information from the postsynaptic cells (See Fig.2 Arrow:2). Suggested by this phenomenon, we could improve the modification rule of connections. A model in which a new rule was adopted has the ability to reconstruct its network so as to adapt itself to different environments even after the self-organization has already once been completed [6].

4.1 A learning rule suggested by Schneider's hypothesis

It is well known that the axons of nerve cells show complicated branching during ontoge­nesis. We may obtain a key for clarification of the mechanisms of self-organization if we study how nerve fibers develop and grow. In addition, it was recently found that even mature organisms show sprouting of nerve fibers and formation of synapses.

Schneider [7] examined the effect of injury in the develop ping visual system of mammal (hamster) and morphologically observed that a significant re-organization occured in the neural network. Based on this study, he proposed a hypothesis that the following two effects are involved in the nervous system.

One of them is called the prunning effect. When a garden tree is partially trimmed, the growth of the branches in the remaining part is promoted as if to make up for the trimmed part. Schneider suggests that a similar phenomenon occurs also in the nervous system.

114

Conservation of a minimum quantity of terminal arborization

Competition for available terminal space

Figure 5: Schneider's hypothesis

To assume nerve fibers which from synapses as shown in the left half of Fig.5 (upside), partial injury which makes synapse formation impossible in that area, results in an increase in the synapses in the remaining area as shown in the right half of the same figure.

The other effect is called the competition for available terminal space effect. Since the cell surface which can accept synaptic connections is finite, the area available for synaptic formation is finite. Therefore, fibers compete with each other to obtain the space for synapse formation (See Fig.5 lower side). If some free space is provided by the injury of some fibers, other fibers occupy the free space and use it for synapse formation.

Making use of these two effects, we propose a new rule for the modification of con­nections. Corresponding to the prunning effect, we assume a mechanism; the cells, which have not formed sufficient connections with its post-synaptic cells, are influenced strongly to reinforce the connections to its postsynaptic cells. Corresponding to the competition for available terminal space effect, we assume a mechanism; connect ability of the cell which have already received many synaptic connections is reduced. If these mechanisms work, nerve fibers show not only competition for terminal space, but there is also cooperation between nerve fibers to form synapses jointly so as to secure a certain number of synapses in total.

If such a new learning rule is adopted, the ability to reconstruct the network according to environmental changes can be added to a neural network model; that is, even after self-organization has once been completed using a certain set of patterns, the network can reinforce weak connections or form new connections in response to a new pattern set.

115

Stimulus patterns ( 1 )

Stimulus patterns (2)

Figure 6: Stimulus patterns

4.2 Experiment of self-organization

Detailed structure of the network is described in Fukushima and Miyake [6]. We tested the performance of this model by computer simulation using an IBM/370 model 138. First, self­organization was allowed to proceed through repeated presentation of 5 different patterns as shown in Fig.6(1). Then the network was made to re-organize using 6 different patterns as shown in Fig.6(2).

After 20 presentations of 5 patterns, we found that no cell responded to two or more patterns. This result shows that the network can distiguish 5 patterns well after its self­organization.

The model is superior to conventional models especially when self-organization is al­lowed to proceed again, using a new pattern set. For example, the network is allowed to repeat self-organization, using 6 patterns (5 previous patterns plus a new pattern 8 which is a variant of one of the previous patterns 0 as shown in Fig.6(2). As a result, a cell which selectively responds to the pattern 8 is formed, besides the already formed cell which is selectively responsive to the pattern O.

In the conventional learning system, which was based only on one-directional feedfor­ward information flow at synapses, re-organization frequently did not lead to the formation of a new cell that responded selectively to 8 ; instead, it often caused the cell responding to 0 to become a cell which responded to both 0 and 8.

This new learning rule can change the network structure in order to adapt itself to new environments even after it has once completed self-organization under a certain environ­ment.

5 Neural network with an ability of rapid self-organizatioI

We discuss a network which processes information by repeatedly comparing input with the memorized pattern information. In this network, feedback inhibitory modifiable connec­tions are paired with excitatory modifiable connections (See Fig.2 Arrow:3) [8].

UNFAMILIAR WORD

FAMILIAR WORD

116

PRESENTATION REPRODUCTION

~ ~ I I~STORE~ I :: I: I I I

I I I I

I

Figure 7: Bechtereva's experiment

5.1 Bechtereva's experiment

In patients who had had electrodes implanted for treatment, Bechtereva [9] recorded the activity of the nerve cells in the subcortical area of the brain. After letting the patients hear and memorize various words or word-like sounds, they were instructed to reproduce the stimulus signals later. In this way, the responsiveness of the neural activity was studied. Fig.7 schematically shows the result from this experiment.

As shown in this figure, some specific cells continuously generated pulses at high fre­quency (similar to the envelop of the stimulus sound wave) during the time span from the end of stimulus presentation to the reproduction, if the sound given was not famil­iar to the patient, for example, if it was a foreign word whose meaning was unknown to the patient. On the other hand, when a word familiar to the patient was given, this cell sharply reduced its pulse frequency upon completion of word presentation, and it gener­ated only a few pulses until it began again to generate pulses at high frequency during the reproduction.

This finding indicates that when an unknown sound is given the information of the stimulus has to be preserved in its original form, and, for this purpose, the sound needs to be repeatedly remembered, while when a familiar word is given, only the address information (information which indicates in which part of the long-term memory the word is stored) has to be preserved.

These findings allow us to conjecture that a top-down inhibitory effect based on compar­ison with long-term memory is also effective during feature-extracting of visual patterns, and that this effect suppresses the activity when known information is input, so that the processing efforts can be concentrated on unknown information.

117

5.2 Simulation of self-organization

Unlike the network of the conventional type of cognitron [3] which has feedforward synaptic connections, the new network has feedback inhibitory modifiable connections, which are paired with the excitatory modifiable connections (See Fig.2 Arrow:3). Detailed description was appeared in Miyake and Fukushima [8].

We tested the performance of this model by computer simulation, using a mini-computer PDP 11/34. In this experiment, four patterns ("X", "Y", "T", "Z") which have many common elements in their shapes were presented in a cyclic manner, and we observed how self-organization of the network proceeds.

In the conventional cognitron [3], a network for extraction of common elements of patterns were preferentially formed when two or more patterns having many common elements were repeatedly presented. As a result, even the last layer cells frequently made a similar response to two or more different patterns after completion of self-organization. That is, the cognitron sometimes judged two or more patterns, which had many common elements, as the same patterns in the process of self-organization. Once this happened, the network could not distinguish among these patterns any more even when these patterns were later presented in different ways.

When a pattern which is unfamiliar but which is similar to memorized patterns in many points is presented to our new model, the model pays attention to the features specific to the unfamiliar pattern. Therefore, this model is expected to acquire an ability to distinguish between resembling patterns after short-term self-organization.

In the computer simulation, four patterns "X", "Y", "T" and "Z", which have many common elements, were repeatedly presented in this order; self-organization of the net­work was allowed to proceed. Fig.8 schematically shows the net stimulus on the process of self-organization. It is seen that the elements common with memorized patterns are suppressed by feedback inhibition, and that only the cells recognizing the features specific to the current input pattern continue to respond. In this way, self-organization proceeds efficiently, paying attention to novel features of the input patterns.

In this model, we assume the presence of efferent connections paired with afferent connections. According to the recent physiological studies, such reciprocal connections are found in the cortex of mammals [10] [11]. Considering this finding, it appears legitimate for us to assume the presence of such pairs of connections in this network.

6 Discussion

All the models discribed above succeeded in realization of flexible pattern information processing functions by some feedback mechanisms.

The study aimed at applying the functions of the biological systems was initiated in the 1960s. However, this study did not proceed so smoothly as expected. Investigators con­centrated their efforts on developping commercial machines for specific purposes, without sufficiently studying the basic principles of pattern recognition.

In very recent years, investigators have begun to notice that they should pay more

118

t 1 2 3 4

LEARNING X Y T Z PATTERN

NOVEL X • • T . ,. .y • FEATURE • • • ~

Figure 8: Net stimulus (novel feature) to the network on the process of self-organization

attention to the basic principles if they want to develop machines having the ability of recognizing real world (visual and auditory) patterns which is found even in children. In addition, the limit of the Neumann type computational principles has begun to be recognized, and there is a growing trend for commercialization of computers with massively parallel processing.

In the brain, numerous cells (which correspond to small processing units) work in parallel via the neural connections and achieve high-level information processing func­tions. In this sence, we may say that the models proposed in this paper will provide new bisic principles for the operation of future parallel processing computers. It is expected that systematic application of parallel processing computers will allow cmmercialization of epoch-making machines which can easily recognize visual and speech patterns.

7 Acknowledgements

The authors are indebted to Mr.Masahiko Okawa, Dr.Eiji Yodogawa, Dr.Takashi Fujio, Dr.Jun-ichi Ujihara, Mr.Hideo Kusaka for their encouragements and for providing this opportunity of reporting; to Mr.Takayuki Ito for valuable discussions.

References

[1] C. Blakemore and G.F. Cooper, "Development of the brain depends on the visual environment," Nature, vol.5270 , pp.477-478, 1970.

119

[2] T. Kasamatsu and J. Pettigrew, "Restoration of visual cortical plasticity by local microperfusion of Norepinephrine," J. Compo Neurol., vol.185 , pp.163-181, 1979.

[3] K. Fukushima, "Cognitron: A Self-organizing multilayered neural network," Bioi. Cybern., vol.20 , pp.121-136, 1975.

[4] D.O. Hebb, "Organization of behavior," John Wiley and Sons, 1949.

[5] K. Fukushima and S. Miyake, "A self-organizing neural network with a function of associative memory: Feedback-type Cognitron," Bioi. Cybern., vol.28 , pp.201-208, 1978.

[6] K. Fukushima and S. Miyake, "Self-organizing of a multilayered neural network," Proc. Intern. ConI. on Cybernetics and Society, pp.28-33, 1978.

[7] G.E. Schneider, "Early lesions of superior colliculus: Factors affecting the foundation of abnormal retinal projections," Brain, Behavior and Evolution, vol.8 , pp.73-109, 1973.

[8] S. Miyake and K. Fukushima, "A neural network model for the mechanism of fea­ture extraction - A self-organizing network with feedback inhibition -," Bioi. Cybern., vol.50 , pp.377-384, 1984.

[9] N.P. Bechtereva, "Biological expression of long-term memory activation and its pos­sible mechanisms," Brain Res. Monogr., volA, pp.3ll-327, 1979.

[10] J. Tigges, W.B. Spatz and M. Tigges, "Reciprocal point-to-point connections between para-striate and striate cortex in the squirrel monkey," J. Compo Neurol., vol.148 , pp.481-490, 1973.

[11] M. Wong-Riley "Reciprocal connections between striate and prestriate cortex in squir­rel monkey as demonstrated by combined preoxidase histochemistry and autoradiog­raphy," Brain Res., vol.147 , pp.159-164, 1978.

Part 2 Visual Function

Interacting Subsystems for Depth Perception and Detour Behaviorl

Michael A. Arbib Center for Neural Engineering

University of Southern California Los Angeles, CA 90089-0782, USA

ABSTRACT

Where many models of depth perception focus on the processing of disparity cues alone. we

here present two models of depth perception. the Cue Interaction model and the Prey· Localization

Model. which involve cooperative computation using both disparity and accomodation as sources of

depth information. We then introduce models of detour behavior in which such depth schemas

can function as subsystems.

1. The Problem of Depth Perception

The problem for many models of binocular perception is to suppress ghost targets.

The essence of the Dev scheme (Section 2) was to have those neurons which

represent similar features at nearby visual directions and approximately equal

depths excite each other, whereas those neurons which correspond to the same visual

direction but different depths were (via interneurons) mutually inhibitory. 2 In

this way, neurons which could represent elements of a surface in space will

1 Preparation of this paper was supported in part by NIH under grant 7 ROI NS24926 from

NINCDS. My grateful thanks to Donald House and Renu Chipalkatti with whom the research

reported herein was conducted. Much of the argument here is developed at greater length and in

richer detail in the forthcoming volume The Metaphorical Brain 2: An Introduction to Schema

Theory and Neural Networks to be published by Wiley-Interscience.

2 For the psychophysical basis for the model, see Julesz 1971. For related models. see Sperling

1970, Nelson 1975. and Marr and Poggio 1977. Subsequent models developed in the light of new

findings on human psychophysics include Marr and Poggio 1979, Mayhew and Frisby 1979. 1981

and Prazdny 1985. 1987. However. our emphasis in the present paper is on models which depend

on the interaction between binocular and monocular depth cues. and which are motivated by

experiments on depth perception in frog and toad.

124

cooperate, whereas those which would represent paradoxical surfaces at the same

depth will compete. The result is that, in many cases, the system will converge to an

adequate depth segmentation of the image. However, as we shall now see, such a

system may need extra cues. For example, in looking at a paling fence, if several

fenceposts are matched with their neighbors on the other eye in a systematic

fashion, then the cooperative effect can swamp out the correct pairing and lead to

the perception of the fence at an incorrect depth. In animals with frontal facing

eyes such ambiguity can be reduced by the use of vergence information to drive the

system with an initial depth estimate. For example, Hirai and Fukushima 1978 make

explicit use of cues from vergence, favoring solutions with lower parallax. Another

method is to use accommodation information to provide the initial bias for a depth

perception system; this is more appropriate to the amphibian, with its lateral-facing

eyes. It is the latter possibility we pursue here, as part of our continuing concern

with Rana Computatrix, the study of computational models of visuomotor coordination

in frog and toad as a test-bed for organizational principles for the vertebrate brain

(Arbib 1982, 1987). Rana computatrix presents the exciting challenge of "evolving"

an integrated account of a single animal, integrating different aspects of vision with

mechanisms for the control of an expanding repertoire of behavior.

Ingle 1976 observed that a monocular frog can snap fairly accurately at prey

presented within the field of its one eye, suggesting that it estimates depth from

monocular cues. Collett 1977 used experiments with prisms and lenses placed in front

of a toad's eyes to show that, in its binocular field, the toad relied mainly on

stereopsis, but that the monocular toad did make depth judgments based on

accommodation. The problem, then, was to design a model which would function on

accommodation cues in the monocular animal but which would nonetheless be most

dependent upon binocular cues within the binocular field. (At present, we treat Ran a

computatrix as an approximation to both frog and toad. Future modeling will

introduce specializations to represent the peculiarities of different subspecies.)

With this as background, we study two models of depth perception in the toad due

to House, together with an outline of their mathematical analysis by Chipalkatti and

Arbib. In House's first model, a small patch in the neural map encodes a single

angular direction but a full range of depth, with the proportion of cells representing

nearer depths much greater than the proportion of cells representing further

depths. In House's second model, there is no explicit map. Rather, each tectum

localizes a prey target on its 2-dimensional map, and it is then up to the motor system

125

d exchlltory field

Figure 1. The Dev model for stereopsis

involves competition along the disparity

axis and cooperation along the depth axis. x

c:======U=~r=~--x vlx' inhibitory field

to compute the 3-dimensional location from the disparity of these 2 signals. After we

have studied these two models, we will turn to their putative role in the toad's ability

to detour around barriers to reach its prey.

2. The Dev Model

As background for the Cue Interaction model, we briefly recall the formulation

of the Dev model of stereopsis due to Amari and Arbib 1977. The model (Fig. 1)

receives input from two one-dimensional retinas, each with coordinate axis x. It

comprises a two-dimensional "excitatory field" where the membrane potential ud(x,t)

of the cell at position (x,d) represents the confidence level at time t that there is an

object in direction x on the left retina whose disparity on the right retina is d; and a

one-dimensional inhibitory field with v(x,t) the activity of the cell at position x at

time t. We think of d as taking a set of discrete values. The output of these cells are

given as f(ud(x,t» and g(v(x,t» respectively, where feu) = 1 if u > 0 else 0, while g(v) = v if v > 0 else O. The model can then be expressed mathematically as:

'tUduldt = -u + w!,i<f(u) - w2*g(v) - hu + s

'tUdV/dt = - v + Ldf(ud) - hv'

Here hu and hv are threshold levels (constants) and sd is the input array. We set

RL(x) = 1 if some object projects to point x on the Left retina else 0

126

and similarly for RR. We then define the stereo input to the model to be

sd(x) = RL(x) RR(x+d)

which is 1 only if there is an object at position x on the left retina as well as at x plus

disparity d on the right retina. A more refined model would make sd (x) a measure of

the correlation of the features of the pattern near position x on the left retina and

those centered at x+d on the right retina.

In our mathematical analysis, x is a continuous variable, and the convolutions are

given by integrals running from -co to +00:

[WI *f(u)]d(x,t) = f WI (x - x')f[ud(x,t)]dx'

[w2*g(v) ](x,t) = f w2(x - x')f[v(x,t)]dx'

In computer simulations, x is a discrete variable, and the convolutions are given

by summations extending over the range of the masks (assumed centered at 0):

[wI*f(u)]d(x,t) = Lx' WI(X - x')f[ud(x,t)]

[ W2*g(v) ](X,t) = Lx, W2(X - x')f[v(x,t)]

with some suitable convention about edge effects.

Our convention is that when we add arrays, the operation supplies missing

variables in lower dimensional arrays. Thus, if (ignoring the time variable t) we

have arrays a(x,d), b(x) and c (the last being just a constant), we have that

[a + b + c](x,d) = a(x,d) + b(x) + c.

Finally, note that the operations f and g act componentwise (e.g., f(ud)(x,t) =

feud (x,t»), while :td reduces a 2-dimensional array feud) to a one dimensional array

:td feud) (again ignoring t):

Figure 2. The Cue Interaction Model.

Layer A represents an inference

system that provides monocular depth

cues from lens accommodation. and

layer D represents an inference system

that provides binocular cues from

disparity matching. Layers M and S

represent spatially organized fields

over which two depth-mapping

processes operate. Arrows from layers

A and D to these fields indicate that

field M receives only monocular depth

127

,.

II

s

D

cues. and field S receives only binocular cues. The ovals and arrows between fields M and S

indicate mutual excitatory interconnections that map each local region (oval) of one field onto the

corresponding local region of the other field. By means of these interconnections. points of high

excitation in one field provide additional excitation to corresponding points in the other field.

Competition among depth estimates within each field assures that points excited in only one field

will have little chance to sustain this excitation when there are other points receiving stimulation

in both fields. (House 1982)

3. The Cue Interaction Model

The first model integrating accomodation and disparity cues presented here. the

Cue Interaction Model (House 1982. 1984; Chipalkatti and Arbib 1987b). uses two

systems, each based on Dev's stereopsis model, to build a depth map. One is driven by

disparity cues, the other by accomodation cues, but corresponding points in the two

maps have excitatory cross-coupling (Fig. 2). The model is so tuned that binocular

depth cues predominate where available, but monocular accomodative cues remain

sufficient to determine depth in the absence of binocular cues. The model produces a

complete depth map of the visual field, and so is appropriate for building a

representation of barriers for use in navigation. At the top is an accommodation­

driven field, M, which receives information about accommodation and which - left to

its own devices - would sharpen up that information to yield relatively refined depth

estimates. Below, we see the type of system, S, posited by Dev to use disparity

information and suppress ghost targets. However, the systems are so intercoupled

that a point in the accommodation field M will excite the corresponding point in the

128

disparity field S, and vice versa. Thus a high confidence in a particular (direction,

depth) coordinate in one layer will bias activity in the other layer accordingly. The

result is that the system will converge to a state affected by both types of information

- although the monocular system can, by itself, yield depth estimates.

S is thus given by

'tvav/at = -V + Ldf(Sd) - hv

which just differs from the Dev model in the addition of the coupling term Km s fc (M)

from M, where the nonlinearity fc is given by fc(S) = 0 if M < 0 else Mlhsat if 0 !> M !>

h sat else 1.

M is given by

'tuaU/at = -U + Ldf(Md) - hu

which differs from S in that the input is now provided by an accomodation measure:

aL(x,d) = exp[{ -(d-da(x»/B )2]

where da(x) is the actual distance of an object in direction x from the left eye. and B is

a suitable constant. We set object in direction x from the left eye if there is no object

in direction x from the left eye. Thus aL(x.d) !> 1. and equals 1 only for d = da(x). It

represents the sharpness of the image in direction x when the focal length of the

left eye is set at d. Similarly, we may define aR (x,d) for the right eye.

Fig. 3 shows stages in the processing by this model of a scene comprising a fence

and two worms. The left-hand column of each Fig. 3b shows the accommodation

(above) and the disparity field (below) for the fence information. In the top image of

Fig. 3b. we see the initial state of the accommodation field. The information is

blurred. representing the lack of fine tuning offered by accommodation. Below, we

see the intial state of the stereopsis field. The targets are better tuned. but they offer

ghost images in addition to the correct images. Fig. 3f shows the outcome of such

interaction. We see that virtually all the ghost fence targets have also been

suppressed. In addition. we see that the accommodation information has been

sharpened considerably. The information is now precise and unambiguous, and thus

'~--~.li~---.l·11 '~------~I)~------~.

'~--~.1 il-~"""""'----rI.11 , l'I~~o,,-,o,,"--~.

, c-"'==-" J il-__ .L>o .... o'---r.11 , c-"'==-" 1'.1-1 _....i.O .... O'-'--4.

129

Figure 3. The time course of the Cue Interaction model is shown from its initially-inert state (a)

and immediate response to input to a satisfactory depth segmentation (f) here. All figures are in

the retinal angle vs. disparity coordinate system. Successive figures are temporally spaced 1.4

field time-constants apart. In each subfigure, the left column shows the barrier fields and the

right the prey fields; in each case, the upper two-dimensional grid shows the level of excitation of

the accomodation field, and the lower the disparity field. The line-graphs under the grids

indicate the intensity and localization on the retinal angle axis of excitation in the inhibitory

pools (House 1982).

130

can be used to guide the further behaviour of the animal.

While the above model is of interest in its own right - as a model specific to the

study of the amphibian (we postulate that such a system processes barrier

information), and as an indication of the class of stereopsis models based on multiple

cues - I want to stress here its more general significance. Cooperative computation

is a general principle of brain operation, with different sensory systems providing

different patterns of information to be factored into the determination of the overall

behaviour of the organism (Arbib 1981). Since we explicitly designed the model of

Fig. 2, we know that one layer represents accommodation information, while another

represents disparity information and we can clearly see the differences in these

types of representation in Fig. 3b. However, Fig. 3f represents the sort of state of

activity that is much more likely to be seen during the ongoing behaviour of the

system, and here we see that both surfaces represent pooled information based on the

interaction between the layers, rather than representing information directly

supplied by sensory systems. This clearly indicates the dangers of experimentation

based on feature analysis without related high-level modelling. As we can see,

feature analysis of Fig. 3f would simply show cells responsive to information

available at a specific depth and visual direction. Only a far more subtle analysis,

guided by a model of the kind presented here, would allow the experimenter to

discover that although much of the time the two surfaces exhibited congruent

activity, one was in fact driven primarily by accommodation, while the other was

driven primarily by disparity information.

With this background, we turn to a brief statement of results from the

mathematical analysis of Chipalkatti and Arbib 1987b. First, we introduce:

Assumption 1: hm > 0; hs > 0; 1 > hu > 0; 1 > hv > O.

The assumption that each h > 0 ensures the stability of the system in the absence of

input. The assumption that hu and hv are less than 1 ensures that when a single cell

is excited in both the Sand M fields, near equilibrium, fields U and V yield a positive

output g(U) and g(V) respectively.

131

Theorem 1: For competition along the depth dimension, in which all stimulation

lies along a fixed "x-slice" we have:

(a) The quiescent state of the system, for which no modules are excited, is in stable

equilibrium for inputs ad and sd satisfying ad < hm and sd < hs for each d (and the

given x).

(b) If ad = 0 for d .. dl and sd = 0 for d .. d2, then the system has an excited equilibrium

state with Md1 , Sd2 > 0 for inputs satisfying ad > ~ - Wm - Ksm and sd > hs -Ws - Kms·

Because of the coupling term between M and S, activity is easier to achieve if d1 =

d 2 , but discordant activity can be maintained if the two fields receive sufficiently

strong inputs for d1 .. d2 , which leads to an undesired situation. However:

Theorem 2: Let sm a x and am a x be the maximal stimuli to the fields Sand M

respectively. Given the conditions

Wm-hm+~ax< 0 <Wm-~+Ksm

W s - hs + smax < 0 < W s - hs + Kms

activity can occur in both fields in response to a single stimulus in each field only

when it is a pair of coupled points in both fields that is active.

In view of this, we impose further constraints on the parameters:

Assumption 2: 0 < W m < hm and hm < Ksm ; while 0 < W s < hs and hs < Kms .

With this, we can prove:

Theorem 3: Let sm a x and am a x be the maximal stimuli to the fields Sand M

respectively. With input restricted to a single x, modules for at most one d can be in

an excited state at stable equilibrium if both wm2 > Wm - hm + Ksm + amax and ws2 >

W s - hs + Kms + smax·

Theorem 4: When discordant stimuli are supplied to the two fields, say ai > 0 and Sj >

0, the selected module depends on the strength of the input.

132

• For small inputs and weak cross-coupling. no module is active at equilibrium.

• For strong inputs and large cross-coupling gains. we have at equilibrium that:

Module i is excited in both fields if ai > Sj and Kms > Sj .

No activity can be sustained if ai = Sj.

Module j is excited in both fields if ai <Sj and Ksm > ai.

We have seen that active points compete with each other along the d-dimension

by trying to suppress the activity of other points and by inhibiting unexcited points

from becoming active. However. due to excitatory interactions. nearby points along

the x-dimension tend to excite each other. Adapting techniques set forth in Amari

and Arbib 1977 for the Dev model. we define

Wm1(x) =Jo X wm1(x')dx' and Wm2(x) =Jo X wm2(x')dx'

and similarly for Wsl and Ws2. We then set

and introduce:

Wm(oo) = Wm1(00) - Wm2(00)(1- hu)

Ws(oo) = Ws1 (00) - WS2(00)(1 - by)

We may then complement our analysis of the d-dimension with the following:

Theorem 5: Let stimulation be restricted to the x-dimension for a fixed d. If a

stimulus is applied to M such that at equilibrium the excitation lies in the interval

[xm l' xm2] with xml - xm2 = am' then the induced excitation in the field S must lie

within the interval [xm l' xm 2] and at its endpoints. the magnitude of the

cross-coupling input must be less than hs and the field M there must lie in the

interval (0. hsat). (Recall that hs a t is the lowest value of M for which the

cross-coupling fc(M) reaches its saturation level of 1.)

With the assumptions in place for the separate analysis of the x-dimension and

d-dimension. we can indeed prove that the Cue Interaction model satisfactorily

processes the "two-worm" situation as simulated in Fig. 3.

133

Theorem 6: For two targets placed at (xl ,d 1) and (x2 ,d2 ) in the visual field, the

peaks corresponding to the "ghost targets" in the field S are suppressed at

equilibrium, assuming that all points in the two fields are initially in the same state.

The model succesfully prevents the emergence of

targets, e.g., a fence, as well. The two fields strongly

in both dimensions. Along the competition-dimension

"ghost images" for multiple

influence each other's activity

they cooperatively select the

surviving module, whereas along the cooperation-dimension they directly control

each other's excitation lengths. These properties of the model lead to the suppression

of activity due to cues representing "ghost targets."

4. The Prey Localization Model

Collett and Udin 1983 showed that, for the task of unobstructed prey-catching,

toads are able to make accurate binocularly-based depth estimates even after n.isthmi

(NI), the major cross-tectal binocular relay, has been lesioned. Collett, Udin and

Finch 1986 report behavioral studies with two toads with lesions which destroyed or

disconnected most of both NI. To find out whether binocular cues remain effective

after NI lesions, they tested prey-catching behaviour when the toads viewed prey

through prisms which changed horizontal binocular disparities or through convex

lenses which altered the accomodative state of the eyes. In both cases, there is a

conflict between monocular and binocular cues. As we have seen, Collett 1977 found

that binocular cues predominate in the normal animal; the present study showed this

also to be true in the NI-lesioned toads. Collett and Udin postulated that the toad may

use triangulation to locate the prey, rather than a process of disparity matching,

much as the mantid has been hypothesized to form depth estimates by comparing

output signals from the two optic lobes (Rossel 1983). They also found that toads

undershoot their prey equally whether the disparities imposed by prisms are

horizontal, vertical or oblique - contrary to mammalian disparity detectors which can

only operate if there is reasonable vertical alignment between stimuli on the two

retinas. In particular, Collett et al doubt that a point in the visual field is resolved by

the tectum into its horizontal and vertical components. Rather, they offer for

consideration the notion that the tectum codes position in polar coordinates, with

disparities measured as the difference between the radial coordinates of a point in

each eye.

134

Figure 4. The Prey Localization Model. Lenses are coupled so that they are accommodated to the

same depth. Imagers produce visuotopically mapped output signals whose intensity is

proportional to the crispness-of-focus at each image point. Pattern recognizers produce

visuotopically mapped output signals that are strongest wherever the image most closely

corresponds to the recognizer's matching criteria. Prey selectors take their input from the

recognizers and from cross-coupled connections with each other. Their outputs are highly

selective, giving a high weight to points receiving the maximum input stimulation and suppressing

all others. Since the prey selectors take their input both from the pattern recognizers and from

cross-coupling they are biased in favor of points receiving strong stimulation on both sides of the

binocular system. The accommodation controller converts the weighted image coordinates from

both prey selectors into an estimate of depth. It then uses this calculated depth to adjust the

lenses. (House 1988)

The Prey Localization .Model (House 1984, 1988), incorporates the triangulation

hypothesis. Each side of the brain selects a prey target based on output of the

contralateral retina. and computes a depth estimate by triangulation to adjust lens

focus. If the selected retinal points correspond to the same prey-object. then the

depth estimate will be accurate and the object will be brought into clearer focus.

"locking on" to the target. If the points do not correspond. the resulting lens

adjustment will tend to bring one of the external objects into clearer focus. and the

two halves of the brain will tend to choose that object over the other. However.

Caine and Gmberg 1985 find that frogs with lesions of NI failed to respond to either

threat or prey stimuli in the corresponding region of the visual field (contradicting

Collett and Udin). while exhibiting normal barrier-avoidance and optokinetic

nystagmus.

135

We now provide mathematical details of the model shown in Fig. 4. For n = 1, r, the

prey selector of each eye has a one-dimensional excitatory field Bn (x,t) and a single

inhibitory element Un:

'tUdUJdt = -Un + J f[Bn(x;)]dx - hu· This is is the model of Didday 1976, with the addition of cross-coupling terms (which

we posit are mediated by n. isthmi)

where wi(x-x') represents the cross-coupling input from a point x' in one selector to

the point x in the other selector. The output from each prey selector is used to

determine the "average" retinal angle of the targets:

9n(t) = tan-1 ( Jf[Bn(x,t)] sin(x/c)dx / Jf[Bn(x,t)] cos(x/c)dx }

where c is a conversion factor which converts retinal coordinates into retinal angles.

Once the angles 91(t) and 9r(t) are known, the corresponding disparity is calculated

by

From Db we infer the corresponding depth db by triangulation. The job of the

accomodation controller is then to adjust the focal length dc of the lens to better

match this depth estimate:

'ta adc(t)!dt = -dc(t) + db(t).

House 1988 provides computer simulations which explore a variety of properties of

the model, while Chipalkatti and Arbib 1987a give a stability analysis of its equilibria.

4. Introducing Schemas

Neither of our depth models can, by itself, fully explain the complete range of

data on the depth resolving system of toads. The Cue Interaction Model successfully

integrates binocular and accommodation cues in a way which allows it to replicate

behavioral data. However, it relies on a neural connectivity that does not appear to be

136

necessary for binocular depth perception. The Prey Localization Model successfully

addresses the Collett-Udin data, and integrates both depth cue sources. However, it is

not capable of operating in a purely monocular mode. Further, this model does not

allow us to address as broad a range of visual data as does the first model. In particular,

it locates only a single point in space and is, therefore, not well suited to locating

barrier-like objects. We argue that, instead of there being a single general

depth-perception mechanism, there are various neural strategies funt-doning either

cooperatively or alternatively to cope with the vast array of visuo-motor tasks

required of the freely functioning animal. In particular, we currently view the Cue

Interaction Model as our best model of mapping the presence of barriers in space

while, as its name suggests, the Prey Localization Model is our current theory of how a

prey is located in the space near the frog or toad. We now wish to see how these might

function as subsystems within a larger model which accounts for certain aspects of

detour behavior in Rana computatrix. But first I need to introduce a way of thinking

which characterizes these subsystems as schemas.

In this paper, I shall mainly speak of schemas as functional units for the analysis

of animal brains, leaving it in most cases to the reader to draw the parallel

implications for neural computing and perceptual robotics. Perceptual and mo to r

schemas (Arbib 1981) are defined as units of perceptual analysis and of motor control,

respectively, to account for the ability of the system to serve these requirements.

Multiple brain regions contribute to a given schema, and it is in the interaction of

these schemas that sensory guidance of motor behavior emerges. The task, then, for

the brain modeller is to not only identify potential schemas which represent the

motor behavior, but to also show how these schemas map into the known neural

circuitry.

A set of bas i c motor schemas is hypothesized to provide sensorimotor neural

activity corresponding to simple, prototypical patterns of movement. These

elementary "building blocks" of motor behavior combine with perceptual schemas to

form coordinated control programs which interweave their activations in accordance

with the current task and sensory environment. Thus motor schemas in general may

be either basic, or built up from other schemas as coordinated control programs.

While basic motor schema activations may be sensory-driven, schema activations in

general are largely task-driven, reflecting the intention-related goals of the

individual and the physical and functional requirements of the task.

137

Schemas may be instantiated. For example, given a schema that represents

generic knowledge about some domain of interaction, we may need several active

copies of the schema - which we call schema instances each suitably tuned, to

subserve our perception of several instances of that domain. We postulate that the

brain can support concurrent activity of many schemas for the recognition of

different objects, and the planning and control of different activities. In summary,

then, the analysis of interacting computing agents called schema instances can serve

as a valuable intermediary between overall specification of some animal, human, or

robot behavior and the neural networks (or other low-level machine instructions)

that subserve it. We add that an object may have different representations

appropriate to different tasks, and, within such a representation, different coordinate

systems may be set up for different parts of the object and/or different stages of task

execution.

The schemas of Rana computatrix various functional subsystems underlying

visuomotor coordination in the frog and toad have the very important but special

property that each schema is implemented within fixed neural circuitry. (This does

not preclude the use of a given region of the brain by more than one schema.) In

contrast with this situation, when we turn to the study of perceptual and motor

schemas in perceptual robotics, we consider systems in which it is not so plausible that

the same circuitry is used every time a copy of the schema is activated. The analysis

of a schema-based system for high-level vision (see, e.g., Appendix 1 of Arbib 1987)

presents processes whereby the "model of the environment" can be formed as an

assemblage of instances of perceptual schemas. This raises the problem of schema

instantiation in neural nets which is, I claim, one of the most important open

problems to be faced in bridging between cognitive science and neuroscience (cf.

Bamden 1987). How does the brain mobilize the appropriate assemblies of neurons

when many copies of the schema appear to be active at the same time? How

economically can inferential and other mechanisms respond to recruited - and so in

some sense unpredictable - assemblies? How are recruited assemblies demobilized?

This is especially difficult when an assembly shares many neurons with other

assemblies. But this is a topic for future papers.

While there is some consensus that we have separate schemas for mapping the

position of barriers and for localizing prey, there is no consensus as to the neural

substrates of these schemas. However, the power of our schema methodology is that it

allows us to model one structure or function within the context of a schematic

138

description of its neural environs without requiring a fully articulated neural

description of how those environs operate. All that matters is that the information

flow between the specified subsystem and its environs be adequately represented.

Thus, in developing our barrier negotiation models, we have made the assumption that

toads are able to infer the depth of barriers and prey simultaneously and that this

information is either determined by different neural substrates or is at least separable

by object category - but our use of these "depth schemas" does not depend on any

specific choice of neural mechanism for their implementation. Let us now see how

these depth schemas enter into a variety of models of detour behavior.

Why more than one such model? It is our intention that, by considering a variety

of models of detour behavior, we can create a space of alternatives in which the design

of a rich set of neuroethological and neurophysiological experiments will be possible.

At this preliminary stage of the search for the neural substrates of detour behavior, it

is premature to focus on a single model. It is hoped that the contrast between these

models will serve to stimulate the design of new behavioral and physiological

experiments. We also stress that the models are not tightly constrained, in that they do

not attempt to specify what particular neurons are doing in the posited behaviors.

Rather, they represent processing schemes which could plausibly be carried out in

neural structures, and thus represent postulates that there are populations of neurons

which carry out the indicated operations. We expect that the refinement of our

models will go hand in hand with the development of further data of this kind, and

that theory and experiment will provide each other with important stimulation.

S. Schemas for Detour Behavior

To see how we may use schemas to move beyond models of the recognition of

visual patterns that serve to trigger stereotyped, though appropriately spatially

directed, responses, we now consider situations in which the animal exhibits

behavior which takes account of a complex spatial context. Specifically, we shall

start from data on a toad viewing a vertical paling fence behind which there is a

worm. It has been shown that the animal may either snap directly at the worm, or

may detour around the barrier. However, it will not go around the barrier if there is

no worm behind it. Thus, we may still see the worm as triggering the animal's

response, but we now see a complex trajectory dependent upon the relative spatial

position of worm and barrier.

Figure 5. The right hand side of the

figure shows the trajectory of a toad

which has sighted a number of meal­

worms behind a paling fence (the row

of dots) and then detours around the

fence to approach the prey. Note that

when it stops (the prey no longer being

in view) its position shows that it has

retained a representation of prey

position, which is relatively accurate

despite the intervening movement of

the toad (Collett 1982).

139

start

10cm

A first view of these data is given in Fig. 5 from Collett (1982). The row of dots

indicates a paling fence. The two circles indicate two alternative placements of

worms which are to attract a toad's attention, while the T indicates an opaque barrier

which prevents the toad from seeing the worms after it has moved from the start

position. The position of the toad is represented by a dot for its head and a line for its

orientation. The sequence of such "arrows" on the right-hand side of the figure

indicates successive positions taken by the toad in a single approach to the prey. The

animal sidesteps around the barrier, pauses for several seconds, and then continues

to a position at which it stops, pointing in approximately the direction of the worm -

but note that, due to the opaque barrier, the worm is no longer visible. On the

left-hand side of the figure, we indicate the position of the toad on a number of

different occasions, at the pause. The dashed arrows correspond to the nearer

position for the worm, the solid arrows correspond to the position of the pause for

the further position. What is of interest is that even though the worms are no longer

visible to the toad at the time of the pause, the orientation of the animal correlates

well with the position of the target. Thus, we must not only explain how it is that the

140

.) b)

11111111111· 111111

'--orr c)

cI)

11111111111 111111

1

~ Figure 6. Epstein's 1979 model of detour behavior: Each prey-object provides a broad peak of

excitation to the tectum; each barrier a narrow trough of inhibition. The tectum then acts as a

maximum selector to choose a prey or barrier-edge as the target of the first move. In (a-c) the

central object shows the position and orientation of a gerbil, the diamond-shaped icons represent

food pellets, and vertically hatched regions represent barriers. The curves below the gerbil show

spatially distributed patterns of excitation (regions above the dotted line), and inhibition (below

the dotted line) elicited in the model by the configuration of pellets and barriers. In (a) the

presence of three prey objects results in an overlapping pattern of excitation, whereas in (b) the

barriers result in a trough of inhibition extending a small distance beyond each barrier end. The

net effect of summing excitation due to pellets and inhibition due to barriers is shown in (c). The

presence of inhibition leaves the maximally excited position to the right of the left fence. The

curves traced in (d) show the time-course of the model in response to the stimulus pattern in (c).

The time dimension t is drawn going into the paper, the horizontal axis represents the spatial

dimension and the vertical axis the level of excitation. The curve of input vector S (bottom) simply

shows the stimulus pattern of (c) held constant as time advances. The curve of excitation level U

(top) shows that the model eventually converges with all orientations suppressed except for the

one corresponding with the initial maximal input. According to the assumptions of the model, this

triggers a tum to the right edge of the left barrier.

141

animal chooses whether to proceed directly toward the prey or to sidestep around the

barrier, but also come to understand how the position of the target can be encoded in

such a way as to be available to guide the animal's behavior even if the target does

not continue to be visible.

Didday (1976) gave a simple model of the tectum as a row of neurons selecting its

maximal stimulus. Epstein (1979) adapted this model by positing a more structured

form of input (Fig. 6). Each visible prey-stimulus provides a tectal input with a

sharp peak at the tectal location corresponding retinotopically to the position of the

stimulus in the visual field, with an exponential decay away from the peak. Each

barrier-stimulus provides a trough of inhibition whose tectal extent is slightly

greater, retinotopically, than the extent of the barrier in the visual field. When the

model tectum acts upon the resultant input pattern, it will choose either a prey-locus

or the edge of a barrier as the maximum. Thus, Epstein's model can exhibit choice of

the direction of the prey or the barrier edge, but not the spatial structure of the

behavior.

Given that the behavior of the toad - whether to approach the prey directly, or to

detour around the barrier depends upon the distance at which the worms are

behind the barrier, a full model of this behavior must incorporate an analysis of the

animal's perception of depth. To address this, Arbib & House (1987) gave two models

for detour behavior which make use of separate depth maps for prey and barriers.

In the first, the Orientation Model, the retinal output of both eyes is processed for

"barrier" and "worm" recognition to provide separate depth mappings for barrier

and worm. We suggest that the animal's behavior reflects the combined effects of

prey "attraction" and barrier "repulsion". Formally, generalizing Epstein's model,

the barrier map B is convolved with a mask I which provides a (position-dependent)

inhibitory effect for each fencepost; the worm depth map W is convolved with a

mask E which provides an excitatory effect for each worm. The resultant total map

T = B*I+W*E

is then subject to further processing which will determine the chosen target. E is

an excitatory mask which projects broadly laterally, and somewhat less broadly

towards the animal. I is an inhibitory mask such that there is a short distance

behind the edge in which there is little inhibition (to model the fact that the toad does

snap at a worm close behind a fence), after which inhibition is equally strong at all

distances. The total excitation T is summed in each direction, and then a maximum

selector network chooses that direction with maximal activity. If this corresponds to

142

the prey, the animal will approach and snap, otherwise, further processing is

required.

The full detour behavior exhibited here is quite complex: the animal does not

simply orient towards the prey or the end of the barrier; rather, if it does not proceed

directly toward the prey, it sidesteps around the barrier orienting in a way that

depends upon the position of the target and the length of the sidestep. We postulate

that each component of the behavior (sidestepping, orienting, snapping, etc.) is

governed by a specific motor schema. We then see detour behavior as an example of

the coordination of motor schemas (Arbib, 1981), where the sidestepping schema acts

to modulate the orienting schema. Ingle (1983) has observed that a lesion of the

crossed-tectofugal pathway will remove orienting; lesion of the crossed­

pretectofugal pathway will block sidestepping; while lesion of the

uncrossed-tectofugal pathway will block snapping.

6. Schemas for Prey-Acquisition

Lara et al. 1984 offer an alternative model of detour behavior in the presence of

barriers in which recognition of gaps is an explicit step in detour computation.

(Consider a human walking through a doorway: The analogous claim is that we

recognize the opening we can walk through, rather than avoiding the door-frame:)

It is thus a challenge to experimentalists to design ways to discriminate between the

hypotheses that "the brain 'recognizes' barriers as inhibitory" and that "the brain

'recognizes' gaps as excitatory" in the cooperative computation of behavior. The

same paper also reports models at the level of interacting schemas, rather than

layers of neuron-like elements for prey-acquisition in environments containing

chasms as well as barriers, and for predator-avoidance. Future research will test

these models by developing their possible neural instantiations. Lara et al. (1984)

also offer models - at the level of interacting schemas rather than layers of

neuron-like elements for prey-acquisition in environments containing chasms as

well as barriers, and for predator-avoidance. We now tum to a presentation of the

schemas for prey-acquisition by toads posited in this approach.

Fig. 7 presents the prey acquisition schema. It is activated by the instantiation of

the perceptual schema for Prey , but the actual motor schemas executed will depend

on whether further perceptual processing activates the "free prey" perceptual

143

'fI, -... "il' S

o Goal,laleIGS) o Schema IS)

6 l'erceptualsc_IPS)

o Molor schema IMSI

Figure 7. If prey is in the visual field. the orient motor schema 0 R is activated. The consequent

unfolding of action depends on whether a barrier (activating perceptual schema barr ) or a chasm

(activating perceptual schema chasm ) or "free space" (activating perceptual schema FP ) is

perceived. See the text for further details. (From Lara et al. 1984).

schema (perceptual schema FP representing a situation in which no obstacles

intervene between the animal and its prey. perceptual schema barr w h i c h

represents a situation in which a barrier intervenes. or perceptual schema C has m

which represents an intervening barrier. The activation of the corresponding

coordinated control program - Prey-No Obstacle • Prey-Barrier or Prey-Chasm

respectively - is represented in Fig. 7 as the outcome of competitive interaction

between the three perceptual schemas:

• If perceptual schema FP is instantiated. it generates a parameter d for the

distance of the prey. If d is small (d-) then the animal fixates. snaps and swallows

the prey to achieve the goal state. If d is larger (d+) then the animal approaches the

144

prey. The arrow to ~ indicates that the animal will return to the circle marked ~

above: so long as the prey remains visible. the animal will continue to approach it

until d is small enough for it to snap and swallow the prey. It might be argued that

both of the ~ -returns in Fig. 8 should be replaced by returns to a since a barrier

might be interposed or become visible after the animal has executed an approach

motor schema. In any case. such returns (to be discussed further below) represent

the behavioral fact that the toad seems to execute a single behavior (which may itself

be composite. directed by a coordinated control program. as in fixate~snap-Hwallow)

to completion. with the passage to the next requiring a fresh perceptual trigger. By

contrast. humans formulate some overall plan to be pursued and modulated in the

light of changing circumstances as when. going to get some object. we initiate a

search strategy should the object not be in its expected place.

• If perceptual schema barr is instantiated. it makes available 2 parameters. dp 0,

the distance of the prey behind the barrier. and h. the height of the barrier. If d po

is small or negative (dpo-). the toad approaches the barrier and proceeds as if no

barrier were present (transfer to ~). If the prey is further behind the barrier and

the barrier is low (dpo+. h-). the toad will approach and jump the barrier. while if the

barrier is high (dpo+. h+). the animal will detour. More specifically. instantiation of

the perceptual schema for a gap will elicit the motor schema for orienting to (OR g )

and approaching (Appr g ) that gap. while the absence of a gap. It seems necessary

to include the "gap" at the end of a barrier as also activating the gap perceptual

schema. In either dpo+ case. control is returned to b after execution of the specified

motor schema .

• Finally. if perceptual schema Chasm is activated. it makes available parameters

representing the depth d and width w of the chasm. If the depth is small (d-) the

toad will walk across the floor of the chasm; if the chasm is deep but not wide (w-. d+).

the toad will leap the chasm; but if the chasm is wide and deep (w+. d+). the animal

will not approach the chasm.

However. general considerations (Arbib 1981) suggest that the animal's behavior

should depend on the representation of the environment by an assemblage of

schemas embodying the spatial relations between multiple objects in the

environment. rather than sequential activation of a perceptual schema for one prey

and then at most one obstacle. In fact. in the description Lara et al. (1984) give of the

Figure 8. Computer simulation, using

a program elaborating the schemas of

Figure 7, of a toad's behavior in response

to a worm in the presence of barriers.

In (a), the toad always prefers the closer

gap; in (b), the further gap is deeper and

is the one chosen. (From Lara et al. 1984).

145

_11_.,

___ .'_ II

4:-. __ --1

--.-

• ~ .....

b

actual implementation of gap approach, they do use a mode of competitive interaction

which is only hinted at in the Fig. 7 by the various arrows labeled "" to express

competitive interactions. More generally, the idea seems to be that the gap schema is

instantiated for each gap in the environment, each schema is given an activity level

based on its position relative to toad, prey and other gaps, and it is the most active gap

schema which provides the parameters for the next motor schema activation. Once it

has executed this motor schema, the animal's behavior is determined ab initio by the

activation of its perceptual schemas in its new situation.

Note that this explicit view of motor schema activation by the "winner" (cf. the

Didday model of prey-selection) of perceptual schema competition obviates the

"return to f3" approach raised above, since we simply postulate that, once triggered, a

motor schema executes to completion with the parameters supplied by the perceptual

schemas, whereupon activation of perceptual schemas is re-initiated to trigger the

next motor schema. As noted above, this seems an appropriate model for frog and

toad - the schema assemblage is completely "refreshed" after each integral action.

This is in contrast to the action-perceptual cycle stressed in, e.g., human behavior, in

which activation of perceptual schemas serves to update an existant schema

assemblage and coordinated control program, rather than necessarily to create new

ones.

An example of two trajectories predicted by the model of Lara et a1. (1984) is

146

presented in Fig. 8. In (a), the "toad" chooses the gap nearer the prey, but in (b), the

presence of the wide gap in the further fence biases the animal's "choice" to favor

the other gap in the nearer fence. Clearly, such predictions can be used to test the

model, and data culled from observations of animal behavior in this fashion can be

used to refine the algorithmic specification of the constituent schemas, just as

psychophysical data can be used to tune schema-based models of visual function in

humans. Such specifications can serve as the endpoint of analysis for the ethologist

uninterested in neural mechanisms; but for the neuroethologist, they can provide

the formal "specification" of the task whose implementation in neural circuitry is to

be analyzed.

7. Path-Planning and Detours

The Path-Planning Model (Arbib and House 1987) is described in terms of an

array of elements which are somewhat more abstract than individual neurons. It

remains an open question as to how best to map this model onto a neural network, and

whether this can be done in a manner consistent with available physiology. We

associate with each position in the ground plane a vector to indicate the preferred

direction of motion of the animal were it to follow a path through the corresponding

point. For conceptual simplicity, the coordinate system used in this model is the

Cartesian (x,y) system. Our task with this model will be twofold: To specify how the

vector field is generated, and to specify how the vector field is processed to detennine

the appropriate parameters for the coordinated activation of motor schemas. In the

technical jargon of differential geometry, then, the neural surface corresponds to a

manifold representing space in some internal coordinate system, while the firing of

a group of neurons associated with a particular coordinate is to represent the vectors

of a tangent field, or flow. The question is how those local vectors are to be integrated

to detennine an overall trajectory for the animal.

Our provisional choice is as follows: A single prey will set up an attractant field,

in which from every point in the animal's representation of space there is an arrow

suggesting a choice of movement toward the prey, with the length of the vector (the

strength of choice for a movement in the given direction) being the greater, the

closer is the point to the prey. We have associated a repellant field with a single

fencepost, with the strength of the field contributing mostly to the determination of a

lateral movement relative to the position of the fencepost from the viewpoint of the

147

animal. Finally. the animal's representation of itself within this field consists of a set

of vectors radiating out in all directions from the animal's current position with a

decay similar to that for the prey field.

The total field may be interpreted as representing the "net motor effect" of the

scene upon the animal. whether the animal is an essentially ballistic creature like a

frog or a toad. or a more "tracking" creature like a gerbil. In the case of the gerbil

we would postulate that the vector field is integrated to yield a variety of trajectories.

with a weight factor for each trajectory. We would then see that this field has two

"bundles" of trajectories receiving high weight. that bundle which goes round the

left end of the barrier to approach the worm. and that which goes around the right

end of the barrier to approach the worm. Thus. if we change "worm" to "sunflower

seed." we would posit that the gerbil actually builds within its brain a representation

of the entire path. one of the paths is selected. and this path regulates the pattern of

footfalls that will move the animal along this trajectory. In yet more sophisticated

models. we could see the path not as being generated once and for all. but rather as

being dynamically updated on the basis of optic flow as the animal proceeds along a

chosen direction.

In toad. however. we postulate that the vector field is processed not to yield a

continuous trajectory - or a bundle of continuous trajectories of which one is to be

chosen - but rather serves to generate a map of motor targets. appropriately labeled

as to type. The divergence operator is a likely candidate for this form of processing.

Once a suitably constructed representation of a vector field is set up. the computation

of divergence is a simple local process which may be carried out in the parallel

distributed fashion associated with neural mechanisms. Further. the divergence of a

vector field is a scalar field. The negative of the divergence will contain peaks where

the flow lines in the field tend to converge and valleys where they tend to diverge.

Fig. 9 is displays the negative of the divergence of the net field for the cases of a

fence with a central gap (Fig.9a). a solid fence near to the prey (Fig.9b). a solid fence

behind a fence with a gap (Fig.9c). and a cage (Fig.9d). Our research has not yet

resulted in isolating the most suitable algorithm for extracting a path from data of

this sort. However. these preliminary results are suggestive of a strong agreement

with the behavioral data. In particular. the powerfully attractive quality of fence

gaps noted by Collett 1982 is especially apparent.

Figure 9.

.•••• f.' .... "" ... , "" .... ", " ... :::;-10::::: .. ,. ~, , ... • , " \ ~ 'I. •

• '"1 ,.', · . ~, "., ~ .... , "",

.. \' ..... " ,

148

"" i . i

.' -l I

_.",.""- I" ::::

.. , \ ",.

.,.\ "" ... .)

Although an analysis has not yet been completed to identify a specific means for

deriving motor activity from the vector fields, these figures indicate that the relevant information

is efficiently encoded by the vector model. (a) Fence with gap. (b) Fence near to prey. (c) Fence

with gap in front of a solid fence. (d) Cage. The model results are consistent with behavioral

results. (Arbib House 1987)

The vector model differs significantly from the simple orientation model of

Section 5. Here visual stimuli are not seen as setting up a simple decision surface

which can be processed to select among several optional actions. Rather, what is set

up is a spatially encoded map of potential motor activity which in some sense is the

net result of the interaction of all of the pertinent visual stimuli. Although in the

simple Cartesian representation used in this paper the vectors are described in terms

of components of forward and lateral motion, there is no reason to expect that the

nervous system would encode Vector quantities in this way. What is more likely is

149

that they would be encoded in terms of the various types of schematized motor

patterns available to the animal. For instance. a particular vector could be

envisioned as having components governing side stepping. turning. and snapping.

The coordinate system for such a vector field would. most appropriately. be body

centered rather than eye centered.

We close by pointing out the relevance of this model of detour behavior for

technological applications. Arkin 1988 discusses AURA. an AUtonomous Robot

Architecture which provides the control system for a mobile robot. A subsystem

called the Navigator conducts off-line path planning on the basis of a terrain map

using relatively conventional AI (Artificial Intelligence) techniques. Another

subsystem. called the Pilot. extracts more specific features. e.g.. landmarks. to

elaborate the plan into motor schemas. with perceptual schemas embedded within the

motor schemas. Such schemas include stay-an-path. move-to-goal. move-ahead.

find-landmark. and avoid-obstacles. What is significant here is that a potential field

is given for each motor schema. In this way. high-level path planning is

model-driven (at a level "above" that posited in Rana computatrix to date). but

execution is data-driven (though in the "trajectory" mode of the gerbil. rather than

the "ballistic" mode of the toad). This system has been used with a real mobile robot

traversing the open spaces and corridors of the University of Massachusetts at

Amherst. proving the relevance of Rana computatrix for perceptual robotics.

References

Amari. S-I. Arbib. M A (1977) Competition and cooperation in neural nets. In: Metzler.

J (ed) Systems neuroscience. Academic Press. London. New York. pp 119-165

Arbib. M A (1981) Perceptual structures and distributed motor control. In: Brooks. V B

(ed) Handbook of physiology - The nervous system II. Motor control

Physiological Society. Bethesda MD. pp1449-1480

American

Arbib. M.A. (1982) Modelling neural mechanisms of visuomotor coordination in frog

and toad. In Competition and Cooperation in Neural Nets (S. Amari and M.A. Arbib.

Eds.). Lecture Notes in Biomathematics Vol. 45. Springer-Verlag. pp. 342-370.

Arbib. M A (1987) Levels of modeling of mechanisms of visually guided behavior. Bch.

Brain Sci 10:407·465

Arbib. M A (in press) The Metaphorical Brain 2: An Introduction to Schema Theory

and Neural Networks Wiley Interscience.. New York

Arbib. M A. House, D H (1987) Depth and detours: an essay on visually-guided behavior.

150

In: Arbib, M A, Hanson, A R (eds) Vision, brain, and cooperative computation. A

Bradford Book/MIT Press, Cambridge,MA pp 129-163

Arkin, R.C., 1988, Neuroscience in motion: the application of schema theory to mobile

robotics, in Visuomotor Coordination: Amphibians, Comparisons, Models, and

Robots (I.-P. Ewert and M.A. Arbib, Eds.), Plenum Press.

Caine, H S, Gruberg, E R (1985) Ablation of nucleus isthmi leads to loss of specific

visually guided behavior in the frog Rana pipiens. Neurosci Lett 54:307-312

Cervantes-Perez, F (1985) Modelling and analysis of neural networks in the visuomotor

system of anuran amphibia. Ph.D. Thesis and COINS Technical Report 85-27,

Computer and Information Science Department, Univ of Massachusetts at Amherst

MA

Chipalkatti, R, Arbib, M A (1987a) The prey localization model: a stability analysis, Bioi

Cyber. 57:287-300.

Chipalkatti, R, Arbib, M A (1987a) The cue interaction model of depth perception: a

stability analysis. I Math Bioi (in press)

Collett, T. (1977) Stereopsis in toads Nature 267: 349-351

Collett, T (1982) Do toads plan routes? A study of the detour behaviour of Bufo Viridis.

I Comp Physiol 146: 261-271

Collett, T., (1987) Binocular depth vision in arthropods, Trends in Neuroscience , Volume

10, Number I, pp. 1-2.

Collett, T., and Udin, S. (1983)The role of the toad's nucleus isthmi in prey-catching

behaviour. Proceedings of second workshop on visuomotor coordination in frog

and toad: Models and Experiments (R. Lara and Arbib, M.A., Eds).

COINS-Technical Report 83-19, University of Massachusetts, Amherst,

Massachusetts

Collett,T.S., Udin, S.B. and Finch, D.I.(l986) A possible mechanism for stereopsis in

anurans.

Dev, P. (1975) Perception of depth surfaces in random-dot stereograms: A neural

model Int. 1. Man-Machine Studies 7: 511-528.

Didday, R L (1970) The simulation and modelling of distributed information processing

in the frog visual system. Ph.D. Thesis, Stanford University

Didday, R. (1976) A model of visuomotor mechanisms in the frog optic tectum. Math

Biosci 30: 169-180

Epstein, S (1979) Vermin users manual. Unpublished MS Thesis, Department of

Computer and Information Science, Univ of Massachusetts at Amherst MA

Hirai, Y., and Fukushima, K. (1978) An inference upon the neural network finding

151

binocular correspondence Bioi. Cybernetics , 31 :209-217.

House, D. (1982) The frog/toad depth perception system - A cooperative/competitive

model In Proceedings of the workshop on visuomotor coordination in frog and

toad: Models and Experiments (Arbib, M.A., Ed.). COINS Technical Report 82-16,

University of Massachusetts, Amherst, Massachusetts

House, D. (1984) Neural Models of Depth Perception in Frog and Toad, Ph.D. Dissertation,

Department of Computer and Information Science, University of Massachusetts at

Amherst

House, D (1988) A model of the visual localization of prey by frog and toad. Bioi Cyber

58: 173-192

Ingle, D. (1976) Spatial visions in anurans. In: K. Fite (ed) The amphibian visual

system, Academic Press: New York, pp 119-140

Ingle, D J (1983) Visual mechanisms of optic tectum and pretectum related to stimulus

localization in frogs and toads. In: J -P Ewert, Capranica, R R, Ingle, D J (eds)

Advances in vertebrate neuroethology . Plenum Press, New York, pp 177-226

Julesz, B. (1971) Foundations of Cyclopean Perception Chicago: Univ. of Chicago Press

Lara, R, Arbib, M A (1982) A neural model of interaction between tectum and

pretectum in prey selection. Cognition and Brain Theory 5:149-171

Lara, R, Arbib, M A, Cromarty, A S (1982) The role of the tectal column in facilitation of

amphibian prey-catching behaviour: a neural model J Neurosci 2: 521-530

Lara, R, Carmona,M, Daza, F, Cruz, A (1984) A global model of the neural mechanisms

responsible for visuomotor coordination in toads. J Th Bioi 11 0: 5 87- 618

Marr, D., and Poggio, T. (1977) Cooperative computation of stereo disparity. Science

194: 283-287.

Marr, D. and Poggio, T. (1979) A computational theory of human stereopsis, Proc.

Roy. Soc. Ser. B , 204:301-328.

Mayhew, J.E.W. and J.P. Frisby (1981) Towards a Computational and Psychophysical

Theory of Stereopsis, Artificial Intelligence , 17:349-385.

Nelson, J.I.(1975) Globality and stereoscopic fusion in binocular vision!. Th Bio. 49:1-88

Prazdny, K.(1985) Detection of Binocular Disparities, Bioi. Cybern. 52:387-395.

Prazdny, K. (1987) On the coarse-to-fine strategy in stereomatching, Bull. Psycho nomic

Soc. , 25:92-94.

R5ssel, S. (1983) Nature 302:821-822.

Sperling, G. (1970) Binocular vision: a physical and a neural theory Am. J. Psych. 83:

461-534

Role of Basal Ganglia in Initiation of Voluntary Movements

Okihide Hikosaka Department or Physiology

Toho University School or Medicine Tokyo,Japan

Abstract. A motor system called the basal ganglia facilitates movement initiation by removing its powerful inhibition on other motor areas. It may also facilitate activity in the cerebral cortex with disinhibition and ensure sequential processing of motor signals.

Multiple brain areas related to saccadic eye movement

A brain structure called the basal ganglia is an assembly of several nerve cell nuclei located at the base of the brain. It has been well known to clinicians that the basal ganglia are indispensable for us to move our body parts. A number of brain diseases, such as Parkinson's disease or Huntington's disease, affect the basal ganglia and thus render victims unable to move or unable to supress involuntary movements. Different types of approaches have clarified that the basal ganglia are intricately connected with the cerebral cortex and that neurons in different nuclei of the basal ganglia indeed carry motor signals [2]. Given many beautiful brain architectures repeatedly revealed with modem anatomical techniques, however, I am aware that our under­standing of the brain is far from complete. It is probably in the connection of neurons as a whole, rather than fine structures of individual brain nuclei, where we can find an appropriate language to describe the functions of the brain. In this article I attempt to characterize the motor function of the basal ganglia, especially with reference to the neural networks between the basal ganglia and the cerebral cortex.

A suggestion that the basal ganglia might be related to eye movement came from the discovery that the substantia nigra, a part of the basal ganglia, has fiber connections to the superior colliculus [3,14]. A number of brain areas are related to saccadic eye movements. In the cerebral cortex are the frontal eye field [1], parietal association cortex, and the recently found supplementary eye field [19]. The frontal eye field, especially, has been related to "voluntary initiation" of saccadic eye movements. A subcortical structure in the midbrain called the superior colliculus or optic tectum is another important area for saccadic eye movement [21, 25].

The output of the superior colliculus is directed to the brainstem reticular formation which contains burst neurons generating a pulse for a saccadic eye movement. The signal processing occurring along this tecto-reticulo-oculomotor pathway is purely motor. The question I wanted to solve was one step further upstream in the oculomotor signal processing. It was how a saccadic eye movement is initiated, not just how it is generated.

Superior colliculus mediates saccadic oculomotor signal

Before going into the basal ganglia, I would like to show the basic function of the superior colliculus (Fig. 1). The superior colliculus is a small protrusion of the brainstem and is a laminated structure. The superficial layer receives direct fiber connections from the retina in a retinotopic manner, and the cells there respond to a visual stimulus within a small area in the contralateral

154

visual field called "visual receptive field" and are therefore purely visual. The intermediate layer, on the other hand, is largely motor. The cells show a burst of spike

activities before a saccadic eye movement if it is directed to an area in the contralateral visual field called "movement field." This saccade motor signal is sent to the brains tern reticular formation and is shaped up to be a pulse output to the extraocular muscles. There is a beautiful matching between the visual receptive field in the superficiallayer and the movement field in the underlying intermediate layer.

It might appear from this scheme that a visual signal originating from the retina could be converted to a saccade motor signal via a top-to-down interlaminar connection. This might occur in special occasions, especially in lower animals, but what's happening is not really that simple [21]. The superior colliculus is a crossroad of sensory-motor signals and has massive, heteroge­neous connections with other cortical and subcortical structures. Two major inputs to the intermediate layer have been identified, those from the frontal eye field [12] and those from the substantia nigra [14], and they have contrasting effects on the superior colliculus.

FIXATION SACCADE

1 TA~ET

l' ~~ET

-I

Fig. 1 Role ofthe superior colliculus in the initiation of visually guided saccade. If a light stimulus (target) is presented in the right visual field (left), cells within a small area in the superficial layer of the left superior colliculus (denoted as V) are activated. This is followed by a burst of spikes in the cells in the underlying intermediate layer (S)(right). This information is sent to the brainstem saccade generator on the right side and is used to generate a saccade to the target. The impulse activities of these cells are shown schematically at the bottom.

It is not easy to demonstrate that a neuron has a motor role. The animal must be free to move voluntarily, yet the relationship between the electrode and the neuron must be kept stabilized. The animal's head must be immobilized especially when eye movements are studied. The animal must be trained to repeat voluntary movements. Robert H. Wurtz [24] devised an ingenious method to

155

control animal's eye movements, as shown in Fig. 2. Here, if a monkey, sitting in a chair, presses a lever, a small spot of light appears at the center of the screen in front of the monkey. After a random period of time this light spot becomes slightly dim for a brief period, and if the monkey releases his hand from the lever he gets rewarded with a drop of water. If the spot jumps to another location, the monkey naturally moves his line of sight to refixate the spot by making a saccadic eye movement.

While the monkey was performing this kind of task, we inserted a microelectrode into the basal ganglia to record electrical activity of single neurons. This is how we investigate information carried by neurons.

Substantia nigra inhibits superior coIIiculus

The pars reticulata of the substantia nigra (SNr) is one of the busiest areas in the brain. Neurons in this nucleus show action potentials incessantly with the rate of up to 100 times per second. They do so even when the animalis sleeping. Such high background activity point to an important aspect of basal ganglia function, which I will show later.

H----.~ ________ ~/ ,-_____ --J!

V--.J

Fig. 2 Saccade task. H and V indicate schematic horizontal and vertical eye positions.

Fig. 3 shows spike activity of a single substantia nigra neuron. The monkey repeated saccades 12 times to a visual target contralateral to the side where the neuron was recorded. The results are shown as a raster display. Like other substantia nigra neurons, this neuron showed tonic, high frequency spike discharges, but stopped discharging after the onset of the target [6-8]. The cessation of the cell activity was followed by a saccade to the target.

Nearly half of substantia nigra neurons showed essentially the same pattern of activity change. It could be a visual response to the saccade target or could be a motor response time-locked to the saccade itself. Now, what does this mean?

156

I can answer this question by comparing the substantia nigra activity with superior colliculus activity, as illustrated in Fig. 4. The upper part shows activity of another substantia neuron, and the lower part shows activity of a superior colliculus neuron. They are aligned on the onset of saccade, but in this case saccade to a remembered target. I compared these two neurons because Robert Wurtz and I were able to prove electrophysioiogically that the substantia nigra neuron pro­jected its axon to the site where the superiorcolliculus neuron was recorded, therefore presumably connecting to this very neuron [9].

FIXATION POINT F -.Jr------"

T~---------------TARGET POINT

EYE POSITION J ,

FIT , ... , " , 1

, ' , .. \ I

.....

I DIM :.wARD

f,------s

hr' ht

",

Fig. 3. Saccade-related activity of a substantia nigra pars reticulata cell. Its spike activity is shown as raster displays, each dot indicating a single action potential. The monkey repeated saccades to a contralateral target, and the results are aligned on the onsets of the target (left) and on the onsets of the saccades (right). In the left raster, the onsets of the saccades are also indicated by small vertical bars. Below the rasters are averaged spike histograms and cumulative spike histograms. Calibration on the left of the histograms indicates 100 spikes/sec/trial.

A striking feature is evident from this comparison. While the substantia nigra neuron was tonically active, the superior colliculus neuron was nearly silent. Before the saccade, the substantia nigra neuron stopped discharging while the colliculus neuron showed a burst of spike activity. This result strongly suggested that the nigrocollicular connection is inhibitory [15].

When the monkey is not making an eye movement, substantia nigra neurons keep inhibiting superior colliculus neurons with their high background activity. In fact, the relationship is probably reversed: because of the tonic inhibition, the superior colliculus neurons are disenabled so that no saccade is elicited. Once the substantia nigra neurons stopped discharging and the tonic inhibition is removed, the superior colliculus neurons get ready to be excited and therefore are likely to produce a saccade.

The next question was obviously how the substantia nigra neurons stop discharging. The substantia nigra is one of the two major output stations in the basal ganglia, andis known to receive fiber connections from other parts of the basal ganglia. The caudate nucleus is one of these areas.

157

Basal ganglia may initiate a movement by disinhibition

In contrast to the substantia nigra, the caudate nucleus is an extremely quiet area. When inserting a microelectrode into the caudate, we passed by many neurons without noticing their presence simply because they did not show a single action potential. Nonetheless, Masahiro Sak­amoto and I found a cluster of neurons that were related to saccadic eye movements [4].

SNr

sc

EYE H

{)

1··.·

~ D

v-----...+-----

Fig.4. A substantia nigra cell (SNr) decreases while a superior colliculus cell (SC) increases its activity before a contralateral saccade. A vertical bar on each raster line indicates the onset of target

F'~ ________ ~ __ __

T ______ ~.L-________________ .... ~

100 ItSlC ILC-'.

F'~~== ,,, '/ I' , ,I

,'1,1

r ". ':',,':'1.:'

100 "SEC

Fig. 5. Activity of a caudate cell selectively related to memory-guided saccade. On the left, while the monkey was fIxating, another spot of light (T) was flashed indicating the position of a future target. The monkey remembered its position and, when the fIxation point (F) disappeared, made a saccade to the position. On the right, the target appeared as the fIxation point went off; a following saccade was guided by the visual information. Calibration: 50 and 100 spikes/sec/trial.

158

Fig. 5 shows an example. This typically quiet caudate neuron showed spike discharges just before a saccade to a contralateral target, only when it was remembered (left), not actually present (right). If we stimulated the site where this caudate neuron was recorded, the activity of saccade­related nigra neurons was suppressed. This experiment strongly suggested that the saccade­related depression of substantia nigra cell activity is the result of an inhibition by caudate neurons including this one.

From these experiments emerged a scheme shown in Fig. 6. The caudate nucleus and the substantia nigra are both included in the basal ganglia. They constitute two serial inhibitions: caudate-nigral and nigro-collicular. The nigro-collicularinhibition is tonically active whereas the caudate-nigral inhibition becomes active only phasic ally. Therefore, disinhibition is the way in which the caudate acts on the superior colliculus; and the substantia nigra determines the depth of the inhibition to be released. Functionally, this disinhibition acts to open the gate for saccade initiation. But this is not the sole function of the basal ganglia, as I will show later.

Fig. 6. Neural mechanism in the basal ganglia for the initiation of saccade. Excitatory and inhibitory neurons are indicated by open and filled circles, respectively. SC: superior colliculus. FEF: frontal eye field. PS: cortical area around the principal sulcus. SC: saccade generator in the brainstem reticular formation. The "axon" of substantia nigra (SNr) neuron is made thicker than others to indicate its high background activity.

Fig. 7 extracts the basic mode of operation of the basal ganglia, that is, disinhibition. There are two important aspects in this scheme: a tonic component and a phasic component. The tonic component acts to suppress the output. This is necessary because the superior colliculus is continuously under excitatory bombardments from a number of brain areas and without this suppressive mechanism the animal would be forced to make saccadic eye movements incessantly and uncontrollably. In pharmacological experiments Robert Wurtz and I have shown that this is indeed the case [10, 11]. The stronger the tonic component, the more effective is the suppression.

159

The second, phasic component opens the inhibitory gate, producing an output. Interestingly, the effectiveness of the phasic component depends on the strength of the tonic component: the stronger the tonic inhibition, the more effective and more clear-cut becomes the output which is released from the tonic inhibition.

Basal ganglia activity selective for memory-guided movement

We now face the fact that the basal ganglia system is only a part of the brain, as illustrated in Fig. 8. It interacts with other brain areas in a number of ways, and this is where the basal ganglia system reveals its unique role. As I indicated before, the superior colliculus is the site where many different types of information converge; the retina and the cerebral cortex including the frontal eye field and parietal association cortex.

The unique feature of the basal ganglia input is its suppressive nature. Most of the other areas provide the superior colliculus with excitatory signals: each of them tells or suggests the colliculus to make a saccade. Motor signals are distributed everywhere in the brain: there is probably no area that emits a holistic motor command. From such chaotic urges to move comes the necessity and importance of the basal ganglia suppressive mechanism.

Caudate

(

SNr Superior Colliculus

I-----4C===

Fig. 7. Tonic inhibition (top) and disinhibition (bottom) of the superiorcolliculus by the basal ganglia. High activity is indicated by a thicker "axon".

As I have shown, the suppression is not the sole function of the basal ganglia: it removes the suppression and thereby contributes to the movement initiation. An important question is, "How unique is the motor signal originating in the basal ganglia?"

I have indicated that the basal ganglia contain a number of saccade-related neurons. However, as already shown in Fig. 5, the neural activity in the basal ganglia was often selective for a saccade which was made to the remembered position of a visual stimulus. Such a neuron showed no activity if the monkey made saccades to a visual target no matter where the target appeared (Fig. 5, right).

This kind of selectivity at first appeared peculiar, probably because we had been implicitly postulating holistic command neurons. If we look at the signal from the input side, not from the output side, the selectivity may not be peculiar or surprising: a memory-related signal is directly used to initiate a saccadic eye movement and the saccade anticipates the appearance of the target (Fig. 5, left).

stimulus selection T T

[ visual inputs

160

short-term memory

(+ )

Superior Colliculus

I----------'~~[ saccadic eye) movement

Fig. 8. Hypothetical neural mechanisms underlying initiation of saccade.

Basal ganglia may modulate activity in cerebral cortex

A question then arises: "Is such memory-related or anticipatory activity used just for prepara­tion or initiation of movements?" I have characterized the basal ganglia as a serial disinhibitory mechanism through which information is passed from the cerebral cortex to lower motor centers. However, if we look at the fiber connections of the basal ganglia, a different and more complex scheme emerges, as shown in Fig. 9.

In addition to the superior colliculus, the substantia nigra projects to parts of the thalamus [13]. A considerable portion of the thalamus functions as relay stations of specific sensory information, but the parts receiving basal ganglia information are called non-specific thalamic nuclei. In short, their functions are unknown. They are mutually connected with the cerebral cortex [18]. The non­specific thalamic nuclei in tum project back to the caudate [16]; the cerebral cortex, especially non-specific association cortex, also projects to the caudate [17, 20]. These connections would complete the neural circuits involving the basal ganglia. Although admittedly oversimplified, this scheme gives us many hints about how the basal ganglia might work.

The first hint derived from this scheme is that the basal ganglia have access to the neural events in the cerebral cortex. The basal ganglia activity may not simply be the result of the cortical activity: it could change the cortical activity by the return connection. Here I assume that the mutual connections between the thalamus and the cerebral cortex are excitatory. Also, I assume that there are excitatory mutual connections within the cerebral cortex. Such mutual excitation or positive feedback would act to hold neural information and subserve the neural basis of memory.

Now let us concentrate on the connection from the basal ganglia to the cerebral cortex (Fig. 10). Here the target of the basal ganglia disinhibitory mechanism is the thalamo-cortical circuits. The substantia nigra [22], and probably also the globus pallidus [23], would normally keep suppress­ing the thalamo-cortical activity. If caudate neurons fire, the tonic inhibition would be removed transiently and the thalamo-cortical activity would be set off. Is there such activity in the caudate?

161

Masahiro Sakamoto and I found an interesting group of caudate neurons [5], and an example is shown in Fig. 11. In the delayed saccade task, a spot of light is flashed while the monkey is fixating to give him a future target position. A number of caudate or substantia nigra neurons responded to such a target cue if it was in the cell's receptive field (Fig. 11, left); but if the same spot was given at the end of the fixation period so that the monkey no longer needed to remember its location and just simply responded to it by making a saccade, the neuron never responded to the spot of light (Fig. 11, right). It was as if the cell's activity was used to encode the stimulus location into memory so that the monkey could use it afterwards.

I speculate that the caudate activity might be explained by the scheme in Fig. 10. The basal ganglia normally suppresses the activity of thalamo-cortical circuits underlying memory. but

anticipation.

Sensory _In.:,.p_ut_s ___ -"\.

Limbic Inputs ':"-'---f

I

Caudate

Thalamus

SNr

'-'--------SNc

Frontal Eye Field

Superior Colliculus

Saccadic Eye Movement

Fig. 9. Interaction between the basal ganglia and the cerebral cortex. The cerebral cortex and the thalamus are assumed to be mutually connected in an excitatory manner so that they are shown as a single area with an excitatory feedback. SNc indicates the substantia nigra pars compacta which exerts strong modulatory effects on a large part of the basal ganglia; this is distinct from the substantia nigra pars reticulata (SNr) which is a major output area of the basal ganglia.

I would like to point out another aspect of this collateral projection. Since the main stream of the basal ganglia is directed to the lower motor center, the information directed to the thalamo­cortical circuit could be regarded as "corollary discharge." The corollary discharge is often postulated such that the sensory systems know what kind of movement is going on and thereby can change their coordinate systems or sensitivities beforehand. I agree that this function would be important, but it is still bound to the present and past. In view of the predictive nature of basal ganglia activities, the function of this corollary discharge should be extended to the future. Since basal ganglia activity is already anticipating future events and preparing for next movements, the corollary discharge would evoke cortical activity anticipating further in the future.

This kind of process would go on sequentially, and this is probably what underlies a complex movement.

162

Cortex-Thalamus ry Caudate I

~.

SNr

Fig. 10. The cerebral cortex-thalamus as a target of the basal ganglia.

I /

F~""""~ ____ __ F~""""~ ____ _ T ____ ~.L-______________ ~ ____ __ T ________________ •• __ ~ .... _

[~ ILC-22S 100 "SEC

Fig. II. Memory-contingent visual response of a caudate cell. The neuron responded to a spot oflight (n only when the monkey had to remember its location as the target for a future saccade (left), but not when the monkey responded to it immediately (right).

163

Neural mechanism of learned / voluntary movement

If we incorporate the pathway returning to the basal ganglia, yet another mode of basal ganglia operation could be speculated (Fig. 12). This is a loop including two inhibitions, and could act as a flip-flop circuit. In the resting state, the thalamo-cortical activity would be suppressed and hence the caudate receives no excitation. If somehow the caudate is excited, the substantia nigra output would be suppressed and the thalamo-cortical activity would be released from the inhibition. This state is stable because the substantia nigra would be kept inhibited by the caudate. This mechanism would ensure that a selected cortical activity is maintained for a while, probably until a next trigger signal is fed in.

Cortex-Thalamus

caudat~. SNr

Fig. 12. Basal ganglia-cortex loop system may act to stabilize neural information as a step of sequential motor acts.

In fact, some of the caudate neurons maintain activity as if holding specific information, as illustrated in Fig. 13. For example, this neuron's activity was set off when the target disappeared and the monkey made a saccade to search for it, and continued until the expected target appeared. This is the period when the monkey had to concentrate on searching, presumably with the search image of the target. It would be unwise to switch to another state, and the basal ganglia flip-flop pathway would prevent this.

164

/ F=-liiiIiiIliiiIiiIiiiiiiiIiL~ __ _ T ____ ~IL_ ____________________ ~.,. .... __ ___

I '" ,. __ " •• - .... _ ...... IOt," I I IIIN I. II , •••• _ •• I .... • ........... . I "" •. " ..... . I "t ... "." ••••••

' •... :,',. ~""';'''I''''~ , \ .............. . .. I .111. ",., ". I II ...

• 1 •• :.:: .... : .. ":.:::::" \ .. ... . .............. .

ILC-I&8 100 "SEC

Fig. 13. Caudate nemal activity related 10 expectation of visual target. A target appeared after a long time gap after the fixation point went off. and the monkey. after making a saccade 10 the remembered position (small vertical bar). waited for the target; the neuron showed discharges continually until the target appeared.

A considerable portion of caudate neurons show activity before an external event: they know what happens next. With such predictive information, the basal ganglia may allow a movement to be triggered.

Fig. 14 summarizes this review. There could be complex interactions between sensory input, limbic input, internal cortical activity, basal ganglia activity, and motor output.

(1) Sensory input may trigger interrial cortical activity. (2) Cortical activity may allow sensory signal to go through (stimulus selection or selective

attention). (3) Basal ganglia may allow cortical activity (by disinhibition). (4) Basal ganglia may hold cortical activity (by flip-flop operation). (5) Basal ganglia activity is influenced by cortical activity. (6) Basal ganglia and hence cortical activity are modulated by limbic input. (7) Basal ganglia may open the gate for movement based on predictive, internal information.

I have described short-term changes of basal ganglia neural activities. However, prediction or anticipation is based on long -term memory of task procedures. One might think that the long-term memory is just the result of cortical activities. But now that we have gone through several possible interactions in the brain, it seems no longer tenable that memory is created solely in the cerebral cortex. Even if we accept the hypothesis that plastic changes in synapses occur in the cerebral cortex, but not in the basal ganglia, the basal ganglia could still be as important as the cerebral cortex for the formation of memory since it ensures the signals go through repeatedly in the cerebral cortex until the involved synapses are structurally enhanced.

165

Sensory System Cortex

Thalamus

SNr

SNc

Fig. 14. Complex neural networks underlying voluntary saccadic eye movement.

Superior Colliculus

The prediction of future events is based on experience that has been repeated many times. How the basal ganglia react in a given circumstance, therefore, depends on such experience and determines how the animal reacts. This aspect is especially important when a young, growing animal acquires a variety of movement pattern. It may determine "habit" or "movement reper­tory." Furthermore, in view of the possible contribution of the basal ganglia to internal, cortical activity, this process may involve how the animal reacts internally or mentally. It may even determine "way of thinking" or "character."

References

1. Bruce, C.J. and Goldberg, M.E. Primate frontal eye fields. I. Single neurons discharging be­fore saccades. J. Neurophysiol. 53: 603-635, 1985.

2. DeLong, M.R. and Georgopoulos, A.P. Motor functions of the basal ganglia. In: Handbook of Physiology, The Nervous System, edited by V.B. Brooks. Bethesda, MD: Am. Physiol. Soc., 1981, sect. 1, part 2, vol. II, chapt. 21, pp. 1017-1061.

166

3. Graybiel, A.M. Organization of the nigrotectal connection: an experimental tracer study in the cat. Brain Res. 143: 339-348, 1978.

4. Hikosaka, O. and Sakamoto, M. Cell activity in monkey caudate nucleus preceding saccadic eye movements. Exp. Brain Res. 63: 659-662, 1986.

5. Hikosaka, O. and Sakamoto, M. Neural activities in the monkey basal ganglia related to atten­tion, memory and anticipation. Brain Dev. 8: 454-462,1986.

6. Hikosaka, O. and Wurtz, R.H. Visual and oculomotor functions of monkey substantia nigra pars reticulata. I. Relation of visual and auditory responses to saccades. J. N europhysiol. 49: 1230-1253,1983.

7. Hikosaka, O. and Wurtz, RH. Visual and oculomotor functions of monkey substantia nigra pars reticulata. II. Visual responses related to fixation of gaze. J. Neurophysiol. 49: 1254-1267, 1983.

8. Hikosaka, O. and Wurtz, R.H. Visual and oculomotor functions of monkey substantia nigra pars reticulata. III. Memory-contingent visual and saccade responses. J. Neurophysiol. 49: 1268-1284,1983.

9. Hikosaka, O. and Wurtz, RH. Visual and oculomotor functions of monkey substantia nigra pars reticulata. IV. Relation of substantia nigra to superior colliculus. J. Neurophysiol. 49: 1285-1301,1983.

10. Hikosaka, O. and Wurtz, R.H. Modification of saccadic eye movements by GAB A-related substances. I. Effect of muscimol and bicuculline in the monkey superior colliculus. J. Neuro­physiol. 53: 266-291, 1985.

11. Hikosaka, O. and Wurtz, RH. Modification of saccadic eye movements by GAB A-related substances. II. Effects of muscimol in the monkey substantia nigra pars reticulata. J. N europhys­iol. 53: 292-308, 1985.

12. Huerta, M.P., Krubitzer, L.A. and Kaas, J.H. Frontal eye field as defined by intracortical micro stimulation in squirrel monkeys, owl monkeys, and macaque monkeys: I. subcortical connections. J. Compo Neurol. 253: 415-439,1986.

13. Ilinsky, I.A., Jouandet, M.L. and Goldman-Rakic, P.S. Organization of the nigrothalamo­cortical systems in the rhesus monkey. J. Compo Neurol. 236: 315-330, 1985.

14. Jayaraman, A., Batton, RR and Carpenter, M.B. Nigrotectal projections in the monkey: an autoradiographic study. Brain Res. 135: 147-152, 1977.

15. Karabelas, A.B. and Moschovakis, A.K. Nigral inhibitory termination on efferent neurons of the superior colliculus: an intracellular horseradish peroxidase study in the cat. J. Compo Neurol. 239: 309-329, 1985.

16. Parent, A., MaCkey, A. and De Bellefeuille, L. The subcortical afferents to caudate nucleus and putamen in primate: a fluorescence retrograde double labeling study. Neurosci. 10: 1137-1150,1983.

167

17. Percheron, G., Yelnik, J. andFran~ois, C. A Golgi analysis of the primate globus pallidus. ill. Spatial organization of the striato-pallidal complex. J. Compo Neurol. 227: 214-227, 1984.

18. Schell, G.R. and Strick, P.L. The origin of thalamic inputs to the arcuate premotor and supplementary motor areas. J. Neurosci. 4: 539-560, 1984.

19. Schlag, 1. and Schalg-Rey, M. Evidence for a supplementary eye field. J. Neurophysiol. 57: 179-200,1987.

20. Selemon, L.D. and Goldman-Rakic, P.S. Longitudinal topography and interdigitation of cor­ticostriatal projections in the rhesus monkey. 1. Neurosci. 5: 776-794, 1985.

21. Sparks, D.L. Translation of sensory signals into commands for control of saccadic eye move­ments: role of primate superior colliculus. Physiol. Rev. 66: 118-171, 1986.

22. Ueki, A. The mode of nigro-thalamic transmission investigated with intracellular recording in the cat. Exp. Brain Res. 49: 116-124, 1983.

23. Uno, M. and Yoshida, M. Monosynaptic inhibition of thalamic neurons produced by stimu­lation of the pallidal nucleus in cats. Brain Res. 99: 377-380, 1975.

24. Wurtz, R.H. Response of striate cortex neurons during rapid eye movements in the monkey. J. Neurophysiol. 32: 975-986,1969.

25. Wurtz, R.H. and Albano, J.E. Visual-motor function of the primate superior colliculus. Annu. Rev. Neurosci. 3: 189-226, 1980.

Neural Mechanisms of Attention in Extrastriate Cortex of Monkeys

Robert Desimone Laboratory of Neuropsychology, NIMH, Bethesda

Jeffrey Moran Laboratory of Clinical Studies, DlCBR, NIAAA, Bethesda

Hedva Spitzer Technion, Haifa, Israel

Abstract: Neuronal recordings in extrastriate cortex of

awake monkeys have shown that sensory processing is under

the control of selective attention. Selective attention

serves to remove irrelevant information from the receptive

fields of extrastriate neurons and sharpen their selectivi­

ty for visual features. These effects of attention may

explain both why we have little awareness of unattended

stimuli, and why our resolution of spatial location and

visual features is improved inside the focus of attention.

In spite of the fact that the computing architec­

ture of the brain is massively parallel, our ability to

process incoming sensory information in parallel is in some

ways surprisingly limited. It is not possible, for exam­

ple, to comprehend several conversations simultaneously, or

to recognize within the same instant more than one or two

objects in a crowded scene. Thus, much of the information

impinging on our sensory surfaces must ultimately be fil­

tered out centrally.

In vision, some filtering of sensory information

occurs preattentively as a result of both early neuronal

mechanisms, such as edge enhancement, and later mechanisms,

such as figure/ground separation, or image segmentation.

Yet, even after figure/ground separation, there are normal­

ly far too many figures to be fully processed and reach

170

awareness at once. Attentional processes are required to

select which of the many figures will achieve preeminence

at a given moment. A similar process is required in oculo­

motor and motor systems, since it is not possible to orient

the eyes or body to every potential target in the environ-

ment at the same time. Indeed, attentional processes in

the sensory and motor systems must be closely linked, since

orienting responses generally follow shifts of attention.

In our work, we have tried to understand how atten­

tion affects sensory processing at the level of individual

neurons in the extrastriate cortex of awake, behaving mon­

keys. The cortical areas we studied lie along the occipi­

totemporal pathway, which is critical for object recogni­

tion [5,10,11,12,25,26]. The pathway begins with projec­

tions of the primary visual cortex (VI) to areas V2 and V3,

and continues with the projections of these areas to area

V4, and of V4 into the inferior temporal cortex. Each of

these forward-going connections is reciprocated by a

backward-going connection.

One striking feature of the occipitotemporal pathway

is that the average receptive field size increases from one

area along the pathway to the next. In the central visual

field representation of striate cortex, receptive fields of

individual neurons are often less than 0.50 wide, whereas

in V2 they are typically 0.5-1.00 , in V4 they are 1-40 ,

and in the inferior temporal cortex they are often 250 or

more [5]. One advantage of large receptive fields is that

they allow for cells to communicate information about the

global properties of objects rather than just local fea-

tures. Another potential advantage is that they may pro-

vide a representation of objects that is invariant over

changes in retinal position [7]. Yet, with respect to

attention, the increase in receptive field size along the

occipitotempora1 pathway presents a paradox, since it im­

plies that cells are confronted by more and more informa­

tion at successive stages of visual processing, rather than

less.

To gain some insight into how the visual system con­

trols processing within large receptive fields, we first

studied the effects of attention on neuronal responses when

171

there was more than one stimulus located inside a neuron's

receptive field [13].

ment was as follows.

The general strategy of the experi­

While the animal fixated on a small

target, we first mapped the :::ell's receptive field with

bars of various colors, orientations, and sizes, generated

on a computer graphics display. Based on the cell's re­

sponses, we selected a set of stimuli each of which was

effective in eliciting a response from the cell and a set

that was ineffective. An effective stimulus was then pre­

sented at one location inside the receptive field and an

ineffective stimulus at another. The monkey was trained on

a task that required it to attend to the stimulus at one

location and ignore the stimulus at another. The monkey's

attention was "covert",

ted. After a block

switch its attention

trials with attention

since eye movements were not permit­

of trials, the monkey was cued to

to the other location. Blocks of

directed towards one or the other

location were alternated repeatedly. Since identical senso­

ry conditions were maintained in the two types of blocks,

any difference in the response of the cell could be attrib­

uted to the effects of attention.

The task used to focus the animal's attention on a

particular location was a modified version of delayed

matching-to-sample. While the monkey held a bar and gazed

at a fixation target (fixation was monitored with a magnet­

ic search coil), a sample stimulus appeared briefly at one

location and half a second later a test stimulus appeared

briefly at the same location. When the test stimlus was

identical to the preceding sample (a "matching" trial), the

animal was rewarded if it released the bar immediately;

when the test stimulus differed from the sample (a "non­

matching" trial), the animal was rewarded only if it de­

layed release for 700 msec. Irrelevant stimuli were pre­

sented at the unattended location simultaneously with the

presentation of both the sample and test stimuli at the

attended location, affording two opportunities to observe

the effects of attention on each trial.

The first cortical region in which we studied cells

was area V4, the last known retinotopically organized area

within the occipitotemporal pathway. We found that the

FIX + •

1 0

.- RF

. - - - - - - - - -+- -.

l~ __ ~ ____ !

172

~ EFFECTIVE ~ SENSORY

STIMULUS

O INEFFECTIVE SENSORY STIMULUS

• ,----------, I • , I , I I I I I

'--- '

Fig. 1. Effect of selective attention on the responses of a neuron in extrastriate area V4. The neuronal respons­es shown are from when the monkey attended to one location inside the receptive field (RF) and ignored another. At the attended location (circled), two stimuli (sample and test) were presented sequentially, and the monkey responded differently depending on whether they were the same or different. Irrelevant stimuli were presented simultaneous­ly with the sample and test but at a separate location in the receptive field. In the initial mapping of the field, the cell responded well to red bars but not at all to green bars. A horizontal or a vertical red bar (effective senso­ry stimuli) was then placed at one location in the field and a horizontal or a vertical green bar (ineffective senso­ry stimuli) at another. When the animal attended to the location of the effective sensory stimulus at the time of presentation of either the sample or test, the cell gave a good response (left), but when the animal attended to the location of the ineffective stimulus, the cell gave only a small response (right) even though the sensory conditions were identical to the previous condition. Thus, the re­sponses of the cell were determined predominantly by the attended stimulus. The horizontal bars under the histo­grams indicate the 200 msec period when the sample and test stimuli were on. Because of the random delay between the sample and test presentations, the histograms were synchro­nized separately at the onsets of the sample and test stimu­li (indicated by the vertical dashed lines) .

173

locus of the animal's attention within the receptive field

of the neuron from which we were recording had a large

effect on the neuron's response. When an effective and

ineffective sensory stimulus were presented within the

receptive field, and the animal attended to the effective

stimulus, the neuron responded well. When the animal at­

tended to the ineffective stimulus, however, the neuron's

response was greatly attenuated, even though the effective

(but ignored) sensory stimulus was still present within the

receptive field (Fig. 1). The neuron responded as if the

receptive field had contracted around the attended stimu­

lus, so that the influence of stimuli at other locations in

the field was reduced or eliminated (Fig. 2).

An even more dramatic illustration of this phenomenon

was observed for a few cells that gave different temporal

patterns of response to different stimuli. For example,

when one cell was tested with individual bars within the

receptive field, it gave an on-response to blue bars and a

mixed on- and off-response to yellow bars. When a blue and

a yellow bar were simultaneously presented within the

field, and the animal attended to just the blue bar, the

cell gave just an on-response, as if the yellow bar was no

longer inside the field. Conversely, when the animal at­

tended to the yellow bar, the cell gave a mixed on- and

off-response similar to the response it had given when just

the yellow bar was inside the field. Thus, by its pattern

of response, the cell appeared to communicate to the rest

of the brain the properties of just the attended stimulus.

One surprising result of the study was that the locus

of attention affected the responses of V4 neurons only when

both the attended and ignored stimuli were located inside

the receptive field. If one stimulus was located inside

the field and one located outside, it seemed to make no

difference to the neuron which stimulus the animal attend­

ed. Apparently, the neural mechanism for attention works

only over a very short span of cortex in V4, possibly at

the level of a V4 "hypercolumn". This result would seem to

conflict, however, with the results of numerous psychologi­

cal studies which have shown that attention to one object

attenuates the processing of other objects throughout the

•

R.F.

r-----l-, 1 01 II i 1 1 1 1 1 1 L ________ 1

174

• •

Fig. 2. Schematic representation of the effects of attention on cells in V4. When two stimuli are located within the receptive field (R.F.), and the animal attends to just one, cells respond as if their receptive field had contracted around this attended stimulus. In the absence of attention, or if attention is directed outside the recep­tive field, the response of a cell will reflect the proper­ties of all the stimuli located within its field. In such a case, information about an individual stimulus, including its specific location, may be lost.

entire visual field, not just nearby objects.

The solution to this puzzle was found in the inferior

temporal cortex, the cortical area that receives projec­

tions from V4. The receptive fields of inferior temporal

neurons invariably include the center of gaze and are often

so large as to cover essentially the entire visual field.

When we recorded from inferior temporal neurons, we found

that the responses of cells were gated by attention to

stimuli throughout at least the central 12 0 of both visu-

al hemifields (the limits of our video display). Thus, it

is likely that in the inferior temporal cortex, attention

gates processing throughout the visual field.

To determine where along the occipitotemporal pathway

attention first affects sensory processing, we recorded

from neurons in both VI and V2. It was not possible to

test the effects of attention with two stimuli inside the

receptive field of a VI neuron, since the receptive field

size was so small that the animal could not selectively

attend to just one of the stimuli. When one stimulus was

175

inside the field and one just outside, the monkey was able

to perform the task, but, as in V4 under this condition,

attention had little or no effect on the cells. In V2,

receptive fields were just large enough for the animal to

attend selectively to one of two stimuli located inside the

receptive field. Unlike the case in V4, however, we found

little or no effects of attention under this condition.

Furthermore, we found no effect of attention when one stimu­

lus was located inside the field and one outside. While

effects of attention in Vl or V2 may ultimately be discov­

ered in other situations (see for example, [1,3]), it seems

that the gating of responses within the receptive field

begins beyond V2, probably at the level of area V4.

Our results indicate that selective attention serves

to remove unwanted information from receptive fields as a

result of a two-stage process in extrastriate cortex. The

first stage works at the level of V4 over a small spatial

range but at a high spatial resolution. The second stage

works at the level of the inferior temporal cortex over a

large spatial range, possibly the entire visual field.

The consequences of such an attentive mechanism are

twofold. First, the gating of responses to unattended

stimuli within the receptive field serves to reduce unwant­

ed information from a cluttered visual scene. This effect

of attention presumably underlies the attenuated processing

and reduced awareness for unattended stimuli shown psy­

chophysically in humans. Second, the fact that neurons

res'pond to an attended stimulus as if their receptive

fields had contracted around it may allow for cells to

communicate information with high spatial resolution in

spite of their large receptive fields. Thus, the visual

system may "have its cake, and eat it too", in that it

achieves the advantages of large receptive fields without

incurring the expected costs in spatial resolution, at

least within the focus of attention. However, outside the

focus of attention large receptive fields may indeed incur

a cost. Given that attention works over only a limited

spatial range in V4, one might expect that spatial resolu­

tion outside the focus of attention to be poor, a possibili­

ty that is consistent with the results of a number of psy-

176

chophysical stQdies. JQlesz [8], for example, has shown

that focal attention is required to perceive the spatial

arrangement of local line elements, and Triesman and her

colleagues [24] have found that the features of stimuli

outside the focus of attention may be perceived in the

wrong locations and consequently form "illusory conjunc­

tions" .

In addition to improving spatial resolution, psy­

chophysical studies in humans have suggested that increas­

ing the amount of attention devoted to a stimulus may en­

hance the processing of that stimulus [9,17]. Likewise, in

monkeys, there is some suggestive neurophysiological evi­

dence that increased attention might enhance processing, in

that neurons in certain cortical areas show different de­

grees of responsiveness depending on whether the monkey is

idle, engaged in a detection task, or engaged in a discrimi­

nation task [15,20,23]. It has not been clear from these

physiological studies, however, whether neuronal responsive­

ness varies with changes in state, level of arousal, the

specific task required of the animal, or the amount of

attention devoted to the stimuli. To test specifically

whether increasing the amount of attention devoted to a

stimulus affects how it is coded within the visual system,

we studied the responses of neurons in area V4 to stimuli

presented within the context of the same perceptual task at

two levels of difficulty [22].

The basic task used was the same delayed

matching-to-sample task that we used to study spatial atten­

tion, except that no irrelevant stimuli were presented.

The general strategy of the experiment was as follows. On

each trial of an experimental session, the sample and test

bars were chosen from a small set of different bars varying

in either color or orientation. The sample and its match-

ing test stimulus in the easy condition were identical to

those in the difficult; the two levels of task difficulty

were determined by the nature of the nonmatching test stimu­

lus. In the easy condition, the nonmatching test stimulus

differed from the sample by 900 of orientation or about

77 nm in wavelength. In the difficult condition, the non­

matching test stimulus varied from the sample by only

177

22.5 0 or 19 nm in wavelength. All cells were tested

under both conditions, some first in the difficult condi-

tion and some first in the easy. We focused our analyses

on the neuronal responses to the sample stimuli, since the

sample stimulus presentations were identical across the two

conditions.

Behavioral evidence indicated that the animals did

indeed process the stimuli differently within the two condi­

tions. Difficult probe trials inserted occasionally within

the easy condition were performed with far more errors than

when the same trials were presented in the context of the

difficult condition. A signal detection analysis of these

results showed that in the difficult condition the animals

adopted a stricter internal criterion for discriminating

matching from nonmatching stimuli, and also that the dis­

criminability of the stimuli increased significantly (d'

2.11 versus 1.7); that is, the animals' internal representa­

tions of the stimuli were better separated, independent of

the criterion used to discriminate them.

Consistent with the behavioral data, the physiologi­

cal recordings showed that neuronal responses in the diffi­

cult condition were stronger and more tightly tuned to the

stimuli than were responses to the same stimuli in the easy

condition (Fig. 3). Compared to the neuronal discharges in

the easy condition, there was an 18% median increase in

firing rate to the optimal stimulus in the difficult condi-

tion as well as a 20% decrease in bandwidth. Interesting-

ly, the percentage increase in responsiveness and percent­

age decrease in bandwidth in the difficult as compared with

the easy condition were about the same in magnitude as the

change in d' measured behaviorally.

One possible explanation for the enhanced responsive­

ness in the difficult condition was an increase in general

arousal, which might cause an improvement in the responsive­

ness of all cells in V4. Alternatively, the neuronal

enhancement might have been restricted to just those cells

whose receptive fields contained the stimuli to which the

animal was attending. To decide between these possibili­

ties, we recorded from cells while the animal performed an

easy versus difficult task on stimuli outside the receptive

A fixation

/

178

B

u w

60

~ 40 Cfl w ~

a::: Cfl 20

... "'11

I' ", I "

I /'/ , ....

EASY ....... -

o~~~~~~~~~

o 45 90 135 ORIENTATION (deg)

Fig. 3. Example of responses from a V4 neuron that were stronger and more selective when the animal was per­forming a difficult discrimination than when it was perform­ing an easy discrimination based on the same stimuli. (A) The stimuli were red bars varying in orientation, centered within the receptive field. (B) The tuning curve shows the firing rates to the set of stimuli when each was presented as the sample to be discriminated from a subsequently pre­sented bar differing by 90 0 of orientation (easy) or 22.50 (difficult).

field of the cell while irrelevant stimuli were presented

inside the field. We reasoned that if the difficult task

increased the responsiveness of all the neurons in V4 due

to arousal, then responses to even the unattended stimuli

inside the field should be enhanced in the difficult condi­

tion. However, the results showed that when the task stimu­

li were presented outside the receptive field of the record­

ed neuron, the difficult task no longer caused any enhance­

ment of responses or sharpening of tuning curves for the

stimuli located inside the field. Thus, increased atten­

tion appears to enhance the responsiveness and sharpen the

selectivity of only the cells that are processing the at­

tended stimulus and not the cells processing ignored stimu­

li.

179

One way of interpreting the narrowing of tuning

curves with increased attention that we found in this study

is that attention served to contract the orientation and

color "receptive fields" of the cells in V4. In this

sense, the results complement our earlier finding that

spatial attention serves to contract the spatial receptive

fields of cells in V4 and the inferior temporal cortex

[13) . Together, the results suggest that one function of

attention is to increase resolution, be it in the spatial

or feature domains. A similar conclusion was reached earli­

er by Bergen and Julesz [2), based on psychophysical re­

sults from humans.

Now that we know that sensory processing in extrastri­

ate cortex is under the control of cognitive factors, the

challenge is to understand the neural mechanisms underlying

this control. Given the extensive behavioral and neurophys­

iological evidence that the posterior parietal cortex is

involved in the control of spatial attention [14,17,18,21),

one might expect the posterior parietal cortex to mediate

the effects of spatial attention within the occipitotempo­

ral pathway. Such an influence must necessarily be indi­

rect, however, as there are no direct connections between

the posterior parietal cortex and the bulk of the inferior

temporal cortex (Ungerleider, personal communication).

Another promising candidate is the pulvinar, since portions

of the pulvinar are reciprocally connected with both V4 and

the inferior temporal cortex. Crick [4] has proposed a

model whereby the pulvinar and the lateral reticular nucle­

us of the thalamus together serve to modulate cortical

activity, and Walter Schneider (personal communication) has

developed a neural network model of thalamo-cortical inter­

actions underlying attention. Further, lesions or chemical

deactivation of the pulvinar seem to impair certain aspects

of attention, measured behaviorally [6,16,19). Yet, even

if the pulvinar is involved in attentional modulation, it

is certain to be itself under the control of numerous brain

systems. Working out the circuitry underlying attention is

a challenge for the future.

180

ACKNOWLEDGEMENTS

We thank Mortimer Mishkin for his support during

all phases of this work and Leslie G. Ungerleider for valu­

able comments on an earlier version of the manuscript.

H. S. was supported in part by the United States-Israel

Binational Science Foundation.

BIBLIOGRAPHY

[1] C.H.Anderson and D.C.Van Essen. Shifter circuits: A

computational strategy for ~ynamic aspects of visual

processing. Proceedings of the National Academy of

Science, 84, 6297-6301, 1987.

[2] J.R.Bergen and B.Ju1esz. Rapid discrimination of

visual patterns. IEEE Transactions on Systems, Man,

and Cybernetics, SMC-13, 857-863, 1983.

[3] R.Boch. Behavioral activation of visual neurons in

monkey striate cortex. Society for Neuroscience

Abstracts, 12, 1367, 1986.

[4] F. Crick. The function of the thalamic reticular com­

plex: The searchlight hypothesis. Proceedings of the

National Academy of Sciences, 81, 4586-4590, 1984.

[5] R.Desimone, S.J • Schein, J . Moran and L.G. Ungerleider.

Contour, color, and shape analysis beyond the striate

cortex. Vision Research, 25, 441-452, 1985.

[6] R.Desimone, L.Thomas, M.Wessinger and W. Schneider,

unpublished data.

[7] C.G.Gross and M.Mishkin. The neural basis of stimulus

equi valence across retinal translation. In S. Harnad,

R.Doty, J.Jaynes, L.Goldberg and G.Krauthamer (Edi­

tors), New York: Academic Press, pp. 109-122, 1977.

[8] B.Julesz. Textons, the elements of texture percep­

tion, and their interactions. Nature, 290, 91,97,

1981.

[9] D. Kahneman. Attention and Effort. Englewood Cliffs:

Prentice-Hall, 1973.

181

[10] J.H.R.Maunsell and W.T.Newsome. Visual processing in

monkey extrastriate cortex. Annual Review of Neuro­

science, 10, 363-401, 1987.

[11] Mishkin, M. A memory system in the monkey. Philosophi­

cal Transactions of the Royal Society of London B,

298, 85-95, 1982.

[12] M.Mishkin, L.G.Ungerleider, K.A.Macko. Object vision

and spatial vision: Two cortical pathways. Trends in

Neuroscience, 6, 414-417, 1983.

[13] J .Moran and R. Desimone. Selective attention gates

visual processing in the extrastriate cortex. Sci­

ence, 229, 782-784, 1985.

[14] V.B.Mountcastle. Brain systems for directed atten­

tion. Journal Royal Society of Medicine, 71, 14-28,

1978.

[15] V. B .Mountcastle, B.C.Motter, M.A. Steinmetz and A.K.

Sestokas. Common and differential effects of atten­

tive fixation on the excitability of parietal and

prestriate (V4) cortical visual neurons in the

macaque monkey. Journal of Neuroscience, 7,

2239-2255, 1987.

[16] S.Petersen, D.L.Robinson and J.D.Morris. Contribu­

tions of the pulvinar to visual spatial attention.

Neuropsychologia, 25, 97-105, 1987.

[17] M.l.Posner and D.E.Presti. Selective attention and

cognitive control. Trends in Neuroscience, 10, 13-17,

1987.

[18] M.I.Posner, J.A.Walker, F.A.Friedrich and R.D.Rafal.

How do the parietal lobes direct covert attention?

Neuropsychologia, 25, 135-145, 1987.

[19] R.D.Rafal and M.I.Posner. Deficits in human visual

spatial attention following thalamic lesions. Proced­

ings of the National Academy of Sciences, 84,

7349-7353, 1987.

[20] B. J. Richmond and T. Sato. Enhancement of inferior

temporal neurons during visual discrimination. Jour­

nal of Neurophysiology, 6, 1292-1306, 1987.

182

[21] D.L.Robinson, M.E.Goldberg and G.B.Stanton. Parietal

association cortex in the primate: Sensory mechanisms

and behavioral modulations. Journal of Neurophysiolo­

gy, 41, 910-932, 1978.

[22] H. Spitzer, R.Desimone and J . Moran . Increased atten­

tion enhances both behavioral and neuronal perfor­

mance. Science, 240, 338-340, 1988.

[23] H.Spitzer and B.Richmond, in preparation.

[24] A.Treisman. Features and objects in visual process­

ing. Scientific American, 114-125, 1986.

[25] L.G.Ungerleider and M.Mishkin. Two cortical visual

systems. In D.J.Ingle, M.A.Goodale, R.J.W.Mansfield

(Edi tors), Analys is of Visual Behavior. Cambridge:

MIT Press, pp. 549-586, 1982.

[26] L. G. Ungerleider. The corticocortical pathways for

object recognition and spatial perception. In

C.Chagas, R.Gattass, C.Gross (Editors), Pattern Recog­

ni tion Mechanisms. Vatican City: Pontifical Academy

of Sciences, pp. 21-37, 1985.

Neuronal Representation of Pictorial Working Memory in the Primate

Temporal Cortex

Yasushi Miyashita Department of Physiology

School of Medicine University of Tokyo, Japan

It has been proposed that visual memory traces are located in the temporal lobes of the cerebral cortex, as electric stimulation of this area in humans results in recall of imageryl. Lesions in this area also affect recognition of an object after a delay in both humans 2 ,3 and monkeys4-7, indicating a role in working

memory of images 8 . Single-unit recordings from the temporal

cortex have shown that some neurons continue to fire when one of two or four colors are to be remembered temporarily9. However, neuronal responses selective to specific complex objects lO- 18 , including hands lO ,13 and faces 13 ,16,17 cease soon after the

offset of stimulus presentation lO- 18 • These results left it open whether any of these neurons could serve memory of the object. We

have recently found a group of shape-selective neurons in an anterior ventral part of the temporal cortex of monkeys that exhibited sustained activity during the delay period of a visual

working memory task 19 ,20. The activity was highly selective for

the pictorial information to be memorized and was independent of the physical attributes such as size, orientation, color or position of the object. These observations indicate that the delay activity represents the working memory of categorized percept of a picture. This article discusses the implications of

these findings in the cognitive neuroscience.

1. Generation of the colored fractal stimuli

As the visual stimuli to be memorized, we generated color

fractal patterns. Fig.l shows the flow-chart of the generating

algorithm. The algorithm produces a number of patterns (usually

100 patterns) when a 32-bit integer is given as a seed of random

numbers. The coordinates of the corner points of a pattern are

YES

random number

get the edge size

random number

get the number of edges

184

random number

get the number of recursion

random number

get colour

Figure 1. The algorithm generating colored fractal stimuli

represented as an list. When input list of x,y points is given, a recursive subroutine generates an output list twice as long by inserting intermediate points. The intermediate points lies on a line which is the perpendicular bisector of the line between two adjacent input points. The program was implemented in a image processing system (Gould/DeAnza IP8500). Examples of the colored fractal patterns thus generated are shown in Fig.3. Our aim for generating these artificial stimuli in the memory task was to use a class of well-defined, complex, but reproducible stimuli which the monkey has never seen in his life.

L E VE R

WARNING

SAMPLE

185

I II II I I

I -----+~------------------------------------~~~r---

L..--____ ~

Figure 2: Sequence of events in a trial of the memory task

Lev, lever press by the monkey. War, warning green image (0.5sec). Sam, sample stimulus (0.2sec). Mat, match stimulus (0.2sec) following a delay of 16sec. Cho, choice signal of white image. Lowest trace, the events-chart used in Fig.3, 4 & 7.

2. Behavioral task and recording of neural activities

In a trial of our visual working memory task, sample and match

stimuli were successively presented on a video monitor, each for

0.2sec at a 16sec delay interval (Fig.2). The stimuli were newly

selected for each trial among the 100 colored fractal patterns.

Two monkeys (Macaca fuscata) were trained to memorize the sample

stimulus and to decide whether the match stimulus was the same or different. Each sample picture was paired with an identical match

stimulus and one which was not identical. For each trial, the

identical or non-identical match stimulus was assigned randomly.

If the match stimulus was different from the sample, the monkey

could release the lever and touch the video screen to obtain a

fruit juice. If the test stimulus was the same as the sample, the

monkey was to keep pressing the lever until the choice signal was turned off. When the monkey released the lever before the choice

signal, the trial was cancelled. The monkeys performed the task

at 85-100% correct. Error trials were excluded from the analysis

of this report. In the training sessions and the period for

searching units, the stimuli were presented in a fixed

In the recording session, long sequence of trials

entire repertory of stimuli was run repeatedly.

seq,uence.

using the

When some

pictures were found to elicit stronger responses than others, the

experimenters selected the relevant pictures and ran shorter

sequences with fewer stimuli.

186

Extracellular spike discharges of 188 neurons were recorded from the anterior ventral part of the temporal cortex of these monkeys with standard physiological techniques 22 • Recording of electrooculograms revealed no systematic differences in eye position which could be related to differential neural responses described below.

3. Neural discharges related to the working memory Figure 3 illustrates reproducible stimulus-dependent discharges during the delay obtained in one cell for 4 different sample stimuli (prominent in a, but virtually ineffective in b, c & d). Trials of the same sample stimulus were originally separated by intervening trials of other sample stimuli, and these were sorted and collected by off-line computation. A time course of the delay activity in Fig.3a is shown as a spike density histogram (Bin width, 200msec) in Fig.4, as contrasted with those for 6 other ineffective stimuli. These histograms are representative of those accumulated with 57 other sample stimuli. Only 2 of the 64 tested stimuli (Fig.s) were followed by especially high delay activity (>10 imp/sec).

These delay activities do not represent mere sensory afterdischarge9 ,11 for the following reasons . First, the high rate of firing did not decline throughout the whole 16sec delay period (Fig.4). Second, firing frequency exhibited during the delay was not necessarily correlated with that during the stimulus presentation (Fig.4 & 5). Third, the delay activity in some neurons started after a latency of a few seconds following stimulus presentation (not shown). Thus, it is concluded that the delay activity is not a passive continuation of the firing during

- --- - ---_. :;~~~~~~~~.:

.. I _

Figure 3: Stimulus selectivity of delay discharges Spike discharges of a cell are shown in cases when the monkey memorizes different fractal patterns. Each rastergram consists of trials whose sample stimulus is shown above. Trials of the same sample stimulus were originally separated by intervening trials of other sample stimuli, and these were sorted and collected by off-line computation.

187

imp. 5-1

20

10

o J1 _____ Lr-

L...J

1 sec

Figure 4: Time course of delay discharges Spike density histogram for the trials with different stimuli were superposed . Bin width, 200ms.

sample

the sensory stimulation, but represents a mnemonic activity to retain visual information .

Of the 188 neurons tested, 144 showed a correlation between firing and one or more events of the trials. Among the 144 cells, 95 showed a sustained increase or decrease of discharge frequency during the delay period, while the others fired only during stimulus presentation. In 77 of these 95 cells, the discharge

Figure 5: Distribution of average delay spike frequency following 64 different sample pictures with which more than 4 trials were tested. Ordinate, number of sample pictures used as stimuli .

-o ... II)

.Q

E ::::I c:

o average delay response

188

frequency varied depending on sample stimuli, but the remaining 18 did not exhibit such selectivity. In many of the 77 selective

cells, only a few pictures elicited a strong delay activation

such as shown in Figs.4 & 5. It is notable that the optimal picture differed from cell to cell, and that the whole population

of the optimal pictures for the 77 cells covered a substantial

part of the repertory of the pictorial stimuli.

4. Categorized percept of a picture is memorized

For further analysis of triggering features of the delay

responses, sample pictures were manipulated in the following

manner (Fig.6): 1) stimulus size was reduced by half, 2) stimuli

were rotated by 90 degrees in a clockwise direction, 3) colored

stimuli were transformed into monochrome by referring to a

pseudo-color look-up table, and 4) stimulus position was changed

on the video monitor (a 0.2sec stimulus presentation time is

short enough to exclude the contribution of saccadic eye

movement). Figure 7 illustrates responses of a neuron which

consistently fired during the delay after one particular picture

but not after others, irrespective of stimulus size (Figs.7Ab & Bb), orientation (Figs.7Ac & Bc), or color (Figs.7Ad & Bd). Simi tar tolerance of responses was observed in a majority of the

tested delay neurons: to size in 16/19, to orientation in 5/7, to

color-monochrome in 15/20, and to position in 8/13 cells. In

STIMULUS TRANSFORMATIONS

ORIENTATION S I Z E POSITION MONOCHROME

Figure 6: Transformations of visual stimuli

b im.,. .... 10

5

189

b- - - - - - - -0.. - - - - - - -6.._ _ _ _ ! 10.---------------.0.---------___ :-rO'--________ - - -

oL-~-~~~~~~~==~~~~~~·-II III Iv

Figure 7: Response tore lance under stimulus transformation in size, orientation or color.

A, histograms similar to Fig.4, but for 5 different sample pictures. (a), control responses. (b) shows effects of stimulus size reduction by half. (c) and (d) show effects of stimulus rotation by 90 degrees in a clockwise direction and of color-to­monochrome transformation. B, average delay spike frequencies as a function of stimulus transformation (a, original; b, size reduction; c, rotation; d, color to monochrome). Responses to 7 different sample pictures (including 5 shown in A) are plotted with different symbols. Error bars indicate standard deviations for 4-15 trials.

other neurons, manipulation of the most effective stimulus reduced or abolished the thereby-evoked delay discharge.

5. Comparison with other temporal cortical neurons In the inferior temporal cortex, shape-selective neuronal discharges have been reported for Fourier descriptors l2 , face I3 ,16,17, hands 10 ,13 or stimuli used in a discrimination task I4 ,15, although all of these were sensory responses evoked during presentation of stimulus. The relative selectivity of these sensory neurons remained invariant over changes of size, position, orientation or contrast I2- 15 ,17. The present delay responses may be derived through such sensory responses, inheriting from them the tolerance to such stimulus transformations. It is notable that, for the Fourier descriptor neurons l2 , the absolute level of the response varied much over changes in stimulus size, while this was not the case for many of

190

the present neurons. This may suggest that the present neurons

represent more abstract properties of objects (like shape

percept) than do such sensory neurons.

In the color-selective delay neurons previously described 9 ,

time courses of sustained delay discharge were similar to those found

Fig.4 in the present shape-selective delay neurons (compare our or 7 with Fig.7 or 6 of Ref.9). The differential delay

activity in the color task was mainly found in the cortex of the lower bank of the superior temporal sulcus 9 ,

posteriorly and dorsally to the presently explored

color cells seemed to be scattered9 , whereas the

tended to group in smaller areas.

6. Functional role in the pictorial memory

lying more

area, and the

present cells

A majority of our shape-selective delay neurons were recorded

in TEav23 (or TE1-TE224) and some in TG v23 and in area 35. These

areas are anatomically designated as the last link from the visual system to limbic memory systems 4 ,23,25,26. Neurons in

these areas were visually responsive with field 27 • Impairment of the recognition memory

lesions including these areas4 - 7 , consistent

a large receptive

task resulted from

with the presently

postulated mnemonic role of neurons in these areas.

The present results suggest that pictorial working memory is

coded by temporary activation of an ensemble of neurons in the

region of the association cortex that processes visual

information 4 ,9,28, rather than by neuronal activity in a brain

area specialized for working memory. Although each neuron in the

ensemble has highly abstract and selective coding features,

representation of the memory of a picture seems to be distributed among a number of neurons. We need to know how the distributed

information is decoded for subsequent decision processes 29 • This work was supported by the grants from Inamori Foundation and

from the Japanese Ministry of Education, Science and Culture

(No.62124056 & No.62570052).

REF ERE N C E S

1. Penfield, W. & Perot, P. Brain 86, 595-697 (1963).

2. Kimura, D. Arch. Neurol. ~, 48-55 (1963).

3. Milner, B. Neuropsychologia £, 191-209 (1968).

4. Mishkin, M. Phil. Trans. R. Soc. London, Ser. B 298, 85-95

(1982).

5. Gaffan, D. & Weiskrantz, L. Brain Res. 196, 373-386 (1980).

6. Sahgal, A., Hutchison, R., Hughes, R.P. & Iverson, S.D. Behav.

Brain Res. ~, 361-373 (1983).

191

7. Fuster, J.M., Bauer, R.H. & Jervey, J.P. Exp. Neurol. 11, 398-409 (1981).

8. Warrington, E.K. & Shallice, T. Quart. J. Exp. Psychol. 24, 30-40 (1972).

9. Fuster, J.M. & Jervey, J.P. J. Neurosci. 1, 361-375 (1982). 10. Gross, C.G., Rocha-Miranda, C.E. & Bender, D.B. J.

Neurophysiol. 35, 96-111 (1972). 11. Gross, C.G., Bender, D.B. & Mishkin, M. Brain Res. 131, 227-

239 (1977). 12. Schwartz, E.L., Desimone, R., Albright, T.D. & Gross, C.G.

Proc. Natl. Acad. Sci. U.S.A. 80, 5776-5778 (1983). 13. Desimone, R., Albright, T.D., Gross, C.G. & Bruce, C. J.

Neurosci. ~, 2051-2062 (1984). 14. Sato, T., Kawamura, T. & Iwai, E. Exp. Brain Res. 38, 313-319

(1980) . 15. Iwai, E. Vision Res. l2, 425-439 (1985). 16. Perret, D.I., Rolls, E.T. & Caan, W. Exp. Brain Res. ~, 329-

342 (1982). 17. Rolls, E.T. & Baylis, G.C. Brain Res. 65, 38-48 (1986). 18. Baylis, G.C. & Rolls, E.T. Exp. Brain Res. 65, 614-622

(1987) . 19. Miyashita, Y., Cho, K. & Mori, K. Soc. Neurosci. Abstr. 11,

608 (1987). 20. Miyashita, Y. & Chang, H.S. Nature 221, 68-70 (1988). 21. Miyashita, Y. & Mori, K. Electroencheph. Clin. Neurol. in

press (1988). 22. Miyashita, Y. & Nagao, S. J. Physiol. 351, 251-262 (1984). 23. Turner, B.H., Mishkin, M. & Knapp, M. J. Compo Neur. 191,

515-543 (1980). 24. Seltzer, B. & Pandya, D.N. Brain Res. 149, 1-24 (1978). 25. Herzog, A.G. & Van Hoesen, G.W. Brain Res. 115, 57-69 (1976). 26. Van Hoesen, G.W. & Pandya, D.N. Brain Res. 95, 1-24 (1975). 27. Desimone, R. & Gross, C.G. Brain Res. 178, 363-380 (1979). 28. Anderson, J.R. Cognitive Psychology and its Implications,

Freeman and Company, San Francisco and London (1980). 29. Coltheart, M. Phil. Trans. R. Soc. London, Ser. B 302, 283-

294 (1983).

Part 3 Motor Control and the Cerebellum

Hierarchical Learning of Voluntary Movement by Cerebellum and Sensory Association Cortex

MitsuoKawato* Michiaki lsobe t

Ryoji Suzuki:!:

Department ofBiophysica1 Engineering Faculty of Engineering Science

Osaka University, Japan

Abstract

In earlier papers, we have proposed the feedback-error-Iearning of inverse dynamics model of the musculoskeletal system as heterosynaptic learning scheme in the cerebrocerebellum and the parvocellular part of the red nucleus system, and the iterative learning in the parietal association cortex. In this paper, we applied hierarchical arrangement of these two neural network models to learning trajectory control of an industrial robotic manipulator. We found that the hierarchical arrangement of the cerebellar and cerebral neural networks not only increased control stability but also dramatically improved accuracy of control and reduced required learning time.

1 Introduction

A computational study [8] reveals that the central nervous system (eNS) must solve the following three computational problems (see Fig. 1) at different levels: (1) de­termination of a desired trajectoryi n the visual coordinates, (2) transformation of the visual coordinates of the desired trajectory into the body coordinates and (3) generation of motor command. To illustrate the model consider an arm movement reaching to a glass on a table. First, one desirable trajectory in task-oriented co­ordinates must be selected from out of an infinite number of possible trajectories which lead to the glass, with spatial coordinates provided by the visual system (de­termination of trajectory). Second, the spatial coordinates of the desired trajectory must be reinterpreted in terms of a corresponding set of body coordinates, such as joint angles or muscle lengths (transformation of coordinates). Finally, motor com­mands (e.g. torque) must be generated to coordinate the activity of many muscles so that the desired trajectory is realized (generation of motor command).

Several lines of experimental evidence suggest that the necessary informations in Fig. 1 for this computational model are internally represented in the brain [9]. First, Flash and Hogan [3] proposed the "minimum jerk model" for trajectory planning of human multijoint arm movement, which strongly indicates that the movement

·to whom correspondence should be addressed:Present address: ATR Auditory and Visual Per­ception Research Laboratories, Twin 21 Bldg. MID Tower, Shiromi 2-1-61, Higashi-ku, Osaka 540 Japan

tPresent address: Mitsubishi Electric Corporation, Amagasaki, Hyogo, 661 Japan I Present address: Department of Mathematical Engineering and Information Physics, Faculty of

Engineering, University of Tokyo, Tokyo, 113 Japan

,.' " ,-

I

I , I I , ,

I I

\ \ \ \

\ \ , ,

... -

196

goal of movement

I trajectory determination

desirable trajectory in task-oriented coordinates

, , ,

I coordinates transformation

desirable trajectory in body coordinates

" I

motor command (muscle torque)

generation of motor command

Figure 1: A computational model for voluntary movement.

is fist planned at the task-oriented coordinates (visual coordinates) rather than at the joint or muscle level. Second, the presence of the transcortical loop (i.e. the negative feedback loop via the cerebral cortex) indicates that the desired trajectory must be represented also in the body coordinates, since signals from proprioceptors are expressed in the body coordinates. Finally, Cheney and Fetz [2] showed that discharge frequencies of primate corticomotoneuronal cells in the motor cortex were fairly proportional to active forces (torque). Consequently, the CNS must adopt, at least partly, the step-by-step computational strategy for control of voluntary movement.

Based on physiological information and previous models, we have proposed com­putational theories and neural network models which account for these three prob­lems.

1. A minimum torque-change model was proposed as a computational model for trajectory formation, which predicts a wide class of trajectories in human multi-joint arm movements [18]. They showed that the trajectory formation problem can be considered as an energy minimization problem with nonlinear constraints given as the dynamics of a controlled object, and we [11] recently proposed a neural network model for trajectory formation, which learns the energy to be minimized from movement examples, and then minimizes the learned energy by parallel computation based on network dynamics.

2. An iterative learning scheme was proposed as an algorithm which simultane­ously solves the coordinates transformation and the control problem [7,10].

197

This algorithm can be regarded as a Newton-like method in function spaces. Because short term memory of time histories of trajectory and torque are required for the iterative learning control, we propose that the sensory asso­ciation cortex of the cerebrum executes the iterative learning control.

3. A neural network model for generation of motor command was proposed [8]. This model contains internal neural models of the musculoskeletal system and its inverse-dynamics system. The inverse-dynamics model is acquired by heterosynaptic plasticity using a feedback motor command (torque) as an error signal. Since real time heterosynaptic plasticity is required for the feedback-error-Iearning, the cerebellum and the red nucleus are supposed to be the locus of this motor control and learning. This neural network model has been successfully applied to trajectory control of an industrial robotic manipulator [12].

It must be noted that we do not adhere to the hypothesis of the step-by-step information processing shown by the three straight arrows in Fig. 1. Rather, we [18] proposed a learning algorithm which calculates the motor command directly from the goal of the movement represented by some performance index: i.e. minimiza­tion of mean square torque-change (broken and curved arrow in Fig. I). Further, as shown by a solid curved arrow in Fig. 1, motor command can be obtained di­rectly from the desired trajectory represented in the task-oriented coordinates by the iterative learning algorithm [7,10].

In this paper, we discuss control and learning performance of hierarchical ar­rangement of the iterative learning neural network (cerebrum) and the feedback­error-learning neural network (cerebellum). The iterative learning is very precise and rapid to converge but has no capability of generalization. The feedback-error­learning, on the other hand, is rather sloppy and requires long learning time but has great ability to generalize learned movement. We will show that hierarchical arrangement of the two neural networks complement with each other and produces ideal control and learning performance. This result reveals superiority of the fun­damental and phylogenically necessary design concept of the brain to hierarchically overlay a phylogenically new and rather unstable but precise network upon a phy­logenically older and dull but robust network. Application of the combination of the cerebral and cerebellar neural networks to trajectory control of an industrial robotic manipulator is described.

2 Computational framework for the problem of control

Let us consider the problem of motor control in a computational framework. There exists causal relation between the motor command and the resulting movement pattern. Let T(t} denote time history of the motor command (torque) and O(t} denote time history of the movement trajectory in a finite time interval t E [0, t fl. The causal relation between T and 0 can be written as G(T(·)} = 00 using a functional G. If a desired movement pattern is denoted by Od(t), the movement error is defined as F(T} = (}d - G(T) = (}d - o. The problem to generate a motor command Td , which realizes the desired movement pattern Od, is equivalent to find an inverse of the functional G. In other words, it is equivalent to find a root of the

198

functional F. Once the inverse of G is acquired, the required motor command can be calculated as Td = G-1«(Jd) since (J = G(Td) = G[G-l«(Jd)] = (Jd holds.

Because controlled objects in motor control have many degrees of freedom and nonlinear dynamics, the functionals G and F are also nonlinear. Furthermore, since the exact dynamics of the controlled objects (arms, limbs, robotic manipulators) are generally unknown, we do not know exact forms of G or F. Consequently, it is practically very difficult to calculate the inverse of G or the root of F.

We proposed a neural network model and the feedback-error learning rule, by which an internal neural model of inverse dynamics of the motor system (i.e. G-1)

is acquired during execution of movement [8], and applied it for trajectory control of an industrial robotic manipulator [12]. Furthermore, we [7,10] proposed an iterative method to obtain the root of the functional F even if the dynamics of the controlled system is unknown. This is mathematically a modification of the well known Newton method and is called Newton-like method. Let the space ofthe motor command be denoted by C :3 T and that of the movement pattern by P :3 (J. The functional F determines an error associated with a specific motor command.

F:C~P. (1)

The Newton method to find a root of F is given as follows.

1"'+1 = 1'" + aT = Tn - F'(1"')-l F(1"'). (2)

However, this scheme cannot actually be used since we do not know the dynamics of the controlled system and hence we do not know the derivative of the functional; F'. Instead, we can utilize the following Newton-like method, in which an approximation ME L(P, C) of FH is used. Here, L(P, C) is a space of linear operator from P to C and M can be somehow computed.

(3)

3 Hierarchical neural network model for control and learn­ing of voluntary movement

Ito [5] proposed that the cerebrocerebellar communication loop is used as a reference model for the open-loop control of voluntary movement. Allen and Tsukahara [1] proposed a comprehensive model, which accounts for the functional roles of several brain regions (association cortex, motor cortex, lateral cerebellum, intermediate cerebellum, basal ganglia) in the control of voluntary movement. Tsukahara and Kawato [17] proposed a theoretical model of the cerebra-cerebella-rubral learning system based on recent experimental findings of the synaptic plasticity in the red nucleus, especially on the sprouting phenomena. Expanding on these previous mod­els, we propose a neural network model for the control of and learning of voluntary movement, shown in Fig. 2. This neural network model is based on various phys­iological and morphological information, especially on the importance of synaptic plasticity and of the cerebra-cerebellar communication loop in the motor learning of voluntary limb movements.

In our model, the association cortex sends the desired movement pattern (Jd expressed in the body coordinates, to the motor cortex, where the motor command, that is torque T to be generated by muscles, is then somehow computed. The actual

CEREBRO-CBM &PARVO RN

SPINO-CBM &MGN RN

T

199

idea

e d :desired trajectory

T

SENSORY­ASSNCX AREAS 2.5. 7

e q

somatosensory feedback

visual feedback

e :movement pattern

I I I I I I I I I I I I I I I I I I _________________ 1

q:movement pattern in visual coordinates

Figure 2: A hierarchical neural network model for control and learning of volun­tary movement. The model is composed of the following four parts: (1) The main descending pathway and the transcortical loop designated by heavy lines, (2) The spinocerebellum-magnocellular red nucleus system (SPINO-CBM & MGN RN) as an internal neural model of dynamics of the musculoskeletal system, (3) The cere­brocerebellum-parvocellular red nucleus system (CEREBRO-CBM & PARVO RN) as an internal neural model of inverse-dynamics of the musculoskeletal system, (4) The sensory-association cortex (SENSORY-ASSN CX, AREAS 2, 5, 7) as an as­sociative memory for iterative learning control. These subsystems of the CNS are phylogenically and ontogenetically older in this order. For the dynamics model and the inverse-dynamics model of the musculoskeletal system, the inputs used for cal­culating the outputs are designated by solid arrows and lines (T for dynamics model and 9d for inverse-dynamics model) and the inputs for synaptic modification (i.e. teaching or error signals) are designated by dotted line and thin arrow (9 for dy­namics model and Tj for inverse-dynamics model). ASSN CX=association cortex; MOTOR CX=motor cortex.

200

motor pattern () is measured by proprioceptors and sent back to the motor cortex via the transcortical loop. Then, feedback control can be performed utilizing error in the movement trajectory (}d - (). However, feedback delays and small gains both limit controllable speeds of motions.

The spinocerebellum-magnocellular part of the red nucleus system receives in­formation about the results ofthe movement () as afferent input from the propriocep­tors, as well as an efference copy of the motor command T. Within this system, an internal neural model of the musculoskeletal system is acquired. Once the internal model is formed by motor learning, it can provide an internal feedback loop.

The cerebrocerebellum-parvocellular part of the red nucleus system develops extensively in primates, especially in man. It receives synaptic inputs from wide areas of the cerebral cortex and does not receive peripheral sensory input. That is, it monitors both the desired trajectory (}d and the feedback motor command Tf but it does not receive information about the actual movement (). Within the cerebrocerebellum-parvocellular red nucleus system, an internal neural model of the inverse-dynamics of the musculoskeletal system is acquired. The inverse-dynamics of the musculoskeletal system is defined as the nonlinear system whose input and output are inverted (trajectory () is the input and motor command T is the output). Note that the spinocerebellum-magnocellular red nucleus system provides a model of the direct dynamics of the musculoskeletal system. Once the inverse-dynamics model is acquired by motor learning, it can compute a good motor command T; directly from the desired trajectory (}d.

During visually guided voluntary movement, the parietal association cortex re­ceives both the visual and somatosensory informations about controlled objects (arms and hands). We propose that some parts of sensory association cortex (ar­eas 2, 5 and 7) solve the problems of coordinates transformation and generation of motor command simultaneously by an iterative learning algorithm, as shown by a solid curved arrow in Fig. 1. This is a trial and error type learning of a single motor pattern, such as repetitive training of golf swing. That is, the amount of mo­tor command needed to coordinate activities of many muscles is not determined at once, but in a step-wise, trial and error fashion in the course of a set of repetitions. In this motor learning, short term memory of time histories of the trajectory and the torque are required. Because of this, we suppose that the iterative learning is conducted in sensory association cortex of the cerebrum, instead of the cerebellum, the red nucleus or the hippocampus. The area 2 is supposed to be involved in motor learning in the body coordinates. The areas 5 and 7 are supposed to be involved in motor learning in the visual coordinates.

In this paper we concentrate on the problem of generation of motor command. So, hereafter, we will consider only the area 2 of the cerebrum for the iterative learning in the body coordinates, and the cerebrocerebellum and the parvocellular part of the red nucleus for the feedback-error-Iearning.

4 Feedback-error-Iearning of inverse-dynamics model

Let us consider a neuron with synaptic plasticity, which approximates the output z(t) of an unknown nonlinear system by monitoring both the input u(t) and the output z(t) ofthis system. The input u(t) to the unknown nonlinear system is also fed to n subsystems and is nonlinearly transformed into n different inputs x/(t)(l =

201

1, ... , n) to the neuron with plasticity. That is, instantaneous firing frequencies of n synaptic input fibers to the neuron are designated by :l:1(t), ... , :l:n(t). Let w, denote a synaptic weight of the /-th input. Average membrane potential y(t) of the neuron is the sum of n postsynaptic potential. For simplicity, we assume that the output signal of the neuron y(t) (instantaneous firing frequency of the neuron) is equal to its average membrane potential. In vector notation, we have the following equations.

X(t) = [:1:1 (t), :l:2(t), ... , :l:n(t)]t,

W [WI, W2, ... , Wn]t, y = WtX(t) = X(t)tw.

Here, t denotes transpose. The second synaptic input to the neuron is an error signal (e.g. climbing fiber input for the Purkinje cell), and is given as an error between the output of the neuron y(t) and the output from the unknown nonlinear system z(t): s(t) = z(t) - y(t).

Based on physiological information about the heterosynaptic plasticity of the red nucleus neurons and the Purkinje cells, we assume that the l-th synaptic weight w, changes when the conjunction of the l-th input :I:,(t) and the error signal 8(t) occurs:

TdW(t)/dt = X(t)s(t) = X(t)[z(t) - X(t)tW(t)]. (4)

Here, T is a time constant of change of the synaptic weight. If u(t) is a stochastic process, then X(t) and z(t) are also stochastic processes. In this case, we can prove the following theorem about the convergence of the synaptic weight W(t) using Geman's [4] result, if X and z are mixing random processes.

Theorem 1 If the time constant T of change of the synaptic weight is sufficiently long compared with the "rate" of mixing of X and z, then the synaptic weight W converges in mean to the value for which a mean square error of the output E[(z_y)2] is minimum.

Because the time constants of physiologically known synaptic plasticities are suffi­ciently long (from a few hours to a few weeks) compared with temporal patterns of movement (several hundreds ms), the assumption of the theorem is satisfied. It is worthwhile to note that the averaged equation of EqA gives the steepest descent method and the convergence is global.

4.1 Learning of inverse-dynamics model by feedback motor command as an error signal

For simplicity, we consider a three degree of freedom manipulator as a controlled object. Although it is much simpler than musculoskeletal systems such as the human arm, they both have several essential features such as nonlinear dynamics and interactions between multiple degrees of freedoms in common.

The simplest block diagram for acquiring the inverse dynamics model of a con­trolled object by the heterosynaptic learning rule is shown in Fig. 3a. As shown in Fig. 3a, the manipulator receives the torque input T(t) and outputs the resulting trajectory O(t). The inverse dynamics model is set in the opposite input-output direction to that of the manipulator, as shown by the arrow. That is, it receives the

a

b

202

-Ti(t) inverse

b dynamics model

terror

+

manipulator I T(t) e (t)

r---------------------------------------------------~ , . • • •

Tf(t)

Til

+

inverse dynamics model

manipulator

Figure 3: Learning schemes of the inverse-dynamics model of the controlled system. (a) The simplest learning method. The arrow shows the direction of signal flow in the inverse-dynamics model. (b) The feedback-error-learning scheme and inter­nal structure of the inverse-dynamics model for a three degree of freedom robotic manipulator.

203

trajectory as an input and outputs the torque 1;(t). The error signal 8(t) is given as the difference between the real torque and the estimated torque: 8(t) = T(t) -1;(t). However, this architecture does not seem to be used in the CNS because of the fol­lowing three reasons. First, after the inverse-dynamics model is acquired, large scale connection change must be done for its input from the actual trajectory to the desired trajectory, so that it can be used in feedforward control. This change of connections must be very precise so that the i-th component of the actual trajectory corresponds to the same i-th component of the desired trajectory. It is very hard to imagine such large scale connection changes occur in the CNS while preserving the minute one-to-one correspondence. Second, we need other supervising neural net­work which determines when the connection change should be done, in other words when the learning is regarded completed. Third, this method which separates the learning and control modes can not cope with aging changes of the manipulator dynamics or a sudden change of a payload.

Fig. 3b shows the detailed structure of the inverse-dynamics model and the block diagram which illustrates arrangement of the inverse-dynamics model, the controlled object (manipulator) and the feedback loop. This block diagram corre­sponds to a subsystem of our comprehensive neural-network model shown in Fig. 2, which includes the motor cortex, the transcortical loop and the cerebrocerebellum­parvocellular red nucleus system. Please note that the arrangement and structure of the model shown in Fig. 3b is completely different from those of the model shown in Fig. 3a. First the input used for calculating the output is different in the two models. It is (J in Fig. 3a while it is (Jd in Fig. 3b. Second the error signal for synaptic modification is different. It is the error T - T; in Fig. 3a but it is the feedback torque Tf in Fig. 3b. Third the directions of the information flow within the inverse-dynamics model are opposite.

The total torque T(t) fed to an actuator of the manipulator is a sum of the feedback torque Tf(t) and the feedforward torque 1;(t), which is calculated by the inverse-dynamics model. The inverse-dynamics model receives the desired trajec­tory (Jdj, (j = 1,2,3) represented as the three joint angles as input and monitors the feedback torque Tf(t) as the error signal.

This architecture for learning of the inverse-dynamics model has several advan­tages over other learning schemes shown in Fig. 3a, or those of Psaltis et al. [14] and Rumelhart [15]. First, the teaching signal or the desired output is not required. Instead, the feedback torque is used as the error signal. That is, back-propagation of the error signal through the controlled object [14] or through a mental model of the controlled object [15] is not necessary at all. Second, the control and learning are done simultaneously. It is expected that the feedback signal tends to zero as learning proceeds. We call this learning scheme as feedback error learning empha­sizing the importance of using the feedback torque (motor command) as the error signal of the heterosynaptic learning.

4.2 Application to trajectory control of a robotic manipu­lator

We [12] applied the neural network shown in Fig. 3b to trajectory control of an industrial robotic manipulator (Kawasaki-Unimate PUMA 260). The neural net­work model was implemented in a microcomputer (Hewlett Packard 9000-300-320). Although the manipulator has six degrees of freedom, for simplicity only the basal

204

a Joint! Joint2 Joint3

l:} Z ct

., N

N

., CD

,~~--~-r-.--~~ a.BB 6.2121

TIMECSEC. )

l:} Z ct .,

N

I~-.--r--r~--~~ 21.2121 S.Be

TIMECSEC. )

l:} Z ct ., ., ~~-.--~~~--.-~

21.2121 6.2121 TIMECSEC. )

b Joint! Joint2 Joint3

l:} Z ct

., N

N

., CD

I+--r--~,--.--~~ 0.00 6.00

TIMECSEC.)

l:} Z ct

., CD

N

., N

I~-.--r--r~--~~ B.B" 6.210

TIMECSEC. )

l:} Z ct

., ., to

., ., ~+--.--~~~--.-~

0.2121 S.I2J2I TIMECSEC.)

Figure 4: Control performance of a faster and different movement than the training pattern, before (a) and after (b) 30min feedback-error-learning. Desired trajectories for the three joint angles are shown by chain curves, and the realized trajectories by solid curves. The unit of the ordinate is radian.

three degrees of freedom were controlled in the present experiment. In the nonlinear dynamics equation ofthe ~anipulator, the input voltage to the motor is represented as a linear summation of various nonlinear terms. We chose these 43 nonlinear terms as the nonlinear transformation of subsystems in Fig. 3b.

The feedback torque (feedback voltage input to the motors) was chosen as a sum of proportional and derivative terms:

(5)

here Vjj is the feedback voltage input to the j-th motor, 1J4i is the j-th joint angle of the desired trajectory and IJj is the j-th joint angle of the realized trajectory. The feedback gains were set as Kp = 60 [volt/rad] and Kd = 1.2 [volt/sec rad].

A desired trajectory lasting for 6 sec was 300 times given to the control system repeatedly. So, the learning time was 30 min.

Results of the learning experiment can be summarized as follows.

1. The inverse-dynamics model was acquired by repetitively experiencing the single motor pattern during 30 min learning. The mean square error of the trajectory (lJd -1J)2 decreased. That is, the control performance improved considerably during learning. When control depended only on the feedback, overshoots and oscillation of the realized trajectory were observed, but after 30 min learning the desired trajectory and the realized trajectory were hard to be separately seen. However, even after 30 min learning, slight errors in the later and steady part of the trajectory were observed.

205

2. During 30 min learning, the feedforward torque increased while the feedback torque VJ decreased considerably. This implies that as motor learning pro­ceeded, the inverse-dynamics model gradually took the place of the feedback loop as a main controller.

3. Once the neural network model learned some movement, it could control quite different and faster movements. It has capability to generalize learned move­ments. We examined whether the neural network model after 30 min of learn­ing a single pattern. could control a quite different movement pattern (Fig. 4 chain curve) which was about twice as fast as the training pattern. Fig. 4 compares control performance for this test movement before (a) and after (b) 30 min learning. The three joint angles of the desired trajectory during 6 sec test pattern are designated by chain curves, and those of the realized trajec­tory by solid curves. Before learning (a), delays and overshoots were often observed, and the realized trajectory deviated considerably from the desired trajectory. However, after learning (b), the actual trajectory almost coin­cided with the desired trajectory. This control capability for quite different and faster movements than the training pattern is one of the most outstanding features of the cerebellar neural network model.

5 Iterative learning control by sensory-association cortex

5.1 Newton-like method within body coordinates

The dynamics of the musculoskeletal system or the robotic manipulator is described by the following differential equation.

dO/dt = h(0,0)+R-1(0)T, dO/dt = O+WI,

8(0) = 0, 0(0) = o. (6)

Here, T is torque input (i.e. motor command), iJ is velocity, 0 is position (e.g. joint angle, muscle length), WI is the initial (angular) velocity, and R is an inertia matrix, which is positive definite and always invertible. Also the product Rh is a summed torque of centripetal, Coriolis, frictional and gravitational forces, and hand R-1

are assumed continuously differentiable. As explained in section 2, the problem to find the desired torque Td which realizes the desirable velocity Od, and the desirable trajectory Od, within a finite time interval [0, t f ], is equivalent to obtaining the root of a nonlinear functional F(T) = Od - 0. It can be solved by the following Newton method.

Tn+1 = Tn + R(on)[d(Od _ iJn)/dt Deh(on, on) X (Od - on) De{h(on, on) + R-l(on)r} X (Od - on)]. (7)

Here, Tn, on, on are torque, velocity and trajectory during the n-th iteration: In this scheme, the motor command in the (n+l)-th iteration is a sum of the motor command in the n-th iteration plus the three modification terms which are, respec­tively, proportional to acceleration, speed and position errors between the desired trajectory and the realized trajectory in the n-th iteration.

206

It is not so difficult to see that Eq. 7 is the Newton method Eq. 2 in function spaces. However, this scheme can not be realized since the dynamics of the con­trolled object is unknown, and hence the matrices R, Deh, D9(h + R-1T") in Eq. 7 are unknown. On the other hand, the following Newton-like method can be actually implemented.

Tn+1 = T" + R(0)[d(8d - 8n )/dt D8h(0, 0) x (8d - 8n )

D9h(0, 0)(6d - 6n )]. (8)

Here, R(O), R(0)D8h(0, 0), R(0)D9h(0, 0) are matrices estimated at the initial state. They are transformations, respectively, from acceleration, velocity and position to torque. This kind of Newton-like method can be formally formulated as Eq. 3.

The Newton-like method Eq. 8 can be practically utilized since the transforma­tion matrices can be easily estimated from the following step response experiment. Let ;8(t) denote the step response of velocity when a unit-step torque is applied only to the i-th actuator with the initial condition (8,6) = (0, ( 0) at time t = O. Then, the transformation matrices can be approximately estimated as follows.

R(60 )

R(60 )D8h(0, ( 0 )

(18(0+),28(0+), ... m 8(0+»-1 -(18(00)'28(00), ... m 8(00»-1. (9)

Here, n is the number of actuators. We assume that gravitational force is compen­sated beforehand. For an industrial robotic manipulator (PUMA-260), we succeeded in relatively precise estimation of the transformation matrices by this method.

The following theorem can be proved regarding the convergence of the scheme Eq.8.

Theorem 2 If the dynamics equation 6 is dissipative, input T is bounded, and II - R(0)R-1(6)1 < 1/3 holds, then the Newton-like method Eq. 8 converges expo­nentially, regardless of the starting point TO.

The condition of the theorem is satisfied for usual industrial robotic manipulators with high reduction-ratio gears from motors to joints, because the high reduction­ratio dramatically weakens the nonlinearity of the manipulator dynamics. Con­sequently, the inertia matrix R of usual industrial robotic manipulators does not change much for various postures 6. But the assumption of the theorem is not sat­isfied for direct-drive manipulators or human arms. When we [10] used the scheme Eq. 8 for control of a relatively small movement of a model direct-drive manipulator, the iteration converged quickly only in a few repetitions. However, when the scheme Eq. 8 was applied to a large movement, the repetitions diverged as anticipated.

As described above, for iterative learning control of manipulators with low reduction-ratio gears (with strong nonlinearity) or human arms, we need more so­phisticated scheme than Eq. 8. In the scheme Eq. 8, the transformation matrices were estimated at only the initial posture and they were used throughout the entire movement. However, for controlled objects with strong nonlinearity, the transfor­mation matrices change substantially along the desired trajectory. To solve this problem we can estimate the transformation matrices at various postures and can use their interpolations so that the Newton-like method gets closer to the original

207

Newton method Eq. 7. Interpolated approximations of matrices R, RDiJh are de­noted by R, RDiJk. The Newton-like method which is closer to the original Newton method can be derived using these interpolated matrices:

Tn+! = rn + R(on)[d(8d - 8n)/dt - DiJk(O, on) X (8d - 8n )]. (10)

The modified Newton-like method was certain to converge for manipulators even with low reduction-ratio gears [10]. We proved the convergence of this modified method under the conditions that R is similar to R and the motor command in the first iteration is close to the real root.

The iterative scheme Eq. 8 itself is only adaptation, since experiences obtained in the course of repetitions can not be used for control of a different movement pat­tern. However, acquisition of the transformation matrices continues irrespectively of movement patterns, and it can be utilized for control of any movement trajectory. So, this corresponds to learning.

6 Hierarchical arrangement of cerebral and cerebellar neu­ral networks

The feedback-error-Iearning neural network has capability to generalize learned movements, but requires long (from 30 minutes to a few hours) learning time and even after the long learning time it can not completely realize the desired trajec­tory. On the other hand, the iterative learning scheme converges exponentially (i.e. trajectory error decreases exponentially) within several repetitions of a single move­ment. So, the learning time usually does not exceed 1 minutes. However, it might become unstable if the motor command in the starting iteration is too far from the true solution. Furthermore, experiences obtained in execution of various movement patterns can not be utilized for control of a new movement pattern. That is, the iteration must be repeated again from the very beginning.

As shown in the comprehensive neural network model of Fig. 2, the iterative learning neural network (area 2 of cerebrum), the feedback-error-Iearning neural network (cerebrocerebellum and parvocellular red nucleus) and the negative feed­back loop (transcortical loop) are hierarchically arranged and are used for motor control at the same time. These three networks are phylogenically and ontogenet­ically newer in this order. We expect that the neural networks arranged hierar­chically complement with each other and provides an ideal control and learning paradigm. In this section, we study control and learning performance of the hier­archical arrangement of the three networks in trajectory control of the industrial robotic manipulator. Fig. 5a shows block diagram of hierarchical arrangement of the three networks in manipulator control. Fig. 5b illustrates computational proce­dure within one iteration of the iterative learning represented by Eq. 10. The input voltage v(n) fed to the motor of the manipulator in the n-th iteration is a sum of the four kinds of voltages; the modification motor command V;~n) from the iterative learning network, the feedforward voltage Vi, from the inverse-dynamics system, the feedback voltage Vi:) and a real time gravity compensating voltage ~~n).

v(n) = V(n) + 11; + V(n) + v(n) . .t .. fb gr • (11)

The gravity compensation voltage is not contained in the comprehensive neural network model in Fig. 2 and can be put into the feedforward torque from the inverse­dynamics model [12]. It is used only for convenience in the present experiment.

a BLOCK DIAGRAM

inverse-dynamics model

I

tIlf I I

I L ____ .,

I I I I I I

ad I

+

feedback a(n)

b ITERATIVE LEARNING

208

Vis

v(n) fb

gravity compensation

Manipulator

(n) iterative ViI learning

V Inn)

V(n+1) II

a (n)

Figure 5: Trajectory control of the industrial robotic manipulator by hierarchical arrangement of cerebral, cerebellar and feedback neural networks. (a) Block dia­gram of hierarchical arrangement of the three kinds of networks and the controlled object (manipulator). The iterative learning network is simply denoted by V;~n). (b) Computation procedure for calculating motor command during one iteration of the learning algorithm.

209

The feedforward voltage V;, is calculated by the inverse-dynamics model from the desired trajectory 9d while monitoring the feedback voltage V}:) as the error signal, as explained in section 4. The modification command V;~"+1) from the iterative learning network is calculated from the trajectory error e(") = 9d - 9(n) in the prior iteration as described by Eq. 10 in section 5 and as shown in Fig. 5b. In Fig. 5b R represents abbreviation of R(9"), and RD represents abbreviation of R(9")Deh(0, 9") in Eq. 10. The feedback voltage V}:) is computed in real time by the feedback loop as described by Eq. 5 with Kp = 2.0 and Kd = 1.2. It must be noted that the feedback voltage in the n-th iteration is used for calculation of the iterative command in the (n+1)-th iteration:

v;~n+1) = v~n) + R(9nHrFe(")jdt2 - D9h(0,9n)de(n)jdt}

= V;~n) + vj;) + R(9n){d2e(") jde - Deh(O, 9n)de(n) jdt}. (12)

This equation is different from the original Newton-like method with interpolated matrices Eq. 10 in that the feedback voltage VJ~) is incorporated into the right-hand side. In the control experiment, this modification was found to be useful for more rapid and stable convergence of the iterative learning.

We applied the hierarchical model shown in Fig. 5 to trajectory control of the industrial robotic manipulator, PUMA-260. First we omit the inverse-dynamics model. That is, the manipulator was controlled by the iterative learning network and the feedback loop. The transformation matrices R(9n) and R(9")Deh(0, 9") were estimated at 25 different postures (5 different 92 x 5 different (3 ) and their interpolations R(8n ) and R(8n)Deh(0, 8") were calculated as double trigonometric Fourier series in (82 , ( 3 ) because of periodicity of 2'11'. Fig. 6 shows results of this experiment. The three joint angles of the desired movement pattern (chain curve, 9d1 (t), 9d2 (t), 8d3(t» and the realized movement pattern (solid curve, 91(t), 92(t), 93(t» during 6 sec of a movement pattern are shown. In the first iteration (Fig. 6a), the manipulator was moved only by the feedback loop. As can be seen, only by the feedback control, the realized trajectory substantially deviated from the desired trajectory. On the other hand, even in the second iteration (Fig. 6b), the desired trajectory and the velocity time course were realized almost satisfactorily.

Second, the feedback-error-Iearning network is incorporated and the whole hi­erarchical model shown in Fig. 5a was used for control of the manipulator. Fig. 7 shows the results of this ''final'' experiment. Fig. 7a shows trajectories in the first iteration of this experiment. That is, this is the result obtained only by the 30 minutes feedback-error-Iearning. As can be seen, it is difficult to separately see the desired trajectory and the realized trajectory. However, in the later and steady part of the movement (after 5 sec) the first and the second joint angles are slightly different from the objective values. However, even in the second iteration (Fig. 7b), the realized and desired trajectories completely coincide with each other within the accuracy of this figure.

Since accuracy of figures such as Fig. 6 and 8 is very limited, we more closely examine the performance of the hierarchical control using a table of mean magnitude of error of the three joint angles in various iterations (see Table 1). Table 1a shows the mean magnitude of error of the three joint angles 184i - 9il (j=1,2,3) in the first to sixth iteration in the experiment using the feedback loop and the iterative learning network shown in Fig. 6. The mean magnitude of error is represented in

210

a b

W~ W~ -.J. -.J. l:J- (j-

Z Z IT: IT:

r r (J1tsl (J1tsl crw crw H • H • LL, LL,

0.00 6.00 0.00 6.00 TIME (SEC) TIME (SEC)

W(S) Wtsl -.Jw -.Jw l:J"': l:J"': z z IT: IT:

Q Q Z Z O(S) Otsl U~ U~ W- W-(J11 (J11

0.00 6.00 0.00 6.00 TIME (SEC) TIME (SEC)

W~ W~ -.J. -.J. l:J- l:J-Z Z IT: IT:

Q Q cr tsl cr tsl Hw Hw I' I' 1-, r,

0.00 6.00 0.00 6.00

TIME (SEC) TIME (SEC)

Figure 6: Trajectory control of the industrial robotic manipulator by only the cere­bral (iterative learning) neural network model. (a) Time courses of the three joint angles during 6 sec movement executed only by the feedback control (the transcor­tical loop ). Desired trajectories are shown by chain curves and realized trajectories by solid curves. The unit of the ordinate is radian. (b) Control performance after single iteration of the movement. Note that the trajectory error was larger than that of Fig. 8b, although the realized trajectories were satisfactorily close to the desired trajectories.

211

a b

w~ w~ ...1. ...1 • 0- 0-z Z IT: IT:

f- f-UllS) UlIS) D:::Ul D:::Ul H H· u..j u..j

0.00 6.00 0.00 6.00 TIME (SEC) TIME (SEC)

WIS) WIS) .JUl -1 Ul 0": 0": z z IT: IT:

~ ~ z Z o IS) o IS) u Ul u Ul W- W-Ull Ull

0.00 6.00 0.00 6.00 TIME (SEC) TIME (SEC)

W~ W~ ...1. -1 . 0- 0-z Z 0: 0:

~ ~ D:::IS) D:::IS) HUl HUl I· I· f-j f-j

0.00 6.00 0.00 6.00 TIME (SEC) TIME (SEC)

Figure 7: Trajectory control of the industrial robotic manipulator by hierarchical arrangement of the cerebral and cerebellar neural network models. (a) Time courses of the three joint angles during 6 sec of a single repetition of the training movement pattern after 30 min feedback-error-Iearning. Control is executed only by the cere­bellar and the feedback networks. This corresponds to the first trial of the control experiment by the hierarchical arrangement of the cerebral and cerebellar neural networks. Desired trajectories are shown by chain curves and realized trajectories by solid curves. The unit of the ordinate is radian. (b) The iterative learning neural network is overlaid upon the feedback-error-Iearning neural network. This figure shows the result in the second iteration. The realized trajectories were almost identical to the desired trajectories.

212

a ---> error of 1st iteration <---

[1]= 11.22178 deg. count = 1456.2 [2]= 7.68733 deg. count = 1494.2 b [3]= 4.11372 deg. count = 491.3 ___ > (------> error of 2nd iteration (---

[1]= 2.33470 deg. count = 303.0 [1]= [2]= 0.73316 deg. count = 142.5 [2]= [3]= 0.96459 deg. count = 115.2 [3]=

error of 1st 121.77242 deg. 0.3211217 deg. 121.18246 deg.

iteration count = count = count =

100.2 62.4 21. 8

(------> error of 3rd iteration (--- ---> [1]= 1. 23473 deg. count = 160.2 [1]= [2]= 1.12930 deg. count = 219.5[2]= [3]= 0.42413 deg. count = 50. 7 [~~:>

error of 2nd 121.1217113 deg. 121.16623 deg. 121.05560 deg.

iteration count = count = count =

iteration count =

9.2 32.3 6.6

---) error of 4th iteration <--- [1]= [1]= 0.22162 deg. count = 28.8 [2]= [2]= 0.42209 deg. count = 82.0 [3]= [3]= 121.18732 deg. count = 22.4 ___ >

error of 3rd 121.03285 deg. 0.1216117 deg. 0.02733 deg.

count = count =

iteration

(---

4.3 11. 9 3.3

---> error of 5th iteration <--- [1]= [1]= 121.07444 deg. count = 9.7 [2]= [2]= 121.07435 deg. count = 14.5 [3]-[3]= 0.12480 deg. count = 14.9 ----) error of 6th iteration (---

[1]= 0.1213786 deg. count 4.9 [2]= 121.05488 deg. count = 10.7 [3]= 0.1219086 deg. count = 1121.9

error of 4th 121.1213782 deg. 0.1213967 deg. 121.1212442 deg.

count count count =

(---

Table 1: (a) Mean joint angle errors in the first 6 iterations in the first experiment shown in Fig. 6 using only the iterative learning neural network. (b) Mean joint angle errors in the first 4 iteration in the second experiment shown in Fig. 7 using the hierarchical arrangement of the cerebellar and the cerebral networks

degrees and in counts of photo encoders attached to each joint of the manipulator, which measures the joint rotation in discrete steps. As shown in Table la, the error of joint angles decreased considerably in the course of repetitions. But even in the third iteration, relatively large errors are observed.

Table 1b shows the mean magnitude of error in the first to the fourth iteration in the experiment shown in Fig. 7 using the hierarchical arrangement of all the three networks of Fig. 5. As can be clearly seen, the order of error in the first iteration was at least ten times smaller than the experiment without the feedback­error-learning network (compare the first iteration of a and b). The order of error in the third iteration of this experiment is smaller or comparable to the sixth iteration in the experiment without the feedback-error-Iearning. Consequently, the hierarchi­cal arrangement of the cerebral, cerebellar and feedback neural networks not only increases control stability but also dramatically improves accuracy of control and shortens required learning time.

7 Discussion

4.9 7.7 2.9

We applied the hierarchical combination of the cerebral, cerebellar and feedback neural networks to trajectory control of the industrial robotic manipulator. It was found that hierarchical arrangement of the three neural networks complement their shortcomings with each other and produces ideal control and learning performance. That is, they are stable, robust, precise and has great capability to generalize learned movements and can cope with any inexperienced movements with some extent of accuracy. Please note that in all experiments described in this paper joint angles and voltages to motors are sampled with 10 msec interval. Nevertheless, for relatively fast movement, even after the 4-th iteration (it requires only one minute), the error

213

of the trajectory is very close to the minimum accuracy of the photoencoder of the manipulator (see the fourth iteration in Table Ib). This result reveals superiority of the fundamental and phylogenically necessary design concept of the brain to hierarchically overlay a phylogenically new and rather unstable but precise network upon a phylogenically older and dull but robust network.

There are two possibilities about how the cerebellar neural network computes nonlinear transformations of the subsystems. One is that they are computed by nonlinear information processing within the dendrites of neurons [13,6). The other is that they are realized by neural circuits, and are acquired by motor learning. Recently we succeeded in learning control of the robotic manipulator by an inverse­dynamics model made of a three-layer neural network [11,16). In this network, the subsystems performing nonlinear transformation were not used. That is, we did not use any a priori knowledge about the dynamical structure of the controlled object. We used a modification of the heterosynaptic learning rule described in this paper still using the feedback torque command as the error signal. This neural network control is very appealing for application to feedforward control of a large scale complex system whose fundamental structure is not even known.

In summary, hierarchical arrangement of the iterative and heterosynaptic neural networks is one of the promising schemes for the future control of not only a direct drive manipulator, but also of a large-scale complex system, whose dynamics is not known.

References

[1] G.!. Allen and N. Tsukahara, "Cerebrocerebellar communication systems," Physio/. Rev., vol.54 , pp. 957-1006, 1974.

[2] P.D. Cheney and E.E. Fetz, "Functional classes of primate corticomotoneuronal cells and their relation to active force," J. Neurophysiol. vol.44 , pp. 773-791,1980.

[3] T. Flash and N. Hogan, "The coordination of arm movements; An experimen­tally confirmed mathematical model," J. Neurosci., vol. 5 , pp. 1688-1703, 1985.

[4) S. Geman, "Some averaging and stability results for random differential equa­tions," SIAM J. Appl. Math., vol. 36 , pp. 86-105, 1979.

[5) M. Ito, "Neurophysiological aspects of the cerebellar motor control system," Intern. J. Neurol., vol.7 ,pp. 162-176, 1970.

[6] M. Kawato, T. Hamaguchi, F. Murakami and N. Tsukahara, "Quantitative analysis of electrical properties of dendritic spines," Bioi. Gybern., vol. 50 , pp. 447-454, 1984.

[7] M. Kawato, H. Miyamoto, M. Isobe and R. Suzuki, "Learning control of hand in task-oriented coordinate by iteration - a Newton-like algorithm -," Japan lEGE Technical Report, vol.MBE85-91 , pp.83-92, (in Japanese) 1986.

[8] M. Kawato, K. Furukawa and R. Suzuki, "A hierarchical neural-network model for control and learning of voluntary movement," Bioi. Gybern., vol. 57 , pp. 169-185, 1987.

214

[9] M. Kawato, "Adaptation and learning in control of voluntary movement by the central nervous system," Advanced Robotics, vol. 2 , in press, 1988.

[10] M. Kawato, M. Isobe, Y. Maeda and R. Suzuki, "Coordinates transformation and learning control for visually-guided voluntary movement with iteration: A Newton-like method in a function space," Bioi. Cybern., in press, 1988.

[11] M. Kawato, Y. Uno, M. Isobe and R. Suzuki, "A hierarchical neural network model for voluntary movement with application to robotics," IEEE Control Systems Magazine, in press, 1988.

[12] H. Miyamoto, M. Kawato, T. Setoyama and R. Suzuki, "Feedback-error­learning neural network for trajectory control of a robotic manipulator," Neural Networks, in press, 1988.

[13] T. Poggio and V. Torre, "A theory of synaptic interactions," In: Theoreti­cal Approaches in Neurobiology. pp. 28--46, Reichardt, W.E., Poggio, T., eds. Cambridge: MIT Press, 1981.

[14] D. Psaltis, A. Sideris and A. Yamamura, "Neural controllers," Proc. IEEE 1st Int. Con/. Neural Networks, San Diego, USA., June 21-24, vol. IV , pp. 551-558, 1987.

[15] D.E. Rumelhart, "Learning sensorimotor programs in parallel distributed pro­cessing systems," this proceedings, 1988.

[16] T. Setoyama, M. Kawato and R. Suzuki, "Manipulator control by inverse­dynamics model learned in multi-layer neural network," Japan IEICE Technical Report, vol. MBE87 , (in Japanese) 1988.

[17] N. Tsukahara and M. Kawato, "Dynamic and plastic properties of the brain stem neuronal networks as the possible neuronal basis of learning and memory," In: Competition and Cooperation in Neural Nets. pp. 430-441, Amari, S., Arbib, M.A. eds., New York: Springer, 1982.

[18] Y. Uno, M. Kawato and R. Suzuki, "Formation and control of optimal tra­jectory in human multijoint arm movement - minimum torque-change model -," BioI. Cybern., submitted, 1988.

1. Introduction

A Model for Oblique Saccade Generation and Adaptation

Masahiko Fujita Nagasaki Institute of Applied Science

Nagasaki, Japan

In 1975, D.A. Robinson developed a model to illustrate the

generation of saccadic eye movements. This model has been a poten­

tial working hypothesis for the study of saccades. It explains that

a saccadic control system is essentially a position-servo system

with a high-gain forward path. The characteristics inclu~e the

following two points: 1) A neural integrator, which eliminates

steady-state positional error in the servo system, is involved in

the concept of a "final common path". The path compensates for the

dynamics of the oculomotor system. 2) Pause neurons suppress the

instability of the high-gain servo system by placing a deadbdnd

element in a high-gain forward path. In spite of logical complete­

ness, the desirable position signals or feedback positional error

signals cannot be found in the actual brain stem circuitry, as was

expected from the model.

The SC (superior colliculus) is the motor center of the saccade

system in the brain stem. Electrical stimulation of the SC causes a

saccadic eye movement with short latency. The amplitude and direc­

tion of the saccades are a function of the site of stimulation in

the SC. Just prior to the naturally occurring saccades, correspond­

ing units in the SC discharge at a high frequency. Sparks and his

colleagues (1980,1983) precisely analyzed the neural signals in the

SC related to saccadic eye movements. Following their work, much

effort has been made to describe the saccade system. In particular,

a model, proposed by Scudder(1984), contains the SC as a motor

center and demonstrates a good simulation of the neural signals in

the brain stem.

The most important problem in the saccade study is how to

reveal the neural mechanism of spatial-to-temporal transformation;

visual or spatial information of the gaze shift in specific ampli­

tude and direction must be transformed into corresponding temporal

signals of oculomotor neurons. The present report proposes what

neural mechanisms from the SC to the premotor neurons can execute

216

this transformation. A possible circuit diagram of a 2-dimensional

saccade system is described on the basis of Robinson's and Scudder's

models. Furthermore, experimental results of adaptation in saccadic

eye movements (Optican and Robinson,1980) are explained by the

function of the cerebellar vermis as an adaptive gain controller.

2. Basic Mechanism of Spatial-to-Temporal Transformation

-- From the SC to the premotor neurons --

The SC is located in the midbrain. It is differentiated into

seven alternating fibrous and cellular layers. The superficial

layers receive direct inputs from the retina. A well-defined retino­

topic map of the contralateral visual field is formed in which the

central visual field is represented anteriorly, while the peripheral

visual field is represented posteriorly. The upper fields are repre­

sented medially, the lower fields laterally. Collicular stimulation

in the deeper layers evokes a saccade to a point in the visual space

from which the superficial layer unit, just superior to the stimula­

tion site, has received retinal input until the saccade starts

(Robinson, 1972, 1981; Sparks, 1986). The motor and visual maps are

aligned in the SC (Fig.1). There is little evidence of direct

connection between the superficial layers and the deeper layers.

Inputs from the FEF (Frontal Eye Fields in the frontal cortex) or

SNr (Substantia nigra in the basal ganglion) to the SC deeper layers

are thought to be saccade motor commands (Wurtz and Hikosaka,1986).

As mentioned above, a site of activation in the SC decides the

amplitude and direction of a saccade. The SC output signal encodes a

"motor error," which indicates a differ-

ence in gaze direction between both ends

of a single saccade. The activated loca­

tion in the SC does not move, while the

ongoing saccade reduces the positional

error (Sparks and Mays,1983; Sparks and

Porter,1983). Discharge rates of the

output units do not vary with respect to

saccade size or direction, but follow a

rather definite temporal pattern denoted

by hIt) at time t, as described in sec­

tion 2.2. Therefore, the SC cannot be an

organization that generates the feedback

postional error signal as defined in

Robinson's model.

VISlW.. FIELD

Fig.l The motor and visual maps in the SC. Stimulation sites a, band c in the left SC generate correspond­ing saccades a,b and c, respectively.

217

The SC output signals are transmitted to premotor neurons in

the pons and midbrain, and eventually transformed to driving signals

of horizontal and vertical oculomotor systems. In the brain stem

saccade system, three types of neurons are distinguished: long-lead

burst neurons (LLBNs) which increase their discharge rates more than

12 msec prior to a saccade; medium-lead burst neurons (MLBNs) which

discharge at high frequencies during a saccade; and pause neurons

(PNs) which cease discharge during the period of MLBNs burst

discharge but, otherwise, discharge at a constant rate. The

following describes models of saccade amplitude coding and

direction coding in neural circuits.

2.1 Differential Distribution of the Output Fibers in the SC Output

Layers -- Amplitude Coding --

Collicular outputs activate premotor neurons in the brain stem.

These output units seem to have identical activity patterns (Sparks

and Mays,1980). This leads us to the postulation that the number of

output units or fibers decides a saccade amplitude. We propose and

discuss a neural mechanism in the following which realizes this idea

of amplitude coding.

<Amplitude coding hypothesis of the SC>

1) In the SC output layer, the density of the output units or output

fibers at a particular location is proportional to the amplitude of

a saccade which the very location evokes when activated.

2) In the SC output layer, the area activated by a motor command

from the FEF or other areas becomes a definite size.

The above hypothesis apparently realizes amplitude coding.

Indirect evidence of the coding is discussed in the following:

(1) Injections of HRP (horse radish peroxidase) into the PPRF

(paramedian pontine reticular formation) caused a greater density of

labeling in the posterior colliculus than in the anterior colliculus

(Edwards,1980). This data indicates a differential distribution of

collicular output units, and suggests a basis for the spatial-to­

temporal transformation (refer to the discussions summary by Fuchs,

1981 ) •

(2) Differential distribution of visual nerve fibers seems to exist

throughout the superficial layer of the SC. Retinotopic mapping from

the retina to the SC describes precise distribution of the fiber

density. Direction coordinates of the mapping have constant inter­

vals, while the amplitude coordinates have approximately intervals

of a logarithmic scale (Schwarz,1977). If we assume that fiber

density on the retina is inversely proportional to the amplitude

218

coordinate value, then as Fig.2

shows, in the SC superf icial

layer the density of axon col­

laterals from the retina becomes

proportional to the amplitude

coordinate value.

(3) Eye movement field patterns

of the SC uni ts (Fig.3 in Sparks,

1976) suggest that the area acti­

vated by a saccade motor signal

is of a constant size, irrespec-

Fig.2 Assumption of nerve fiber projection from the retina into the superior colliculus.

tive of different saccade amplitudes or directions.

(4) The neural field with the lateral inhibition system will main­

tain a local excitation of a definite size decided by the network

factors (Amari, 1977). If we assume this type of inhibition system

in the SC deeper layers, we can explain not only the constancy in

the size of the excitation area, but also several lines of experi­

ments on the SC by mathematical modeling, that is, the inactivation

of the PNs by stimulating the SC (Raybourn and Keller,1977),and the

interaction of double-point collicular stimulation (Robinson, 1972).

2.2 Population Coding in the LLBN -- Direction Coding --

In Fig.3 the nerve fibers are projected in parallel to a neuron

pool. We expect the following principle by mathematical modeling.

When an input signal traverses the central fibers of the tract, it

will activate many dendrites in the neuron pool, and the total

activity of the neuron pool will increase. Inversely, an input

signal along the more peripheral fibers of the tract will cause less

activity in the neuron pool. The total amount

of activity depends on the location of the

input signal, as shown in the lower part of

Fig.3. This activity indicates spatial-to­

temporal transfor-mation. Taking the highest

value at the middle, and assigning a value 0

to both ends, a convex function of the trans­

formation resembles a trigonometric cosine

function, and constitutes a neural basis of

direction coding in the present model. Ampli­

tude and direction decoding processes may be

carried out simultaneously in a premotor

neuron pool. Their integrated process is

discussed on the possible neural structure

Fig.3 Simple principle of population coding for spatial-to-temporal transformation.

219

from the SC to a premotor neuron pool. Complex calculations of

neural signals can be simplified by several assumptions without

contradicting essential arguments.

The present model assumes the following: A saccade command

(r,e) from higher centers, where r denotes the amplitude and e denotes the direction, activates a small area in the SC. Briefly

maintained excitation in this area sends excitatory signals to LLBN

pools. The number of nerve fibers in a state of excitation is

proportional to the amplitude r. Assume that, in an LLBN pool, the

distribution of input fibers from the SC is uniform. Then a small

rectangle ( log(r)±-[;/2, 6±-M2 ) in the SC, an area of [;2, will be

projected into a rectangle ( r±-rM2, 8 ±-M2 in an LLBN pool, an

area of rt,2.

Figure 4 illustrates an LLBN pool, in which flat dendrites are

of various sizes and placed parallel to a definite plane. Input

fibers are projected into this neuron pool, perpendicularly to these

flat dendrites, and make synapses at crosspoints. For simplicity,

the following is stated:

(1) An LLBN dendrite is rectangular in shape. Let the coordinates of

four corners be denoted by (P i ,0 j ), where i,j=1,2. The size and

location of these dendrites are randomly

distributed within a definite range: Then

the conditions hold that,

0< P1 <P 2 <R and -8<01 <02<8,

where Rand 8 are, for instance, equal to

90 deg.

(2) An LLBN responds in proportion to the

size of an overlapped area by the

dendrite and the excitatory input area of rf',2.

The total activity, Y(r,8,t), of the

LLBN pool at time t in a saccade command

(r,8) can easily be calculated as

Y(r,8,t)=Kf(r)g(8)h(t),

where

f(r)=r 2 (R(1-[;/2)-r(1-6», 9(8)=(8 2 _8 2 )/8 2

and K=682.

The variable g(8) is parabolic. If

8 = 90 deg, then g(8) approximates cos8.

Since a real amplitude coordinate system

in the SC differs from a logarithmic

scale for small r, the output component

hIt)

ILLBN ,-I ,,-/~\\~~~ I L -- I I

i~ ~t/

f{YlgIBlh(t)

Fig.4 Population coding in the LLBN pool.

220

fIr) may not be proportional to r2. Saturation in fIr) may corre­

spond to the fact that large saccades are likely hypometric, but a

decline in fIr) near r=R is unnatural. However, fIr) increases

monotonously in a range from 0 deg to 60 deg when R=90 deg, where

normal saccades occur.

2.3 A Hypothesis of Nerve Fiber Projection from the SC to the LLBN

Nerve fiber projection from the SC to four LLBN pools, shown in

Fig.S, accounts for the generation of oblique saccades. For gener­

ating a horizontal component of a saccade, the SC projects output

fibers into a contralateral LLBN pool. For vertical components, both

medial halves of the SCs project fibers side by side into another

LLBN pool to generate an upward component of saccades. Descending

nerve fiber bundles from the lateral halves of both SCs first cross

each other, and then travel in parallel into another LLBN pool to

generate a downward component of A saccades. By these proj ections, the sC 5 C horizontal and vertical components

become proportional to cos e and sin e , respectively.

Figure Sb shows a possible 3-

dimensional form of the nerve fiber

projection. Each of the four fiber

bundles in the vertical system will be

activated in saccades in a specific

direction. An arrow at the end of each

bundle indicates the direction. Four B neuron pools were found in the vertical

premotor system. These neuron pools

have specific on-directions in approx­

imately the same planes as the two

vertical semicircular canals, and show

burst discharge activity during the

fast phase of the vestibular nystagmus

(Vilis et al.,1986; Hepp et al.,1986).

These locations and the corresponding

directions indicate the same arrange­

ment as the vertical system illustrated

in Fig.Sb.

Fig.5 Hypothesis of nerve fiber projection from the superior colliculi to four LLBN pools, which accounts for the generation of two­dimensional saccades.

221

3. Overall Organization of the Brain Stem Saccade System

3.1 The Logical Structure

Total activities of the LLBN pools at time t are nearly equal

to rh(t)cos 6 in a horizontal system, and rh(t)sin 6 in a vertical

system. If these driving power signals are stored in the individual

integrators and independently consumed by negative feedback as the

eye movement proceeds in each direction, then an accurate 2-

dimensional saccade will follow.

Figure 6 illustrates the above circuit. At the end of a

saccade, the integrated value of an input signal amounts to rHcos6

or rHsin6, where H=Jh(t)dt. This value is almost canceled out with

the integrated value of a negative feedback signal from amplifier A.

The same amplifier simultaneously sends this signal to a final

common pathway and increases the integrator to the same value,

rHcos 6 or rHsin6, which represents a component of the final eye

position. When H is correctly adjusted, an accurate saccade will

occur. Factors such as amplifier A do not cause al terations in the

final values because of a local feedback system, as described by

Robinson(1975), Scudder(1982) and Fuchs et al.(1985). If integrator

G contains a leak, then 500 msec, for instance, is long enough for

its time constant, since a

saccadic eye movement does not

usually last more than about

100 msec. To maintain new eye

positions, integrators in the

final common pathways must not

contain any leak. Experiments

have confirmed the existance of

this type of integrating func-

tion in'the cat brain stem

(Cheron et al.,1986; Fukushima,

1986) •

Another model, Fig.7, is

obtained from the a Fig.6 model

with minor modifications. In

this new model, final values of

the 2-dimensional components

maintain an equal value rH

which causes the saccade to

miss the target, unless the

saccade transmission is stop­

ped. A saccadic eye movement

Fig.6 Block diagram of two-dimensional saccade system. HM: horizontal moto­neuron system. VM: vertical motoneuron system. T: time constant in a final common pathway.

Fig.7 Alternate version of block dia­gram of two-dimensional saccade system.

222

begins when the PNs cease to suppress amplifier A. Until that time,

driving power signals, rh(t)cos e and rh(t)sine, become highly

integrated so as to produce the correct initial eye velocity. The

saccade will then gradually turn toward an incorrect point. Re­

discharging of the PNs at the proper time will stop the eye movement

and form a correct saccade.

Finally, the PNs function can be summarized as follows. The

model illustrated in Fig.6 can produce correct saccades,

irrespective of arbitrary pause timing in the PNs firing. The output

of integrator G in a horizontal or a vertical system encodes the

residual size of an ongoing saccade in the corresponding direction.

For a correct saccade, re-discharging of the PNs must take place

during the proper timing when the activity of G decreases

sufficiently. In the model outlined in Fig.7, sufficient increasing

of the activity of G must precede the PNs pause in firing in order

to achieve a powerful and correct initial velocity.

3.2 Neural Implementaion

One of four LLBN pools is described in Fig.8 with connections

from the SC to the MLBN circuits. Two separate groups of LLBNs are

assumed. The dendrites of the first

group are irregular in shape and size.

The second group sums up exci ta tory

outputs from the first group spatially

and temporally. 'i'his type of summation

can be realized with large dendrites and

long time constants in response. The

second group receives inhibitory signals

as well from inhibitory MLBNs which are

activated by excitatory MLBNs. Oculo­

motor neurons are activated by the

excitatory MLBNs. This circuit realizes

Fig.6 diagram.

If negative feedback signals from

the inhibitory MLBNs are transmitted to

the first group of LLBN, their suppres­

sion will have an effect only on the

LLBNs which have been activated by the

SC excitatory signals. Therefore factors

cos e and sin e will appear on feedback

pathways, as indicated in Fig.7. The

present model estimates eye movement

,,- --------, ~ r\~--~~ ----l:: r<,:>-------+_ ------::. --..C

Fig.8 Neural circuit of a premotor neuron pool.

~e lliJ!LJJt

~r

Fig.9 Estimated eye movement fields, corresponding to rectangular models of LLBN dendrites.

223

fields (EMFs) of the LLBNs in a horizontal system, as illustrated in

Fig.9. These EMFs are similar in shape to those observed by Hepp and

Henn(1983). The present model asserts that irregularity of the shape

and size of the EMF reflects that of the LLBN dendrites and consti­

tutes a neural basis for the population coding.

4. Possible Mechanism of Adaptive Modification in Saccades

Partial tenectomy of a monkey's eye muscle system causes its

saccade to become dysmetric. The saccades become hypometric, pulling

in the direction of the tenectomized muscle. The monkey foveates a

target after corrective saccades. Gradually the saccades become

normometric. In about 5 days, the monkey foveates a target in a

single saccade. If the monkey has suffered a cerebellectomy, the

saccadic eye movements never restored to a normometric stage

(Optican and Robinson, 1980). Such adaptive modification, possibly

by the cerebellar vermis, is explained in the following paragraph.

Ito(1984) proposed the concept of the CNMC (cerebellar cortico­

nuclear microcomplex) as a functional unit of the cerebellum

(Fig.10). According to his theory, motor signals have a sidepath

through the corresponding CNMC of the cerebellum. Changes in

transmission efficacy in the cortex modify the input-output relation

of the CNMC. We propose here the hypothesis of the CNMC related to

the saccade as follows:

1) The SCs project outputs not only to premotor neurons in the brain

stem, hut also to the cerebellum (CNMC of the cerebellar vermis and

the fastigial nucleus) as a sidepath. The NRTP (Nucleus Reticularis

Tegmenti Pontis) is a possible relay site in the sidepath.

2) The projection into the CNMC has the

same type of topographic mapping as the SC.

3) Projection from the CNMC to the four

premotor neuron pools exists; the form of

the projection is almost the same as that

illustrated in Fig.5 when the SCs are

replaced by the CNMCs.

With these assumptions, it is natural­

ly expected that a micro-stimulation in the

CNMC evokes a saccadic eye movement

topographically the same as the SC. When

the transmission efficacy in a small region

of the CNMC changes, the motor signals

passing through the area will be altered

Fig.IO Sidepathmodel of CNMC in the saccade system. P: cerebe 11 arcortex (vermis), F:cerebellar nucleus (fastigial n.).

224

and all corresponding saccades will be modified in amplitude and

possibly in direction as well. Nearby saccades will also be

modified, since their motor signals pass through a portion of the

area the modified saccade passes, owing to topograhical organization

of the nervous system.

Conclusions

The major problem concerning the relationship of the SC with

other oculomotor areas is how information about the direction and

amplitude of the saccades is decoded (Sparks,1986). The present

report answers this question by mathematical modeling and several

hypothesis of the interconnections between the SC and premotor

neurons. A simple population coding is essential for spatial-to­

temporal transformation in this model, as well as the projection

hypothesis from the SC to the premotor neuron pools.

An area activated by higher centers is assumed to be of a

definite size, possibly because of the reciprocal inhibition system

in the SC. Further studies of the inhibition system in the SC will

coincide wi th early works by Didday (1976) and Arbib (1982) on the

sensory motor integration in the tectum of the frog, which

corresponds to the SC of the primate, and will shed some light on

the structure of the SC.

References

Amari, S.(1977) Dynamics of pattern formation in lateral-inhibition type neural fields. Biol. Cybern. 27, 77-87.

Arbib, M.A. (1982) Modelling neural mechanisms of visuomotor coor­dination in frog and toad. In: Competition and Cooperation in Neural Nets (Amari, S. and Aribib, M.A., Eds.), Lecture Notes in Biomathematics 46, Springer-Verlag, 342-370.

Cheron, G., Godaux, E., Laune, J.M. and Vanderkelen, B. (1986) Lesions in the cat prepositus complex: Effects on the vestibulo-ocular reflex and saccades. J. Physiol. 372, 75-94.

Cheron, G., Gillis, P. and Godaux, E. (1986) Lesions in the cat prepositus complex: Effects on the optokinetic system. J. Physiol. 372, 95-111

Didday, R.L. (1976) A model of visuomotor mechanisms in the frog optic tectum. Math. Biol. 30, 169-180.

Edwards, S.B. (1980) The deep cell layers of the superior colliculus: their reticular characteristics and structural organization. In: The Reticular Formation Revised: Specifying functions for a Non-specific System ( Hobson, J.A. and Brazier, M.A. Eds.), New York: Raven, 193-209.

225

Fuchs, A.F. (1981) Discussion summary. In: Progress in oculomotor research (Fuchs, A.F. and Becker., W., Eds.), New York: Else'lier-North Holland, 115-121.

Fuchs, A.F., Kaneko, C.R.S. and Scudder, C.A. (1985) Brainstem con­trol of saccadic eye movements. Ann. Rev. Neurosci. 8, 307-337.

Fukushima, K. (1986) The interstitial nucleus of Cajal and its role in the control of movements of head and eyes. Prog. Neurobiol. 29, 107-192.

Hepp, K. and Henn, V. (1983) Spatio-temporal recording of rapid eye movement signals in the monkey paramedian pontine reticular formation (PPRF). Exp. Brain Res. 52, 105-120.

Hepp,K., Vilis and Henn, V. (1986) Vertical and torsional rapid eye movement generation in the riMLF. Soc. Neurosci. Abstr.12, 1182.

Ito, M. (1984) The Cerebellum and Neural Control. Raven Press, New York.

Mays, L.E. and Sparks, D.L. (1980) Dissociation of visual and saccade-related responses in superior colliculus neurons. J. Neurophysiol. 43, 207-232.

Optican, L.M. and Robinson,D.A. (1980) Cerebellar-dependent adaptive control of primate saccadic system. J. Neurophysiol. 44, 1058-1076.

Raybourn, M.S. and Keller, E.L. (1977) Colliculoreticular organiza­tion in primate oculomotor system.J. Neurophysiol.40, 861-878.

Robinson, D.A. (1972) Eye movements evoked by collicular stimulation in the alert monkey. Vision Res. 121, 1795-1808.

Robinson, D.A. (1975) Oculomotor control nisms of Ocular Motility and Their (G.Lennerstrand and P.Bach-y-Rita, 337-374.

signals. In: Basic Mecha­Clinical Implications Eds.), Pergamon, Oxford,

Robinson, D.A. (1981) Control of eye movements. In: Handbook of Physiology. The Nervous System ( Brooks, V.B. Ed.), Washington, DC: American Physiological Society, 1275-1313.

Schwartz, E.L. (1977) Spatial mapping in the primate sensory projec­tion:Analytic structure andre levance to perception. Biol. Cybern. 25, 181-194.

Scudder, C.A. (1984) A different local feedback model of the saccadic burst generator. Soc. Neurosci. Abstr. 10(2), 910.

Sparks, D.L., Holland, R. and Guthrie, B.L. (1976) Size and distri­bution of movement fields in the monkey superior colliculus. Brain Res. 113, 21-34.

Sparks, D.L. and Mays, L.E. rela ted burst neurons Brain Res. 190, 39-50.

(1980 ) in the

Movement fields of saccade­monkey superior colliculus

Sparks, D.L. and Mays, L.E. (1983) Spatial localization of saccade targets. I. Compensation for stmulation-induced perturbations in eye position. J. Neurophysiol. 49, 45-63.

226

Sparks, D.L. and Porter, J.D. (1983) Spatial localization of saccade targets. II. Activity of superior colliculus neurons preceding compensatory saccades. J. Neurophysiol. 49, 64-74.

Sparks, D.L. (1986) Translation of sensory signals into commands for control of saccadic eye movements: Role of primate superior colliculus. Physiological Reviews 66, 118-171.

Vilis, T., Hepp, K., Schwarz, U., Henn, V. and Haas, H. (1986) Unilateral riMLF lesions impair saccade generation along spe­cific vertical planes. Soc. Neurosci. Abstr. 12, 1182.

Wurtz, R.H. and Hikosaka, O. (1986) Role of the basal ganglia initiation of saccadic eye movements. Progress in Research 64, 175-190.

in the Brain

Cerebellar Mechanisms in the Adaptation of Vestibuloocular Reflex

Yasushi Miyashita and Koichi Mori Department of Physiology

School of Medicine University of Tokyo, Japan

A lot of efforts have been devoted to identify neuronal circuitry responsible for adaptation of the vestibuloocular reflex (VOR) and to build a control system model for the whole VOR system. However, a roles of urgent to One clue involvment

controversy still exists in evaluating the crucial the cerebellum in the VOR adaptation, and it seems solve the controversy on a sound experimental basis. for solution may be obtained from examination of

of the inferior olive in the VOR adaptation, as the inferior olive has been assumed to play a key role in learning mechanisms of the cerebellar cortex. This article attempts such an examination through data collected in studies on adaptive modification of the horizontal VOR (HVOR) under combined visual-vestibular stimulations in rabbits, cats and monkeys. These data are further viewed in the light of recent studies of effect of olivo-cerebellar impulses which induce long­term modification of synaptic transmission in the cerebellar cortex.

1. Lesion and Stimulation Experiments on the Inferior Olive Lesion experiments of the cerebellar flocculus have been shown to abolish the VOR monkeys 1S, but it Since Maekawa information flow

adaptation in rabbits 9 , in cats21 and in accompanies a decrease of the VOR gain 9 . & Simpson (1973) found powerful visual

to the flocculus relayed at the dorsal cap of the inferior olive, a simplified neuronal circuit diagram, as in Fig. 1, has been refered to in examining how the whole VOR system works under adaptive conditions. Ito & Miyashita (1975) found that interruption of the climbing fiber visual pathway at the rostral inferior olive abolished adaptation of the HVOR, while it

228

Figure 1 Neuronal circuity for the cerebello-HVOR system. III, VI, oculomotor and abducens cranial nuclei.

did not affect the dynamic characteristics of either VOR or optokinetic eye movement (OKR) in rabbits. This rostral olivary lesion eliminated the visual response of the dorsal cap neurons, but did not affect their tonic activi ty 20. Lesion of the dorsal cap itself abolished HVOR adaptability in rabbits, although it also impaired dynamic characteristic of the VORIO and OKR3. This implies a possible complication which would occur when we affect tonic activity of the olivary neurons, as was proved to be true in the case of olivary lesion by 3-acetylpyridine 14 . Haddad et al. (1980) also demonstrated that dorsal cap lesion abolished VOR adaptability in cats. In monkeys, no data have yet been available as to effects of interruption of the visual climbing fiber pathway on VOR adaptability. Reversible paralysis of olivary neurons in cats by local drug injection induced a transient increase in VOR gain. Stimulation of climbing fibers induced VOR gain decrease in alert cats when the stimulus frequency was increased up to 40-60 Hz4. Climbing fibers were thus claimed to code "a gain-modulating signal", which multiplied the head velocity signal coded on parallel fiber discharges. This theory predicts a change of climbing fiber discharges during the adaptation of eye movement. However, this prediction failed to be confirmed by flocculus single unit recordings either in VOR adaptation 24 ,25 or OKR adaptation (Nagao, in preparation). The cause of these acute disturbances upon VOR dynamics is not clear at the present; but the effect of climbing fiber paralysis might be related to the withdrawal of tonic climbing fiber activi ty 14 (see later). The effects of climbing fiber stimulation are more difficult to interpret

229

because the natural firing frequency of climbing fibers during VOR adaptation is around 1-2 Hz and hardly exceeds 10 Hz (see Fig. 2A).

2. Neuronal Connections around Dorsal Cap The visual pathway to the dorsal cap has been analyzed well in cats and rabbits. Hoffman et al. (1976) suggested the Nucleus of Optic Tract (NOT) to be a relay nucleus from the retina to the dorsal cap in cats. Maekawa & Takeda (1977) found labeled neurons in the Dorsal Terminal Nucleus (DTN) as well as in the NOT after injection of HRP into the dorsal cap in rabbits. However, it should be noted that climbing fibers from the dorsal cap terminate only at a part of the flocculus9 . In cats, ventrolateral outgrowth of the principal olive, rostral areas of the principal and medial accessory olive, also send olivo­floccular fibers which terminate in 8 separate longitudinal bands6 . At the present stage of investigation, no experimental data are available to suggest what kind of information these 'extra-dorsal cap' floccular afferents carry. On the other hand, from HRP studies, floccular Purkinje cells which project to different brainstem nuclei (medial and superior vestibular nuclei, nucleus prepositus hypogrossi, cerebellar lateral nucleus etc) were also found to be localized differentially in rabbits, in cats and in monkeys 1,9. Ito, Orlov & Yamamoto (1982) demonstrated in rabbits that single-pulse microstimulation of the flocculus (5-30 rA) inhibited a vestibuloocular reflex from the horizontal canal to the medial and lateral rectus muscle. This stimulation was effective where and only where the stimulating microelectrode could record climbing fiber response evoked by stimulation of the ipsilateral retina. Thus, floccular Purkinje cells which receive dorsal cap climbing fiber afferents are connected to the secondary relay neurons of the horizontal vestibuloocular reflex, and other Purkinje cells may be related to different target neurons.

3. Climbing Fiber Responses of Floccular Purkinje Cells In the monkey flocculus, it has recently been shown that complex spike discharges are modulated effectively during visual­vestibular adaptive stimulation24 ,25. In this experiment, sinusoidal vestibular stimulation was applied by rotating a turntable to which a monkey chair was attached, and for visual stimulation a cylindrical screen with a checkerboard pattern was rotated sinusoidally. Either one of the following two types of combined visual-vestibular stimulation was applied to induce an adaptive gain change of the horizontal VOR: 1) 200 turntable rotation combined 1800 out of phase with 200 screen rotation (outphase combination); 2) 50 0 turntable rotation

230

A

Figure 2 : Modulation of complex spikes during combined visual­vestibular stimulation25 •

Upper trace in A represents spike density histogram (pulse/sec) for complex spikes in a H-zone Purkinje cell during inphase combination. Middle and lower traces, screen velocity (o/sec) and head velocity (o/sec). Calibration, 1 second. B, C, complex spike modulation in H-zone Purkinje cells. D, E, simular to B, C, but for modulation in non-H-zone Purkinje cells. B, D, under outphase combination. C, E, under inphase combination.

combined in phase with 500 screen rotation (inphase combination). In 48 Purkinje cells tested, the response pattern of the complex spike was very different in two floccular areas, one where micro stimulation through the recording electrode (less than 40 uA) induced abduction of the ipsilateral (to the flocculus) eye (H-zone), and the other where the microstimulation induced vertical or no eye movements (non-H-zone). When Purkinje cells were sampled in the H-zone, modulation of the complex spike was regularly inphase with head rotation (mean ± S.E. for phase shift 5.4±22.3 0 , for amplitude 62.9±22.2%; n=lO) under the outphase combination (Fig. 2B), whereas under the inphase combination (C) it was regularly out-of-phase (phase shift l72±l5.30 , amplitude 62.9±24.l%; n=l7). In non-H-zone Purkinje cells, the modulation of complex spike discharge was very weak. And, if it ever occurred, its phase shift was very different from that of H-zone Purkinje cells (Fig. 2D,E). In the dark, no modulation of complex spikes was observed during vestibular stimulation. Thus, the modulation elicited during visual-vestibular adaptive stimulation was presumed to represent optokinetic responsiveness. The complex spike discharge in the H-zone is, then, enhanced during contralateral and depressed during ipsilateral movement of the visual environment. This H-zone specific responsiveness of complex spikes is same in its direction selectivity as that of rabbits'

231

Purkinje cells20 ,23. It is also the same as that of NOT neurons and dorsal cap neurons 2 in rabbits. It is reasonable to conclude that in monkeys, as well as in rabbits and cats, H-zone Purkinje cells receive visual climbing fiber afferents through the neuronal circuit shown in Fig. 1, although we should wait for

experimental anatomical evidence to confirm the connections in

monkeys.

4. Adaptive Change of Simple Spike Responses in VOR H-zone Purkinje cells in monkeys consistently changed their simple spike responses to head rotation in parallel with adaptive VOR gain changes and in a direction opposite the complex spike modulation 24 ,25. In this experiment, each session of experiments began when a Purkinje cell activity was recorded stably. Then, the HVOR gain and spike discharge of the Purkinje cell was sampled during sinusoidal rotation in the dark (500

0.3Hz) (Fig. 3A). After combined visual-vestibular stimulation (outphase or inphase combination, see above) for one hour, the HVOR and spike discharge of the same Purkinje cell were again tested (Fig. 3B). If the Purkinje cell had been lost in the

course of the combined adaptive stimulation, the session was

terminated. Fig. 3 illustrates for an H-zone Purkinje cell. When

inphase combination of visual-vestibular adaptive stimulation effectively decreased the HVOR gain in one hour, simple spike responses to head rotation in the dark, which was originally out

of phase with head velocity, was converted to inphase with head

velocity (Fig. 3A,B,C). Similarly, 20 H-zone cells tested under

inphase visual-vestibular combination shifted their simple spike discharge to augment inphase modulation. By contrast, for 15 H­zone cells tested under outphase visual-vestibular combination,

the shift of simple spike modulation was in the opposite direction. However, in non-H-zone Purkinje cells, these systematic changes of simple spike modulation were not observed. This result differs from previous neuronal recording in monkeys 17

where no systematic shift to out phase direction was detected for simple spike modulation in the dark during adaptation to high VOR gain. However, in their experiment climbing fiber response was reported to be 'extremely weak' and only 12 out of 128 tested

Purkinje cells modulated (phase shift not described) their

complex spike discharge l8 . These features resemble those of our non-H-zone Purkinje cell responses. Miles et al. (1980a) also

claimed that a simple spike modulation shift, if it ever

occurred, might be a secondary effect of VOR gain change because

simple spike response to head rotation contains a component

proportional to eye velocity. However, in rabbits' H-zone Purkinje cells, the eye-velocity sensitive component accounts for

only about one-fourth of the Purkinje cell responses to head

A H

232

180

c VOR Gain

1.0

0.5

0·0 before

D 90·

A

after

35%

A~A A f1:. A •

o·

Figure 3 : Changes in simple spike modulation of a Purkinje cell correlated to adaptation of the horizontal VOR25.

A, B, head velocity (H), eye velocity (E) in degree/sec, and simple spike discharge (SS) in pUlse/sec recorded in the dark before (A) and after (B) one hour adaptation of the VOR. Time calibration, lsec. C, gain change in the adaptation of the horizontal VOR. D, polar diagram for modulation of simple spike discharge of Purkinje cells induced by head rotation in the dark. Values for modulation estimated in five successive measurements are indicated by open triangles (before) and closed (after the adaptation). Arrow connects mean values.

rotation 19 . In monkeys' H-zone cells, elimination of the eye velocity component by lesion of the vestibular nuclei preserved the modulation shift of simple spikes in VOR adaptation24 ,25. Thus, the modulation shift of simple spikes in VOR adaptation at least contains primary plastic events in the cerebellar cortex, which contribute to control brainstem reflex pathways from the

233

vestibular canal to ocular motoneurons (Fig. 1).

Taken together with the complex spike responses discussed in the previous section (Fig. 2), these results imply that when complex spikes modulate out phase during visual-vestibular adaptive stimulation, the simple spike response to head rotation in darkness becomes more inphase, and that when complex spikes

modulate inphase, the simple spike response becomes more

outphase. These reciprocal patterns of complex spike modulation

and simple spike modulation shift are consistent with the

assumption that the synaptic transmission from the vestibular

mossy fibers through parallel fibers to Purkinje cells

(represented by simple spikes) is depressed by conjoint

activation of climbing fibers through visual pathways 9. Next, we

will discuss the direct evidence at the cellular level for the

assumption.

5. Effects of Climbing Fiber Activation on Simple Spikes

The climbing fiber impulses produce both a powerful EPSP and a

prolonged plateau-like potential in dendritic branches of Purkinje cells. In addition to this primary effect, climbing

fiber impulses produce depressant or facilitatory after-effects

on the simple spike activity of Purkinje cells 16 , and interact with parallel fiber-Purkinje cell transmission 5 ,12,13,22. A

sustained inhibitory after-effect is represented by a release

excitation, which occurs subsequent to destruction or cooling of

the inferior olive. The spontaneous discharge rate of simple

spikes doubles, and the typical irregular simple spike discharge

changes to a regular steady firing pattern. These drastic effects may provide a cellular basis of the difficulty in eye movement

study which attempts to evaluate the functional role of the

olivo-cerebellar system by lesion of the inferior olive itself,

as already pointed out in the first section.

The interaction of the climbing fiber impulse with parallel

fiber-Purkinje cell transmission can provide a synaptic basis of

the plasticity assumption described in the previous section. Ito,

Sakurai & Tongroach (1982) first tackled this challenge by

electrically stimulating a vestibular nerve (20 Hz), which

projects to the flocculus as mossy fibers, conjointly with

climbing fibers (4 Hz), and they found that conjunctive

stimulation depressed the Purkinje cell responsiveness to

vestibular nerve stimulation for more than one hour. As the

conjunctive stimulation did not affect the responsiveness of

putative basket cells, it was concluded that signal transmission

at parallel fiber-Purkinje cell synapses undergoes a long-term

depression after conjunctive activation of the climbing fiber on

the Purkinje cell.

Next, this conclusion was more directly tested by stimulating

234

A E

Beto re _" .. -1tt++i11 I 1-11 I ..-..-.IIIJ.WIi 111 ......... 111 11 ...... 11111 ........ 1 III u.w11l1l.LJJJ<1i ItU-.w1llu.w1l 111.1..-1 ~ 1 15mv

8

I

c

15 min ,HIli I o

30 min

I'

~ ~10mV 20 s

1illUljllllillJUIIUIllillltlliJitiJil

1IIIIJIIIIIUJIIlIJlIIIIIIIIIIIIJI'liLlL

F

~ 120mV

+

G

1 ~ 15mv

H

~ 1

Is mV

10 ms

Figure 4 : Cerebellar synaptic plasticity revealed by intra­dendritic recording in in vitro slices 22.

Effect of conjunctive climbing fiber-parallel fiber stimulation. A-D, strip-chart records of the membrane potential of an impaled Purkinje cell taken a few minutes before, during, IS min after and 30 min after conditioning, respectively. In A, C and D upward deflections indicate parallel fiber-mediated e.p.s.p.s, and downward deflections, the hyperpolarization induced by intracellular injection of d.c. current steps (O.SnA, 200 ms). E, G and H are averaged e.p.s.p.s obtained during recording of A, C and D, respectively. F, a record during the conjunctive stimulation (4 Hz for 2S s) shown in B. Upward and downward arrows indicate climbing fiber and parallel fiber stimulation, respectively.

parallel fibers through microelectrodes placed in the molecular layerS ,13. Two separate parallel fiber beams impinged dendritic arborization of a single Purkinje cell. The firing index of the Purkinje cell from each beam was used as an index of parallel fiber-Purkinje cell synaptic transmission. When one of the two beams was stimulated in conjunction with the inferior olive, the transmission from that parallel fiber beam was specifically depressed; the unchanged. This

transmission from the other beam remained specificity suggests that the effect of a

conjunctive stimulation is not due to general depression in Purkinje cells, which occurs after the repetitive stimulation of

QI "0 ::I

120

~ 110 a. E co 100 ci vi ci eli .s

90

235

3

~ 80 c: co ~ u QI

f6' ... c: QI

~ QI 0..

70

60

50

T -20 -10 o 10 20 30 40 50

Figure 5: Time course of plastic changes of e.p.s.p amplitude after various types of conditioning 22 .

e, conditioned with conjunctive climbing fiber-parallel fiber stimulation; 0, with parallel fiber stimulation paired with white matter stimulation just subthreshold for climbing fibers; 0 ,with climbing fiber stimulation alone. Time 0 indicates the end of conditioning. Vertical bars represent ±S.E.M. The figure attached to each plotted point represents the number of cells involved in the measurements. Statistical comparisons were made between the pre-conditioning data and post-conditioning data for each plotted point (SAS GLM ANOVA). *, P<0.05; **, P<O.Ol; ***, P<O.OOl.

climbing fibers (8-10 Hz), but that the effect is localized to a site in the parallel fiber-Purkinje cell synapses involved in conjunctive stimulation. It should be noted that this long-term depression did not occur when either parallel fibers or climbing fibers were stimulated without conjunction. Recently, these results with in vivo extracellular recordings were more directly confirmed by intracellular recording of parallel fiber EPSPs in in vitro slices of the guinea-pig cerebellum22 • Since the parallel fiber synapses on the Purkinje cell were located on the dendrites, microelectrodes were inserted into the dendrite rather than the soma of the Purkinje cells to observe the EPSP without attenuation through membrane cable properties. Parallel fibers were stimulated with an electrode placed near the pial surface of the molecular layer, while climbing fibers were activated by the stimulation of the white matter. Conjunctive stimulation of climbing fibers and parallel

236

fibers at 4 Hz for 25 s induced depression of parallel fiber­mediated EPSPs in Purkinje cells, both in the peak amplitudes and in the slopes (Fig.4). The depression was about 30% on average

and lasted for more than 50 min (Fig.5). No such depression occurred when the intensity of the white matter stimulation was set just subthreshold for the climbing fiber innervating the Purkinje cell under study. Instead, the parallel fiber-mediated

EPSPs were moderately potentiated for a period ranging from 10 to 50 min (Fig.5).

6. Conclusions

The importance of the olivo-floccular system in the VOR

adaptation was first demonstrated by lesion experiments in rabbits and cats. Multiple olivo-floccular projections which

terminate differentially in longitudinal bands were also revealed in cats and rabbits. Corresponding experimental data in monkeys are not yet available. However, the complex spike discharge of a single Purkinje cell recorded in the monkey flocculus revealed specific optokinetic responsiveness and functional

localization similar to those in rabbit flocculus. Simple spikes response to head rotation in darkness were found to undergo systematic modulation shift during adaptive gain change

of the HVOR in the Purkinje cells which were selected according to the visual climbing fiber input and the inhibitory effect upon

horizontal VOR pathways. This observation in monkeys accords with

those in rabbits. Climbing fiber activation of a Purkinje cell results in short and

long-term after-effects on the Purkinje cell activity. Among them, conjunctive stimulation of climbing fibers and parallel

fibers was shown to produce a long-lasting depression of parallel fiber-Purkinje cell synapses, which does not occur by parallel

fiber stimulation alone. These results are consistent with the hypothesis that the flocculus Purkinje cells adaptively control

the HVOR through their simple spike activity under the

influence of retinal error signals conveyed by visual climbing

fiber pathways.

REF ERE N C E S

1. Balaban, C. D., Ito, M. & Watanabe, E. Neurosci. Lett. 12, 101-105 (1981).

2. Barmack, N. H. & Hess, D. T. J. Neurophysiol. 43, 151-164

(1980). 3. Barmack, N. H. & Simpson, J. I. J. Neurophysiol. 43, 182-206

(1980).

237

4. Demer. J. L. & Robinson, D. A. J. Neurophysiol. 47, 1084-1107 (1982) •

5. Ekerot,C.-F. & Kana. M. Brain Res. 342, 357-360 (1985). 6. Gerrits, N. M. & Voogd, J. Neuroscience L, 2971-2991 (1982). 7. Haddad, G. M., Derner, J. L. & Robinson, D. A. Brain Res. 185,

265-275 (1980). 8. Hoffman, K. P., Behrend, K. & Schopprnann, A. Brain Res. 115,

150-153 (1976). 9. Ito, M. The cerebellum and neural control. Raven Press. (1984). 10. Ito, M, & Miyashita, Y. Proc. Jpn. Acad. 21, 716-720 (1975). 11. Ito, M., Orlov, I. & Yamamoto, M. Neuroscience L, 1657-1664

(1982) • 12. Ito, M., Sakurai, M. & Tongroach, P.

324, 113-134 (1982). J. Physiol.(London)

13. Kano, M. & Kato, M. Nature 325, 276-279 (1987). 14. Karachot, L., Ito, M. & Kanai, Y. Exp. Brain Res. 66,

229-246 (1987). 15. Lisberger, S. G., Miles, F. A. & Zee, D.S. J. Neurophysiol.

52, 1140-1153 (1984). 16. McDevitt,C.J., Ebner, T.J. & Bloedel, J.R. Brain Res. 237,

484-491 (1982). 17. Miles, F. A., Braitman, D. J. & Dow, B. M. J.Neurophysiol.

43, 1477-1493 (1980a). 18. Miles, F. A., Fuller, J. H., Braitman, D. J. & Dow, B. M.

J. Neurophysiol. 43, 1437-1476 (1980b). 19. Miyashita, Y. Exp. Brain Res. 55, 81-90 (1984). 20. Miyashita, Y. & Nagao, S. Neuroscience Res. 1, 223-241

(1984) . 21. Robinson, D. A. J. Neurophysiol. 39, 954-969 (1976). 22. Sakurai, M. J.Physiol.(London) 394, 463-480 (1987). 23. Simpson, J. I. & Hess, R. In: Baker, R. & Berthoz, A.(eds),

Control of Gaze by Brain Stem Neurons, Elsevier, Amsterdam. 351-360 (1977).

24. Watanabe, E. Brain Res. 297, 169-174 (1984). 25. Watanabe, E. Neuroscience Res. 1, 20-38 (1985).

A Kalman Filter Theory of the Cerebellum

Michael Paulin California Institute of Technology

Pasadena, CA.

Abstract. A variety of evidence suggests that the cerebellum is directly involved in certain sensory tasks. The specific hypothesis developed in this article is that the cerebellum is a neural analog of a Kalman-Bucy filter, whose function is to estimate state variables of the motor system and of external dynamical systems.

1. Introduction.

Cerebellar dysfunction causes a loss of coordination of movements and a decreased ability to automate movement patterns. Sensory deficits are not seen. I will argue, however, that the cerebellum is directly involved in certain sensory tasks such as predicting trajectories and analyzing the mechanical properties of objects.

The argument is in two parts. First, the assignment of motor control and learning functions to the cerebellum is based on circumstantial evidence: cerebellar dysfunction affects motor control and learning. Yet in some species, notably the weakly electric teleosts, the cerebellum is associated with purely sensory systems. Reconsideration of the way in which the cerebellum is associated with various motor systems suggests that assigning the cerebellum a purely sensory function could explain its apparent motor functions.

The second part of the argument is from stochastic, multivariate control theory. Tracking and controlling dynamical systems requires accurate knowledge of system variables, including some which may

240

not be directly observable. The set of variables required to predict or control a process is called the state vector. The Kalman-Bucy filter (KBF) is a state estimator for dynamical systems. For sensory estimation and motor control by the nervous system there must be a neural analog of the KBF.

Together, these two lines of reasoning suggest an hypothesis: the cerebellum is a neural analog of a KBF. In this paper I will expand on this hypothesis and show that it is testable.

2. Correlates of Cerebellar Development.

Cephaloscyl/ium is a small, sedentary elasmobranch whose "hunting" strategy consists of lying in wait at night for a prey to drift in front of its nose, then drawing in the prey with an electroreceptor -mediated "yawn reflex" [1]. Cephaloscyllium has a relatively large, well developed cerebellum, as do other elasmobranchs [2].

The african mormyrids and the south american gymnotids are freshwater teleosts which have independently evolved "gigantocerebella". These fish have unexceptional motor behaviors in comparison to similar fish. Mormyrids and Gymnotids explore their environment by measuring distortions in electric fields generated by electric organs in their tails [3].

Frogs orient towards prey with coordinated leg and trunk movements, and strike with rapid ballistic tongue movements. The frog cerebellum is small, almost vestigial, with a poorly differentiated, primitive neuronal organization [4].

The rat cerebellar cortex, particularly the hemispheres and posterior vermis, has strong inputs from superficial sensors in the perioral regions [5]. There is a similarity between the way the rat uses its perioral sensors and the way the weakly electric teleosts use their electric organs to explore the environment. The motor aspect of the sensory task is greatly simplified for the fish because the electric organ has only one degree of freedom in its response, the discharge waveform is uniform and it is generated by a uniform control signal [3], yet relatively the fish has a very much larger cerebellar cortex than the rat. Proprioceptive inputs from the head and particularly the jaws, which should be quite important for mouth movement control,

241

are relatively rare in the rat cerebellar cortex [5].

In primates the hemispheres of the cerebellar cortex receive major sensory inputs from the hands and fingers [6]. Movements of the hands and fingers are finely controlled, and these movements are used for active exploration in the same way that the rat uses its mouth and the weakly electric teleosts use their electric organs.

All of this suggests that the apparent correlation between cerebellar development and the complexity of fine motor control might be due to an association between certain sensory systems and fine motor control systems which are used together, and that the real function of the cerebellum may be a sensory one. The sensory function would be particularly obscure in the common laboratory animals - rats, cats and primates - which use finely controlled movements, whose control requires accurate dynamic state estimation, for sensory purposes.

3. Optimal State Estimation.

The mean of a set of observations Xl .•. xN estimates the true

value of the observed variable x,

1 N ~=N LX. (1)

i = 1 1

In using (1) we assume a model of the process which generates x, and a model of the measurement process:

x = 0, (2)

where • i is unbiased, random error in the ith measurement. The first

equation expresses the assumption that the observed variable is constant. If the • i 's form a Gaussian white noise sequence then ~ is

the minimum variance unbiased, or optimal, estimator of x given Xl

. .. xN' Now suppose that observations Xl .•• xN are made at

equal time intervals. Let ~ (N I M) represent the optimal estimate of

x (N) based on the first M observations. Equation (1) can be rewritten in recursive form:

242

2 (N IN) = ~(N IN-1) + (lIN) [xN-~ (N I N-1)] (3).

The optimal estimate based on N observations is obtained by adding a

correction term to a prediction based on the first N -1 observations. The correction term is proportional to the difference between the prediction, ~(NIN-1), and the measurement, xN' If the variance of

the measurements is cr2 then the variance of the optimal estimate at

the Nth time step is cr 2/N, which is the measurement error variance multiplied by the weight of the error correction term in (3). That is, the weight of the correction term is the model-to-data noise ratio.

Note that (3) is a re-arrangement of (1). The advantage of writing it in this form is that we can apply the same approach to estimating variables which are generated by stochastic, dynamic processes. That is, variables which are outputs of systems of differential equations with noise, generalizing (2).

Any system of linear ordinary differential equations can be written as a larger set of first order equations (For an example , see [7]). These first order equations can be written in matrix-vector form

x(t) = Ax(t) + Bu(t)

y(t) = Cx(t) (4)

in which the outputs y (t) of the original set of differential equations are recovered from the state vector x by the second equation. When these equations to have additive noise,

x(t) = Ax(t)+ Bu (t) + Gv(t)

y (t) = Cx (t) + w(t) (5) •

the filtering problem is to estimate the state vector x (t) from measurements of the system output. y (t ). If the noise terms v (t ) and w (t) are Gaussian then the optimal estimator for x (t) is the Kalman-Bucy filter (KBF). The KBF is a continuous-time extension of the Kalman filter [8]. which estimates states of systems governed by

243

difference equations. Derivation of the KBF [9, see e.g.10] is beyond the scope of this article, but the recursive estimator (3) is in fact a Kalman filter for (2). In general the KBF resembles (3) sufficiently well that by following the reasoning which leads to (3), one can develop a good intuition for what the KBF does and how it works.

The KBF for (5) is:

• ~(t) = ~(t) + Bu(t) + K[y(t)-~(t)] (6a)

(6b)

(6c)

Where Q and It are the covariance matrices of v and" respectively. Explicit time dependencies of the matrices on the right hand sides of these equations have been omitted for clarity. Comparing (3) with (6a'. it can be seen that in each case the optimal state estimate is formed by making a prediction based on the model and the current state estimate, and modifying this prediction by a correction term based on the difference between predicted and actual measurements. The weight of the correction term, K (t), is computed by (6c) and is related to the prediction error covariance P, which is computed by the matrix Ricatti equation (6b).

The underlying principle of the KBF is that it recursively fits the solution to the equations of motion (6) which best fits the incoming data. By extrapolating along the fitted trajectory, a KBF can predict future states of the system. The predictions of a KBF are conditionally optimal, minimum variance unbiased estimates of the true future state of the system. The KBF is an extension of the Wiener filter, developed to predict aircraft flight paths for fire control during world war II [11].

A simple interpretation of the KBF is that it is a dynamic model of the observed process which is made to mimic that process by driving it with an error signal. The factor K (t) in equation (6a), called the Kalman gain, specifies the strength of the coupling between the observed system and the observer or KBF. K (t) varies continuously during filtering, even if the observed system has fixed dynamics. An alternative heuristic interpretation is to think of (6) as defining a continuous trade-off between model predictions and data, which

244

depends on the assumed relative reliability (covariances) of the model and the data.

Because the transfer function of a KBF varies continuously the KBF is often called an adaptive filter. This usage of "adaptive" differs from the usage in physiology. The KBF is adaptive because it knows in advance what the equations of motion of the observed system are. For example, the equations of motion of an aircraft are nonlinear, but locally linear so that the craft can be stabilized by a linear regulator. The linear regulator requires a state estimate, which can be provided by a KBF designed using the aircraft's equations of motion linearized along a specified trajectory. Although the linearized equations of motion are time-varying, they are always the same for a particular maneuver. A KBF can optimally estimate aircraft state during a maneuver as long as it recognizes (or is told) what maneuver is being performed and as long as it has learned (or has been programmed with) the optimal filtering dynamics for that maneuver.

To those unfamiliar with estimation theory, it may seem like "cheating" for the estimator to know in advance what the equations of motion of the observed system are. But Kalman's theorem shows that it is necessary to build information about the observed system into the observer and, if you recall eqn (3), even the simple problem of estimating a constant from unbiased measurements involves a strong assumption about equations of motion of the observed variable, i.e. that it really is a constant. The process of determining the equations of motion of an unknown system is called system identification. If there were such a thing as a neural analog of a KBF, then adaptation or learning would correspond to system identification.

4. The Vestlbulo-Ocular Reflex.

The vestibulo-ocular reflex (VOR) helps to stabilize the eyes in space during head movements. In a classical, deterministic control theory framework the dynamics of eye stabilization are easy to model. Ignoring the geometry of eye movement control and the fact that multiple sensory systems are involved (not because these factors are unimportant but because they clutter the analysis of dynamics at this stage), we can say that the aim of the VOR is to match eye rotational velocity to head rotational velocity during head movements. The transfer function of the VOR, then, should be TVOR(s) = 1.

245

This top-level description of VOR dynamics can be decomposed into components. The transfer function of the vestibular system, TVES' can be measured by recording from vestibular primary afferents while moving the head [12]; and that of the eye motor plant, T EYE, by

recording eye movements while stimulating eye motor nuclei or nerves [13]. The transfer function of the central nervous system component of the VOR, TCNS' can be estimated from TVOR = T EYE TeN S Tv E S. From such measurements and calculations, it

appears that T C N S is a leaky integrator with a time constant of

several seconds [14].

Implicit in the feedforward control model of VOR is that the control task, the desired eye trajectory, is known exactly, measured and provided to the brain by the vestibular apparatus. But how accurate is the vestibular data? Phase estimates in estimated cat semicircular canal primary afferent transfer functions have standard errors of several degrees across the band from O.01-4Hz [12]. Averaging of primary afferent signals undoubtedly takes place, but there must be some uncertainty or noise in semicircular canal primary afferent signals. No matter how small this noise level is, theoretically it could be reduced even further using Kalman filtering.

Because of the small time delay in the reflex path between the labyrinth and the eyes (on the order of 10msec in mammals [15]), the sensory signal runs a few msec behind that which the motor output controller ideally requires, so there is an opportunity for prediction to improve VOR performance. It has been suggested that an analog of Taylor-series prediction might be used in the VOR [16], but differential operators are very sensitive to data noise and this approach is not practical [11, 17]. Arbib and Amari [17] and Paulin [18] noted that Kalman filtering techniques could perform this prediction task and suggested that it may be fruitful to consider neural analogs of the Kalman filter as models for VOR and cerebellum.

The one major difference between the feedforward control approach and the KBF approach to modelling VOR is that a feedforward controller is a compensator for sensor (vestibular) and actuator (eye motor) dynamics, while a KBF is a predictor for an external dynamic process (head movements). The KBF makes good predictions if its assumptions are correct, but may make very bad predictions if they are not. The hypothesis that a KBF analog exists in the VOR control

246

network can be tested by looking for errors in compensatory eye movements when the pattern of head movements changes unexpectedly.

Collewijn [19] has adapted the rabbit VOR by prolonged slow oscillation on a turntable at 116 Hz. After such training, the rabbits generate 116Hz compensatory eye movements during oscillation on the turntable in the dark, even when the turntable is oscillated at other frequencies, and sometimes showed spontaneous 1/6 Hz eye oscillations. This nonlinear response property can be explained ad hoc by adding a "pattern generator" to the feedforward control model. On the other hand, the KBF model predicts this behavior. The sinusoid could be tracked by the VOR with tiny errors during training and eventually the dynamics of a KBF to optimally predict the signal could be learned. By equations (6), once the correct KBF dynamics for this situation have been learned, incoming data is used initially to put the KBF onto the correct trajectory (correct phase) but the prediction error covariance falls rapidly and so does the weight of the correction term. Soon after the movement is initiated, it is generated almost entirely by the "internal model". This happens regardless of the actual pattern of head movement, because KBF dynamics are set or selected on the basis of expected head movement dynamics, not actual head movements. Given their recent experience, it is not surprising that the rabbits expect 1/6 Hz sinusoidal stimuli when they are placed on the turntable. Their VOR dynamics are optimized for this stimulus, thus producing 1/6 Hz sinusoids in response to other stimuli.

The rabbits must use cues other than the vestibular data itself to determine what dynamics are appropriate for a given situation. It should be possible to condition the rabbits with two different turntable frequencies and two different external cues (e.g. lights or tones). After conditioning in the light, the VOR in the dark should switch to a particular frequency when the corresponding cue is presented, independently of actual turntable frequency. Stronger experimental tests could be designed by using complex turntable movements and predicting from equations (5) and (6) exactly how a KBF would respond to unexpected head movement dynamics.

Other "strange" aspects of VOR dynamics are straightforward properties of the KBF. Frequency selectivity, in which the VOR selectively alters its gain near a particular frequency after prolonged single-frequency oscillation while a subject is wearing reducing

247

lenses [20], is the same phenomenon as "prediction" or "pattern generation" just discussed. That is, it is due to the filter tuning its dynamics to match the dynamics of the process which generates the sensory data. The KBF theory predicts that some frequency-selective gain change will occur after prolonged single-frequency oscillations in the light even without special lenses. "Phase crossover", a change in phase responses near single-frequency gain changes [20], is a consequence of the link between gain and phase curves of linear systems [21], and will occur in any linear or nearly linear dynamical system including a steady-state KBF.

5. Cerebellar Function.

Evidence for a KBF analog in the VOR implies the existence of a KBF analog in the cerebellum, since the VOR is a modified cerebellum (or vice versa) in which the vestibular nuclei correspond to the deep cerebellar nuclei. This is also consistent with the evidence in section 1 suggesting that the cerebellum is really a sensory processor whose correct operation is essential for motor control.

KBFs are widely applied in guidance and control systems to provide state estimates to feedback regulators which stabilize the systems. Symptoms of KBF dysfunction in such control systems include the classical symptoms of faulty feedback regulation - tremor, overshoot and undershoot - which were noted long ago to be similar to the symptoms of cerebellar dysfunction [11].

If the cerebellum is a KBF analog which provides state estimates to the motor system, then without a cerebellum smooth, coordinated movement would be impossible. Unable to provide the spinal motor system with the state predictions it needs to generate accurate movements, the cerebellar-damaged patient would have to resort to isolated, short movements whose consequences would be somewhat predictable. This also is a symptom of cerebellar dysfunction. According to the KBF theory, it is not because the cerebellum normally coordinates or "glues" individual movements together, but because the cerebellar patient - lacking vital information for multivariate control - is forced to break extended smooth movements with many degrees of freedom into small, manageable pieces.

The sensory nature of cerebellar function may be difficult or

248

impossible to observe directly in human subjects, as our sensory systems are all linked to motor or homeostatic feedback control systems and any observed deficits could be attributed to a loss of "cerebellar motor function". However, decreased ability to follow and predict acoustic pitch or intenSity modulations, rapid changes in thermal stimuli or passive tactile stimuli may be noted in cerebellar patients.

6. Cerebellar Learning

Cerebellar dysfunction causes learning deficits. VOR adaptation slows or stops when the cerebellar cortex is damaged [6,15). The cerebellum has a role in conditioned reflex learning [22). However, it is important to distinguish between the function of the cerebellum (by hypothesis, dynamic state estimation and prediction) and the fact that the cerebellum develops and acquires that function through ontogeny and experience. No observations support the claim [6, 22) that the cerebellum has a 'special role in motor learning, except as a consequence of its special dynamic role.

First, if the cerebellum is damaged, those aspects of motor control which require an intact cerebellum cannot be learned - you simply cannot learn how to do a particular task if you don't have the machinery to perform the task. Second, consider what happens if we let down the front tires on an automobile. Not only does an experienced driver have trouble with fine control of movement, but a novice driver has trouble learning how to control its movements. The fact is that the tire valve is part of the motor learning system, but a careful analysis of automobile function would correctly dismiss this "fact" as trivial and misleading. The learning deficit is due to the inability to precisely relate control actions to sensory consequences of those actions, and has nothing to do with the modifiability or intelligence of the damaged component.

Specific synaptic modification rules have been demonstrated in the cerebellar cortex [6, 23). These rules are similar to the Marr-Albus-Fujita cerebellar learning rules [6]. But recovery from peripheral vestibular damage does not require an intact cerebellar cortex, and neural changes associated with such recovery are not restricted to the cerebellum or cerebellar cortex [24, 25). Muscles respond adaptively to changing demands placed on them; fast muscle

249

fibers gradually acquire physiological, biochemical, histochemical and ultrastructural properties of slow fibers when they receive stimulation similar to that normally received by slow fibers [26]. In Bullfrogs, the statacoustic nerve sprouts after it has been sectioned and normal vestibular function is recovered [27]. There is only one component of the VOR - the labyrinth - for which there is not direct evidence of localized, adaptive dynamic modification, and this may be because nobody has looked for it there (such adaptation could easily be accomplished by control of endolymph viscosity, for example). Evidence overwhelmingly favours the view that learning in sensorimotor systems is distributed among all components, including the cerebellum but not restricted to the cerebellum or even the eNS, in adult animals.

What rules govern learning in the cerebellum? The most popular learning rules for models of adaptive sensorimotor control and cerebellar learning have been error feedback correlation rules, that is, the perceptron rule and its relatives the Widrow-Hoff "adaline" rule and backpropagation [6, 28]. An alternative type of learning rule is reinforcement learning [29], in which elements actively probe to determine their contribution to system performance. The error correlation rules have faster convergence and are (arguably) more elegant than reinforcement, but they require error evaluators or "teachers", while reinforcement systems do not. Error correlation rules are not suitable as models of learning by the cerebellum, but a version of reinforcement learning may be.

The VOR is a special case in which there seems to be a natural "teacher" - retinal image slip - which could drive an error correlation adaptive mechanism. For example if the cerebellar cortex is an analog of an adaptive array filter [30] so that its output may be written

~ = LW.X. J J

(7)

in terms of element transfer functions x i with weights "i' then

mean squared error

(8)

where y * is the intended output, can be minimized using the learning rule

250

~w j = - ex ~e(t)xj(t)dt (9)

Where aCt) = yet) -y* (t) and the update "j -> "j + L\"j

is made at the end of time step T [30]. Equation (9) is derived by substituting (7) into (8) and differentiating with respect to "j.

Thus. rule (9) causes the weights "j to move towards the

mean-squared error minimizing point along a steepest descent path of the gradient of mean squared error. Experimental results show that VOR parameters move approximately on a steepest descent path of

HUMAN

o 1t PHASE LAG

CAT

o

Figure 1: When humans or cats are fitted with prisms which reverse the visual field from left to right. the VOR undergoes a remarkable change in which the gain first decreases rapidly. then the phase reverses and the gain slowly increases again, so that the reflex runs "backwards". This plot shows data digitized from plots published by Melvil/-Jones [321. smoothed and re-plotted in the gain-phase plane. along with level curves of mean squared error in retinal image slip as a function of VOR gain and phase. Human gain and phase measurements during adaptation are shown on the left half of the plot. cat measurements on the right in "mirror image" coordinates. Figures alongside the adaptation trajectory represent time in days since prisms were donned. • marks the optimal gain-phase point while wearing prisms.

251

mean squared retinal image slip during learning [31; Figure 1], indicating that some kind of gradient-tracking mechanism is involved in VOR learning.

However, there is a time delay between output from the cerebellum and the return of corresponding error signals, mainly in muscle activation and in retinal slip transduction. We must re-write (9) as

(10)

where ~'t is the time delay. In complex s-plane (Laplace transform) notation (used interchageably with time domain notation in [30]), the

delay factors out of (10) as a complex exponential exp (- i..1't)

The change in "j will not be in the direction of steepest descent of v,

but at an angle ~t to the gradient. The parameters should spiral

towards the optimal pOint, and above some frequency, ~ 't > x/2, so the weight vector will actually begin to spiral away from the optimal point. Readers not familiar with Laplace notation may simply note that if e (t) is a sinusoid then above some frequency, • (t) is

negatively correlated to e (t -~'t) so the learning rule (10) causes the weights to diverge and the mean squared error to increase.

A number of experimenters have reported that in mammals there is a time delay in optokinetic feedback of about 70 msec [15, 33, 34, 35]. Collewijn [36] measured slip latencies in the rabbit nucleus of the optic tract of 60 plus or minus 10 msec, and argued that the closed-loop delay in the optokinetic system is greater than 80 msec [37]. Barmack and Hess [38] reported 40-180 msec latencies in the Inferior Olive, which is the putative source of error signals to the cerebellar cortex in error correlation models of VOR learning [6]. Shorter latencies (around 30 msec) have been reported for responses to flash stimuli, but these do not take into account either the time delay in muscle activation or in slip detection in the retina. It seems reasonable to conclude that there is a time delay of at least 40 msec in the closed loop from the cerebellum and back again, via eye movements and retinal slip. 40 msec is 900 at 6.25 Hz, so compensatory responses above 6.25 Hz. cannot have been learned by a rule similar to (9). If the delay is actually near 70 msec, then the adaptation limit of (10) is near 3.5 Hz.

The human VOR is compensatory above 6.25 Hz. Several groups have

252

reported compensatory responses up to 10 -15 Hz [39, 40, 41, 42) and, in one experiment [43), it was shown to be compensatory up to 30 Hz. The rabbit VOR improves with frequency up to 10Hz and is very accurate at that frequency [44). Monkeys have good compensatory VOR at higher frequencies also [45, 46].

There can be no doubt that error correlation is not involved in VOR learning at high frequencies. Whether it is involved at low frequencies can be convincingly tested by using sinusoidal training stimuli and looking for the predicted bias from the steepest descent path during learning. In any case other mechanisms must be involved, unless the VOR is "hard wired" to be compensatory at high frequencies.

As mentioned above, a possible alternative class of learning rules for VOR adaptation is reinforcement learning. The underlying principle of reinforcement learning is that neurons are as self-interested agents with preferences for certain inputs over others. They actively probe by modifying their outputs and observing the consequences for their own inputs, selecting self-adjustments which have "pleasant" consequences. This is analogous to human search strategy in, for example, setting the controls on a stereo: make small adjustments to each knob, continue in the same direction if the sound improves and go back the other way if it gets worse. The searcher does not need to understand how knob-turning is related to sound output, nor even to have a preconceived idea of what the "best" sound is. For detailed discussion of this approach, see [29). I will proceed by giving an example of a reinforcement learning rule similar to those studied by Barto et al [29).

Suppose that performance of a neural net is a smooth function of parameters p 1 PN' and that each neuron receives this

performance signal. For simplicity, I assume that there is one parameter per neuron, although the argument is easily generalized to multiple parameters per neuron. If neuron i changes its parameter by an amount .1Pi' the change in performance v (p) is

(11 )

and the product of this with the original perturbation is

253

approximately

av 2 AVAPi :::: api (Ap) (12)

If the neurons all make independent, zero-mean perturbations in their values, then the conditional expected value of the product L\VL\p i

for neuron i given L\Pi is the same as before, because the expected

change in V (p ) due to perturbations by other neurons is zero. Therefore, performance can be improved by making small test perturbations L\'1'P i and following up with "permanent" changes

based on the consequences of the probe, by the rule

Pi = Pi (13)

The expected adaptation path is a steepest descent path of the performance function, but the actual path will vary randomly around that path because of the stochastic gradient evaluation procedure.

Adaptive array filters using rules similar to this parameter perturbation rule have been implemented in telecommunications systems [47]. Widrow-Hoff (adaline) filters (Le. using rule (9)) are often preferable because they converge very quickly (within a few cycles of the input signal after a change in external parameters -another reason to suspect the claim that VOR learning, which is slow, uses this learning rule), but the parameter perturbation mechanism can be applied in cases where the adaline rule cannot. In particular, there can be long time delays between the initial probing parameter perturbations and permanent modifications.

For example, a network can "behave" on a time scale of seconds, yet perturb its parameters only once an hour. Even if the network does not find out for a minute or two about the outcome of its actions it can still improve its performance, hour-by-hour, by applying rule (13), because at the end of each hour most of the expected change in performance is due to the change in parameters made at the beginning of that hour. This assumes that the task is fixed on a time scale somewhat longer than a week. In practice the best time scale (bandwidth) for perturbations depends on the bandwidth of the signal being filtered and on the size of time delays in performance feedback.

254

Performance of a KBF can be evaluated without an external error measurement or performance monitoring system. From equation (6a) it can be seen that error evaluation is intrinsic to the filter. If the system model is correct, that is, if the signal being filtered was actually generated by a system whose equations of motion are as

assumed in the filter design, then ~ is an unbiased estimate of x with the specified error variance and

n (t) = Y (t) - ex (t) (14 )

is white noise. Intuitively, if the model is correct then differences between model predictions and observations are random. The signal n (t) is called the innovations. Whiteness and lack of bias in the innovations is a measure of filter performance. In principle an adaptive KBF can be constructed by computing a measure of temporal correlation and/or bias in the innovations and applying a reinforcement learning rule similar to (13). Oman [7] has suggested that a mechanism similar to this may be involved in vestibular reflex adaptation and in the etiology of motion sickness, which often accompanies adaptation to novel visual-vestibular environments.

7. Implementation.

The differential equations (6) defining the KBF can be implemented in an analog electronic network, althou.gh it is simpler and more flexible to program discrete-time versions of these equations into a digital filter. How might a neural network (I mean with real neurons, not "neurons" as conceived by connectionist modellers) be constructed so that its dynamics are a workable approximation to (6)? I have no answer to this question at present, but I will suggest some considerations which may be important.

The KBF is an integral operator, and current efforts to model the "neural integrator" of the VOR are relevant. These efforts are based around the principle that long time constants necessary for VOR control could be constructed by positive feedback in neural circuit loops. The main problem considered by modellers has been to find a way of doing this which leads to robust, stable circuits, which explains observed VOR dynamics and which uses realistic neuron components and circuit connectivity. Good models satisfying these

255

criteria have been constructed [49,50], according to which integration on the VOR pathway is due to positive feedback between vestibular nuclei (VN) and nucleus prepositus hyperglossi, and to crossed reciprocal VN-VN connections. The equivalent for cerebellum in general is the proposal [51,52] that long time constants could be constructed in positive feedback loops between deep cerebellar nuclei, reticular nuclei and the red nucleus.

According to the KBF theory, VOR and cerebellar dynamics (transfer function) should vary rapidly during movements. Assuming that the integrative dynamics are indeed constructed in extracortical feedback loops, this could be acheived in various ways by the cerebellar cortex. What the cortex would have to do is alter time constants in the underlying loops by altering loop gains, or equivalently by shunting loop activity. Tsukahara [53] found that when the cerebellar cortex is pOisoned with picrotoxin, electric shocks delivered to the red nucleus cause an explosive buildup of red nucleus - cerebellar interpositus nucleus activity, showing that time constants in this loop are normally controlled by cerebellar cortical inhibition of the interpositus. Waespe, Cohen and Raphan [54] have reported that the nodulus and uvula of the cortex of the vestibulocerebellum seem be involved in rapid adjustments of VOR time constants during head movements.

Thus, it is plausible that long time constants required for KBF dynamics are constructed in loops between cerebellar nuclei, reticular nuclei and the red nucleus, and that the function of the cerebellar cortex is to vary filter dynamics during filtering operations in order to optimize the accuracy of filter output. Boylls [51] developed a model of cerebellar involvement in locomotion in which climbing fiber inputs from the inferior olive cause temporary, localized lifting of cerebellar cortical inhibition of interpositus neurons, leading to a buildup of interpositus-red nucleus activity. A mechanism similar to this could be useful in a KBF model of cerebellum, in which incoming climbing fiber signals lead to rapid corrections of filter dynamics. This would mean that the inferior olive, or some earlier element, acts as a bias detector, triggering corrections to filtering dynamics when significant errors are detected. The correcting signal is also a performance signal, telling the cerebellar cortex that it has chosen the wrong dynamics for the current task.

A design feature that is probably relevent to cerebellar function is

256

array filter architecture [47]. There are two principle advantages of array filters, these are robustness - the system fails gracefully when components fail catastophically - and the fact that the system is linearly parameterized even if the elements themselves are highly nonlinear. Adaptive arrays with nonlinear elements can approximate arbitrary nonlinear functionals [55] and predict signals generated by nonlinear processes [56]. Given the parallel organization of the nervous system in general, and the cerebellum in particular, it is difficult to ignore adaptive arrays as possible realistic models for cerebellar structure and function.

8. Conclusion.

It is not necessary to treat the relatively large cerebella of sluggish elasmobranchs and the giant cerebella of weakly electric teleosts as anomalous, requiring their own special theory which gives the cerebellum a specific function related to electrolocation. A simple explanation which encompasses these cases and the apparent motor role of the cerebellum in "higher" vertebrates is that the cerebellum is a sensory processor for estimating and predicting states of dynamical processes. Cerebellar state estimation is applied to the motor system, providing motor pattern generators with state variables which they need to produce smooth, coordinated multivariate movements. It may also be applied for purely sensory tasks, tracking and predicting external dynamical processes such as moving objects or temperature fluctuations.

Estimation and prediction by the cerebellum may be more sophisticated than the analogous function of a KBF. For some situations, notably for VOR control and regulating limbs during movements, the KBF may be a very good abstract model of the role of the cerebellum. How this could be implemented as a neural net is a problem for future research.

Acknowledgements

This report is based on material originally published in (18), supported by a grant from the Medical Research Council of New Zealand to John Montgomery. I would like to thank John Montgomery and Michael Arbib for their support, and Jim Bower for useful discussions of cerebellar sensory function.

257

Bibliography

[1] T. C. Tricas. Bioelectric-mediated Predation by Swell Sharks, Cephaloscyllium ventriosum. Copeia 4, 1982.

[2] R. G. Northcutt. Brain Organization in the Cartilaginous Fishes. In: The Sensory Biology of Sharks, Skates and Rays. E.S. Hodgson and RF Mathewson (eds.). ONR, 1978.

[3] T.H. Bullock and W. Heiligenberg. Electroreception. Wiley, 1986. [4] R.R. Llinas, J.R. Bloedel and D.E. Hillman. Functional Characterization of

Neuronal Circuitry of Frog Cerebellar Cortex. Journal of Neurophysiology 32, 1969.

[5] W. Welker. Spatial Organization of Somatosensory Projection to Rat Cerebellar Cortex: Functional and Connectional Implications of Fractured Somatotopy (Summary of Wisconsin Studies). In: New Concepts in Cerebellar Neurobiology, Alan R. Uss Inc, 1987.

[6] M. Ito. The Cerebellum and Neural Control. Raven, 1984. [7] C.M. Oman. A Heuristic Mathematical Model for the Dynamics of Sensory

Conflict and Motion Sickness. Acta OtoLaryngologica s392, 1982. [8] R.E. Kalman. A New Approach to Unear Filtering and Predition

Problems. Journal of Basic Engineering, March 1960. [9] R.E. Kalman and R.S. Bucy. New Results in Unear Filtering and Prediction

Theory. Journal of Basic Engineering, March 1961. [10] G.L. Goodwin and K.S. Sin. Adaptive Filtering, Prediction and Control.

Prentice-Hall,1984. [11] N. Wiener. Cybernetics. 1948. [12] D.L.Tomko, R.J. Peterka, R.N. Schor and D.P. O'Leary. Response Dynamics of

Horizontal Canal Afferents in Barbiturate Anaesthetized Cats. Journal of Neurophysiology 45, 1981.

[13] M.G. Paulin and J.C. Montgomery. Elasmobranch Eye Motor Dynamics Characterized using Pseudorandom Stimulus. Journal of Comparitive Physiology A 158, 1986.

[14] M.G. Paulin and J.C. Montgomery. A Vestibulo-Ocular Reflex with no Head Movement. Biological CyberneticS 55, 1986.

[15] D.A. Robinson. Adaptive Gain Control of the Vestibulo-Oculer Reflex by the Cerebellum. Journal of Neurophysiology, 36,1976.

[16] A.J. Pellionisz and R.R. Uinas. Brain Modeling by Tensor Network Theory and Computer Simulation. The Cerebellum: Distributed Processor for Predictive Coordination. Neuroscience 4, 1979.

[17] M.A. Arbib and S. Amari. Sensori-Motor Transformations in the Brain (With a Critique of the Tensor Theory of Cerebellum). Journal of Theoretical Biology 112,1985.

[18] M.G. Paulin. Cerebellar Control of Vestibular Reflexes. Ph.D. TheSis. University of Auckland, 1985.

[19] H. Collewijn. Integration of Adaptive Changes of the Optokinetic Reflex, Pursuit and the Vestibulo-Ocular Reflex. In: A. Berthoz and G. Melvill-Jones (eds.) Adaptive Mechanisms of Gaze Control. Elsevier, 1985.

[20] S.G. Usberger, FA Miles and L.M. Optican. Frequency-Selective Adaptation: Evidence for Channels in the Vestibulo-Ocular Reflex? Journal of Neuroscience 3,1983.

[21] J.J. DiStefano, A.R. Stubberud, I.J. Williams. Feedback and Control Systems. Schaums Outline Series, McGraw-Hili, 1967.

[22] R.F. Thompson. The Neurobiology of Learning and Memory. Science 233,1986.

258

[23] M. Kano and M. Kato. Quisqualate Receptors are Specifically Involved in Cerebellar Synaptic Plasticity. Nature. 325,1987.

[24] R.R. Llinas and K. Walton. Vestibular Compensation: A Distributed Property of the Central Nervous System. In: Integration in the Nervous System. H. Asanuma and V.J. Wilson (eds.).lgaku-Shoin, 1979.

[25] R.R. Uinas and A.J. Pellionisz. Cerebellar Function and the Adaptive Feature of the Central Nervous System. In: Adaptive Mechanisms of Gaze Control. A. Berthoz and G. Melvill-Jones (eds.). Elsevier, 1985.

[26] S. Salmons. Functional Adaptation in Skeletal Muscle. In: The Motor System in Neurobiology. Evarts, Wise and Bousfield (eds.). Elsevier, 1985.

[27] A. Newman, A. Kuruvilla, A. Pereda and V. Honrubia. Regeneration of the 8th Cranial Nerve. I. Anatomical VerifICation in the Bullfrog. Laryngoscope 96, 1986.

[28] J.L. McClelland, D.E. Rumelhart and the PDP Research Group. Parallel Distributed Processing, volume 1. MIT Press, 1986.

[29] A.G. Barto (ed.). Simulation Experiments with Goal-Seeking Adaptive Elements. DTIC Technical Report AFWAL-TR-84-1022, 1984.

[30] M. Fujita. Adaptive Filter Model of the Cerebellum. Biological Cybernetics, 45, 1982.

[31] M. Fujita. Simulation of Adaptive Modification of Vestibulo-Ocular Reflex with an Adaptive Filter Model of the Cerebellum. Biological CyberneticS 45,1982.

[32] G. Melvill-Jones. Plasticity in the Adult Vestibulo-Ocular Reflex Arc. Philosophical Transactions of the Royal Society, London (B). 278, 19n.

[33] R.H.S. Carpenter. Movements of the Eyes. Pion, 19n. [34] P.J. Dallos and R.W. Jones. Learning Behavior of the Eye Fixation and Control

System. IEEE AC-8, 1963. [35] G.J. St. Cyr and D.H. Fender. Nonlinearities in the Human Oculomotor System:

Time Delays. Vision Research, 24, 1969. [36] H. Collewijn. Direction-Selective Units in the Rabbits Nucleus of the Optic

Tract. Brain Research 100, 1975. [37] H. Collewijn. The Modifiability of the Adult Vestibulo-Ocular Reflex. TINS 1,

1979. [38] H.H. Barmack and D.T. Hess. Multiple-Unit Activity Evoked in Dorsal Cap of

Inferior Olive of the Rabbit by Visual Stimulation. Journal of Neurophysiology 43,1980.

[39] D. Hyden, B. Larsby and L.M. OcIkvist. Quantification of Eye Movements in Light and Darkness. Acta Otolaryngologica s406, 1984.

[40] U. Reker. The High Frequency Umit of the Fundamental Vestibulo-Ocular Reflex. Arch. Oto-Rhino-Laryngology 239, 1984.

[41] A. Benson. Compensatory Eye Movements Produced by Angular Oscillation. Proc. XXV IUPS, Munich, 1971.

[42] A.A. Skavenski, R.M. Hansen, R.M. Steinman and B.J. Winterson. Quality of Retinal Image Stabilization During Small Natural and Artificial Body Rotations in Man. Vision Research 19, 1979.

(43] G.M. Gauthier, J. Piron, J. Roll, E. Marchetti and B. Martin. High Frequency Vestibulo-Ocular Reflex Activation THrough Forced Head Rotation in Man. Aviation, Space and Environmental Medicine 55, 1984.

[44] B.J. Winterson, H. Collewijn and R.M. Steinman. Compensatory Eye Movements to Miniature Rotations in the Rabbit: Implications for Retinallrnage Stability. Vision Research 19, 1979.

(45] E.L. Keller. Gain of'the Vestibulo-Oculer Reflex in Monkey at High Rotational Frequency. Vision Research 18, 1978.

259

(46) J.M. Furman, D.P. O'leary and J.W. Wolfe. Application of Unear Systems Analysis to the Horizontal Vestibulo-Ocular Reflex in the Alert Rhesus Monkey using Pseudorandom Binary Sequence and Single Frequency Sinusoidal Stimulation. Biological Cybernetics 33, 1979.

(47) D.H. Brandwood and C.J. Tarran. Adaptive Arrays for Communications. Proc lEE F 129, 1982.

(49) H.L. Galiana and J.S. Outerbridge. A Bilateral Model for Central Neural Pathways in Vestibulo-Ocular Reflex. Journal of Neurophysiology 51, 1984.

[50) S.C. Cannon, D.A. Robinson and S. Shamma. A Proposed Neural Network for the Integrator of the Oculomotor System. Biological Cybernetics 49 1983.

[51) C.C. Boylls. A Theory of Cerebellar Function with Applications to Locomotion. I. The Physiological Role of Climbing Fiber Inputs in Anterior Lobe Operation. COINS Technical Report 75C-6, 1975.

[52) C.C. Boylls. A Theory of Cerebellar Function with Applications to Locomotion. I. The Relation of Anterior Lobe Climbing Fiber Function to Locomotor Behavior in the Cat. COINS Technical Report 76-1, 1975.

[53) N. Tsukahara. The Properties of the Cerebello-Pontine Reverberating Circuit. Brain Research 19, 1972 ..

[54) W. Waespe, B. Cohen and T. Raphan. Dynamic Modification of the Vestibulo-Ocular Reflex by the Nodulus and Uvula. Science 228,1985.

[55) G. Palm. On the Representation and Approximation of Nonlinear Systems. Part I: Discrete Time. Biological CyberneticS 34, 1979.

[56) A. Lapides and R. Farber. Nonlinear Signal Processing Using neural Networks. Prediction and System Modeling. Los Alamos National Laboratory Preprlnt LA-UR-87-2662,1987.

Conditioning and the Cerebellum

John W Moore and Diana EJ Blazis University of Massachusetts, Amherst, Massachusetts

Mathematical models of learning that describe the effects of training with no re­gard for motor output are incomplete and consequently difficult to represent within natural systems. This chapter summarizes our efforts to reconcile a model that describes real-time topographical features of the classically conditioned nictitating membrane response (NMR) of the rabbit with knowledge about the cerebellum, the brain region thought to be most crucial for NMR conditioning. The NMR is a pro­tective response resulting from retraction of the eyeball and the passive sweeping of the NM over the eye. The conditioned NMR is a graded, adaptive response.

Our strategy for modeling the conditioned NMR was to constrain the Sutton­Barto (SB) model (Sutton and Barto, 1981; Barto and Sutton, 1982) to predict re­sponse topography in simple conditioning situations involving a single conditioned stimulus (CS) and unconditioned stimulus (US). The original SB model was pre­sented in the context of the extensive behavioral literature on NMR conditioning. Constraints were derived partly from electrophysiological experiments conducted on awake, behaving rabbits (Berthier and Moore, 1986; Desmond and Moore, 1986). The physiologically constrained SB model retains the ability of the original to de­scribe multiple-CS phenomena such as blocking, conditioned inhibition, and higher­order conditioning. We refer to this variant of the SB model as the Sutton-Barto­Desmond (SBD) model (Blazis and Moore, 1987; Moore, Desmond, Berthier, Blazis, Sutton, and Barto, 1986).

As shown in Figure 1, a conditioned NMR begins well after CS onset and rises gradually in a ramped or S-shaped fashion within the CS-US or interstimulus in­terval (lSI). The CR attains a maximum at or near the temporal locus of the US, and then decays rapidly during the post-US period. This pattern of response to­pography is also reflected in the activity of neurons that have been identified in single-unit recording studies as being linked to the generation of CRs. For example, Desmond (1985; see also Desmond and Moore, 1986) described the activity of brain stem neurons recorded during classical conditioning of the rabbit NMR with a 350 ms tone CS (lSI of 350 ms). In a typical cell, spikes began to be recruited about 70 ms after CS onset. About 150 ms after CS onset, spike recruitment increased sharply and continued to increase throughout the remainder of the lSI. The mo­mentary rate of firing prior to the US rarely exceeded 200-Hz. After US offset, firing initiated by the US declined toward a baseline rate of about lO-Hz.

The Model

The SBD model is capable of modeling response topography because it assumes that the internal representation of a CS at the site of learning is a template for CRs observed at the periphery. The variable in the model representing the ith member of a set of CSs is denoted Xi' When C Si begins, the variable t in Equations 1-3,

262

ACQUISITION EXTINCTION

8' rial, 1 (Trial 1 /\

I I

8' )Trial, 2 rial 2

/\ I I I

8' rial,6 A-- rial 10

~

8'

(Trial 25 (Trial 25

CS US CS CS On On On Off

Figure 1. Simulated NMR topographies in acquisition and extinction. S' is the output variable of the Sutton-Barto-Desmond model of NMR conditioning obtained with a 250 ms lSI during acquisition and extinction in a forward-delay paradigm. US duration = 30 ms (from Blazis & Moore, 1987).

263

which represents successive time steps of 10-ms duration, is set equal to 1. The value of Xi is 0 when t = 1 and remains 0 until t = 7, i.e., 70 ms after the external onset time of C Si. At this point, Xi increases in an S-shaped fashion and levels orr at a maximum value of 1.0 by t = 25 (250 ms after C SI onset) and remains at this value until C Si offset, at which time Xi begins to fall exponentially to o. Thus, according to Equation 2, the output of the model, s(t), conforms to the temporal map or template provided by Xi. As the number of training trials increases, the variable V;(t) increases, and the CR becomes increasingly robust. This process is reversed over a series of extinction trials as shown in Figure 1.

As in the SB model, learning in the SBD model occurs according to a modified Hebbian rule which states that changes of the synaptic weight of CS i , the ith of a set of potential CSs, denoted ~ V;, are proportional to the product of the eligibility of CS;'s input to the learning element, Xi, and the difference between the current output, s(t), and the trace of preceding outputs, s(t) (defined below). At time t, ~ Vi is computed as follows:

~V;(t) = c[s(t) - S(t)]Xi(t), (1)

where c is a learning rate parameter, 0 < c ::; 1. Changes in V; occur both during and after the occurence of CSij the US is

important only insofar as it affects the term s-s during computational epochs (time steps). The fully parameterized version of the model described in other reports (e.g., Moore et ai, 1986) is based on time steps of 10 ms. Thus, a single training trial with a CS-US interval of 350 ms might involve over 400 computations of ~ V;, depending on a number of real-time variables such as the rate of decay of the eligibility factor, Xi-

The output of the learning element at time t, denoted s(t), is defined as the weighted sum of inputs from all CSs, where Xi(t) refers to the magnitude of CS i at time t:

n

s(t) = L Vi(t)Xi(t) + >"(t). (2) i=l

N(t) is a US effectiveness variable. The trace of s, denoted s, is computed by:

s(t + 1) = f3s(t) + (1 - f3)s(t), (3)

where 0 ::; f3 < 1. s can be interpreted as the element's prediction or expectation of its output during the current time step.

The parameter f3 determines the rate of decay of s. Wit.h the lO-ms time step assumed in our simulation studies, f3 should range from 0.5 to 0.6. If it exceeds 0.6, the ability of the model to reach stable weights is disrupted and a "blow up" of weights can occur. The large weights result in unrealistic rectangular-shaped response profiles. Values of f3 less than 0.5 result in low amplitude CRs that do not blend with unconditioned responses (URs) and inappropriate negative weights at less-than-optimal CS-US intervals (Blazis and Moore, 1987). Given the 10-ms time step, this narrow range of acceptable f3 values implies that the relationship between sand s can be described in continuous time by an exponential function with a time constant on the order of 30 ms. Hence, for any change in s on a given time step, s closes to within one percent of s within the ensuing 10 time steps, or 100 ms. This relationship imposes a key constraint on circuit models that would

264

describe where s - s is computed and how this term interacts with CS; input, X;,

at sites of synaptic modification (Eccles, Sasaki, and Strata, 1967).

Implementation in Cerebellum

Several laboratories have demonstrated that the cerebellum plays an essential role in the acquisition and generation of conditioned NMRs (Thompson, Donegan, Clark, Lavond, Lincoln, Madden, Manounas, Mauk, and McCormick, 1987; Yeo, Hardiman, and Glickstein, 1984, 1985a-c, 1986). In this section we consider two frameworks for implementing the SBD model in cerebellar cortex. We begin by briefly discussing the hypothesis that changes of V occur through modification of parallel fiber (PF)jPurkinje cell (PC) synapses (Ito, 1984; Thompson, 1986) (sub­scripts denoting different CSs are suppressed in ensuing discussion of variables Vi, Xi, and Xi). A cerebellar PC can in principle receive inputs representing many dif­ferent CSs, the climbing fiber input seems a natural means for providing input from the US, and the cell has basically a single output channel with only limited axon collateralization. Furthermore, cerebellar PCs have been shown to respond to CSs in a CR-related manner (e.g., Berthier and Moore, 1986).

A number of investigators have expressed doubts that learning is mediated by modification of PF JPC synapses (see Bloedel, 1987; Bloedel and Ebner, 1985; Lis­berger, Morris, and Tychsen, 1987; Llinas, 1985), particularly regarding the assump­tion that such learning depends on climbing fiber inputs. We therefore consider the possibility that changes of V occur at mossy fiber (MF)jgranule cell synapses and does not depend on climbing fibers. In addition, we sought an implementation that does not cast climbing fibers as the conveyor of reinforcement. This hypothesis represents a novel approach to cerebellar involvement in classical conditioning.

Figures 2 and 3 lay the groundwork for discussing schemes for implementing the SBD model and NMR conditioning in the cerebellum. Figure 2 summarizes cerebellar and brain stem structures and pathways involved in NMR conditioning (see, e.g., Berthier, Desmond, and Moore, 1987; Thompson, 1986). As noted above, it has been suggested that learning and generation of conditioned NMRs involves cerebellar PCs located in hemispheral lobule VI (HVI). Lesions of HVI have been reported to dramatically attenuate NMRs (Yeo et ai, 1985b) , and single-unit record­ing studies report CR-related patterns of activity by HVI PCs that are consistent with a causal role in this behavior (Berthier and Moore, 1986). In Figure 3, the numbers 1-3 along the top and letters A-D along the left-hand edge provide a set of coordinates that will facilitate discussion. Figure 3 omits some of details in­cluded in most textbook renderings of the cerebellum. For example, climbing fiber synapses onto PCs are not shown. The figure includes only those features needed later for integrating physiological evidence into a plausible circuit diagram for NMR conditioning under the constraints of the SBD model.

Site of Plasticity: Purkinje Cells

Assuming that changes in V occur at PF JPC synapses, where is s - s computed and how does this information reach an involved PC? One option is that s - s is computed within the postsynaptic cell itself and is therefore readily available to modify eligible synapses. Another possibility is that s - s is computed outside the PC and fed back by other circuit elements. This could occur in a number of ways

265

us

SpoV

Figure 2. Summary of cerebellar and brain stem circuitry and information flow medi­ating NMRs. Solid lines indicate strong projections; dashed lines are used to indicate projections that are comparatively weak or not universally agreed upon. The vertical dashed line represents the medial axis of the brain stem. CS information (represented bilaterally) gains access to hemispherallobule VI (HVI) via mossy fibers arising from pontine nuclei (PN). This information, as well as information about the US, also goes to supratrigeminal reticular formation (SR) which is represented bilaterally. SR has been implicated in NMR conditioning as an independent parallel system that appears to be essential for expression of CRs (see Desmond and Moore, 1982; 1986). US information gains access to both SR and HVI via sensory trigeminal neurons. Spinal trigeminal nucleus pars oralis (Spo V) provides synaptic drive to motoneurons in the accessory abducens nucleus (AAN). SpoV also projects to HVI. There is a direct mossy fiber projection and an indirect climbing fiber projection via the dorsal accessory olivary nucleus (DAO). Both sets of projections are bilateral. The output of HVI is relayed to cerebellar nucleus interpositus (IP) and from there to contralateral

red nucleus (RN). RN projection neurons terminate in AAN and SpoV to complete the circuit and initiate a conditioned NMR.

A

B

c

D

266

1 2 3

--------~-+------<~

~: PC axon collateral ~ y

MF

-------------------------------<J G -<J ---1

excitatory

inhibitory

1 ~r®~--.

Figure 3. Summary of cerebellar neural circuitry. A-D represent beams of parallel fibers (PF) in the molecular layer. These synapse onto Purkinje cells (PC) and basket cells (Ba), one of which is indicated on the C beam. Basket cells inhibit off-beam PCs, as exemplified by the basket cell on the C beam and the PC on the D beam. The

latter is shown as inhibiting a projection neuron in cerebellar nucleus interpositus

(IP) which, in turn, excites a projection neuron in contralateral red nucleus (RN)

leading to some here-unspecified response. Mossy fiber (MF) terminals and granule

cells (Gr) occupy the granular layer. Three granule cells are shown, and two receive

inhibitory input from Golgi cells (Go). Both Golgi cells are excited by PF beams.

The Golgi cell under 2 is shown receiving inhibitory input from a PC axon collateraL

267

(Figure 4). For example, the PC might send an axon collateral to local circuit elements that provide feedback for computing 8 - 8. PC axon collaterals have been reported as terminating on Golgi cells, basket cells, granule cells, and other PCs. Were we to rule out feedback from PC axon collaterals, the two remaining sources of feedback are climbing fibers and mossy fibers. For example, feedback information could arise as efference from collateral output from Spo V in the course of driving AAN motoneurons: As Figure 2 indicates, in addition to its role as the locus of interneurons mediating unconditioned reflexive extension of the NM to direct stimulation of the eye, SpoV projects to HVI of cerebellar cortex. The projection is either a direct one via mossy fibers, or indirect via climbing fibers originating in the dorsal accessory olive (DAO), the source of climbing fibers to HVI. Both projections could be involved in computing 8 - 8.

Having designated SpoV as a likely source of feedback used to compute 8 - 8,

consider the various ways this information might reach a PC for modification of PF fPC synapses. These are summarized in Figure 4. Figure 4A indicates several alternative means by which 8 - 8 might attain access to the PC, including the possibility noted above that it is computed within the cell. The extracellular routes include parallel fibers (PF), the climbing fiber (CF), or an indirect route via a basket cell (Ba). Yet another set of possibilities, shown in Figure 4B, is that one of the variables, either 8 or -8, is generated within the PC and the other term is contributed extracellularly. All of the schemes illustrated in Figure 4 require that computation of 8 - 8 does not compromise the PC's assumed role in generating CRs.

Site of Plasticity: Granule Cells

Consider now an implementation of the SBD model in which learning occurs at MF fgranule cell synapses, i.e., one synapse before the PF fPC stage of processing. The most compelling scheme suggested by the anatomical and physiological liter­ature is one in which granule cells compute changes in V via convergence of 8 - "8 from Golgi cells and x conveyed by mossy fibers. This convergence of 8 - 8 and x implements the Hebbian mechanism assumed by the model. Golgi cells appear to be particularly suitable for computing 8 - 8 for several reasons: They receive in­put from parallel fibers; mossy fibers, and collateral inputs from PCs and climbing fibers. Hence, in principle they could provide sites of convergence of information about 8 and 8. They are capable of modulating their output to reflect their input with little noise or signal distortion (Miles, Fuller, Braitman, and Dow, 1980; Schul­man and Bloom, 1981), a feature that suggests they could transfer feedback about NMR topography with high fidelity.

According to a study by Eccles, Sasaki, and Strata (1967), the temporal course of Golgi cell inhibition of information flow through the granular layer resembles the relationship between the variables sand 8 in the time domain. We suggest that Golgi cells compute 8 by acting on granule cells that receive 8 information simultaneously from mossy fibers. (Possible circuits 'by which 8 and 8 converge onto other Golgi cells for computation of 8 - s will be considered later.) Figure 5 summarizes how mossy fiber input carrying 8 information might be used to compute 8 in the way suggested by the Eccles et al (1967) study: 8 information carried by mossy fibers is converted to 8 by the action of Golgi cells. The model assumes that Golgi cells that convert 8 to s are activated by parallel fibers. A group of

A

(s - s)? PF ~

~

(CF

268

x ~ PF

B

A ~sors? 1 PF

s or s1 PF ~

(s-s)? sors1

CR CR ~ excitatory

--; inhibitory

Figure 4. Summary of possible sites of convergence of SBD model variables x and s - 8

for weight changes mediated by parallel fiber/PC interactions. Diagrams A and B

represent PCs with two parallel fiber (PF) inputs, a climbing fiber (CF) input, and a basket cell (Ba) input. In Figure 4A s - s might be transmitted to the PC by any route or it might be computed intracellularly. In Figure 4B the two components of

s - 8 are dissociated from each other so as to illustrate the possibility that each is contributed from a different source.

granule cells (Gr), represented in the lower left hand portion of the figure, receives mossy fiber input carrying s information as feedback from SpoV (coordinate Cl). The output of these granule cells passes s information through the granular layer with no distortion to form parallel fiber beam B. Beam B excites Golgi cells (Go) that impinge on members of a second class of granule cells that also receive s via mossy fibers from SpoV. We emphasize that these Golgi cells and the second class of granule cells receive s simultaneously, i.e., within the same 10 ms time step. The action of the Golgi cells on the second group of granule cells converts s to s. The output of the second group of granule cells forms the parallel fiber beam labeled A which transmits s to other circuit elements.

The circuit model assumes that learning occurs at synapses of granule cells that receive mossy fiber input labeled x (coordinate C3). Mechanisms that implement the eligibility of these synapses for modification, x, presumably reside within these granule cells. The factor s - s is contributed by Golgi cells that function as dif­ferential amplifiers. These Golgi cells receive s as excitatory input from Beam A (coordinate A2), as described above. In Figure 5 they are shown receiving s as an inhibitory input from a PC axon collateral such as the one indicated at coordinate B3. Notice that the Golgi cell under 2 in the figure is actually computing s - s.

269

1 2 3

A ------ , 8~ ~

'Y B

C

D

t t 8 8

r

y

~~ On damped PC lag

-4- 8

lag VX~

x

~ 'Y lead

8 8

----------------------<~7 lead

CR~~--0 Figure 5. Implementation of SBD model at mossy fiber/granule cell synapses. A-D are

parallel fiber beams as in Figure 3. From right to left: the variable 8 is fed back to cerebellar cortex by mossy fibers arising in brain stem spinal trigeminal nucleus pars oralis (SpoV in Figure 2) in two streams. One stream gives rise to parallel fibers that drive PCs with a firing pattern that lags the CR. This beam (B) excites Golgi cells (Go) that impinge on granule cells (Gr) excited by the other stream carrying 8 and thereby convert it into a beam of parallel fibers (A) carrying 8 information. This beam drives PCs with a firing pattern that lags the CR and is damped relative to the firing patterns of PCs on beams Band C. Beam A contributes 8 to Golgi cells that compute 8 - 8. The other term for this computation, 8, is provided either by axon collaterals from lag PCs on the B beam or by climbing fibers. These Golgi cells pass 8 - 8 to granule cells that receive CS information, x, and thereby mediate weight changes at these mossy fiber/granule cell synapses to the extent that they are eligible for modification. These granule cells give rise to a beam of PFs (C) and drives PCs proportionally to V x and with a firing pattern that leads the CR. This beam

also excites basket cells (Ba) that inhibit tonic firing of PCs on the D beam. These "off PCs of the lead type" disinhibit IP neurons and thereby initiate the sequence of motor commands that result in a CR.

270

We have referred to the computation as being s - s in interests of clarity. Because Golgi cells are inhibitory, the computation is effectively one of s - s with respect to the granule cells receiving x.

Because Golgi cells are inhihitory interneurons, when s - s is positive the tonic output of the Golgi cells is modulated downward, thereby disinhibiting granule cells that receive x on which they impinge. This disinhibition causes an increase in the weight of MF jgranule cell synapses to the extent that they are eligible for change. Similarly, when s - s is negative the tonic output of these Golgi cells is modulated upward. This increases granule cell inhibition and decreases the weight of eligible MF jgranule cell synapses. Pbssible mechanisms for bidirectional weight changes in Hebbian synapses are suggested by experimental work on associative long-term potentiation in hippocampal slices (Kelso, Ganong, and Brown, 1986) and by theoretical analyses of calcium dynamics in dendritic spines (Gamble and Koch,1987).

CR-Related PC Activity: Lead, Lag, and Damped Lag

The circuit model in Figure 5 implies the existence of various types of CR-related firing patterns by PCs. For example, the PC on the s beam (A) is labeled "damped lag" because its firing pattern during a CS presentation would lag behind a CR and would also rellect the slow rise and decay of spike recruitment implied by the relationship between sand s in the model. The PC on the s beam (B) is labeled "lag" because its firing rate during a CS presentation would mirror response topography but with a lag by virtue of the fact that s represents efference from the brain stem. CR-related "lead" PCs such as the one on the x beam (C) are also implied by the circuit. Berthier and Moore (1986) observed both lead and lag CR-related PCs in HVI during NMR conditioning. They also observed PCs that decreased their firing before the occurrence of CRs. These CR-related "off-cells" of the lead type on the D beam are posited as being the ones most responsible for generating the CR via Ip and RN.

CR-Related PC Activity: On and Off

As noted, Berthier and Moore (1986) observed PCs that increased their simple spike firing above pre-trial rates on trials with CRs. These cells were designated "on cells". However, CR-related "off-cells", PCs that decreased their firing rate below pre-trial rates, were also observed. Although off-PCs might arise from long term depression of PF JPC synapses (Ito, 1984; Thompson, 1986), the circuit model shown in Figure 6 provides an alternative explanation of off-PCs. These PCs are associated with the beam of parallel fibers labeled D. They become off-cells of the lead type when a CS is presented because of increased inhibition from basket cells (Ba) on the C beam (coordinate D3). (Like PC axon collaterals, basket cell axons tend to project perpendicularly to the longitudinal axis of the parallel fiber beam by which they are activated).

A CR is initiated when basket cell inhibition of PCs on beam D becomes suffi­ciently great to disinhibit IP neurons to which they project. These IP cells project in turn to RN neurons that excite the rellex pathways mediating the CR (Figure 2). Because mossy fibers from pontine nuclei do not send collaterals to deep cerebellar nuclei (Brodal, Dietrichs, and Walberg, 1986), the model does not assume that CSs activate IP neurons. They are assumed to be tonically activated by neural traffic

Lead On-PC

Lead Off-PC

Lag On-PC

Damped Lag On-PC

NM Response

CS+ Trial

271

11111111111111111111111111111111111111111111111111111111 111111111111111111111111

--------~-------~~----CS us

Figure 6. Renderings of CR-related simple spike firing patterns predicted by the circuit model in Figure 5. The baseline firing frequency for all four types of PCs is 100 Hz, and the CS-US interval on reinforced trials (CS+) is 350 ms.

unrelated to a particular stimulus, but this activation is normally suppressed by inhibition imposed by PCs. Hence, a CS releases this inhibition and the level of activation of IP neurons increases sufficiently to drive the RN neurons in the next stage of the efferent pathway of the CR.

Figure 6 summarizes the four types of firing patterns discussed in connection with Figure 5. As in the Berthier and Moore (1986) study, the interval between CS onset and US onset represents 350 ms. The CR in the figure begins 200 ms after the CS. Renderings of PC simple-spike firing were hand-crafted to resemble typical CR-related PC responses; they all assume baseline firing rates of 100 Hz, which is typical of that observed in our recording experiments. The firing patterns are labeled to correspond with the types of PCs indicated in the circuit model shown in Figure 5. Hence, the increase in firing rate of the Lag On-PC begins within a few ms after CR initiation. The increase in firing of the Damped Lag On-PC begins slightly later and persists slightly longer. The increase in firing rate of the Lead On-PC precedes the CR by more than 100 ms, and the Lead Off-PC begins to cease firing at this time, both profiles being typical of CR-elicited firing patterns observed by Berthier and Moore (1986).

Of the approximately 40 CR-related PCs reported by Berthier and Moore (1986), on-cells exceeded off-cells by 3:1, and lead and lag cells were equally distributed (ra-

272

tio of 1:1). Although possibly a coincidence, it is nevertheless interesting that these ratios are implied by the circuit model, provided of course that parallel fiber beams A-C are comparable in terms of number of fibers and levels of activation evoked by the variables s, s, and x, respectively. The correspondence between the model's predictions regarding the statistical distribution of CR-related PC types encourages further experimental tests of the model. Such experiments might provide: (a) re­liable separation of damped-lag PCs from the lag PCs; (b) evidence of Golgi cell activity related to the variables s or s, i.e., the implied but as-yet-unsubstantiated CR-related firing of the lag and damped-lag variety among Golgi cells; (c) evidence of CR-related neural traffic among parallel fibers (beams A-D in Figure 5).

Cerebellar Implementation of Multiple-CS Phenomena

Our effort to implement the SBD model in cerebellar cortex was guided by exper­imental evidence suggesting that this region of the brain is not only essential for robust CRs, but may be a site of learning as well. We have argued that a circuit implementation along the lines of Figure 5 is a promising candidate for implement­ing the model, but our discussion has been limited to conditioning with a single CS. How adequate is this implementation for conditioning protocols involving more than one CS?

The principal multiple-CS phenomena of interest are higher-order conditioning, blocking, and conditioned inhibition. Like virtually all contemporary learning the­ories, including the original SB model (Barto and Sutton, 1982), the SBD model predicts appropriate outcomes in simulations of these multiple-CS protocols (Blazis et ai, 1986; Moore et ai, 1986). The model predicts higher-order condit,ioning because it is basically an S-R contiguity theory of learning, albeit one with an in­formational structure: A second-order CR can be established provided the temporal relationship between the primary and secondary CSs is appropriate and provided the primary, initially trained CS is capable of evoking a CR. Should the primary CS lose its capacity to evoke a CR, e.g., through extinction, the secondary CS would eventually follow suit. The model predicts blocking because of its perceptron-like architecture and the fact that the learning rule in Equation 1 is basically a variant of the Widrow-Hoff LMS rule (see Sutton and Barto, 1981). Conditioned inhibi­tion also follows from Equation 1 because the synaptic weight, V, of a CS that is never reinforced can take on negative value when it is presented in combination with another CS that possesses a consistently positive weight.

The circuit model in Figure 5 could be readily be extended to encompass higher­order conditioning and blocking. All that would be required is a global broadcast of the variables 8 and s over a sufficiently large region of HVI to encompass inputs from many potential CSs. This would permit local computation of s - s by Golgi cells. In the case of higher-order conditioning, synaptic weights at granule cells that receive input from the second-order CS would increase to the extent that they remain eligible for change at the time their associated Golgi cells compute the large negative value of s - s that results from evocation of a CR by the primary CS. Should the primary CS lose weight, the weight of the secondary CS would decline as well. In blocking, the CS combined with the originally trained CS would not accumulate weight gains over reinforced trials as long as the original CS retained the capacity to evoke a CR; a CR sufficiently robust to preclude large values of s - s at the time of US onset. However, extending the circuit model in the manner sug-

273

gested here would not be appropriate because of experimental evidence indicating that blocking, higher-order conditioning, and certain other complex conditioning phenomena involve the participation of other brain region besides the cerebellum, part.icularly the hippocampal formation (Berger, Weikart, Bassett, and Orr, 1986).

The idea of global broadcasting of 8 and s would also allow for the creation of negative weights in granule cells that receive input from a CS assigned the role of conditioned inhibitor. However, in addition to being resistant to subsequent acqui­sition procedures, a conditioned inhibitor must be capable of opposing the evocation of a CR by a conditioned exciter when the two stimuli are presented together. It is not obvious how this would be accomplished in the cerebellar circuit model. It may be inappropriate to alter the present circuit model so as to produce such CR suppression, in any case, because there is no evidence that the cerebellum is in­volved in conditioned inhibition. For example, Berthier and Moore (1986) used a differential conditioning procedure in order to assess the CR-relatedness of cerebel­lar units. Differential conditioning is closely related to conditioned inhibition in that both procedures include reinforced and nonreinforced trials. Although CRs were suppressed on a high proportion of trials to the nonreinforced CS, there were no instances of unit activity related to CR suppression. Furthermore, lesion studies by Mis (1977) and others suggest that conditioned inhibition involves the partic­ipation of brain regions outside the cerebellum (see Yeo, Hardiman, Moore, and Steele-Russell, 1983). If conditioned inhibition involves processes extrinsic to the cerebellum, as seems likely, there may be no need to assume a bidirectional Hebbian mechanism in the model. Learning theorists have long recognized that bidirectional modifiability is not necessary to account for conditioned inhibition (e.g., Moore and Stickney, 1985).

Before moving on, one additional caveat about the SBD model deserves em­phasis. The SBD model is a member of a class of learning rules which allows for complete unlearning. By this we mean that the variable V can in principal decrease back to its inital value of 0.0 given a sufficient number of extinction trials. Other well known models based on the Widrow-Hoff LMS share this feature, including the Sutton-Barto and Rescorla-Wagner models. The difficulty is that conditioning is rather more robust and permanent than this. The residual imprint of acquisi­tion survives even the most extended extinction training. This becomes apparent whenever acquisition is reinstated-CRs begin to emerge in a fraction of the time required during original training. Thus, although the variable V in the model can return to its preacquisition starting point, we must bear in mind that somehow the trace of CR acquisition is retained. This permanent trace or engram is not captured by the SBD model, nor can it be assumed that its anatomical substrate is associated with MF /granule cells sysnapses. The permanent imprint of the CS-US association may involve other cerebellar and brain stem elements, possibly even Purkinje cells as envisioned be Marr and Albus and suggested by the work of Ito and his collaborators.

In sum, the SBD model is a mathematical description of a device capable of simulating an impressive array of facts about NMR conditioning at the behavioral and neurophysiological levels. Despite its potential ability to encompass multiple-CS effects within the framework of either a single neuron or a more elaborate circuit, an implementation of the model confined to the cerebellum is not entirely appropriate for multiple-CS phenomena. This caveat aside, our investigations of the SBD model

274

have nevertheless suggested a novel theory about the locus of synaptic changes for a real instance of conditioning and in a real nervous system. Further theoretical work should move toward a systems level of analysis that might point the way toward to a neural network architecture that not only accounts for phenomenology, but does so in a neurobiologically realistic manner. Further research on the circuit model could be conducted on a time scale compatible with the modeling of neural events such as action potentials, that is, in the domain of microseconds.

Neurobiological Correlates of x: Pons and Hippocampus

Implementing the SBD model in the cerebellar cortex raises questions as to how CS input is shaped so as to yield appropriate response topography, that is, what are the mechanisms that provide the preprocessing of CS inputs to learning elements? The possibility suggested by several investigators that such preprocessing involves interactions among the cerebellum, hippocampus, and pontine nuclei (see Berger, Weikart, Bassett, and Orr, 1986; Schmajuk, 1986). Basically, the idea is that CR templates are constructed in parallel and at multiple levels. At the level of pontine nuclei, stimuli are coded with respect to onsets and offsets. Although this would suffice for adaptive CR topographies in forward-delay paradigms with near-optimal ISIs, more elaborate coding schemes involving the hippocampus and cerebellum are engaged in more complex paradigms such as trace and long-lSI paradigms (see Hoehler and Thompson, 1980; Port, Mikhail, and Patterson, 1985; Port, Romano, Steinmetz, Mikhail, and Patterson, 1986; Solomon, Vander Schaaf, Norbe, Weisz, and Thompson, 1986)

The preprocessor needed by the SBD model should be capable of altering param­eters that shape x, thereby providing the necessary modulation of the amplitude and time course of a response. We have observed that the proper combination such alterations can yield longer-latency CRs (Blazis and Moore, 1987). However, changing the shape of x is not sufficient for appropriate topography in the trace conditioning paradigm, since x begins to decay at CS offset. The pre-processor (hippocampus) might override this problem by shifting x so that the rising phase of the CR begins after CS offset. Implementing a mechanism for changing the shape of, or shifting x could be based on feedback about the adaptability of the template presumably provided by the hippocampus under conditions of non-optimal ISIs or trace conditioning. There is evidence that the contribution of the hippocampus to NMR conditioning might be partially mediated by feedback via projections from cerebellum, as suggested by loss of CR-related hippocampal neuronal firing follow­ing lesions of cerebellum (Clark, McCormick, Lavond, and Thompson, 1984).

The present report has examined our effort to model CR topography by shaping CS input to a learning element in such a way as to provide a template for the response to be learned. The critical questions concern how and where the CS is represented within the brain. At this time, the most promising brain region may be the pontine nuclei, structures which form points of convergence of sensory inputs and possibly response modulating inputs from the hippocampus. Simultaneous recordings from pons and hippocampus, as well as hippocampus and cerebellum, may further elucidate the information processing underlying NMR conditioning.

Acknow ledgements

This research was supported by AFOSR grants 83-0125 and 86-01825, and NSF grant BNS 83-17920. The original simulation program was written by N.E. Berthier.

275

The authors wish to thank A. G. Barto, N.E. Berthier, J.E. Desmond, W. G. Richards, and N. A. Schmajuk for helpful comments and discussions.

References

Barto, A.G., & Sutton, R.S. (1982). Simulation of anticipatory responses in clas­sical conditioning by a neuron-like adaptive element. Behavioural Brain Re­search, 4, 221-235.

Berger, T. W., Weikart, C. L., Bassett, J. L., & Orr, W. B. (1986). Lesions of the retrosplenial cortex produce deficits in reversal learning of the rabbit nic­titating membrane response: Implications for potential interactions between hippocampal and cerebellar brain systems. Behavioral Neuroscience, 100, 802-809.

Berthier, N. E., Desmond, J.E., & Moore, J.W. (1987). Brain stem control of the nictitating membrane response. In I. Gormezano, W. F. Prokasy, & R. F. Thompson (Eds.), Classical Conditioning III. Hillsdale, NJ: Lawrence Erl­baum Associates.

Berthier, N. E., & Moore, J.W. (1986). Cerebellar Purkinje cell activity related to the classically conditioned nictitating membrane response. Experimental Brain Research, 69, 341-350.

Blazis, D.E.J., Desmond, J.E., Moore, J.W., & Berthier, N. E. (1986). Simulation of the classically conditioned response by a neuron-like adaptive element: a real-time variant of the Sutton-Barto model. Proceedings of the eighth annual conference of the Cognitive Science Society, , 176-186.

Blazis, D.E.J., & Moore, J.W. (1987). Simulation of a classically conditioned re­sponse: components of the input trace and a cerebellar neural network imple­mentation of the Sutton-Barto-Desmond model. Technical Report 87-74, De­partment of Computer and Information Sciences, University of Massachusetts, Amherst, MA.

Bloedel, J.R. (1987). Technical comment. Science, 298, 1728-1729.

Bloedel, J.R., & Ebner, T.J. (1985). Climbing fiber function: Regulation of Purk­inje cell responsiveness. In J.R. Bloedel, J. Dichgans, & W. Precht (Eds.), Cerebellar Functions (pp. 247-260). New York: Springer-Verlag.

Brodal, P., Dietrichs, E., & Walberg, F. (1986). Do pontocerebellar mossy fibres give off collaterals to the cerebellar nuclei? An experimental study in the cat with implantation of crystalline HRP-WGA. Neuroscience Research, 4, 12-24.

Clark, G. A., McCormick, D.A., Lavond, D.G., & Thompson, R.F. (1984). Effects of lesions of cerebellar nuclei on conditioned behavioral and hippocampal re­sponses. Brain Research, 291, 125-136.

Desmond, J.E. (1985). The classically conditioned nictitating membrane response: Analysis of learning-related single neurons of the brain stem. Ph.D. Disserta­tion. University of Massachusetts, Amherst, MA.

276

Desmond, J.E., & Moore, J.W. (1982). A brain stem region essential for the classically conditioned but not unconditioned nictitating membrane response. Physiology 8 Behavior, 28, 1029-1033.

Desmond, J.E., & Moore, J.W. (1986). Dorsolateral pontine tegmentum and the classically conditioned nictitating membrane response: Analysis of CR-related activity. Experimental Brain Research, 65, 59-74.

Eccles, J .C., Sasaki, K., & Strata, P. (1967). A comparison of the inhibitory action of Golgi and basket cells. Experimental Brain Research, 3, 81-94.

Gamble, E., & Koch, C. (1987). The dynamics of free calcium in dendritic spines in response to repetitive synaptic input. Science, 236, 1311-1315.

Hoehler, F.K., & Thompson, R.F. (1980). Effect of the interstimulus (CS-US) interval on hippocampal unit activity during classical conditioning of the nic­titating membrane response of the rabbit. Journal of Comparative and Phys­iological Psychology, 94, 201-215.

Ito, M. (1984). The Cerebellum and Neural Control. New York: Raven Press.

Kelso, S.R., Ganong, A.H., & Brown, T.H. (1986). Hebbian synapses in hippocam­pus. Proceedings of the National Academy of Science (USA), 83, 5326-5330.

Lisberger, S.G., Morris, E.J., & Tychsen, L. (1987). Visual motion processing and sensory-motor integration for smooth pursuit eye movements. Annual Review of Neuroscience, 10, 97-129.

Llinas, R. (1985). Functional significance of the basic cerebellar circuit in motor coordination. In J. D. J.R. Bloedel & W. Precht (Eds.), Cerebellar functions (pp. 230-247). New York: Springer-Verlag.

Miles, F.A., Fuller, J.H., Braitman, D.J., & Dow, B.M. (1980). Long-term adap­tive changes in primate vestibuloocular reflex. III. Electrophysiological ob­servations in flocculus of normal monkeys. Journal of Neurophysiology, 43, 1437-1476.

Mis, F.W. (1977). A midbrain-brain stem circuit for conditioned inhibition of the nictitating membrane response in the rabbit (Oryctolagus cuniculus). Journal of Comparative and Physiological Psychology, 91, 975-988.

Moore, J.W., Desmond, J.E., Berthier, N.E., Blazis, D.E.J., Sutton, R.S., & Barto, A.G. (1986). Simulation of the classically conditioned nictitating membrane response by a neuron-like adpative element: Response topography, neuronal firing, and interstimulus intervals. Behavioural Brain Research, 21, 143-154.

Moore, J.W., & Stickney, K.J. (198.5). Antiassociations: Conditioned inhibition in attentional-associative networks. In R.R. Miller & N.E. Spear (Eds.), Infor­mation processes in animals: conditioned inhibition (pp. 209-222). Hillsdale, N.J: Lawrence Erlbaum Associates.

277

Port, R.L., Mikhail, A.A., & Patterson, M.M. (1985). Differential effects of hip­pocampectomy on classically conditioned rabbit nictitating membrane re­sponse related to interstimulus interval. Behavioral Neuroscience, 99, 200-208.

Port, R.L., Romano, A.G., Steinmetz, J.E., Mikhail, A.A., & Patterson, M.M. (1986). Retention and acquisition of classical trace conditioned responses by hippocampallesioned rabbits. Behavioral Neuroscience, 100, 745-752.

Schmajuk, N.A. (1986). Real-time attentional models for classical conditioning and the hippocampus. Ph.D. Dissertation, University of Massachusetts, Amherst, MA.

Schulman, J.A., & Bloom, F.E. (1981). Golgi cells of the cerebellum are inhibited by inferior olive activity. Brain Research, 210, 350-355.

Solomon, P.R., Vander Schaaf, E.R., Norbe, A.C., Weisz, D.J., & Thompson, R.F. (1986). Hippocampus and trace conditioning of the rabbit's nictitating membrane response. Behavioral Neuroscience, 100, 729-744.

Sutton, R.S., & Barto, A. G. (1981). Toward a modern theory of adaptive net­works: Expectation and prediction. Psychological Review, 88, 135-170.

Thompson, R.F. (1986). The neurobiology of learning and memory. Science, 233, 941-947.

Thompson, R.F., Donegan, N.H., Clark, G.A., Lavond, D.G., Lincoln, J.S.,. Mad­den, J., Mamounas, L.A., Mauk, M.D., and McCormick, D.A. Neuronal sub­strates of discrete, defensive conditioned reflexes, conditioned fear states, and their interactions in the rabbit. In 1. Gormezano, W.F. Prokasy, and R.F. Thompson (Eds.), Classical conditioning, 3rd edition. Hillsdale, N.J.: Lawrence Erlbaum Associates, 1987, 371-400.

Yeo, C.H., Hardiman, M.J., & Glickstein, M. (1984). Discrete lesions of the cere­bellar cortex abolish the classically conditioned nictitating membrane response of the rabbit. Behavioural Brain Research, 13, 261-266.

Yeo, C.H., Hardiman, M.J., & Glickstein, M. (1985a). Classical conditioning of the nictitating membrane response of the rabbit: 1. Lesions of the cerebellar nuclei. Experimental Brain Research, 60, 87-98.

Yeo, C.H., Hardiman, M.J., & Glickstein, M. (1985b). Classical conditioning of the nictitating membrane response of the rabbit. II. Lesions of the cerebellar cortex. Experimental Brain Research, 60, 99-113.

Yeo, C.H., Hardiman, M.J., & Glickstein, M. (l98.5c). Classical conditioning of the nictitating membrane response of the rabbit: Ill. Connections of cerebellar lobule flVI. Experimental Brain Research, 60, 114-126.

Yeo, C.H., Hardiman, M.J., Moore, J.W., & Steele-Russell, 1. (1983). Retention of conditioned inhibition of the nictitating membrane response in decorticate rabbits. Behavioural Brain Research, 10, 383.

CONTRIBUTORS

Shun-ichi Amari Department of Mathematical Engineering and Instrumentation Physics University of Tokyo Tokyo, 113 JAPAN

Michael A. Arbib Center for Neural Engineering University of Southern California Los Angeles, CA 90089-0782, USA

Diana E. 1. Blazis Psychology Department University of Massachusetts Amherst, MA 01003, USA

Robert Desimone National Institute of Mental Health Building 9, Room INI07 Bethesda, MD 20892, USA

Masahiko Fujita Department of Mechanical Engineering Nagasaki Institute of Applied Science Abamachi, Nagasaki, 851-01 JAPAN

Kunihiko Fukushima NHK Science and Technical Research Laboratories Kinuta, Setagaya, Tokyo, 157, JAPAN

Okihide Hikosaka Department of Physiology Tooho University School of Medicine 5-21-16 Omori, Tokyo 143 JAPAN

Michiaki Isobe Mitsubishi Electric Corporation Amagasaki, Hyogo, 661 JAPAN

Mitsuo Kawato ATR Auditory and Visual Research Perception Laboratories Twin 21 Bldg., MID Tower Shiromi 2-1-61, Higashi-Ku Osaka, 540, JAPAN

S. E. Hampson Information & Computer Science Department University of California Irvine, CA, USA

Sei Miyake ATR Auditory and Visual Research Perception Laboratories Twin 21 Bldg., MID Tower Shiromi 2-1-61, Higashi-Ku Osaka, 540, JAPAN

Yasushi Miyashita Department of Physiology Faculty of Medicine Uni versity of Tokyo Tokyo, 113 JAPAN

John W. Moore Psychology Department University of Massachusetts Amherst, MA 01003, USA

Jeffrey Moran Laboratory of Clinical Studies DICBR, NIAA Bethesda, MD 20892, USA

Koichi Mori Department of Physiology Faculty of Medicine . University of Tokyo Tokyo, 113 JAPAN

Michael Paulin Biological Sciences California Institute of Technology Pasadena, CA, USA

Nestor A. Schmajuk Center for Adaptive Systems Boston University Boston, MA 02215, USA

Shigeru Shinomoto Department of Physics Kyoto University Kyoto 606, JAPAN

Hedva Spitzer Technion Haifa, ISRAEL

Mriganka Sur Department of Brain & Cognitive Sciences Massachusetts Institute of Technology Cambridge, MA 02139, USA

Ryoji Suzuki

280

Department of Mathematical Engineering and Instrumentation Physics University of Tokyo Tokyo, 113 JAPAN

D. J. Volper Information & Computer Science Department University of California Irvine, CA, USA