the nature of reinforcement in cerebellar learning

NEUROBIOLOGY OF LEARNING AND MEMORY 70, 150–176 (1998)ARTICLE NO. NL983845

The Nature of Reinforcementin Cerebellar Learning

Richard F. Thompson, Judith K. Thompson, Jeansok J. Kim,1

David J. Krupa,2 and Paul G. Shinkman3

Program in Neuroscience, University of Southern California, 3614 Watt Way, HNB 522,Los Angeles, California 90089-2520

In a now classic study, W. J. Brogden and W. H. Gantt (1942, American Journal ofPhysiology, 119, 277–278) demonstrated that movements (limbs, head, eyelid) elicited bydirect electrical stimulation of certain regions of the cerebellum (particularly the ansiformlobe) could be trained to respond to neutral auditory or visual conditioned stimuli withappropriate pairing. In recent work we have replicated these results in detail and presentedconsiderable evidence that the reinforcing or ‘‘teaching’’ pathway so activated for the learn-ing of discrete movements is the inferior olive–climbing fiber projection system to thecerebellum. Very strong evidence now indicates that the memory traces for this ‘‘skilledresponse’’ learning are formed and stored in the cerebellum. The climbing fiber systemand inhibitory pathway from the interpositus nucleus to the inferior olive appear to forma neural instantiation of the Resorla–Wagner formulation of classical conditioning andaccounts for the ‘‘cognitive’’ phenomenon of blocking. It is concluded that reinforcement inthis form of learning is not due simply to contiguity/contingency or to unconditioned stimu-lus aversiveness, per se, but rather to activation of a particular brain circuit, here theclimbing fiber system, a circumstance that may apply to other forms of learning, withother reinforcement circuits, as well. q 1998 Academic Press

Basic associative learning, which results from exposure to relations among events in theworld, is the way organisms, including humans, learn about causal relationships in the world.For both modern Pavlovian and cognitive views of learning and memory, the individual learnsa representation of the causal structure of the world and as a result of experience, thenadjusts this representation to bring it in tune with the real causal structure of the world,thus striving to reduce any discrepancies or errors between its internal representation andexternal reality. (Paraphrased from Rescorla, 1988)

The nature of ‘‘pleasure–pain’’ has been discussed and debated from thebeginning of humanity. Pleasure, reward, reinforcement, incentive, drive, pun-

The research reported here was supported in part by research grants from the National ScienceFoundation (IBN-9215069), The National Institutes of Health (AG05142), The National Instituteof Mental Health (5P01-MH52194), and The Office of Naval Research (N00014-95-1-1152) and bya grant from The Sanyo Co., Ltd., to R.F.T.

Address correspondence and reprint requests to Richard F. Thompson, Neuroscience Program,University of Southern California, 3614 Watt Way, HNB 522, Los Angeles, CA 90089-2520.

1 Present address: Department of Psychology, Yale University, 2 Hillhouse Avenue, New Haven,CT 06520-8205.

2 Department of Neurobiology, Duke University Medical Center, Box 3209, Durham, NC 27710.3 Present address: Department of Psychology, University of North Carolina, CB3270, Davie

Hall, Chapel Hill, NC, 27599-3270.

1501074-7427/98 $25.00Copyright q 1998 by Academic PressAll rights of reproduction in any form reserved.

AID NLM 3845 / 6v16$$$241 09-11-98 16:36:10 nlmoas AP: NLM

151THE NATURE OF REINFORCEMENT IN CEREBELLAR LEARNING

ishment, and many other terms, some with considerable explicit and implicitbaggage, refer to phenomena that clearly play a major role in determiningbehavior, particularly so in the context of learning and memory. This literatureis reviewed in detail and elegantly analyzed in the volume by Judson Brownon motivation (Brown, 1961).

We believe we have succeeded in identifying the neuronal substrate of rein-forcement for one aspect of basic associative learning and memory: classicalconditioning of discrete behavioral responses learned with aversive uncondi-tioned stimuli. Our results apply to any behavioral response so learned, e.g.,limb flexion, eyeblink, head turn, but we use eyeblink conditioning as ourprimary model system and much of our work has been done with rabbits. Tothe extent tested, our results hold equally for all mammals, including humans(Thompson & Krupa, 1994). Not to maintain suspense, the reinforcement path-way is the inferior olive–climbing fiber projection system to the cerebellum.We believe this is the first instance where the reinforcement system has beenfully identified in the mammalian brain for any aspect of learning and memory.Before detailing the empirical evidence for this discovery, a bit of history isin order.

We focus on classical conditioning. Pavlov (1927) was of course the first toapply the term reinforcement to classical conditioning. So far as we can tell,his use of the term entailed very little surplus meaning. In very much the samespirit as B. F. Skinner many years later, Pavlov characterized the temporalrelations of the conditioned stimulus (CS) and the unconditioned stimulus (US)reinforcement, examined schedules of reinforcement, etc. He did not attempta theoretical behavioral analysis of reinforcement, beyond noting that rein-forcers formed a hierarchy of effective USs. Thus, food to a hungry dog was somuch stronger than painful electric shocks to the skin that the latter couldcome to serve as a sought-after CS for a food US.

Gormezano et al. (1983) present a comprehensive discussion of the role ofreinforcement in classical conditioning. As they note, there are two principalviews: contiguity and effect. In contiguity accounts (Estes, 1959; Guthrie, 1930;Pavlov, 1927) the role of the US is to ensure the occurrence of the US/UR inan appropriate temporal relationship to the CS. More recently Rescorla (1967,1968) and others have proposed that the critical relationship is the probabilitythat given the CS, the US/UR will follow, i.e., the contingency.

The classic view of effect or reward stems from Thorndike’s law of effect:

Of the several responses made to the same situation, those which are accompanied or closelyfollowed by satisfaction to the animal will, other things being equal, be more firmly connectedwith the situation so that, when it reoccurs, they will be more likely to recur. . . . (Thorndike,1911, p. 244)

The tasks Thorndike used were of course instrumental rather than classical,where the Law of Effect has been most widely applied. We wish to avoid thelong-standing debate over whether instrumental or classical conditioning ismore fundamental, that is, whether either one underlies the other (see Re-scorla & Solomon, 1967). In the case of classical conditioning of discrete re-sponses, a number of procedural controls and empirical analyses appear to ruleout instrumental contingencies as playing any important role under normalconditions (Gormezano et al., 1983). However, this does not rule out the Law of


152 THOMPSON ET AL.

Effect in classical conditioning. Effective aversive peripheral USs (e.g., cornealairpuff or periorbital shock for eyeblink conditioning), in addition to having acontingent relationship to the CS, also have aversive properties. We hope toshow here that in eyeblink conditioning the aversive and reinforcing propertiesof the US can be completely separated.

We begin by considering briefly the roles of the various components of thebehavior and their neural substrates in eyeblink conditioning.

THE CR

Instrumental views of classical conditioning argue that the occurrence ofthe behavioral CR somehow modifies the reinforcing or aversive properties ofthe US (e.g., Schlosberg, 1937). Early studies by Gantt and associates (Gantt,1937; Light & Gantt, 1936) appeared to rule this out. Thus, they crushed theappropriate ventral roots to the hindlimb of dogs, trained with paw shock (noflexion) and tested CS alone after regeneration (CRs appeared). Perhaps be-cause of the complexities and possible aversive nature of the procedure, thisfinding, and a similar earlier finding using curare by Harlow and Stagner(1933), has not been fully credited. In recent studies, we and others havereversibly inactivated the magnocellular red nucleus, an essential part of theeyeblink efferent CR circuit (see Fig. 1), using either infusion of muscimol(Krupa et al., 1993) or cooling (Clark & Lavond, 1993). These procedures com-pletely prevent expression of the behavioral CR but do not prevent occurrenceof the UR. In fact, muscimol inactivation has no effect at all on performanceof the UR (Fig. 2). When animals are then tested in the absence of inactivationthey have learned fully, i.e., to asymptote, just as would have occurred withnormal expression of the behavioral CR (Fig. 2). The fact that the CR is notperformed at all during learning argues against the instrumental hypothesis.More important, this result rules out performance of the CR, and the rednucleus itself, as playing any significant role in basic learning of the CR.

THE UR

A. Blocking the UR

Some contiguity views of classical conditioning stress the importance of theoccurrence of the UR as well as the US. Again, the Gantt and Harlow studiesnoted above argue against this. In a recent study (Krupa et al., 1996) we usedmuscimol to reversibly inactivate the motor nuclei generating the eyeblinkresponse (ipsilateral seventh and accessory sixth nuclei). Results were striking:over the 5 days of inactivation training (tone CS, corneal airpuff US) animalsshowed no CRs and no URs at all (Figs. 1 and 3). When tested immediatelyfollowing this with no inactivation animals had fully learned the CR and per-formed normal URs. Consequently the occurrence of the behavioral UR (andCR) seems to play no role in learning.

However, these studies do not disassociate the sensory/response elicitingvs aversive properties of the UR. Interestingly, when the motor nuclei wereinactivated (Krupa et al., 1996) the entire ipsilateral side of the face and thepinna were paralyzed. However, the animals still tried to move their heads



FIG. 1. Highly simplified schematic of the essential (necessary and sufficient) brain circuitryinvolved in classical conditioning of discrete responses, e.g., eyeblink response. Laterality is notindicated: Only the cerebellar hemisphere ipsilateral to the trained eye is critical; the necessaryregions of the trigeminal nucleus and motor nuclei are ipsilateral and the necessary regions ofthe inferior olive, pontine nuclei, and red nucleus are contralateral to the trained eye and cerebellarhemisphere. Interneuron circuits are not shown; only net excitatory (r) or inhibitory (¢) actionsof projection pathways are indicated. Shadowed boxes represent areas that have been reversiblyinactivated during training. (a) Inactivation of motor nuclei including facial (seventh) and acces-sory sixth. (b) Inactivation of magnocellular red nucleus. (c) Inactivation of the anterior interpos-itus nucleus of the cerebellum and some overlying cortex of lobule H VI. (d) Inactivation ofventral anterior interpositus nucleus and associated white matter. (e) Complete inactivation ofthe superior cerebellar peduncle (scp), essentially all output from the cerebellar hemisphere. Seetext for details. (From Thompson and Krupa, 1994. With permission, from the Annual Review ofNeuroscience, Vol. 17, copyright 1994, by Annual Reviews.)

away from the corneal airpuff—they still detected the aversive nature ofthe US.

B. Central Elicitation of the UR

Behavioral responses can be elicited by electrical stimulation of central brainstructures. This method was applied to the study of classical conditioning inpioneering studies, e.g., by Brogden and Gantt (1937, 1942) and by Loucks(1935). In brief, Loucks stimulated the leg region of the motor cortex to elicita limb flexion in dogs as a US and paired this stimulus with neutral CSs. Hecould not obtain any significant learning. Brogden and Gantt, on the otherhand, evoked behavioral movements by stimulation of the cerebellum andthese proved to be very effective USs for conditioning (we will return to thisstudy below). More recently, Doty and Giurgea (1961) claimed that stimulationof the motor cortex could serve as an effective US. This contradiction wasresolved in a careful study by Wagner et al. (1967). They showed that when


154 THOMPSON ET AL.

FIG. 2. Effect of muscimol infusion in the cerebellum and red nucleus on conditioned responses(CRs) and unconditioned responses (URs) (see Fig. 1). (A) All animals received an infusion beforetraining on sessions 1 to 6. The cerebellar group (j) (n Å 6) received muscimol infusions into theipsilateral lateral cerebellum (c in Fig. 1), the red nucleus group (m) (n Å 6) received muscimolin the contralateral red nucleus (b in Fig. 1), and the saline group (l) (n Å 6) received 1 ml ofsaline vehicle into the ipsilateral lateral cerebellum (c in Fig. 1). No infusions were administeredon days 7 to 10. All animals received muscimol infusions before session 11. Data are expressedas percentage CRs averaged over all animals in each group for each training session. (B) percentageCRs for sessions 1 to 4 of the saline group and sessions 7 to 10 of the cerebellar and red nucleusgroups. (C) UR amplitudes on airpuff-only test trials during the 6 sessions in which infusionswere administered. There were no significant differences between groups on these days. All datapoints are means { SEM. (Reprinted with permission from Krupa, Thompson, and Thompson,1993, Science 260, 989–991. Copyright 1993 American Association for the Advancement of Sci-ence.)

the cortical US resulted in negative consequences, i.e., loss of balance, it couldserve as a US (Doty results) but if there were no motivational consequences,the same stimulus that elicited limb flexion was not an effective US (Loucksresult). This, incidentally, is a very nice example of how instrumental contin-gencies can influence classical procedures.

In retrospect, the Brogden and Gantt study was extremely important. It wasthe first to show that the cerebellum could play a critically important role inbasic associative learning, long before studies of adaptation of the vestibulo-ocular reflex, learning of multijoint movements, etc., or the classic theoretical


FIG. 3. Effects of muscimol infusion in the motor nuclei on mean percentage CRs and URsfor each training session (see a in Fig. 1). One hour prior to sessions 1–6 and session 10, onegroup of rabbits (muscimol; n Å 6, filled circles) was infused with 3.5 nmol muscimol (in 0.4 mlsaline) into the motor nuclei (MN). The other group (control, n Å 6, open circles) was infused withsaline vehicle. No infusions were administered prior to sessions 7–9. (A) Mean ({SEM) percentageCRs for each group. Rabbits infused with saline vehicle (sessions 1–6) learned the CR normally,reaching asymptotic levels of CR by the end of session 3. In marked contrast, rabbits infused withmuscimol performed the CR at asymptotic levels from the start of training on session 7 (noinfusion). They had fully learned the CR during the previous inactivation sessions. Muscimolinfusions prior to session 10 completely abolished the CR in all rabbits. (B) Mean ({SEM) URamplitude (measured on US-alone trials) for each group. Muscimol infusions prior to sessions 1–6 completely prevented performance of the UR. UR performance on sessions 7–9 (no infusions)did not differ from that of the controls. Infusion of muscimol on session 10 completely abolishedthe UR in all of the animals. (From Krupa, Weng, and Thompson, Behavioral Neuroscience 110,1–9. Copyright q 1996 by the American Psychological Association. Reprinted with permission.)

155

AID NLM 3845 / 6v16$$384509-11-98 16:36:10 nlmoas AP: NLM

156 THOMPSON ET AL.

papers by Marr (1969) and Albus (1971). Brogden and Gantt used a coil im-planted on the skull (in dogs) with electrodes implanted in the cerebellum(mostly in white matter). Current was induced by an external field coil directlyover the buried coil. The intensity of the inducing current was increased untila just noticeable muscle response was evoked. Auditory and visual CSs wereused. CSs were of 2-s duration and the cerebellar stimulus was 0.1 s at theend of the CS. Importantly, a wide range of movements were elicited by stimu-lation and many (but not all) could be conditioned. In particular, movementsof the ipsilateral forelimb and hind limb, eyelid closure, and pinna movementswere all easily conditioned to the neutral CSs. Most of these effective electrodeplacements were in the ansiform lobe. Interestingly, electrodes in the ventralaspect of the vermis generally elicited neck/postural movements and many ofthese were resistant to conditioning. Importantly, neither the animals inLoucks’ study nor those in the study of Brogden and Gantt showed any behav-ioral signs at all that would indicate that the brain stimulus was aversive.These stimuli appear to have no hedonic tone whatever. Similarly, humansreport no aversive properties with such brain stimulation.

Brogden and Gantt draw a key comparison between Loucks’ failure to condi-tion movements by stimulation of the motor cortex as a US and their extraordi-nary success in conditioning the same movements elicited by cerebellar stimu-lation as a US.

If the law of effect (the stamping in of movements under the influence of pain or pleasure)were basic to all learning, then none of our cerebellar animals should have become conditioned.Pain was no more a factor in the responses produced by cerebellar stimulation than thoseproduced by stimulation of the cerebral cortex. The difference in the results, then, is reducedto a difference in the functions of the motor cortex and those of the cerebellar loci which werestimulated in our animals that became conditioned. (Brogden & Gantt, 1942, pp. 15–16)

THE CEREBELLUM AND MEMORY

This finding that the cerebellum played a key role in learning and memorywas widely ignored because of traditional views that the functions of the cere-bellum were limited to coordination and execution of movements. Astonish-ingly, this traditional view of the cerebellum is still held by some cerebellarphysiologists today, in the face of now overwhelming evidence to the contrary.

We have reviewed the evidence that the cerebellum is critically involved inclassical conditioning of discrete responses in detail in recent papers (e.g.,Thompson & Kim, 1996; Thompson & Krupa, 1994; Kim & Thompson, 1997).In brief, small focal unilateral lesions of the cerebellar anterior interpositusnucleus completely and permanently prevent learning and completely andpermanently abolish the memory for classical conditioning of the eyeblinkresponse ipsilateral to the cerebellar lesion, yet have no persisting effect onany aspect of the UR or on the ability of the contralateral eye to learn andremember the CR (McCormick et al., 1982). Neuronal recordings from thissame region of the interpositus nucleus show the development of increasedfrequency of unit responses in the CS–US onset interval over training thatcorrelate closely with the behavioral CR and precede and predict the form ofthe CR within trials (Fig. 4) (McCormick & Thompson, 1984). Similarly, manyPurkinje neurons in cerebellar cortex show learning-induced decreased or in-



FIG. 4. Neuronal unit activity recorded from the lateral interpositus nucleus during unpairedand paired presentations of the training stimuli. The animal was first given pseudorandomlyunpaired presentations of the tone and corneal airpuff, in which the neurons responded very littleto either stimulus. However, when the stimuli were paired together in time, the cells beganresponding within the CS period as the animal learned the eyeblink response. The onset of thisunit activity preceded the behavioral NM response within a trial by 36–58 ms. Stimulationthrough this recording site yielded ipsilateral eyelid closure and NM extension. Each histogrambar is 9 ms in duration. The upper trace of each histogram represents the movement of the NM,with up being extension across the eyeball. (Reprinted with permission from McCormick andThompson, 1984, The Journal of Neuroscience 4(11), 2811–2822.)

creased responses within the CS period that correlate with the behavioral CR.Finally, electrical microstimulation of this same region of the interpositusevokes eyeblinks in naive animals—the circuit is hard-wired from interpositusto behavior.

Reversible inactivation (muscimol, lidocaine, cooling) of this same criticalregion of the interpositus nucleus during training completely prevents learn-ing, with no effect on the UR (there is of course no expression of the CRduring training). After inactivation is removed, these animals learn as thoughcompletely naive with no savings at all (Fig. 2) (Clark et al., 1992; Krupa etal., 1993; Nordholm et al., 1993; see also Hardiman et al., 1996). This resultholds even for very small infusions of muscimol (1.0 nmol in 0.1 ml) limited inspread ([3H]muscimol) to the anterior interpositus and medial region of thedentate nucleus (see Figs. 2 and 5) (Krupa & Thompson, 1997). Even thoughthere was no spread of muscimol to the cerebellar cortex in most of theseanimals, because of the reciprocal connections between the interpositus nu-cleus and the cerebellar cortex, the latter structure cannot be ruled out asplaying a key role. Indeed, Purkinje cell degeneration (pcd) mutant mice thathave no Purkinje neurons in cerebellar cortex (and hence no output from thecortex) are markedly impaired in eyeblink conditioning, although they do showsignificant learning (Chen et al., 1996). In marked contrast to the results ofinactivation of the interpositus nucleus during training, reversible inactivationof the superior cerebellar peduncle (scp), the immediate output from the inter-


158 THOMPSON ET AL.

FIG. 5. Maximal extent of diffusion of very small infusions of 3H-labeled muscimol (1.0 nmolin 0.1 ml) in four rabbits with effective cannulae placements in the anterior interpositus nucleusthat completely prevented learning of the CR (c in Fig. 1). Muscimol diffused throughout theanterior interpositus nucleus and regions of dentate nucleus. In some rabbits, very low concentra-tions of muscimol diffused into the most ventral aspects of ansiform cortex. There was no diffusionof muscimol outside of the cerebellum. (From Krupa and Thompson, 1997.)

AID NLM 3845 / 6v16$$3845 09-11-98 16:36:10 nlmoas AP: NLM


positus (and dentate) nucleus, during training (using TTX), although com-pletely preventing expression of the CR (with no effect on the UR), does notimpair learning at all (Krupa & Thompson, 1995) (Figs. 1 and 6). Animalsdemonstrated learning at asymptote immediately after inactivation was re-moved, just as is true for inactivation of the red nucleus during training (Fig.2). A similar result was obtained with lidocaine inactivation of the immediateoutput of the interpositus nucleus (Nordholm et al., 1993—see Fig. 1d). So thememory trace must be formed at or beyond the cerebellum (localized regions ofinterpositus nucleus and cerebellar cortex) but before the output of the cerebel-lum (superior cerebellar peduncle). The conclusion appears obvious, indeedinescapable.

THE US PATHWAY: THE INFERIOR OLIVE–CLIMBING FIBER SYSTEM

Here we focus on the US pathway, the reinforcing or teaching input to thecerebellar memory trace system. The view that the inferior olive–climbingfiber system is the ‘‘instructive’’ or ‘‘teaching’’ input to the cerebellum for thelearning of movements was the basis of now classic theories of the cerebellumas a learning machine (Albus, 1971; Brindley, 1964; Eccles, 1977; Eccles etal., 1967; Ito, 1972, 1984; Marr, 1969). For recent overviews, see Lavond et al.(1993), Thach et al. (1992), and Thompson and Krupa (1994). In these theories,information about movements, contexts, and other types of sensory informationwas held to be projected to the cerebellum by the mossy fibers, coming fromthe pontine nuclei and other sources that synapse on granule cells in thecerebellar cortex, giving rise to parallel fibers. Information about movementerrors and aversive USs is provided by the inferior olive–climbing fiber system.Purkinje neurons are the only output neurons from cerebellar cortex, providinginhibitory projections to cerebellar and vestibular nuclei. Each Purkinje neu-ron receives¢100,000 excitatory sysnapses from parallel fibers and excitationfrom one, and only one, climbing fiber, an architecture that inspired the classictheories. Both mossy and climbing fibers also send projections to the cerebellarnuclei (e.g., interpositus nucleus).

Physiological studies of multijoint limb movements in monkeys (see Thachet al., 1992) and of adaptation of the VOR (vestibulo-ocular reflex) in rabbits(see Ito, 1984) have suggested that the cerebellum is involved in learning oradapting movements and that the inferior olive–climbing fiber system mightfunction as an error-indicating or error-correcting system. However, until stud-ies of the brain substrates of eyeblink conditioning there was little evidence(except for Brogden & Gantt, 1942) that the cerebellum played a critical rolein basic associative learning and memory (see above).

We first discovered the effects of inferior olive lesions on eyeblink condition-ing (Steinmetz et al., 1984; McCormick et al., 1985). In brief, small localizedelectrolytic lesions were made in the face representation field of the dorsalaccessory olive (DAO) region of the inferior olive contralateral to the trainedeye. If effective lesions were made before training, animals were unable tolearn. If made after training, animals initially showed CRs and then withcontinued paired training gradually showed extinction of the CR (Fig. 7). Theextinction study was done independently in two replications, by David McCor-mick and by Joseph Steinmetz. Behaviorally, the corneal airpuff US was still


160 THOMPSON ET AL.

FIG. 6. Effect of inactivating the superior cerebellar peduncle (scp) on learning (e in Fig.1). (Top) Percentage CRs for all conditioning sessions from all animals with effective cannulaeplacements. Tetrodotoxin (TTX) (inactivating fibers and cell bodies) was infused into the superiorcerebellar peduncle (scp) (all output from interpositus nucleus) of each animal prior to sessions1–6 and session 10. No infusions were administered prior to sessions 7–9. Muscimol was infusedprior to session 11. Animals trained with an auditory white noise (WN) as CS (open squares, nÅ 6) or with electrical microstimulation (stim) of the lateral reticular nucleus (LRN) as CS (solidcircles, n Å 4) performed no significant number of CRs during the first 6 infusion sessions. Onsession 7, the first session without infusion, these animals performed the CR at asymptotic levelsfrom the start of training; they had fully learned the CR during the previous six inactivationsessions. Controls (solid triangles, n Å 6) were infused with TTX and restrained but presentedwith no stimuli during sessions 1–6. These animals performed significantly fewer CRs on session7, their first conditioning session with the auditory CS, and subsequently learned the CR onfollowing sessions. TTX infusions prior to session 10 completely abolished the previously acquiredCR in all rabbits. Infusion of muscimol (inactivating cell bodies but not fibers) prior to session 11had no effect upon the CR in any rabbit. (Bottom) Trials to criterion (8/9 consecutive CRs) for thethree groups. (Reprinted with permission from Krupa and Thompson, 1995, Proceedings of theNational Academy of Sciences, U.S.A., 92, 5097–5101. Copyright 1995 National Academy of Sci-ences, U.S.A.)



FIG. 7. Effect of lesion of the rostromedial inferior olive (dorsal accessory olive) on the percent-age of trials in which conditioned responses were performed. The animals were first trained (P1–P3) and then received either a lesion which included the rostromedial inferior olive contralateralto the trained eye (lesion) or disconnection of the airpuff (control); this was followed by 4 days ofpaired trials to the same eye (L1–L4) for the lesion group or tone-alone trials in the control group.Each data point represents the average of one-half day of training. (Reprinted from Brain Research359, McCormick, Steinmetz, and Thompson, Lesions of the inferior olivary complex cause extinc-tion of the classically conditioned eyeblink response, pp. 120–130. Copyright 1985, with permissionfrom Elsevier Science.)

clearly aversive to the animals after the DAO lesion; i.e., if given the opportu-nity, they would avoid it. The difference was that they were now unable tolearn or remember the CR. The basic finding of extinction of the CR followinglesions of the IO–climbing fiber system was replicated by Voneida et al. (1990)using conditioned forelimb flexion in the cat with lesions of the climbing fibers.To our knowledge, these are the only reports in the literature where brainlesions result in extinction of a learned response with continued reinforcedtraining. Yeo et al. (1986) reported that DAO lesions caused immediate aboli-tion of the CR but their lesions were larger and there were a number of otherprocedural differences. We have more recently made additional inferior olivelesions in trained animals, including chemotoxic lesions that do not destroyfibers of passage, and have replicated the extinction result with small, appro-priately placed lesions; larger lesions can cause immediate abolition (Kim &


162 THOMPSON ET AL.

Thompson, unpublished observations; Mintz et al., 1994), we think by mas-sively inducing abnormal activity in Purkinje neurons (Benedetti et al., 1984).

In the present context, the most important results of the DAO lesion studiesare: (1) effective lesions prevent learning, (2) effective lesions result in extinc-tion with continued paired training, and (3) the US is still aversive but it isno longer reinforcing. The fact that CRs are still present after the effectiveDAO lesion argues strongly that the memory trace is not formed or stored inthe IO. Recording studies are consistent with this conclusion. Unit clusterrecordings from the region of the DAO critical for eyeblink conditioning (deter-mined by the lesion studies) reveal evoked unit activity to the onset of the US(corneal airpuff) prior to learning and no responses at all to the tone CS. Overthe course of training, the US-evoked neural response gradually disappearsas the CR develops and there is never any neural response in the CS–USonset interval when CRs occur (Sears & Steinmetz, 1991; Thompson, 1989a).This is in marked contrast to the neuronal model of the learned response thatdevelops in the interpositus nucleus in the CS–US onset interval over training(see above and Fig. 4). The complete absence of any neural unit activity in theCS period in the region of the DAO critical for learning rules out the possibilitythat any significant component of the memory trace is formed in the DAO.

Electrical microstimulation in the IO–DAO can elicit behavioral responses,i.e., eyeblink, limb flexion, head turn. Whatever response is so elicited caneasily be conditioned to a neutral tone CS (Mauk et al., 1986). A numberof control procedures showed that the effective DAO electrical stimulus wasactivating climbing fibers to the cerebellum and that this cerebellar activationfunctioned as the necessary and effective US (Thompson, 1989a). In a morerecent and detailed replication and extension, we electrically stimulated pon-tine nuclei–mossy fibers projecting to the cerebellum as the CS and inferiorolive–climbing fibers as the US in intact behaving rabbits and obtained normallearning (Steinmetz et al., 1989). To our knowledge, this is the only system inthe brain other than reflex afferents where the exact response elicited by thestimulus can be conditioned to neutral stimuli. The IO–climbing fiber systemseems specialized for the training of discrete precise movements, i.e., motorskills.

The inferior olive–climbing fiber projection system to the cerebellum alsoplays a key role in cerebellar long-term depression (LTD). This process ofsynaptic plasticity is widely viewed as a putative mechanism of memory stor-age in cerebellar cortex (see Ito, 1984). The phenomenon was initially discov-ered by Ito et al. (1982), who reported that conjoint electrical stimulation ofclimbing fibers and mossy (-parallel) fibers at a relatively slow rate (e.g., 4/s)resulted in long-term depression of the parallel fiber synapses on Purkinjeneuron dendrites. This is precisely the mechanism Albus (1971) proposed inhis model of cerebellar learning. The events critical for the establishment ofcerebellar LTD are activation of glutamate AMPA and metabotropic receptorson Purkinje neuron dendrites (normally by release of glutamate at parallelfiber synapses on Purkinje neuron dendrites, but also by application of gluta-mate), together with influx of calcium ions into the Purkinje neurons (normallyby climbing fiber activation but also by depolarization) (Linden & Conner,1995; Ito, 1994).

If LTD is a mechanism underlying some aspects of memory trace formation



in the cerebellar cortex in eyeblink conditioning, then certain temporal con-straints exist. As noted above, when eyeblink conditioning is established byelectrical stimulation of mossy fiber projections to the cerebellum as a CS andclimbing fibers as a US (Steinmetz et al., 1989) normal learning occurs. Aswith peripheral stimuli, the mossy fiber CS onset must precede the climbingfiber US onset by at least 100 ms and best learning occurs with a 250-msprecedence of the CS. But LTD is established by conjoint simultaneous stimula-tion of parallel fibers and climbing fibers, particularly in cerebellar slice stud-ies. We examined this issue in the cerebellar slice, with field potentialrecordings, using parallel fiber and climbing fiber stimulation, and a parallel-fiber-alone stimulation site to control for health of the slices (Chen & Thomp-son, 1995). In brief, we got the best LTD if the parallel fiber stimulus onsetpreceded the climbing fiber stimulation by 250 ms in the absence of GABAantagonists. It turns out that virtually all cerebellar slice studies of LTD usedbiculline or picrotoxin (GABA antagonists) in the bath. Indeed, when we addedbicuculline to the bath we obtained robust LTD with simultaneous stimulationof parallel and climbing fibers. These results suggest that inhibitory interneu-rons in cerebellar cortex (Golgi, basket, and stellate neurons) play a criticalrole in timing processes in cerebellar cortex (see Keele & Ivry, 1990, for areview of putative timing functions of the cerebellum).

In a most interesting paper, Schreurs et al. (1996) applied a standard condi-tioning paradigm to the cerebellar slice—100-Hz parallel fiber stimulation for80 ms followed immediately by 20-Hz climbing fiber stimulation for 100 ms,intertrial interval of 30 s. Control slices were given unpaired stimulation. Thepaired slices showed substantial and persisting depression of Purkinje neuronEPSPs to parallel fiber test stimulation relative to unpaired controls. One istempted to refer to this as eyeblink conditioning in a dish.

Our DAO stimulation result—normal learning in the intact, behaving ani-mal—immediately calls to mind the Brogden and Gantt findings. The obvioushypothesis is that their effective cerebellar US was activating climbing fibers.In recent work we have repeated and extended their observations. In Swainet al. (1992) we directly replicated the results of Brogden and Gantt usingrabbits and facial responses (eyeblink and nictitating membrane extension,movements of the upper lip, and movements of the head) elicited by electricalstimulation of cerebellar white matter as the US, paired with a tone CS. TheseURs were learned as CRs to the CS in a manner comparable with peripheralUSs and they showed extinction and rapid relearning, again as with peripheralUSs. Most recently we used electrical stimulation of cerebellar parallel fibers(concentric cerebellar cortical surface electrode on lobule H VI) as a CS pairedwith cerebellar white matter stimulation as a US (Shinkman et al., 1996).Normal learning occurred in this much reduced preparation. We are confidentthat the parallel fiber stimulation did not spread subcortically and suggest,but cannot yet prove, that the memory trace so established was localized tothe cerebellar cortex.

We have also explored the possible US functions of other components of theessential cerebellar learning circuit, i.e., regions where behavioral responsescan be elicited by electrical stimulation, specifically the red nucleus and inter-positus nucleus, giving standard paired tone CS–brain stimulation US train-ing (see Fig. 1) (Chapman et al., 1988). We note that Nowak and Gormezano


164 THOMPSON ET AL.

(1990) stimulated the trigeminal nucleus as a US. It was effective, as wouldbe expected—it activates projections to the IO as well as the ascending somato-sensory system and the reflex pathways. In our studies we used stimulusintensities just sufficient to elicit a substantial eyeblink response comparableto the corneal airpuff-elicited UR—these intensities were relatively low (e.g.,100 Hz, 0.1-ms pulses, 75–200 mA) (standard tone CS). Results for the magno-cellular red nucleus were very clear—even though robust eyeblinks were elic-ited, no CRs at all occurred in 5 days of training and the animals learnedsubsequently with a corneal airpuff US with no savings at all. In sum, therewas no evidence of any learning at all. Eyeblink-eliciting stimulation of theanterior interpositus nucleus as US similarly did not yield any CRs. Theseanimals did learn with savings when subsequently trained with a cornealairpuff US. However, when animals trained with the airpuff US were shiftedto eyeblink-eliciting interpositus stimulation as a US they extinguished. Weinterpret this result to indicate that the interpositus stimulus activated onlya few climbing fibers (they project to the interpositus nucleus) antidromically,insufficient to establish or maintain the learned response but sufficient toestablish some learning-related neuronal plasticity. Alternatively, the inter-positus stimulus may have induced some plasticity directly in the interpositus.In any case, this result stands in marked contrast to the robust learningestablished by stimulation of the DAO–climbing fibers or cerebellar whitematter as a US.

In complete agreement with Brogden and Gantt we found that stimulationof the DAO (and cerebellar white matter) as a US effective for learning hadno aversive properties whatsoever. We stimulated rabbits (DAO) in a particu-lar place in a test area—they neither avoided nor approached that place andgave no signs of discomfort. Manipulations of the DAO thus yield a doubledissociation: (1) lesion of the critical region of the DAO prevents learningbecause it abolishes the reinforcement but does not abolish the aversive aspectof the US and (2) stimulation of the critical region of the DAO (and cerebellarwhite matter) provides the reinforcement necessary to support the learning ofdiscrete movements but is not aversive. Again, we argue that the IO–climbingfiber system projecting to the cerebellum is a brain system specialized to pro-vide reinforcement for the learning of discrete (skilled?) movements and theonly brain system so specialized.

Recently a knockout mouse that is markedly deficient in the gamma isoformof protein kinase C (PKCg knockout) was developed (Kano et al., 1995; Chenet al., 1995). PKCg is brain-specific and highly expressed in cerebellar Purkinjeneurons (Tanaka & Nishizuka, 1994). As might be expected, these knockoutmice show impaired motor coordination (see Chen et al., 1995). An extraordi-nary fact about these animals is that many Purkinje neurons have multipleclimbing fiber innervation (Kano et al., 1995). Normal mice, like other mam-mals, have only one climbing fiber projecting to each Purkinje neuron. Multipleclimbing fiber innervation is present developmentally but is pruned down toone climbing fiber per Purkinje cell in wild-type mice shortly after birth(Crepel, 1982). Our climbing-fiber reinforcement hypothesis would have tomake the strong prediction that eyeblink conditioning is not impaired in theseanimals in spite of impaired motor coordination. In fact, these animals learneyeblink conditioning significantly faster than do normal wild-type controls



FIG. 8. (Left) PKCg knockout mutant mice show facilitated eyeblink conditioning: PercentageCR by session. Wild-type mice (n Å 12) showed a gradual increase in percentage CR (mean {SEM) over the first 4 days of training. In contrast, mutant mice (n Å 12) exhibited asymptoticCRs by day 2 of training. Percentage CR in mutant mice is significantly higher than in wild-typemice on the second day of training (F(1, 23)Å 8.29; p, .01), while no statistically reliable differenceswere observed on other days. (Right) Mutant mice show more memory saving of conditionedresponses during extinction test. After being switched to CS-alone trials, both groups demonstratea gradual extinction of percentage CR over 4 days. The percentage CR in the mutant group duringextinction is consistently higher than in the wild type (F(1, 23) Å 9.30; p õ0.01). (From Chen etal., 1995.)

(see Fig. 8). Further, they exhibit normal LTD in cerebellar cortex (Chen etal., 1995).

Insofar as theories of reinforcement are concerned, the conclusion seemsclear: Neither contiguity/contingency, per se, nor the Law of Effect is correct.Instead, activation of a particular brain system, the inferior olive–climbingfiber system, affectively neutral, is the necessary reinforcing pathway for clas-sical conditioning of discrete responses. It is of course true that a contingentand appropriate temporal relationship must exist between a CS and activationof the climbing fiber system for learning to occur. These results account clearlyfor the discrepancy between the results of Loucks (1935) and Brogden andGantt (1942) and the report by Bruner (1963) that a light US effective ineliciting an eyeblink response does not support learning. [But Steinmetz andassociates (Rogers et al., 1997) reported eyelid conditioning with a bright lightflash US.] Contiguity/contingency of CS and US/UR, per se, is not sufficientand aversiveness, per se, is not a factor. The fact that manipulations of theproperties of peripheral USs have profound effects on the rate and degree oflearning must now be reinterpreted in terms of effects on neurons in theinferior olive rather than solely in terms of ‘‘aversiveness.’’ However, this isnot to deny that the motivational state of the animal plays an importantrole in behavioral learning performance. Indeed, water-deprived rabbits learneyeblink conditioning significantly faster than satiated rabbits (Berry & Swain,1989) and water-deprived rats learn fear conditioning to context faster thansatiated rats (Maren et al., 1994). This issue of the interaction of inferredlearning and motivational factors in the context of basic associative learninghas been considered in detail by Wagner and associates (see Wagner & Do-negan, 1989; Brandon & Wagner, 1991; Wagner & Brandon, 1989).


166 THOMPSON ET AL.

THE RESCORLA–WAGNER FORMULATION

The most influential behavioral theory of classical conditioning was devel-oped by Rescorla and Wagner in 1972 (we abbreviate it R–W) and elaboratedin many more recent publications (e.g., Donegan et al., 1989; Rescorla, 1988;Wagner, 1981; Brandon & Wagner, 1991; Wagner & Donegan, 1989; Wagner &Rescorla, 1972). The heart of their theory is the error-correcting algorithm. Forthe development of excitatory conditioning, they proposed that the associativestrength that accrues between a stimulus (CS) and its outcome on a trial, DVi ,is given (in simplified terms) by

DVi Å b (l 0 SkVk),

where b is a constant reflecting learning rate, l is the maximum associativestrength that can be established with a given US, and SkVk is the sum of theassociative strengths between the US and all stimuli appearing on that trial.Again, DVi is the increase in associative strength occurring on trial i. We haveargued that our discovery of the reinforcement role of the IO–climbing fibersystem in classical conditioning constitutes the neural instantiation of the R–W theory (Donegan et al., 1989; Gluck et al., 1990; Thompson, 1989b; seealso Wagner & Donegan, 1989, who draw parallels between their ‘‘sometimesopponent process (SOP)’’ theory of conditioning and our cerebellar circuit).

Our basic hypothesis is that the amount of associative strength accruing tothe CS from the occurrence of the contingent US on a given trial is determinedby the number of climbing fibers activated by the US on that trial. Thus, theamount of reinforcement occurring on a given trial is determined by the num-ber of activated climbing fibers and their corresponding actions on Purkinjeneurons relative to the (maximum) number activated by the same US pre-sented without the CS. Note that according to the R–W formulation, theamount of associative strength accruing on initial trials of training is maxi-mum since SkVk is zero or very close to it. As training proceeds, SkVk increasesin value and DVi decreases until in well-trained animals SkVk is very nearlyequal to l and DVi is very close to zero.

This is precisely what we have found with unit recordings in the DAO (seeabove): US onset evokes substantial phasic DAO unit responses at the begin-ning of training and then decrease to zero in well-trained animals on pairedtrials where CRs occur. The evoked unit response remains large on US-alonetrials (Sears & Steinmetz, 1991). Identical results were found for eyeblinkconditioning in the decorticate ferret—inhibition of inferior olive during condi-tioned responses (Hesslow & Ivarsson, 1996). In current studies of responsesof Purkinje neurons we see the same result for the complex spikes evoked byclimbing fiber activation (Krupa, 1993) (see Fig. 9). Of 54 Purkinje neuronsrecorded in lobule H VI, 16 exhibited specific evoked complex spike activity inresponse to US onset during US-alone trials. Of these, 5 were recorded earlyin training and responded to the US with complex spikes on paired trials(before animals performed any CRs). The remaining 11 Purkinje neurons wererecorded in well-trained animals and did not respond to the US with complexspikes on paired trials when the animals displayed CRs, but did show evokedcomplex spikes on US-alone trials.



FIG. 9. Examples of complex spikes recorded from a Purkinje cell in lobule H VI duringairpuff-alone and tone–airpuff paired trials in a trained animal. (In naive animals, the airpuff-evoked complex spikes are observed on both paired CS–US and US-alone trials). In trainedanimals, the airpuff-evoked complex spikes are present on US-alone trials (left). In contrast,during the CS–US paired trials where the animals display CRs, the airpuff-evoked complex spikesare absent (right). Each dot in the raster represents a complex spike discharge. (From Krupa,1993.)

A small point: In our preparation the external eyelids are held open and thecorneal airpuff US is delivered to the temporal region of the cornea, a region notcovered by the nictitating membrane at full extension. Indeed, the nictitatingmembrane response to the corneal airpuff on paired trials and on US-alonetrials can be larger after training than before training (Steinmetz et al., 1992;Ivkovich & Thompson, 1997). Furthermore, a periorbital shock US yields learn-ing identical in all respects to that with a corneal airpuff US.

Activity of neurons in the anterior interpositus does not respond significantlyto the tone CS before training. As learning develops, these neurons massivelyincrease their frequency of discharge in the CS period, as we noted earlier (seeFig. 4), driving the conditioned behavioral response via the red nucleus. Inaddition to their strong excitatory projections to the red nucleus, interpositusneurons have a powerful direct, inhibitory GABAergic projection to the inferiorolive (Andersson et al., 1988; Nelson & Mugnoini, 1987) (Fig. 1). Consequently,as learning develops, the degree of GABAergic inhibition of neurons in the IO(DAO) will increase to the point where US activation of the DAO is completelyinhibited on paired CS–US trials (but of course not on US-alone trials). Thisis precisely what we have observed for activity of relevant DAO neurons andcomplex spikes in Purkinje neurons.

BLOCKING

The negative feedback circuit from interpositus to DAO thus provides a clearneuronal circuit instantiation of the error-correcting algorithm in the R–Wformulation. As we and others have noted (e.g., Gluck et al., 1990), the R–Wformulation is an instance of the least-mean-squares algorithm in adaptivefilter theory and is equivalent to the ‘‘delta rule,’’ a widely used error-correctingalgorithm in connectionist learning theory (Rumelhart & McClelland, 1986;Widrow & Hoff, 1960). Among other accomplishments the R–W theory pro-vides a straightforward explanation of the blocking effect (Kamin, 1969),thought by some to be an instance of a cognitive phenomenon in classicalconditioning.


168 THOMPSON ET AL.

Indeed, a very strong prediction from our hypothesis is that the phenomenonof blocking in eyeblink conditioning can be fully accounted for by the error-correcting cerebellar–inferior olive circuitry (see Thompson, 1989a, 1989b). Inbrief, if an animal is first trained on CS1–US, then given additional trainingon CS1 paired simultaneously with CS2, followed by US, and then tested forresponse to CS2, little conditioned responding is seen compared with animalsnot given the prior CS1–US training (and other appropriate control conditions).The important message of blocking is that the informational context in whicha CS appears determines the degree to which it becomes associated with theUS (as opposed to simply the number of times the CS is paired with the US)—contiguity is necessary to produce conditioning, but not sufficient.

Blocking has become the cornerstone of all models of associative learningthat address the Pavlovian conditioning literature (e.g., Mackintosh, 1975;Pearce & Hall, 1980). In fact, it seems almost obligatory for a model to predictblocking in order to be taken seriously (see also Gluck & Thompson, 1987).One way blocking could occur (Rescorla–Wagner formulation) is that as theCR becomes established to CS1, the associative strength added by additionalpairings of CS1 and the US becomes increasingly less, so that after learningto CS1 is asymptotic, no additional associative strength is added by additionalCS1–US pairings. Hence, if further training is given with CS1 and a new CS2,presented simultaneously, and the US, no associative strength will accrueto CS2–US.

An intervening but essential prediction is that if the GABAergic inhibitoryprojection from interpositus to the DAO mediates the learning-induced shut-down of US-evoked activity in DAO neurons and consequently of evoked com-plex spikes in Purkinje neurons, then blocking GABA inhibition in the DAOshould result in the occurrence of evoked DAO units and complex Purkinjeneuron spikes on paired trials where CRs occur. This is indeed the case (seeFig. 10). In Purkinje neurons that exhibited US-onset-evoked complex spikeson US-alone trials, but no such activity on paired trials where the CR occurred,intraolivary infusions of the GABA antagonist picrotoxin (1 ml, 10 mM, 0.1 ml/min) in the DAO caused the Purkinje neurons to exhibit US-evoked complexspikes on paired trials where they gave behavioral CRs (Kim et al., 1997).This provides another line of evidence, incidentally, that performance of thebehavioral CR, per se, does not alter the stimulus properties of the US.Blocking GABA inhibition in the DAO reinstates the US-evoked activity inthe DAO–cerebellum on CR trials.

To test whether the GABAergic cerebello-olivary projection is indeed in-volved in blocking, rabbits were implanted with unilateral guide cannulaejust above the contralateral inferior olive. Animals were subjected to Kamin’sstandard two-stage blocking procedure (Fig. 11). In the first phase, animalsreceived seven daily sessions of tone–airpuff conditioning. In the second phase,animals underwent five sessions of tone–light–airpuff compound conditioning,while either picrotoxin (PTX) or artificial cerebrospinal fluid (ACSF) was in-fused directly into the inferior olive. Controls experienced only the secondphase of the blocking procedure, i.e., 5 days of tone–light–airpuff compoundconditioning. Afterward, all animals were presented with light–airpuff pair-ings to assess whether conditioning to the light had accrued during compoundconditioning (Kim et al., 1992, 1997).



FIG. 10. Example of an isolated Purkinje cell in a well-trained rabbit before and after infusionsof picrotoxin (PTX), a GABA antagonist, into the inferior olive. During the US-alone trials, thePurkinje cell responded to the airpuff with a complex spike in a time-locked pattern (a). The samecell, however, did not respond to the airpuff with a complex spike when the animal displayed CRsduring the paired CS–US trials (b), just as was true for the neuron in Fig. 9. When PTX wasinfused into the inferior olive, the cell began to respond to the airpuff with a complex spike eventhough the animal continued to perform conditioned responses, indicating that the CR-inducedinhibition of the complex spikes had been prevented by PTX (c). (Reprinted with permission fromKim, Krupa, and Thompson, 1998, Science 279, 570–573. Copyright 1998 American Associationfor the Advancement of Science.)


170 THOMPSON ET AL.

FIG. 11. Experimental design of the blocking study. Animals are given 10 blocks of 10 trialsper day, the first trial CS-alone and the sixth trial US-alone in each block, for a total of 100 trials.Tone CS (T) 1 kHz, 85 dB, 500 ms, light CS (L) 45 Lux, 500 ms, and the corneal airpuff US (A)2.3 N/m2, 100 ms, coterminating with CSs.

Both control and PTX animals exhibited significant learning to the light CScompared with ACSF animals (Fig. 12). In fact, the control and PTX groupsshowed immediate responses to the light CS. Overall percentage CR to thelight CS in the PTX group was not statistically different from controls thatdid not receive tone–airpuff training prior to the compound training (56 and64%, respectively), indicating that blocking did not occur in the PTX group. Incontrast, ACSF animals demonstrated blocking—they did not show evidence ofconditioning to the light (during compound conditioning) and subsequentlylearned the CR to the light over 5 days of light–airpuff training. PTX had noeffect on the performance of CRs and URs during the compound conditioning,indicating that PTX selectively affected blocking.

These data indicate that animals which received ACSF into the inferior oliveshowed blocking—they did not acquire CRs to the light CS when it was pairedwith the airpuff US in conjunction with a previously conditioned tone stimulus.In marked contrast, animals that had PTX infused into the inferior oliveshowed no blocking—they acquired the CR to the light CS during compoundtraining as evidenced by substantial savings during subsequent light–airpufftraining. We argue that infusions of PTX into the inferior olive during com-pound tone–light–airpuff training impeded the tone-induced cerebellar inhibi-tion of US-evoked inferior olive responses, thereby allowing animals to condi-tion to the light CS. Blocking was thus prevented in these animals. In thecontrols that did not receive tone–airpuff training beforehand, conditioning tothe light stimulus occurred during compound training because there was nopreviously established cerebellar inhibition of inferior olive activity in responseto incoming CS information.

CONCLUSIONS

These results (Figs. 10–12) indicate that the GABAergic cerebello-olivaryprojection (Anderssen et al., 1988; Angaut & Sotelo, 1989) plays a crucial rolein mediating blocking in eyeblink conditioning. Different forms of learning,



FIG. 12. Results of the blocking study (see Fig. 11). Mean percentage eyeblink CRs of control(n Å 5), artificial cerebrospinal fluid (ACSF) (n Å 6), and picrotoxin (PTX) (n Å 12) groups duringthe 5 days (L1 to L5) of light–airpuff savings test-training. See text for details. (Reprinted withpermission from Kim, Krupa, and Thompson, 1998, Science 279, 570–573. Copyright 1998 Ameri-can Association for the Advancement of Science.)

dependent on other structures, may employ a similar negative feedback mecha-nism to regulate the US or ‘‘reinforcing’’ input (Graybiel et al., 1994; Schultz,1997; Schultz et al., 1993, 1997). For example, it has been reported that manydopamine neurons in the substantia nigra and the ventral tegmental areashow phasic responses to the delivery of liquid reward in monkeys undergoinga spatial delayed response task. However, once learning is established (i.e.,the animal learns that a light cue predicts the reward), the delivery of thereward no longer elicits phasic responses in dopamine neurons (Schultz et al.,1993, 1997). Such negative feedback circuits in the brain may in fact providethe neuronal instantiation of behavioral characterizations of blocking (Kamin,1968; Mackintosh, 1975; McLaren & Dickenson, 1990; Pearce & Hall, 1980;Rescorla & Wagner, 1972; Wagner, 1981).

Classical conditioning is the simplest form of associative learning by whichanimals, including humans, learn relations among events in the world so thattheir behaviors become better adapted to their environment (Rescorla, 1988).Functionally, blocking plays an important role in how animals process andattend to information in their environment. Animals typically encounter nu-merous stimuli in their environment. Therefore, it is adaptive for animals to


172 THOMPSON ET AL.

respond selectively to those stimuli (e.g., CSs) that reliably predict biologicallysignificant events (e.g., USs). Other stimuli that provide no new useful informa-tion about the US should be disregarded, otherwise animals will be constantlyforming unnecessary associations with various stimuli in their surroundings.The behavioral phenomenon of blocking, which appears to use a heuristicnegative feedback process, may serve to circumvent such redundant learning.ly, to return to the major issue at hand, our results indicate that at least forthe cerebellum and its associated circuitry necessary and sufficient for classicalconditioning of discrete responses, reinforcement is not due simply to contiguity/contingency nor to aversiveness, the affective quality of the US. Instead, it is dueto activation of a particular brain circuit, the inferior olive–climbing fiber projec-tion to the cerebellum, with some degree of contingent relationship to the CS.The climbing fiber system is in fact the reinforcing or teaching pathway for thisform of learning and memory. The cerebellum is perhaps the oldest major struc-ture in the vertebrate brain to have become specialized for learning and memory.If it is a model for other, more recently evolved memory systems, perhaps thesame principle applies—reinforcement is due to activation of particular teachingpathways and not due simply to contiguity/contingency or to any affective aspectof the stimulus or situation, per se. This does appear to be the case for thedopamine system projecting to the striatum (see above) and possibly for thedopamine projection system to the nucleus accumbens.

REFERENCES

Albus, J. S. (1971). A theory of cerebellar function, Mathematical Bioscience, 10, 25–61.

Andersson, G., Garwicz, M., & Hesslow, G. (1988). Evidence for a GABA-mediated cerebellarinhibition of the inferior olive in the cat. Experimental Brain Research, 72, 450–456.

Angaut, P., & Sotelo, C. (1989). Synaptology of the cerebello-olivary pathway. Double labellingwith anterograde axonal tracing and GABA immunocytochemistry in the rat. Brain Research,479, 361–365.

Benedeti, F., Montarolo, P. G., & Rabacchi, S. (1984). Inferior olive lesion induces long-lastingfunctional modification in the Purkinje cells. Experimental Brain Research, 55, 368–371.

Berry, S. D., & Swain, R. A. (1989). Water deprivation optimizes hippocampal activity and facili-tates nictitating membrane conditioning. Behavioral Neuroscience, 103, 71–76.

Brandon, S. E., & Wagner, A. R. (1991). Modulation of a discrete Pavlovian conditioned reflex by aputative emotive conditioned stimulus. Journal of Experimental Psychology: Animal BehaviorProcesses, 17, 299–311.

Brindley, C. S. (1964). The use made by the cerebellum of the information that it receives fromsense organs. IBRO Bulletin, 3, 80.

Brogden, W. J., & Gantt, W. H. (1937). Cerebellar conditioned reflexes. American Journal of Physi-ology, 119, 277–278.

Brogden, W. J., & Gantt, W. H. (1942). Intraneural conditioning: Cerebellar conditioned reflexes.Archives of Neurology and Psychiatry, 48, 437–455.

Brown, J. S. (1961). The motivation of behavior. New York: McGraw–Hill.

Bruner, A. (1963). Investigations of the properties of the unconditioned stimulus in classical condi-tioning of the nictitating membrane response in the albino rabbit. Doctoral dissertation, Indi-ana University.

Chapman, P. F., Steinmetz, J. E., & Thompson, R. F. (1988). Classical conditioning does not occurwhen direct stimulation of the red nucleus or cerebellar nuclei is the unconditioned stimulus.Brain Research, 442, 97–104.

Chen, C., Masanobu, K., Abeliovich, A., Chen, L., Bao, S., Kim, J. J., Hashimoto, K., Thompson,



R. F., & Tonegawa, S. (1995). Impaired motor coordination correlates with persistent multipleclimbing fiber innervation in PKCg mutant mice. Cell 83, 1233–1242.

Chen, C., & Thompson, R. F. (1995). Temporal specificity of long-term depression in parallel fiber–Purkinje synapses in rat cerebellar slice. Learning and Memory, 2, 185–198.

Chen, L., Bao, S., Lockard, J. M., Kim, J. J., & Thompson, R. F. (1996). Impaired classical eyeblinkconditioning in cerebellar lesioned and Purkinje cell degeneration (pcd) mutant mice. Journalof Neuroscience, 16, 2829–2838.

R. E., & Lavond, D. G. (1993). Reversible lesions of the red nucleus during acquisition and retentionof a classically conditioned behavior in rabbit. Behavioral Neuroscience, 107, 264–270.

Clark, R. E., Zhang, A. A., & Lavond, D. G. (1992). Reversible lesions of the cerebellar interpositusnucleus during acquisition and retention of a classically conditioned behavior. BehavioralNeuroscience, 106, 879–888.

Crepel, F. (1982). Regression of functional synapses in the immature mammalian cerebellum.Trends in Neurosciences, 5, 266–269.

Donegan, N. H., Gluck, M. A., & Thompson, R. F. (1989). Integrating behavioral and biologicalmodels of classical conditioning. In R. D. Hawkins & G. H. Bower (Eds.), Psychology of learningand motivation (Vol. 23, pp. 109–156). New York: Academic Press.

Doty, R. W., & Giurgea, C. (1961). Conditioned reflexes established by coupling electrical excitationof two cortical areas. In J. F. Delafresnaye (Ed.), Brain mechanisms and learning (pp. 133–151). Oxford: Blackwell.

Eccles, J. C. (1977). An instruction–selection theory of learning in the cerebellar cortex. BrainResearch, 127, 327–352.

Eccles, J. C., Ito, M., & Szentagothai, J. (1967). The cerebellum as a neuronal machine. Berlin:Springer-Verlag.

Estes, W. K. (1959). The statistical approach to learning theory. In S. Koch (Ed.), Psychology: Astudy of a science (Vol. 2, pp. 380–491). New York: McGraw–Hill.

Gantt, W. H. (1937). Contributions to the physiology of the conditioned reflex. Archives of Neurol-ogy and Psychiatry, 37, 848–858.

M. A., Reifsnider, E. S., & Thompson, R. F. (1990). Adaptive signal processing and the cerebellum:Models of classical conditioning and VOR adaptation. In M. A. Gluck & D. E. Rumelhart (Eds.),Neuroscience and connectionist models (pp. 131–185). Hillsdale, NJ: Erlbaum.

Gluck, M. A., & Thompson, R. F. (1987). Modeling the neural substrates of associative learningand memory: A computational approach. Psychological Review, 94, 176–191.

Gormezano, I., Kehoe, E. J., & Marshall-Goodell, B. S. (1983). Twenty years of classical condition-ing research with the rabbit. In J. M. Sprague & A. N. Epstein (Eds.), Progress in physiologicalpsychology (pp. 197–275). New York: Academic Press.

Graybiel, A. M., Aosaki, T., Flaherty, A. W., & Kimura, M. (1994). The basal ganglia and adaptivemotor control. Science, 265, 1826–1831.

Guthrie, E. R. (1930). Conditioning as a principle of learning. Psychological Review, 40, 355–367.Hardiman, M. J., Ramnani, N., & Yeo, C. H. (1996). Reversible inactivations of the cerebellum

with muscimol prevent the acquisition and extinction of conditioned nicitating membraneresponses in the rabbit. Experimental Brain Research, 110, 235–247.

Harlow, H. F., & Stagner, R. (1933). Effect of complete striate muscle paralysis upon the learningprocess. Journal of Experimental Psychology, 16, 283–294.

Hesslow, G., & Ivarsson, M. (1996). Inhibition of the inferior olive during conditioned responsesin the decerebrate ferret. Experimental Brain Research, 110, 36–46.

Ito, M. (1972). Neural design of the cerebellar motor control system. Brain Research, 40, 81–84.Ito, M. (1984). The cerebellum and neural control. New York: Appleton-Century-Crofts.Ito, M. (1994). Cerebellar mechanisms of long-term depression. In M. Baudry, Davis, & R. F.

Thompson (Eds.), Synaptic plasticity: Molecular and functional aspects. Cambridge, MA: MITPress.

Ito, M., Sukurai, M., & Tongroach, P. (1982). Climbing fibre induced depression of both mossy fibreresponsiveness and glutamate sensitivity of cerebellar Purkinje cells. Journal of Physiology,London, 324, 113–134.


174 THOMPSON ET AL.

Ivkovich, D., & Thompson, R. F. (in press). Motor cortex lesions do not effect learning or perfor-mance of the eyeblink response in rabbits. Behavioral Neuroscience.

Kamin, L. J. (1968). Attention-like processes in classical conditioning. In M. R. Jones (Ed.), Miamisymposium on the prediction of behavior: Aversive stimulation (pp. 9–32). Miami: Univ. ofMiami Press.

Kamin, L. J. (1969). Predictability, surprise, attention and conditioning. In R. M. Church (Ed.),Punishment and aversive behavior (pp. 279–296). New York: Appleton-Century-Crofts.

Kano, M., Hashimoto, K., Chen, C., Abeliovich, A., Aiba, A., Kurihara, H., Watanabe, M., Inoue,Y., & Tonegawa, S. (1995). Impaired synapse elimination during cerebellar development inPKC, mutant mice. Cell 83, 1223–1231.

Keele, S. W., & Ivry, R. B. (1990). Does the cerebellum provide a common computation for diversetasks: A timing hypothesis. In A. Diamond (Ed.), The development and neural bases of highercognitive functions (pp. 179–211). New York: New York Academy of Sciences Press.

Kim, J. J., Krupa, D. J., & Thompson, R. F. (1992). Intra-olivary infusions of picrotoxin prevent‘‘blocking’’ of rabbit conditioned eyeblink response. Society for Neuroscience Abstracts, 18,1562.

Kim, J. J., Krupa, D. J., & Thompson, R. F. (in press). Inhibitory cerebello-olivary projectionsmediate the ‘‘blocking’’ effect in classical conditioning. Science.

Kim, J. J., & Thompson, R. F. (1997). Cerebellar circuits and synaptic mechanisms involved inclassical eyeblink conditioning. Trends in Neurosciences, 20, 177–181.

Krupa, D. J. (1993). Localization of the essential memory trace for a classically conditioned behav-ior. Doctoral dissertation, University of Southern California.

Krupa, D. J., Thompson, J. K., & Thompson, R. F. (1993). Localization of a memory trace in themammalian brain. Science, 260, 989–991.

Krupa, D. J., & Thompson, R. F. (1995). Inactivation of the superior cerebellar peduncle blocksexpression but not acquisition of the rabbit’s classically conditioned eyeblink response. Pro-ceedings of the National Academy of Sciences of the USA, 92, 5097–5101.

Krupa, D. J., & Thompson, R. F. (1997). Reversible inactivation of the cerebellar interpositusnucleus completely prevents acquisition of the classically conditioned eyeblink response.Learning and Memory, 3, 545–556.

Krupa, D. J., Weng, J., & Thompson, R. F. (1996). Inactivation of brainstem motor nuclei blocksexpression but not acquisition of the rabbit’s classically conditioned eyeblink response. Behav-ioral Neuroscience, 110, 1–9.

Lavond, D. G., Kim, J. J., & Thompson, R. F. (1993). Mammalian brain substrates of aversiveclassical conditioning. Annual Review of Psychology, 44, 317–342.

Light, J. S., & Gantt, W. H. (1936). Essential part of reflex arc for establishment of conditionedreflex. Journal of Comparative Psychology, 21, 19–36.

Linden, D. J., & Conner, J. A. (1995). Long-term synaptic depression. Annual Review of Neurosci-ence, 18, 319–357.

Loucks, R. B. (1935). The experimental delimitation of neural structures essential for learning: Theattempt to condition striped muscle response with faradization of the sigmoid gyri. Journal ofPsychology, 1, 5–44.

Mackintosh, N. J. (1975). A theory of attention: Variations in the associability of stimuli withreinforcement. Psychological Review, 82, 276–298.

Maren, S., DeCola, J. P., Swain, R. A., Fanselow, M. S., & Thompson, R. F. (1994). Parallel aug-mentation of hippocampal long-term potentiation, theta rhythm, and contextual fear condi-tioning water-deprived rats. Behavioral Neuroscience, 108, 44–56.

Marr, D. (1969). A theory of cerebellar cortex. Journal of Physiology, 202, 437–470.

Mauk, M. D., Steinmetz, J. E., & Thompson, R. F. (1986). Classical conditioning using stimulationof the inferior olive as the unconditioned stimulus. Proceedings of the National Academy ofSciences of the USA, 83, 5349–5353.

McCormick, D. A., Clark, G. A., Lavond, D. G., & Thompson, R. F. (1982). Initial localization ofthe memory trace for a basic form of learning. Proceedings of the National Academy of Sciencesof the USA, 79, 2731–2742.



McCormick, D. A., Steinmetz, J. E., & Thompson, R. F. (1985). Lesions of the inferior olivarycomplex cause extinction of the classically conditioned eyeblink response. Brain Research,359, 120–130.

McCormick, D. A., & Thompson, R. F. (1984). Neuronal responses of the rabbit cerebellum duringacquisition and performance of a classically conditioned nictitating membrane-eyelid re-sponse. Journal of Neuroscience, 4, 2811–2822.

McLaren, I. P., & Dickinson, A. (1990). The conditioning connection. Philosophical Transactionsof the Royal Society of London, B329, 179–186.

Mintz, M., Lavond, D. G., Zhang, A. A., Yun, Y., & Thompson, R. F. (1994). Unilateral inferiorolive NMDA lesion leads to unilateral deficit in acquisition of NMR classical conditioning.Behavioral and Neural Biology, 61, 218–224.

Nelson, B., & Mugnoini, E. (1987). GABAergic innervation of the inferior olivary complex andexperimental evidence for its origin. In P. Strata (Ed.). The olivocerebellar system in motorcontrol. New York: Springer-Verlag.

Nordholm, A. F., Thompson, J. K., Dersarkissian, C., & Thompson, R. F. (1993). Lidocaine infusionin a critical region of cerebellum completely prevents learning of the conditioned eyeblinkresponse. Behavioral Neuroscience, 107, 882–886.

Nowak, A. J., & Gormezano, I. (1990). Electrical stimulation of brainstem nuclei: Elicitation,modification and conditioning of the rabbit nictitating membrane response. Behavioral Neuro-science, 104, 4–10.

Pavlov, I. P. (1927). Conditioned reflexes: An investigation of the physiological activity of the cere-bral cortex. London: Oxford Univ. Press. [G. V. Anrep, translator]

Pearce, J. M., & Hall, G. (1980). A model for Pavlovian learning: variations in the effectivenessof conditioned but not of unconditioned stimuli. Psychological Review, 87, 532–552.

Rescorla, R. A. (1967). Pavlovian conditioning and its proper control procedures. PsychologicalReview, 74, 71–80.

Rescorla, R. A. (1968). Probability of shock in the prescence and abscence of CS in fear conditioning.Journal of Comparative and Physiological Psychology, 66, 1–5.

Rescorla, R. A. (1988). Behavioral studies of Pavlovian conditioning. Annual Review of Neurosci-ence, 11, 329–352.

Rescorla, R. A., & Solomon, R. L. (1967). Two-process learning theory: Relationships betweenPavlovian conditioning and instrumental learning. Psychological Review, 74, 151–182.

Rescorla, R., & Wagner, A. (1972). A theory of Pavlovian conditioning: Variations in the effective-ness of reinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Currentresearch theory (pp. 64–99). New York: Appleton-Century-Crofts.

Rogers, R. F., Fender, A. S., & Steinmetz, J. E. (1997). Interpositus nucleus contributions to nonso-matosensory-based eyelid conditioning. American Psychological Society Abstracts, 31.

Rumelhart, D. E., & McClelland, J. L. (1986). Parallel distributed processing: Explorations in themicrostructure of cognition. Cambridge, MA: MIT Press.

Schlosberg, H. (1937). The relationship between success and the laws of conditioning. PsychologicalReview, 1937, 379–394.

Schreurs, B. G., Oh, M. M., & Alkon, D. L. (1996). Pairing-specific long-term depression of Purkinjecell excitatory postsynaptic potentials results from a classical conditioning procedure in therabbit cerebellar slice. Journal of Neurophysiology, 75, 1051–1060.

Schultz, W. (1997). Dopamine neurons and their role in reward mechanisms. Current Opinion inNeurobiology, 7, 191–197.

Schultz, W., Apicella, P., & Ljungberg, T. (1993). Responses of monkey dopamine neurons toreward and conditioned stimuli during successive steps of learning a delayed response task.Journal of Neuroscience, 13, 900–913.

Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward.Science, 275, 1593–1599.

Sears, L. L., & Steinmetz, J. E. (1991). Dorsal accessory inferior olive activity diminishes duringacquisition of the rabbit classically conditioned eyelid response. Brain Research, 545, 114–122.


176 THOMPSON ET AL.

Shinkman, P. G., Swain, R. A., & Thompson, R. F. (1996). Classical conditioning with electricalstimulation of cerebellum as both conditioned and unconditioned stimulus. Behavioral Neuro-science, 110, 914–921.

Steinmetz, J. E., Lavond, D. G., Ivkovich, D., Logan, C. G., & Thompson, R. F. (1992). Disruptionof classical eyelid conditioning after cerebellar lesions: Damage to a memory trace system ora simple performance deficit? Journal of Neuroscience, 12, 4403–4426.

Steinmetz, J. E., Lavond, D. G., & Thompson, R. F. (1989). Classical conditioning in rabbits usingpontine nucleus stimulation as a conditioned stimulus and inferior olive stimulation as anunconditioned stimulus. Synapse, 3, 225–232.

Steinmetz, J. E., McCormick, D. A., Baier, C. A., & Thompson, R. F. (1984). Involvement of theinferior olive in classical conditioning of the rabbit eyelid. Neuroscience Abstracts, 10, 122.

Swain, R. A., Shinkman, P. G., Nordholm, A. F., & Thompson, R. F. (1992). Cerebellar stimulationas an unconditioned stimulus in classical conditioning. Behavioral Neuroscience, 106, 739–750.

Tanaka, C., & Nishizuka, Y. (1994). The protein kinase C family for neuronal signaling. AnnualReview of Neuroscience, 17, 551–567.

Thach, W. T., Goodkin, H. G., & Keating, J. G. (1992). The cerebellum and the adaptive coordina-tion of movement. Annual Review of Neuroscience, 15, 403–442.

Thompson, R. F. (1989a). Role of inferior olive in classical conditioning. In P. Strata (Ed.), Theolivocerebellar system in motor control. New York: Springer-Verlag.

Thompson, R. F. (1989b). The essential memory trace circuit and the essential reinforcementsystem for a basic form of associative learning. In T. Archer & L-G. Nilsson (Eds.), Aversion,avoidance and anxiety: Perspectives on aversively motivated behavior (pp. 251–264). Hillsdale,NJ: Erlbaum.

Thompson, R. F., & Kim, J. J. (1996). Memory systems in the brain and localization of a memory.Proceedings of the National Academy of Sciences of the USA, 93, 13438–13444.

Thompson, R. F., & Krupa, D. J. (1994). Organization of memory traces in the mammalian brain.Annual Review of Neuroscience, 17, 519–549.

Thorndike, E. L. (1911). Animal intelligence: Experimental studies. New York: Macmillian.Voneida, T., Christie, D., Boganski, R., & Chopko, B. (1990). Changes in instrumentally and

classically conditioned limb-flexion responses following inferior olivary lesions and olivocere-bellar tractotomy in the cat. Journal of Neuroscience, 10, 3583–3593.

Wagner, A. R. (1981). SOP: A model of automatic memory processing in animal behavior. In N. E.Spear & R. R. Miller (Eds.), Information processing in animals: Memory mechanisms (pp. 5–47). Hillsdale, NJ: Erlbaum.

Wagner, A. R., & Brandon, S. E. (1989). Evolution of a structured connectionist model of Pavlovianconditioning (AESOP). In S. B. Klein & R. R. Mowrer (Eds.), Contemporary learning theories:Pavlovian conditioning and the status of traditional learning theory (pp. 149–190). Hillsdale,NJ: Erlbaum.

Wagner, A. R., & Donegan, N. (1989). Some relationships between a computational model (SOP)and an essential neural circuit for Pavlovian (rabbit eyeblink) conditioning. In R. D.Hawkins & G. H. Bower (Eds.), Computational models of learning in simple neural systems:The psychology of learning and motivation (Vol. 23, pp. 157–203). New York: Academic Press.

Wagner, A. R., & Rescorla, R. A. (1972). Inhibition in Pavlovian conditioning: Application of atheory. In R. A. Boakes & M. S. Halliday (Eds.), Inhibition and learning (pp. 301–336). Lon-don: Academic Press.

Wagner, A. R., Thomas, E., & Norton, T. (1967). Conditioning with electrical stimulation of motorcortex: Evidence of a possible source of motivation. Journal of Comparative PhysiologicalPsychology, 64, 191–199.

Widrow, G., & Hoff, M. E. (1960). Adaptive switching circuits. Institute of Radio Engineers, WesternElectric Show and Convention, Convention Record, 4, 96–194.

Yeo, C. H., Hardiman, M. J., & Glickstein, M. (1986). Classical conditioning of the nictitatingmembrane response of the rabbit IV. Lesions of the inferior olive. Experimental Brain Re-search, 63, 81–92.


the nature of reinforcement in cerebellar learning

Documents