307c17e2e1982db2749a0b97534048b9

cover next page >

Rethinking Innateness

A connectionist perspective on development

title: Rethinking Innateness : A Connectionist Perspective OnDevelopment Neural Network Modeling and Connectionism

author: Elman, Jeffrey L.publisher: MIT Press

isbn10 | asin: 0262050528print isbn13: 9780262050524

ebook isbn13: 9780585020341language: English

subject Nature and nurture, Connectionism, Nativism (Psychology)publication date: 1996

lcc: BF341.R35 1996ebddc: 155.7

subject: Nature and nurture, Connectionism, Nativism (Psychology)

cover next page >

If you like this book, buy it!

< previous page cover-0 next page >

Neural Network Modeling and Connectionism Jeffrey L. Elman, Editor

Connectionist Modeling and Brain Function: The Developing Interface Stephen Jos Hanson and Carl R. Olson, editors

Neural Network Design and the Complexity of Learning J. Stephen Judd

Neural Networks for Control W. Thomas Miller, Richard S. Sutton, and Paul J. Werbos, editors

The Perception of Multiple Objects: A Connectionist Approach Michael C. Mozer

Neural Computation of Pattern Motion: Modeling Stages of Motion Analysis in the Primate Visual Cortex Margaret Euphrasia Sereno

Subsymbolic Natural Language Processing: An Integrated Model of Scripts, Lexicon, and Memory Risto Miikkulainen

Analogy-Making as Perception: A Computer Model Melanie Mitchell

Mechanisms of Implicit Learning: Connectionist Models of Sequence Processing Axel Cleeremans

The Human Semantic Potential: Spatial Language and Constrained Connectionism Terry Regier

Rethinking Innateness: A Connectionist Perspective on Development Jeffrey L. Elman, Elizabeth A. Bates, Mark H. Johnson, Annette Karmiloff-Smith, Domenico Parisi, and Kim Plunkett




Rethinking InnatenessA connectionist perspective on development

Jeffrey L. Elman, Elizabeth A. Bates, Mark H. Johnson, Annette Karmiloff-Smith,

Domenico Parisi, Kim Plunkett

A Bradford Book The MIT Press

Cambridge, Massachusetts London, England




Second printing, 1997 1996 Massachusetts Institute of Technology

All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher.

Printed and bound in the United States of America.

Library of Congress Cataloging-in-Publication Data

Rethinking innateness: a connectionist perspective on development / Jeffrey L. Elman. [et al.] p. cm.(Neural network modeling and connectionism: X) "A Bradford book." Includes bibliographical references and index. ISBN 0-262-05052-8 (hb: alk. paper) 1. Nature and nurture. 2. Connectionism. 3. Nativism (Psychology) I. Elman, Jeffrey L. II. Series: Neural network modeling and connectionism: 10. BF341.R35 1996 155.7dc20 96-15522 CIP



< previous page page_v next page >

Page v

"This is dedicated to the ones we love."




Page v




mojesabzRectangle

< previous page page_vii next page >

Page vii

Contents

Series foreword ix

Preface xi

CHAPTER 1 New perspectives on development 1

CHAPTER 2 Why connectionism? 47

CHAPTER 3 Ontogenetic development: A connectionist synthesis 107

CHAPTER 4 The shape of change 173

CHAPTER 5 Brain development 239

CHAPTER 6 Interactions, all the way down 319

CHAPTER 7 Rethinking innateness 357

References 397

Subject index 443

< previous page page_vii next page >



Page v




mojesabzRectangle

< previous page page_ix next page >

Page ix

Series foreword

The goal of this series, Neural Network Modeling and Connectionism, is to identify and bring to the public the best work in theexciting field of neural network and connectionist modeling. The series includes monographs based on dissertations, extendedreports of work by leaders in the field, edited volumes and collections on topics of special interest, major reference works, andundergraduate and graduate-level texts. The field is highly interdisciplinary, and works published in the series will touch on awide variety of topics ranging from low-level vision to the philosophical foundations of theories of representation.

Jeffrey L. Elman, Editor

Associate Editors:

James Anderson, Brown University Andrew Barto, University of Massachusetts, Amherst Gary Dell, University of Illinois Jerome Feldman, University of California, Berkeley Stephen Grossberg, Boston University Stephen Hanson, Princeton University Geoffrey Hinton, University of Toronto Michael Jordan, MIT James McClelland, Carnegie-Mellon University Domenico Parisi, Instituto di Psicologia del CNR David Rumelhart, Stanford University Terrence Sejnowski, The Salk Institute Paul Smolensky, Johns Hopkins University Stephen P. Stich, Rutgers University David Touretzky, Carnegie-Mellon University David Zipser, University of California, San Diego

< previous page page_ix next page >



Page v




mojesabzRectangle

< previous page page_xi next page >

Page xi

Preface

Where does knowledge come from?

That is the central question we pose in this book.

Human children and adults know many things. They know how to speak a language (many people know several). They knowhow to read maps, and how to go from San Diego to London and Rome. Some of us know how to build cars, and others knowhow to solve partial differential equations.

Most people feel that knowledge comes from two kinds of sources: What is given us by virtue of our nature, and what we knowas a consequence of our nurture. In reality, though, it is far from clear what is meant by either nature or nurture. Nature isusually understood to mean "present in the genotype," and nurture usually means "learned by experience." The difficulty is thatwhen we look at the genome, we don't really see arms or legs (as the preformationists thought we might) and we certainly don'tsee complex behaviors. As we learn more about the basic mechanisms of genetics, we have come to understand that the distaleffects of gene products are highly indirect, complicated, and most often dependent on interactions not only with other geneproducts but also with external events.

Learning is similarly problematic. We know that learning probably involves changes in synaptic connections, and it is nowbelieved that these changes are effected by the products of specific genes which are expressed only under the conditions whichgive rise to learning.

The obvious conclusion is that the real answer to the question, Where does knowledge come from, is that it comes from theinteraction between nature and nurture, or what has been called "epigenesis."

< previous page page_xi next page >


< previous page page_xii next page >

Page xii

Genetic constraints interact with internal and external environmental influences, and they jointly give rise to the phenotype.

Unfortunately, as compelling and sensible as this claim seems, it is less a conclusion than a starting point. The problem does notgo away, it is simply rephrased. In fact, epigenetic interactions must, if anything, be more complicated than the simpler morestatic view that x% of behavior comes from genes and y% comes from the environment. For this reason, the interactionist (orconstructivist) approach has engendered a certainamount of skepticism on the part of developmentalists. To paraphrase DavidKlahr (whose complaint was about Piaget's concepts of assimilation and accommodation), nature and nurture are like theBatman and Robin of developmental theory: They hang around waiting in the wings, swoop in and solve a problem, and thendisappear before they can be unmasked.

In fact, we believe that the interactionist view is not only the correct one, but that the field is now in a position where we canflesh this approach out in some detail. Our optimism springs from two sources. First, there has been extraordinary progress madein recent years in genetics, embryology, and developmental neuroscience. We are beginning to have an idea of how genes dotheir job. In addition, much has been discovered about the cortical basis for complex behavior. We also know more now thanwe did two decades ago about brain development; current evidence suggests a far higher degree of cortical plasticity than wasanticipated, and this has obviously far-reaching consequences for theories of development. An impressive array of tools forstudying brain processes has been developed, which permit non-invasive access to events in the brain and a spatial and temporalgranularity that is quite remarkable.

Second, we have seen in recent years dramatic advances in a framework for computation which is particularly appropriate forunderstanding neural processing. This framework has been variously called parallel distributed processing, neural networkmodeling, or connectionism (a term introduced by Donald Hebb in the 1940's, and the name we adopt here). This approach hasdemonstrated that a great deal of information is latent in the environment

< previous page page_xii next page >


< previous page page_xiii next page >

Page xiii

and can be extracted using simple but powerful learning rules. But importantly, connectionist models also suggest new ways inwhich things can be innate. Furthermore, by using connectionist models together with genetic algorithms and artificial lifemodels, it is possible to study within one and the same simulation evolutionary change at the level of populations of neuralnetworks, maturation and learning in individual neural networks, and the interactions between the two.

This book offers our perspective on development, the nature/ nurture controversy, and on the issue of innateness. The definitionof innateness itself is controversial. We take the question to be essentially how to account for those behaviors which, given thenormal experiences encountered during development, are universal across a species. This is a much broader perspective thanmany might adopt, but it lets us then ask what are the sources of constraint which lead to these universal outcomes.

We take a connectionist perspective, but we are very aware that ours is a specific and significantly enlarged conception of whatconnectionism is. In some ways, it is our view of what connectionism should (and hopefully, will) be. We are convinced thatconnectionism has a great deal to offer for understanding development. We also think that connectionists can only profit fromthe encounter with development. In a very deep sense, we believe that development is not just an accidental path on the wayfrom being small to getting big. Rather, with Piaget, we are convinced that only by understanding the secrets of the process ofdevelopment will we ever understand complex behaviors and adult forms; but our solution will be somewhat different fromPiaget's.

It seems appropriate to say something about how we came to write this book and the process by which it was written. Thereader should know that this book is a truly collaborative effort; thus, chapters are unsigned and each reflects our joint efforts.The collaboration began in the late 1980's. At that time, the John D. and Catherine T. MacArthur Foundation awarded a traininggrant to the Center for Research in Language at UCSD; this was part of the larger MacArthur Network in Transitions fromInfancy to Early Childhood, directed by Bob Emde. The basic goal was to introduce

< previous page page_xiii next page >


< previous page page_xiv next page >

Page xiv

developmentalists to the tools and methodology of connectionist modeling. But the training program was unusual in severalrespects, and reflects the open and innovative approach encouraged by Emde. There was a high degree of flexibility in theprogram. We were able to bring senior as well as junior fellows to UCSD, and for varying degrees of time, depending on theschedules and goals of the fellows. In some cases, the goal was simple literacy in connectionist modeling. In other cases, fellowsdeveloped computer simulations of data they had brought with them. An important component of the program was the set ofsimulation exercises which were created to illustrate properties of connectionist models that are especially relevant todevelopmental theory. These simulations have been extended and amplified and form the core of the companion volume to thisbook.

After several years of the program, a workshop was held at UCSD in 1991. Alumni of the trainingprogram returned for afour-day reunion and presented the work that they had done as a result of their participation in the program. This was anextremely exciting event, because it impressed upon us the extent to which connectionism not only provides a very naturalcomputational framework for modeling many developmental phenomena, but also gives us concepts for rethinking some of theold chestnuts. Furthermore, we realized that a critical mass was building; as a group, we felt we had a great deal to say. Thuswas born the idea of summarizing this work in book form.

About the same time, the organizers of the Society for Research in Child Development invited several of us to organize a specialcolloquium on connectionism and development for their 1992 meeting. Betty Stanton of Bradford Books/MIT Press was presentat that symposium. Afterwards, she enthusiastically suggested that we turn the symposium contents into a book. Having justdecided ourselves that the time was ripe to do this, we were pleased with Betty's excitement and support.

Primarily for logistical reasons, a subset of the original group proceeded to work on the book, with Jeff Elman being chieflyresponsible for coordinating the joint efforts. We realized that we had several goals which would require more than one volume.

< previous page page_xiv next page >


< previous page page_xv next page >

Page xv

First, we wanted to make a theoretical statement; this warranted a volume of its own. Second, we very much wanted to make themethodology accessible to as broad an audience as possible; so we designed a second volume with this pedagogical goal inmind. The second volume contains software and simulation exercises which allow the interested reader to replicate many of thesimulations we describe in the first volume. The software is general purpose and can also be used by readers to carry out theirown simulations. Finally, there is now a considerable body of literature in using connectionism to model development. Althoughwe summarize and refer to much of this literature in the first two volumes, we felt it would be useful to collect some of the bestwork into a reader, the third volume. The first two volumes will be published almost simultaneously; our goal is for the thirdvolume to appear within 18 months.

Our conception of the first volume has changed dramatically in the course of writing. Our original view was that this volumewould bring together chapters written by us as individuals. Our discussions as a group proved so stimulating, however, that wesoon moved to a very different model: A truly coauthored volume reflecting our joint ideas. Of course, this meant having todevelop these joint ideas! As congenial as our viewpoints were and as great the overlap in our attitudes, we discovered that therewere many areas around which we held different opinions, and very many more about which we held no opinions at all. Theprocess of writing the book was thus highly constructive. Our meetings became seminars; the planning of chapter contentsturned into lively discussions of theory. We all have found the process of working on this project to be enormously stimulatingand we have learned much from each other. If we may be permitted a bit of self-appreciation, we are very grateful to each otherfor the forbearance, patience, and graciousness which have made it possible to forge a synthesis out of our differentperspectives. This book is more than could have been produced by any one of us, and all of us feel that we ourselves havegained from the experience.

We owe a great deal to the MacArthur Foundation for their support. The far-sighted approach of Bob Emde, Mark Appelbaum,

< previous page page_xv next page >


< previous page page_xvi next page >

Page xvi

Kathryn Barnard, Marshall Haith, Jerry Kagan, Marion Radke-Yarrow, and Arnold Sameroff was critical in this effort. Withouttheir supportboth tangible as well as intellectualwe could not have written this book.

We also wish to make clear that although the cover of this volume bears the names of only six of us, the ideas within the bookreflect an amalgam of insights and findings garnered from a much larger group. This includes the trainees in the program aswell as other participants in the 1991 workshop: Dick Aslin, Alain Content, Judith Goodman, Marshall Haith, Roy Higginson,Claes von Hofsten, Jean Mandler, Michael Maratsos, Brian MacWhinney, Bruce Pennington, Elena Pizzuto, Rob Roberts, JimRussell, Richard Schwartz, Joan Stiles, David Swinney, and Richard Wagner. The trainers in the program, Virginia Marchman,Mary Hare, Arshavir Blackwell, and Cathy Harris, were more than trainers. They were colleagues and collaborators, and theyplayed a pivotal role in the program and in our thinking. We are also grateful to a number of colleagues with whom we haveinteracted over the years. Some may still not agree with our arguments, while others will undoubtedly recognize some of theirown ideas in the pages that follow. We owe to Jay McClelland the opening sentence of this Preface. We are grateful to thesefriends and hope that our translation of their ideas will not displease them.

In addition, we wish to thank those who read and commented on various sections of this book. Dorothy Bishop, Gergely Cisbra,Terry Deacon, Lucy Hadden, Francesca Happ, Henry Kennedy, Herb Killackey, Jean Mandler, Jay Moody, Yuko Munakata,Andrew Oliver, Adolfo Perinat, Paul Rodriguez, Marty Sereno, JeffShrager, Tassos Stevens, Joan Stiles, Faraneh Vargha-Khadem, and members of the UCSD DevLab have provided us with important and valuable feedback.

In December of 1994, the University of Higher Studies in the Republic of San Marino sponsored a two-day workshop entitled''Rethinking Innateness," where three of our authors were able to air some of the ideas in this book and to listen and learn fromsome of the best cognitive neuroscientists in Italy. Herb Killackey joined us at the San Marino workshop, and we are immenselygrateful to

< previous page page_xvi next page >


< previous page page_xvii next page >

Page xvii

him for his input, and for extensive discussions about the topics covered in Chapter 5.

George Carnevale played a particularly important role in Chapter 4. Much of that chapter draws on his own work (Bates &Carnevale, 1993), and George helped track down a number of errors in earlier drafts (of course, we reserve for ourselves thecredit for those errors that remain). We also thank Jacqueline Johnson for making available the data from Johnson & Newport(1989), which we reanalyze in Chapter 4.

Meiti Opie not only read, commented, and proofed numerous drafts of Chapters 1, 5, and 7, but also served as general editorialassistant. She also tracked down and prepared the references. Meiti's persistence and attention to detail were extraordinary, andvery much appreciated.

Betty and Harry Stanton of Bradford Books/MIT Press have been enthusiastic and eager in their support for this book from itsbeginning. We appreciate their faith in us, and their willingness to believe, as do we, that we have something important andexciting to say. Teri Mendelsohn, and later, Amy Pierce, of MIT Press have been an enormous help in the actual production ofthe first two volumes. A great deal of coordination was required, given the six co-authors, two books, and packaging andproduction of software. Teri and Amy made the job much easier, and their patience and encouragement is much appreciated.

When we began this effort, we did not fully appreciate the difficulties of producing a coauthored book with six authors whowere located in San Diego, Pittsburgh, London, Oxford, and Rome. The logistics of travel, hotel accommodations, and arrangingperiodic meetings were formidable. Bob Buffington, Jan Corte, Miriam Eduvala, Larry Juarez, John Staight, and Meiti Opie ofthe Center for Research in Language at UCSD, and Leslie Tucker at the Cognitive Development Unit, London, all played acritical role in arranging our meetings and making the time together as productive as possible. We are very much indebted tothem for their help.

We acknowledge the financial support which has been provided to the authors and made the research described here possible. Inaddition to the funds from the MacArthur Foundation already men-

< previous page page_xvii next page >


< previous page page_xviii next page >

Page xviii

tioned, this includes support from the Office of Naval Research (contract N00014-93-1-0194) and National Science Foundation(grant DBS 92-09432) to Jeff Elman; the National Institutes of Health (grants NIH/NIDCD 2-R01-DC00216, NIH/NINDS P50NS22343, NIH/NIDCD Program Project P50 DC01289-0351) to Elizabeth Bates; Carnegie Mellon University, the NationalScience Foundation (grant DBS 91-20433), and the Medical Research Council of the United Kingdom to Mark Johnson; aMcDonnell/Pew Visiting Fellowship and the Medical Research Council of the United Kingdom to Annette Karmiloff-Smith;and the Science and Engineering Research Council, the Biological and Biotechnical Research Council, and the Economic andSocial Sciences Research Council to Kim Plunkett.

Finally, we wish to thank Marta Kutas, who is a valued colleague and a treasured friend. She wrote a number of poems on thetheme of innateness, and we are pleased and flattered that she allowed us to choose one to open the book.

While the central arguments and concepts of this book represent our collaborative efforts, in any enterprise involving severalauthors with very different backgrounds there are bound to be areas of disagreement that cannot be resolved. One of theseconcerned the title, with which MJ wishes to put on record his disagreement. In MJ's view the term "innate" is better dispensedwith entirely, as opposed to being re-thought.

< previous page page_xviii next page >


< previous page page_xix next page >

Page xix

< previous page page_xix next page >


< previous page page_1 next page >

Page 1

CHAPTER 1 New perspectives on development

The problem of change

Things change. When things change in a positive direction (i.e., more differentiation, more organization, and usually ensuringbetter outcomes), we call that change "development." This is Heinz Werner's orthogenic principle (Werner, 1948).

Ironically, in the past several decades of developmental research there has been relatively little interest in the actual mechanismsresponsible for change. The evidence of surprising abilities in the newborn, coupled with results from learning theory whichsuggest that many important things which we do as adults are not learnable, have led many researchers to conclude thatdevelopment is largely a matter of working out predetermined behaviors. Change, in this view, reduces to the mere triggering ofinnate knowledge.

Counterposed to this is the other extreme: Change as inductive learning. Learning, in this view, involves a copying orinternalizing of behaviors which are present in the environment. "Knowledge acquisition" is understood in the literal sense. Yetthis extreme view is favored by few. Not only does it fail to explain the precocious abilities of infants and their final maturestates, but it also fails to provide any account of how knowledge is deposited in the environment in the first place.

The third possibility, which has been the position advocated by classic developmentalists such as Waddington and Piaget, is thatchange arises through the interaction of maturational factors, under genetic control, and the environment. The interaction atissue here is not the banal kind where black and white yield gray, but a much more challenging and interesting kind where thepathways from genotype to phenotype may be highly indirect and nonobvious. The




Page 2

problem with this view in the past has been that, lacking a formal and precise theory of how such interactions might occur, talkof "emergent form" was at best vague. At worst, it reduces to hopeless mysticism.

Two recent developments, however, suggest that the view of development as an interactive process is indeed the correct one,and that a formal theory of emergent form may be within our grasp. The first development is the extraordinary progress that hasbeen made in the neurosciences. The second has been the renascence of a computational framework which is particularly wellsuited to exploring these new biological discoveries via modeling.

Advances in neuroscience

The pace of research in molecular biology, genetics, embryology, brain development, and cognitive neuroscience has beenbreathtaking. Consider:

Earlier theories of genes as static blueprints for body plans have given way to a radically different picture, in which genesmove around, recombine with other genes at different points in development, give rise to products which bind directly to othergenes (and so regulate their expression), and may even promote beneficial mutation (such that the rate of mutation may beincreased under stressful conditions where change is desirable).

Scientists have discovered how to create "designer genes." Human insulin can be produced in vats of bacteria, and caterpillar-resistant tomatoes can be grown. And plants have been created which produce biodegradable plastic!

We now possess a complete and detailed picture of the embryology of at least one relatively complex organism (the nematode,C. Elegans). Scientists know, on a cell-by-cell basis, how the adult worm develops from the fertilized egg.




Page 3

Scientists have carried out ingenious plasticity experiments in which plugs of brain tissue from visual cortex (in late fetalrodents) are transplanted to sensorimotor cortex. This has led to the discovery that the old visual cortex neurons start to act likesensorimotor neurons. In other cases, researchers have shown that if information from the eyes is routed to auditory cortex earlyenough, regions of auditory cortex will set up retinotopic maps, and the organism will start to respond to visual stimuli based onmessages going to the "borrowed" cortex. The conclusion many neuroscientists are coming to is that neocortex is basically an"organ of plasticity." Its subsequent specification and modularization appear to be an outcome of developmenta result, ratherthan a cause.

Although the degree of plasticity observed in the developing brain is surprising, the discovery of plasticity in adult mammalshas come as an even greater surprise for those who believed in fixed and predetermined forms of neural organization. Studieshave shown that somatosensory cortex will reorganize in the adult primate to reflect changes in the body surface (whetherresulting from amputation or from temporary paralysis of a single digit on the hand). At first, this kind of reorganization seemedto be restricted to a very small spatial scale (a few microns at most) which suggested that a more transient local phenomenoncould be responsible for the change. More recent evidence from adult animals that underwent amputation more than a decadeprior to testing shows that this reorganization can extend across several centimeters of cortex. There are only two possibleexplanations for a finding of this kind: New wiring can be manufactured and established in the adult brain, or old patterns ofconnectivity can be converted (i.e., reprogrammed) to serve functions that they never served before.

Sophisticated techniques have been developed for "eavesdropping" on brain activity with extraordinary spatial and temporaldetail. Structural Magnetic Resonance Imaging (MRI), for example, provides enough spatial resolution to reveal a flea dancingon the corpus callosum (assuming there were such a flea). Evoked response potentials (ERP) gives us a temporal localiza-




Page 4

tion of brain processes to within thousandths of a second. Positron emission tomography (PET), magneto-encephalography(MEG), and new functional MRI techniques provide a bridge between the precise spatial resolution of structural MRI and thefine temporal resolution of EEG, showing us which parts of the brain are most active during various cognitive tasks. Takentogether, these techniques provide us with potentially powerful tools both for examining the structure and functioning of theliving brain, and its development over time.

These techniques make available a range of data which were simply not accessible even a decade ago. But although some mightlike to believe that theory follows inevitably from data, in fact it is usually the case that data may be interpreted in more thanone way. What are needed are additional constraints. These come from a second development, which is a computationalframework for understanding neural systems (real or artificial).

Neural computation: the connectionist revolution

Coinciding (but not coincidentally) with the dramatic advances in neuroscience, a second dramatic event has unfolded in therealm of computational modeling. This is the re-emergence of a biologically oriented framework for understanding complexbehavior: Connectionism. The connectionist paradigm has provided vivid illustrations of ways in which global behaviors mayemerge out of systems which operate on the basis of purely local information. A number of simple but powerful learningalgorithms have been developed which allow these networks to learn by example. What can be learned (without beingprespecified) has been surprising, and has demonstrated that a great deal more information and structure is latent in theenvironment than has been realized. Consider:




Page 5

Visual cortex in mammals is well known to include neurons which are selectively sensitive to highly specific visual inputs.These neurons include edge detectors, center-surround cells, and motion detectors. Biologically plausible network models havebeen constructed which demonstrate that such specialized response properties do not have to be prespecified. They emergenaturally and inevitably from cells which are initially uncommitted, simply as a function of a simple learning rule and exposureto stimulation (Linsker, 1986, 1990; Miller, Keller, & Stryker, 1989; Sereno & Sereno, 1991). These artificial networks evendevelop the characteristic zebra-like striped patterns seen in ocular dominance columns in real cortex (Miller, Keller, & Stryker,1989).

When artificial networks are trained to compute the 2-D location of an object, given as inputs the position of the stimulus onthe retina and the position of the eyeballs, the networks not only learn the task but develop internal units whose responseproperties closely resemble those of units recorded from the parietal cortex of macaques while engaged in a similar task (Zipser& Andersen, 1988).

Networks which are trained on tasks such as reading or verb morphology demonstrate, when "lesioned," symptoms andpatterns of recovery which closely resemble the patterns of human aphasics (Farah & McClelland, 1991; Hinton & Shallice,1991; Marchman, 1993; Martin et al., 1994; Plaut & Shallice, 1993; Seidenberg & McClelland, 1989).

The rules of English pronunciation are complex and highly variable, and have been difficult to model with traditional ArtificialIntelligence techniques. But neural networks can be taught to read out loud simply by being exposed to very large amounts ofdata (Sejnowski & Rosenberg, 1987).

In learning a number of tasks, children frequently exhibit various "U-shaped" patterns of behavior; good early performance issucceeded by poorer performance, which eventually again




Page 6

improves. Networks which are trained on similar tasks exhibit the same patterns of behavior (MacWhinney et al., 1989; Plunkett& Marchman, 1991, 1993; Rumelhart & McClelland, 1986).

Children are known to go through phases in which behavior changes slowly and is resistant to new learning. At other points intime children show heightened sensitivity to examples and rapid changes in behavior. Networks exhibit similar "readiness"phenomena (McClelland, 1989).

Networks which are trained to process encrypted text (i.e., the words are not known to the network) will spontaneouslydiscover grammatical categories such as noun, verb, as well as semantic distinctions such as animacy, human vs. animal, edible,and breakable (Elman, 1990). A curious fact lurks here which points to the importance of a developing system: Such networkscan be taught complex grammar, but only if they undergo "maturational" changes in working memory or changes over time inthe input (Elman, 1993).

Our perspective

Taken together, these advancesin developmental and cognitive neuroscience on the one hand, and neural computation on theothermake it possible for us to reconsider a number of basic questions which have challenged developmentalists, from a newand different perspective:

What does it mean for something to be innate? What is the nature of the "knowledge" contained in the genome?

Why does development occur in the first place?

What are the mechanisms which drive change?

What are the shapes of change? What can we infer from the shape of change about the mechanisms of change?




Page 7

Can we talk meaningfully about ''partial knowledge?"

How does the environment affect development, and how do genetic constraints interact with experience?

Our purpose in writing this book is to develop a theoretical framework for exploring the above questions and understanding howand why development occurs. We will cover a number of different specific topics in this book, but there are some centralthemes which recur throughout. We would like to identify these issues explicitly from the outset and foreshadow, briefly, whatwe will have to say about each one.

We begin with a discussion of genes. Although we are primarily concerned with behavior, and behavior is a very long way fromgene expression, genes obviously play a central role in constraining outcomes. When we contemplate the issue of innateness, itis genes that we first think of. This discussion of genes will also help us to set the stage for what will be a recurring themethroughout this book: The developmental process isfrom the most basic level upessentially dependent at all times on interactionswith multiple factors.

From genes to behavior

There is no getting around it: Human embryos are destined to end up as humans, and chimpanzee embryos as chimpanzees.Rearing one of the two in the environment of the other has only minimal effects on cross-species differences. Clearly, theconstraints on developmental outcomes are enormously powerful, and they operate from the moment of conception.Furthermore, although there is a great deal of variability in brain organization between individuals, the assignment of variousfunctions (vision, olfaction, audition, etc.) is not random. There are predictable and consistent localizations across the majorityof individuals.

It is easy to state the obvious conclusion, which is that genes play the central role in determining both interspecies differencesand intraspecies commonalities. This is true, but the real question is how, and what the genes are doing. Most developmentalistsagree




Page 8

that a preformationist version of an answer (that these outcomes are contained in an explicit way in the genome) is unlikely to becorrect (although some version of preformation may come close to capturing the nature of development in certain organisms,e.g., nematodes). There is simply too much plasticity in the development of higher organisms (as we shall discuss in Chapter 5)to ignore the critical effect of experience. We know too that there aren't enough genes to encode the final form directly, and thatgenes don't need to code everything. So how do genes accomplish their task?

How genes do their work

Asked what genes do, most people will report the basic facts known since Mendel (although he did not use the term gene),namely, that genes are the basic units of inheritance and that genes are the critters that determine things like hair color, gender,height. Such a view of genes is not incorrect, but it is woefully incomplete, and lurking beneath this view are a number ofcommonly held myths about genes which are very much at odds with recent findings in molecular genetics.

For instance, according to conventional wisdom, genes are discrete in both their effects and their location. Thus, one mightimagine a gene for eye color which in one form (allele) specifies blue eyes, and in another specifies brown eyes. Genes are alsothought of as being discrete with regard to location. As with the memory of a computer, under this view one should be able topoint to some region of a chromosome and identify the starting point and ending point of a gene (which is itself made up of asequence of base pairs).

In fact, the reality of genes and how they function is far more complex and interesting. Consider the following.

Genes are often physically distributed in space. In eukaryotes (e.g., humans, fruitflies, and corn are eukaryotes), DNA has beenfound to be made up of stretches of base pairs called exons, which code for the production of proteins, but which are interruptedby sequences of noncoding base pairs called introns. In some cases, the quantity of noncoding DNA may be more than 100times greater than the coding DNA. What happens during protein synthesis, which is how




Page 9

most genes actually accomplish their work, is that the RNA copy of the gene-to-bewhich includes intronshas to be cut up andrespliced by specialized molecular machinery (see Figure 1.1). The

FIGURE 1.1 DNA often includes nonfunctional base pairs (introns) aswell as sequences which code for proteins and other products (exons).During synthesis, the RNA transcript (but not the original DNA) is cut

and spliced so that only the exons remain; the revised RNA is thenused for actual synthesis.

result is a temporary "cleaned up" version of the gene transcript which can then be used for protein synthesis.

Moreover, the same portion of DNA can be spliced in different ways. For some purposes, a sequence of base pairs may betreated as an intron (noncoding), but for other purposes, the same region may be spliced to yield a different gene transcript andend up as an exon (coding). Finally, although the structure of DNA base pairs is basically stable, some sequences move around.This movement turns out to play a much more important role in genetic expression than was thought when "jumping genes"were first discovered.




Page 10

Genes are not binary in their effects. What about the view of genes as discrete in their effects? This too turns out to be amisleading idea. To be sure, there are base pair sequences which code directly for specific and well-defined traits. But in manycases the encoding is continuously valued. A subsequence of base pairs may be repeated or there may be multiple copies of thegene; this causes more of the protein product to be produced and may result in a more strongly expressed trait.

Genes do their work with other genes. Sometimes, but rarely, it is possible to tie the effects of a single gene's products to someclearly defined trait. However, such "single action" genes either tend to be associated with evolutionarily primitive mechanismsor they work as switches to turn on and off some other function which is coded by a group of genes.

For example, the fruitfly, Drosophila melanogaster, has a gene called Antennapaedia (Antp). If the Antp gene undergoes acertain mutation, then instead of antennae the fruitfly will develop an extra pair of feet growing out of its head where theantennae would be. Notice that this bizarre effect relies on the fact that what the Antp gene does is to regulate the expression ofother gene complexes which actually produce the feet (or antennae). Even simple traits such as eye color in the fruitfly maydepend on joint action of 13 or more genes. Thus, while there are single-action genes, more typical are cases where multiplegenes are involved in producing any given trait, with some genes playing the role of switches which control and regulate theexpression of other genes.

Genes are often reused for different purposes. A very large number of genes in an animal's genome are what one might call"housekeeping genes." They code for the production of basic proteins which function as enzymes, form cellular organelles, carryout cellular metabolic activities, etc.

But Nature is stingy with her solutions. Things which work in one species frequently turn up in very distantly related species.All together, probably something like 5,000 genes are needed by cells in all eukaryotes for housekeeping purposes. Essentiallythe same genes, modified here and there, are shared by all species and cell types. The lesson here is that there is remarkableconservatism and




Page 11

reusing of old solutions. By rearranging and slightly modifying only a few thousand interacting gene complexes, enormousdiversity of structure is possible.

This conservatism does not rule out the abrupt appearance of what seem to be radically new structures, be they language orflippers or wings. There is a great deal of genetic redundancy in eukaryotes. The same gene may appear many times in thegenome, and often slightly different genes produce similar or identical products. This redundancy accommodates many smallchanges in the genome before there is a dramatic shift in phenotype. Thus the appearance of abrupt changes in phenotypicoutcomes may be misleading, and result from much tinier changes at the genetic level. This brings us to the next point.

The relationship between genome and phenotype is highly nonlinear. Although a linear increase in genome size (measured asthe number of DNA base pairs) which correlates with phenotypic size can be observed for simple species (e.g., worms), thisdoes not hold for so-called higher species (see Table 1.1). In the latter case, the relationship is highly nonlinear. In Chapter 4 wewill discuss nonlinear phenomena in some detail. For the moment suffice it to note that one of the most dramatic nonlinearrelationships in nature is that which exists between the genome and the phenotype.

Compare, for example, the genome of the chimpanzee, the Old World monkey, and the human. To the layman's (admittedlybiased) eye, the Old World monkey and the chimp resemble each other much more closely than either species resembles us. Yetgenetically the chimp and the human are almost indistinguishable: We have 98.4% of our genetic material in common,compared with only approximately 93% shared by the chimp and Old World monkey. Humans are also closer to chimps,genetically, than chimps are to gorillas. Whatever differences there are between us and the chimp therefore come down to theeffects of the 1.6% difference.

In Chapter 7, we will discuss the implications of the above facts for what it might mean for a trait or a behavior to be innate.For the moment, the abovewhich reveals only the most modest glimpse of the complexity which underlies genetic functioningisenough to help us make a simple point. Even the simplest questions, what a



CoverTitle 1996ISBNContentsSeries forewordPrefaceCHAPTER 1CHAPTER 2CHAPTER 3CHAPTER 4CHAPTER 5CHAPTER 6CHAPTER 7ReferencesSubject index

307c17e2e1982db2749a0b97534048b9

Documents

previous page page

page page vthis

england previous page

cip previous page cover

page neural network

connectionism jeffrey

bradford book

development jeffrey