event perception: a mind brain ?· event perception: a mind brain perspective jeffrey m. zacks,...

Download Event Perception: A Mind Brain ?· Event Perception: A Mind Brain Perspective Jeffrey M. Zacks, Nicole…

Post on 22-Jun-2018




0 download

Embed Size (px)


  • Event Perception: A MindBrain Perspective

    Jeffrey M. Zacks, Nicole K. Speer, Khena M. Swallow, Todd S. Braver, and Jeremy R. ReynoldsWashington University in St. Louis

    People perceive and conceive of activity in terms of discrete events. Here the authors propose a theoryaccording to which the perception of boundaries between events arises from ongoing perceptualprocessing and regulates attention and memory. Perceptual systems continuously make predictions aboutwhat will happen next. When transient errors in predictions arise, an event boundary is perceived.According to the theory, the perception of events depends on both sensory cues and knowledge structuresthat represent previously learned information about event parts and inferences about actors goals andplans. Neurological and neurophysiological data suggest that representations of events may be imple-mented by structures in the lateral prefrontal cortex and that perceptual prediction error is calculated andevaluated by a processing pathway, including the anterior cingulate cortex and subcortical neuromodu-latory systems.

    Keywords: events, attention, memory, prefrontal cortex, orienting response

    What Is an Event?

    The world as presented to human sense organs is continuous,dynamic, and fleeting. Yet people seem to perceive events asstable entities, to identify parts of events and their relations to otherparts. For example, one might describe baking cookies by listingthe following parts: Preheating the oven, mixing the ingredientsin a bowl, putting the dough on a cookie sheet . . . This couldreflect mere happenstance or accidents of linguistic structure, buta growing body of research suggests that talk of discrete eventsreflects a deeper psychological reality, that people perceive activ-ity in terms of discrete events, that ongoing processing resourcesare devoted to this perceptual process, and that the online percep-tion of events determines how episodes are encoded in memory(Zacks & Tversky, 2001). Thus, events are key components ofperception, attention, and memory.

    In this article, we present a theory of the perception of everydayevents, review psychological data that have informed the theory,and discuss possible neural substrates of the theorys components.We begin with the formal definition of an event: a segment oftime at a given location that is conceived by an observer to have a

    beginning and an end (Zacks & Tversky, 2001, p. 17). Thisdefinition is useful but surely does not exhaust the commonconception of an event. The everyday notion of an event probablyhas a family-resemblance structure, with some highly typicalmembers such as weddings and breakfasts and some atypicalmembers such as the decay of a radioactive atom or the melting ofa pond. Typical events seem to share some common features. Theyrange from a few seconds (eating a strawberry) to a few hours(going for a hike). They are directed toward a goal; the goal of awedding is to formalize a union, and the goal of breakfast is to sateones hunger. Events involve animate agents, often human. Thesefeatures, however, are neither necessary nor sufficient. Someevents are quite short (cutting a ribbon) or quite long (World WarII). Events that are natural occurrences, such as landslides, maylack both goals and animate agents. So, the taxonomic boundariesof the category event are fuzzy. The spatial and temporal bound-aries of events also can be fuzzyit is sometimes difficult to saywhere or when one event ends and another begins. Neither taxo-nomic fuzziness nor boundary fuzziness is a particular problem forthe psychology of events, and both are exactly analogous to thepsychology of objects. Here, we are concerned with the core of thecategory event: events that involve goal-directed human activityand are of modest duration (seconds to tens of minutes). (Forcomparative reviews of conceptions of events in psychology, seeStranger & Hommel, 1996; Shipley, in press.) In the next section,we present a theory of how human observers segment continuousactivity into discrete events. In subsequent sections, we discussevidence in support of this theory and its implications.

    A Theory of Event Segmentation

    Perception can be described as a roughly hierarchical process inwhich sensory information is successively transformed into repre-sentations that form the basis for action. Particularly important arerepresentations of states of the world in the near future, which maybe called perceptual predictions. Perceptual predictions are valu-able because they allow an organism to anticipate the future and to

    Jeffrey M. Zacks and Todd S. Braver, Department of Psychology andDepartment of Radiology, Washington University in St. Louis; Nicole K.Speer, Khena M. Swallow, and Jeremy R. Reynolds, Department of Psy-chology, Washington University in St. Louis.

    Nicole K. Speer and Jeremy R. Reynolds are now at the Department ofPsychology, University of Colorado.

    The research summarized here was conducted with a number of collab-orators, including Margaret Sheridan, Barbara Tversky, and Jean Vettel. Itwas supported in part by the James S. McDonnell foundation, NationalInstitutes of Health (Grant MH62318-01), and National Science Founda-tion (Grant 0236651). The article benefited from the comments of CoreyMaley and Barbara Tversky.

    Correspondence concerning this article should be addressed to JeffreyM. Zacks, Department of Psychology, Washington University, St. Louis,MO 63130-4899. E-mail: jzacks@artsci.wustl.edu

    Psychological Bulletin Copyright 2007 by the American Psychological Association2007, Vol. 133, No. 2, 273293 0033-2909/07/$12.00 DOI: 10.1037/0033-2909.133.2.273


  • plan appropriate actions rather than merely react to incomingstimuli. Such representations are critical for avoiding interceptionby predators, intercepting prey, and coordinating behavior withothers. To the extent that information processing is hierarchical,perceptual predictions arise late in the processing hierarchy be-cause incoming sensory information is transformed to generatepredictions. In addition to being hierarchical, perception can bedescribed as recurrent: Later processing stages affect the flow ofprocessing in earlier stages. Finally, perception can be described ascyclical: Perceptual predictions are constantly compared with whatactually happens, and these comparisons are used to guide ongoingprocessing. These three notionshierarchy, recurrence, and cy-clicalityare working assumptions in many different theories ofperception (Neisser, 1967), neurophysiology (Carpenter, & Gross-berg, 2003; Fuster, 1991), and language processing (van Dijk &Kintsch, 1983). They have been developed perhaps most fully inrecurrent neural network models, which have been applied to wordlearning (Elman, 1990), to action learning (Jordan & Rumelhart,1992), and to event perception (C. Hanson & Hanson, 1996).

    The theory presented here, which we call event segmentationtheory (EST), shares these three properties. In this section, wedescribe the theory in information-processing terms. Later in thearticle, we recast the theory in terms of the neural systems that mayimplement these information-processing components.

    Architecture and Principles

    EST proposes that event segmentation arises from the percep-tual processing stream depicted in Figure 1. Its core is a pathwaywhose input is a set of sensory representations and whose output isa set of perceptual predictions. The sensory inputs correspond tothe information conveyed by the peripheral nervous system to thecortex. In the visual modality, for example, this corresponds tobasic information about brightness, color, and possibly some pre-liminary edge extraction. Sensory inputs are transformed by per-

    ceptual processing to produce multimodal representations withrich semantic content, encoding information such as object identityand location, motion trajectories, and the identities and attitudes ofother people. According to the theory, processing is oriented intime such that it results in predictions about the future state ofperceptual representations. For example, extracting a motion con-tour leads to predictions about the future locations of objects, andinferring the goals of a person leads to predictions about his or herfuture movements.

    We propose that perceptual processing is guided by a set ofrepresentations called event models that bias processing in theperceptual stream. An event model is a representation of what ishappening now, which is robust to transient variability in thesensory input. The stability of event models over time is a sourceof perceptual constancy; an ongoing event is a single entity despitepotential disruptions in sensory input such as occlusion or distrac-tion. In this regard, event models are similar to the object filesproposed to mediate object constancy in visual perception (Kah-neman, Treisman, & Gibbs, 1992) or the short-term action repre-sentations proposed to mediate perceptual constancy in biologicalmotion (Stranger & Hommel, 1996, see section 5). However, eventmodels are hypothesized to be active over much longer timeframes than object files or biological motion representations. Interms of the current theory, object files and biological motionrepresentations are hypothesized to be part of high-level perceptualprocessing components.

    Event models are working memory representations, which areimplemented by transient changes in neural activation rather thanlong-term changes in synaptic weights. They are not necessarilyaccessible to consciousness, though people may have partialawareness of their contents under some circumstances. Eventmodels are multimodal, integrating information from visual, audi-tory, and the other sensory modalities. In these regards they areakin to the representations recently proposed by Baddeley (2000)