1 psychocinematics issues and directions

38
Page 1 of 38 Psychocinematics: Issues and Directions PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013. All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a monograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: Vrije Universiteit Amsterdam library; date: 26 July 2013 Psychocinematics: Exploring Cognition at the Movies Arthur P. Shimamura Print publication date: 2013 Print ISBN-13: 9780199862139 Published to Oxford Scholarship Online: May-13 DOI: 10.1093/acprof:oso/9780199862139.001.0001 Psychocinematics: Issues and Directions Arthur P. Shimamura DOI: 10.1093/acprof:oso/9780199862139.003.0001 Abstract and Keywords Movies offer a unique yet accessible way of understanding human psychology. We often envelop ourselves within a movie, generating the same sensations, thoughts, and feelings as the characters portrayed on the screen. Proposed here are issues and directions for psychocinematics, the scientific analysis of our movie experience. Storytelling, acting, editing, and cinematography are ways in which movies impinge on our psyche. How is our attention and imagination captured by movies? How do they instill emotions, such as suspense, fear, happiness, horror, and surprise? In what ways do personal and cultural knowledge influence our experience? Why are movies such a popular infatuation? These are some of the questions that the field of psychocinematics can address. Keywords: psychocinematics, film studies, psychology, cognitive science, neuroscience, attention, sensation, memory, narrative, emotion, brain IN THE OPENING scene of The Graduate, Benjamin Braddock (Dustin Hoffman) exits a plane and enters the terminal at the LA International Airport. When he steps onto a moving walkway, the camera begins to track alongside at the same pace, which keeps Benjamin’s position fixed and isolated to the far right of an otherwise empty screen (Figure 1.1). The opening credits begin to fill the space as we listen to The Sounds of Silence, Simon and Garfunkel’s anthem to social alienation. Over the soundtrack, a voice from an airport loudspeaker requests: “Please hold onto the handrail and stand to the right. If you wish to pass, please do so on the left.” This beginning sequence anticipates the entire movie, which tells a story of a

Upload: bryan-o

Post on 30-Nov-2015

83 views

Category:

Documents


1 download

DESCRIPTION

1 Psychocinematics Issues and Directions

TRANSCRIPT

Page 1: 1 Psychocinematics Issues and Directions

Page 1 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

Psychocinematics: Exploring Cognition at the MoviesArthur P. Shimamura

Print publication date: 2013Print ISBN-13: 9780199862139Published to Oxford Scholarship Online: May-13DOI: 10.1093/acprof:oso/9780199862139.001.0001

Psychocinematics: Issues and Directions

Arthur P. Shimamura

DOI: 10.1093/acprof:oso/9780199862139.003.0001

Abstract and Keywords

Movies offer a unique yet accessible way of understanding humanpsychology. We often envelop ourselves within a movie, generating thesame sensations, thoughts, and feelings as the characters portrayed on thescreen. Proposed here are issues and directions for psychocinematics, thescientific analysis of our movie experience. Storytelling, acting, editing, andcinematography are ways in which movies impinge on our psyche. How is ourattention and imagination captured by movies? How do they instill emotions,such as suspense, fear, happiness, horror, and surprise? In what ways dopersonal and cultural knowledge influence our experience? Why are moviessuch a popular infatuation? These are some of the questions that the field ofpsychocinematics can address.

Keywords:   psychocinematics, film studies, psychology, cognitive science, neuroscience,attention, sensation, memory, narrative, emotion, brain

IN THE OPENING scene of The Graduate, Benjamin Braddock (DustinHoffman) exits a plane and enters the terminal at the LA InternationalAirport. When he steps onto a moving walkway, the camera begins to trackalongside at the same pace, which keeps Benjamin’s position fixed andisolated to the far right of an otherwise empty screen (Figure 1.1). Theopening credits begin to fill the space as we listen to The Sounds of Silence,Simon and Garfunkel’s anthem to social alienation. Over the soundtrack, avoice from an airport loudspeaker requests: “Please hold onto the handrailand stand to the right. If you wish to pass, please do so on the left.” Thisbeginning sequence anticipates the entire movie, which tells a story of a

Page 2: 1 Psychocinematics Issues and Directions

Page 2 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

newly minted college graduate entering adulthood without any sense ofpurpose or direction. Benjamin is moving though he doesn’t appear to begoing anywhere.

Filmmakers have developed techniques, largely through trial and error,that drive our sensations, thoughts, and feelings. Through acting, staging,sound, camera movements, and editing, movies develop a storyline, orwhat scholars call a narrative, that fully engages us. How do filmmakersheighten our sensations and direct our attention so that the drama keeps usriveted to the screen? How do techniques link events in an often seamlessmanner? How do movies drive our emotions, instilling suspense, laughter,horror, sadness, and surprise along the way? Philosophers, film theorists,psychologists, and recently brain scientists have considered our movieexperience, as, of course, have filmmakers themselves. Why are movies socompelling?

Even at its early beginnings, scholars have considered the nature of ourmovie experience and its impact on society and culture (see Lindsay, 1915;Munsterberg, 1916). Over nearly a century, volumes of scholarly pageshave addressed essential features of our movie experience, such as the waymovies present a “realistic” view of the world or the way we connect withthe characters portrayed. Commonplace now are university departments ofFilm Studies or Media that offer worthy intellectual analyses of such issues.Some film theorists have advocated a psychological or cognitive approach tomovies (Bordwell, 1989; Carroll, 2008; Currie, 1995) as they are particularlyinterested in what goes on in the mind (and brain) of (p.2)

Figure 1.1 Opening scene of The Graduate (1967, Embassy Pictures).

the moviegoer. Until recently, however, only a few scientists have consideredour movie experience as a topic of empirical investigation. The contributorsof this volume—which include film theorists, philosophers, psychologists, andneuroscientists—explore the viability of a scientific approach to our movieexperience or what I call psychocinematics.

Page 3: 1 Psychocinematics Issues and Directions

Page 3 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

What Is Psychocinematics?

As the term implies, psychocinematics seeks to understand the psychologicalunderpinnings of this unique and popular form of entertainment. Whatsets this venture apart from other ways of understanding movies is itsreliance on empirical research. That is, from experimental investigationsthe viewer’s experience is analyzed through objective, systematic, andreplicable measurements. Such investigations cannot address all aspectsof “film studies,” particularly those that pertain to societal ramifications.Yet we experience movies, and thus it is vitally important to understand asprecisely as possible how we engage ourselves mentally as we watch them.With the advent of brain imaging techniques, particularly functional magneticresonance imaging (fMRI), psychological science can now link mental eventswith brain processes. Indeed, it is now possible to have individuals watch amovie in an fMRI scanner and record the brain regions that are active duringthe experience. In this way, psychocinematics can connect minds, brains,and experiences as we watch movies.

Psychocinematics is grounded on a scientific analysis of our aestheticresponse to movies. What exactly is aesthetics has been bandied aboutfor centuries, primarily by philosophers, though recently scientists havetaken part in this discussion (see Shimamura & Palmer, 2012). In essence,aesthetics can be viewed as a hedonic response, which is simply apreference or liking judgment. After a movie, as the closing credits scroll,we often elicit a hedonic response, such as “I liked it” or “That was terrible.”This judgment can be graded from very positive to very negative or fromvery interesting to very dull. We might even be able to articulate the basis onwhich this aesthetic response was made, such as having a satisfying endingor not being able to identify with the protagonist. Our aesthetic responsemay be based on the visuals, the action, or how a movie makes us think orfeel. A primary goal of psychocinematics is to offer a scientific approach tothe aesthetics of movies (see Table 1.1). (p.3)Table 1.1 What Is Psychocinematics?

• Considers the psychological and biological underpinnings of our movieexperience

• Addresses the aesthetic nature of movies from an empirical approach

• Develops movies for psychological research as a means of understandingcognitive processes as they unfold dynamically in time

Page 4: 1 Psychocinematics Issues and Directions

Page 4 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

• Encourages a multidisciplinary approach that includes psychology, cognitivescience, neuroscience, philosophy, film studies, filmmaking, art history, sociology,and other related endeavors

Psychocinematics could have been established very early on had thescholarly world been ready for Hugo Munsterberg’s The Photoplay: APsychological Study, which was originally published in 1916. Munsterbergwas an established leader in applied psychology, having published influentialpapers on the role of psychology in law, education, and business. In ThePhotoplay, he considered a cognitive approach to movies by suggestingthat the art of film is in depicting our mental states rather than reproducingreality. He wrote:

… the photoplay tells us the human story by overcomingthe forms of the outer world, namely, space, time, andcausality, and by adjusting the events to the forms of the innerworld, namely attention, memory, imagination, and emotion.(Munsterberg, 1916, p. 173, italics his)

For Munsterberg, movies work because they model or simulate mentalstates: A close-up shot is akin to focusing our attention; a flashbackmimics remembering; soft focus connotes a feeling. From this view,Munsterberg argued for a psychological analysis of how movies impingeon our perception, imagination, and emotion. He thus formulated theconceptual basis for a research program in psychocinematics. His ideas,however, were largely ignored for over 50 years. This negligence was dueto several factors. First, Munsterberg died less than a year after the bookwas published. Second, mainstream American psychology at that time wasin the throes of behaviorism such that any consideration of mental stateswas viewed as unscientific. Third, he was an outspoken German sympathizerduring World War I, which antagonized many Americans. When his book wasrepublished in 1970, the time was more conducive for his prescient views tobe appreciated by those interested in film theory (see Munsterberg, 2001).

With current advances in cognitive and brain sciences, the time is ripe forworthy empirical investigations of our movie experience. Digital technologyhas made it easier to control and manipulate movie stimuli. When usedas psychological stimuli, movies can help us better understand how weperceive, interpret, and respond to events. Indeed, movies offer a morenatural way of presenting events, especially compared to typical stimuliused in psychological studies (e.g., pictures or words). When events are

Page 5: 1 Psychocinematics Issues and Directions

Page 5 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

presented as movies, the psychological scientist can study mental processesdynamically as they unfold in time.

It is acknowledged that a full understanding of our movie experience requirescollaboration among scientists and nonscientists. Philosophers and filmtheorists have thought deeply (p.4) about our experience with movies,as, of course, have filmmakers themselves. A thorough understanding ofthe psychological underpinnings of our movie experience will depend onworthy interactions among scientists, philosophers, filmmakers, historians,sociologists, and others who can help motivate a psychocinematic approach.

Making Movies

What exactly is a movie? Why do people make movies? These are the kindsof questions that appear simple at first but upon further consideration canbe quite controversial. As our intent is to address the viewer’s experience,we can sidestep these issues and simply refer to our everyday usage of“movies” as moving pictures that many of us watch for enjoyment, interest,and entertainment. We will thus err on the side of inclusion by consideringthe variety of moving images we experience daily, such as those viewed intheaters (e.g., Hollywood features), on television (e.g., sit-coms), and on theInternet (e.g., YouTube videos). Most film scholars make distinctions betweenterms such as movies, film, cinema, video, and motion pictures. For the sakeof simplicity, these terms will often be used interchangeably in this volumeor will be defined specifically within chapters (for deeper conceptualizations,see Carroll, 2008; Mast, 1984; Perkins, 1974).

As we have dispensed with the philosophical issue of what exactly definesa movie, we will accept the fact that not all movies share all of the featuresthat we will consider (indeed, there are movies that don’t move, such asChris Marker’s La Jetée, which is composed almost entirely of a successionof still photographs). Also, within the realm of avant-garde or experimentalcinema, some movies don’t profess to tell a story and instead may offer acommentary on the art form itself. We will acknowledge these exceptions,not necessarily excluding them in our analyses, but we consider the coreinterest of psychocinematics to be the understanding of our most commonmovie experience, which is the viewing of fictional narratives as portrayedby moving images (aka “movies”). Such viewings are, of course, bestrepresented by commercial feature films, and we will consider them asprototypical examples as we encounter the ideas offered in this volume.

Page 6: 1 Psychocinematics Issues and Directions

Page 6 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

Setting the Stage

As unique as movies are as an art form or means of entertainment, theyevolved from earlier traditions, most notably storytelling, theater, andphotography. We may not be fully aware of these influences as we watcha movie, yet our analysis of psychocinematics would be significantlydiminished if we ignored these roots. Storytelling is its earliest influence, asthe central intent of most movies is the telling of a story (for deeper analysis,see Currie, 2012; Robinson, 2005). Certainly with respect to Hollywoodmovies, it is the story and how that story is told that forms the backbone ofour movie experience. We relish a good plot and when someone tells a storywe use our imagination to conjure up our own private movie of the narrative.

Homer’s ancient tales of the Iliad and Odyssey—epic stories of warfareand ships at sea—were the Star Wars of their time. Such narratives,characterized as myths, fables, and legends, convey a culture’s knowledge,morality, and politics by way of a rich oral tradition. Since the advent of (p.5) printed material, we have the means of reading stories to ourselves.What makes a good story? Aristotle in The Poetics suggested that a well-formed plot should be encapsulated as a complete tale with a beginning,middle, and end. Characters must be developed through recognition orrealization (e.g., Luke Skywalker realizing that Darth Vader is his father).There should be surprises and reversals of fortune along the way. Movies, ofcourse, are grounded in storytelling as they follow Aristotle’s guidelines forplot formation and structure. Also, many of the familiar movie genres comefrom literature, such as epic, romance, comedy, Western, mystery, sciencefiction, and horror.

Theater offered the second artistic route to movies. In Greek dramasperformed in Athens 2,500 years ago, in Shakespearean plays performed inLondon 400 years ago, and today as performed in countless cities, peoplehave enjoyed stories as presented by actors performing on stage. Whereasstorytelling relies on the listener (or reader) to imagine the actions of thecharacters portrayed, theater actually presents the actions. Also, theaterprovided some of the essential ingredients for making a movie, including ascript, set direction, actors, and stage design. Indeed, when movies were firstshown, a common term for this new art form was photoplay, suggesting thatmovies were viewed as plays recorded on film. Movies are, of course, muchmore than that as the ability to move the camera and change the sequenceof shots through editing has made movies a unique art form (Dymtryk, 1986;

Page 7: 1 Psychocinematics Issues and Directions

Page 7 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

Murch, 2001). Yet it is easy to see why the term was applied as people go toboth to watch a drama unfold through the performance of actors.

It is the ability of actors to transcend us outside of our current environmentthat makes for good theater (see Goldstein & Bloom, 2011). The illusion ofnaturalness in movement and dialogue and the ability to express both subtleand grand emotions are hallmarks of a fine performance. Many film actorscome with stage training, though, of course, there are differences betweenthe way one performs in front of an audience and the way one performsin front of a camera. Yet the ability to convey lines with a natural sincerityand to move without appearing overly histrionic is an essential feature thatenhances both a staged performance and one on film. Thus, many of therules and techniques of theater acting are relevant to movie acting.

The visual makeup of a staged play, or what scholars often call the mise-en-scène (placing on stage), has been co-opted and applied to movie sets.A movie’s visual aesthetics is based on lighting, object placement, scenery,and the actors’ movements, all of which have been influenced by theatricalperformances. Also, the duties of the film director and set designer havetheir roots in theater, though again there are differences in what they do inthe theater compared to a movie production. Nevertheless, the visual style ofa movie, what even film scholars call the mise-en-scène, has its foundation intheater. Of course, with movies, particularly in the digital age of computed-generated imagery (CGI), there is immense freedom in how the mise-en-scène is arranged.

Without the advent of photography, the third essential route to movies, itis unclear how moving pictures would even have been invented. Moviesare simply still images presented rapidly in sequence, and prior to thedigital revolution, the primary method of creating such images was throughchemical-based photography. Thus, all of the advantages and limitations ofphotography are inherent in the creation and aesthetic appeal of movies.Creative photography offered a basis for frame composition and dramaticlighting. Unlike theater, the (p.6) photographic image could depict a close-up or present an unusual point of view, such as from high above or downbelow. The filmmaker could add stark contrasts of light and dark as was usedin German expressionist films, such as F. W. Murnau’s Nosferatu (1922), andlater in American film noir. Early on, limitations in the process of photographyhampered filmmaking. Movie cameras were bulky and difficult to move.Film emulsions were limited in their sensitivity, and bright lights were oftenrequired to obtain good images. Care had to be taken to ensure contrasts

Page 8: 1 Psychocinematics Issues and Directions

Page 8 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

in light and dark as different color frequencies seen by the eye can appearequally gray on black-and-white film. Thus, the aesthetic appeal of a shotdepended largely on the application of photographic techniques to enhancethe creative vision of the director and cinematographer.

Eadweard Muybridge, sometimes considered the “father of motionpictures” (Hendricks, 1975), photographed animals in motion and inventedan early motion picture projector (see Shimamura, 2002; Solnit, 2004).Muybridge was commissioned by Leland Stanford, the railroad magnateand former governor of California, to demonstrate that a trotting horse hadall four legs off the ground at some point in its gait. He placed camerasalong a racetrack and strung threads across the track, attaching each oneto a shutter so that when the horse trotted along the track it pulled thethreads, snapping pictures in succession. Muybridge proved that a trottinghorse did indeed have all four legs off the ground at one moment in its gait.Moreover, when these images were shown quickly in succession, the animalsappeared to move, thus paving the way for the creation of motion pictures(see Rossell, 1998).

Moving Images

The commercial breakthrough in movies as a form of entertainmentcame with Thomas Edison’s Kinetoscope and the Lumière brothers’Cinématograph. These inventions created a stir during the mid-1890s aspeople flocked to watch moving images. The Lumière brothers had their firstpublic showing in Paris on December 28, 1895, at which 33 people paid 1franc to watch 10 short clips, including their first film, La Sortie de l’UsineLumièreàLyon, which simply showed workers exiting their factory in Lyon(this and other early silent films can be viewed at http://www.YouTube.com).Each movie was hand-cranked by an operator and lasted no more than 50seconds. One clip presented a fictional narrative, L’Arroseur arrosé (TheSprinkler Sprinkled), in which a boy sneaks behind a man watering with agarden hose and steps on the hose to prevent the flow. When the man looksat the hose to see what’s wrong, the boy releases his foot and the man’sface is sprayed…and movie comedy is created.

Moving images are created by the rapid succession of still photographs(or frames). The on-and-off (i.e., flicker) rate must be fast enough so thatwe do not perceive any change in illumination between frames. With theCinématographe, the flicker rate was rather slow, less than 16 framesper second, such that a noticeable flashing or flickering was perceived

Page 9: 1 Psychocinematics Issues and Directions

Page 9 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

(hence the term flicks to refer to these early movies). As movie technologyadvanced, the perception of flicker was abolished with the use of apropellerlike shutter that spun rapidly in front of the lens of a movie projectorso that a movie running at 24 frames per second would actually presenteach frame two or three times, thus increasing the flicker rate to 48 or 72 Hz(cycles per second). Even at these speeds, if we could accurately measurethe light projected onto the screen, there would still be a flicker. We do notperceive the flicker because there is a lingering trace of the visual imagethat is maintained briefly by sensory neurons (a visual echo, if you (p.7)will). This phenomenon is easy to demonstrate with a flashlight or laserpointer. If you turn the light on you’ll see a point, but when you move thelight around very quickly you’ll see a trailing line. That trail of light is thelingering trace, which, of course, is all in your head. Digital displays rely onthe same perceptual processes, though they are presented differently (seeChapter 6).

It is, however, not the phenomenon of visual persistence that gives us theperception of moving images, though many have believed it to be so (fora discussion of this issue, see Anderson & Anderson, 1993). The “illusion”of seeing movement from the rapid succession of still images, or whatpsychologists call apparent motion, is based on a not completely understoodset of processes that was studied by Max Wertheimer, the German Gestaltpsychologist (Wertheimer, 1912). Wertheimer conducted experiments inwhich a vertical line is followed by a horizontal line at various lag timesbetween presentations. If the lag time (interstimulus interval) was very fast(less than 30 ms), then the viewer perceived two simultaneously presentedlines that appeared to form a right angle. In this case, visual persistence ofthe first line overlapped with the presentation of the second line so that thetwo were perceived as being presented simultaneously. If the interstimulusinterval was very long (greater than 200 ms), then the viewer simply saw twolines being presented sequentially. However, between these two intervals,there was a range of lags when the viewer actually perceived the verticalline appear to move and swing down to a horizontal position. The perceptualmechanisms underlying this apparent motion are multifaceted and today stillstudied by psychologists and neuroscientists (see Deco & Roland, 2010).

In making a movie, we start with the shot, the sequence of frames takencontinuously from a camera. Very early movies, such as those shown by theLumière brothers, were single shots taken by a stationary camera. Thesedays, shots can include camera movements, such as tracking (moving theentire camera with the action), panning (rotating the camera head to the left

Page 10: 1 Psychocinematics Issues and Directions

Page 10 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

or right), or tilting (rotating the camera head up or down), as well as variablefocus shots that zoom in or out of a scene. The history of movies is markedby creative applications of camera placement and movement intended tocarry the narrative along in dramatic ways (for a fuller presentation of theseissues, see Bordwell, Staiger, & Thompson, 1985; Cook, 2004).

Editing involves the joining of sequential shots, with each individual jointcalled an edit. The most basic edit is the direct joining of two shots, oftenreferred to as a direct cut or simply a cut. Other shot transitions includethe dissolve, in which the ending of the first shot fades out while the nextone fades in as an overlapping double exposure. This transition is used toindicate that some time has elapsed between shots. The fade to black is afuller dissolve such that for a perceptible moment the screen is completelyblack before the next shot fades in. This transition is used to suggest that aneven greater amount of time has elapsed compared to a dissolve. The wipeis a classic silent era transition in which a noticeable line moves across theshot and acts as a border between the shots, as if the next scene is wipingover the previous one. Dissolves and fades are less used in contemporarymovies, and wipes are almost never used, except to make stylistic referenceto an older way of editing (George Lucas used wipe edits in Star Wars:Episode IV—A New Hope as a way of making reference to earlier sciencefiction movies).

Other editing techniques include cross-cutting, eyeline matches, match-action edits, and shot/reverse shots. In cross-cutting or parallel editing, shotsmove between two different events to show that they are occurring at thesame time. Cross-cutting, especially when (p.8) scenes alternate severaltimes, enhances the dramatic feel of the narrative as it moves along. Eyelinematches are shot sequences in which a character glances somewhere off-screen, and the next shot shows the object of fixation. In a match-actionedit, a shot transition occurs while a character is in midmovement, suchas a shot that begins with a person reaching for the door and cutting toa close-up of a hand grabbing the doorknob. Shot/reverse shots are verycommon during dialogues. It often involves successive “over-the-shoulder”shots. For example, when filming a couple engaged in a conversation, asequence might first show the man talking with the camera positioned overthe shoulder of the woman. When the woman responds, a cut moves to areverse angle shot that shows the woman viewed over the shoulder of theman. All of these editing techniques drive the narrative and help createseamless edits.

Page 11: 1 Psychocinematics Issues and Directions

Page 11 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

In 1927, sound made its way into movie production with The Jazz Singer, amostly silent movie but one that included synchronized dialogue and singing.Sounds are classified in two ways. Diegetic sounds refer to ones that arepart of the scene, such as conversations, footsteps, and clinking glasses.Nondiegetic sounds are not part of the scene, such as a musical soundtrackor the voice of a narrator. The application of both diegetic and nondiegeticsounds offered a rich, new arsenal of expressive tools for the filmmaker.Indeed, the way we integrate visual and auditory sensations as we watchmovies is what makes them a particularly forceful means of presenting astory.

Continuity Editing

From the creativity of early filmmakers, Hollywood studios codified a set ofrules for narrative structure that first depended on scriptwriters conformingto Aristotle’s story arc—that is, delineating a beginning, middle, and end, andoffering surprises and reversals of fortune along the way. The studios thenrelied on directors, cinematographers, and editors to create an audiovisualexperience that presented the narrative without confusing the moviegoer.This “Hollywood style” guaranteed a smooth temporal flow of events witha set of “continuity” rules that made edits appear as seamless, virtuallyinvisible shot transitions. These rules began with the application of standardediting practices, such as cross-cutting, eyeline matches, match-action edits,and shot/reverse shots.

Consider Alfred Hitchcock’s Rope (1948), which runs as if it were filmedas a single shot.1 As such, the duration of time enacted is identical to thetime spent watching. Also, for such a movie the spatial setting is limited asto where a camera could be moved during that time interval. Editing withcuts gives filmmakers a way of introducing lapses in time and changes inspatial setting that flow smoothly. Interestingly, cutting also increases theperceived activity, speed, and enjoyment of a filmed sequence (Kraft, 1986).To create shot transitions with a smooth flow, it is necessary to set up aspatiotemporal context for the viewer. For example, a scene often beginswith an establishing shot, which is a long (wide-angle) shot that sets the“stage” or spatial context of a scene. One rule of continuity editing is toinclude an establishing shot every time the narrative shifts to a differentspatial setting.

(p.9) Once a spatial reference frame has been established, that perspectivemust be maintained as shots move to closer views. Consider an establishing

Page 12: 1 Psychocinematics Issues and Directions

Page 12 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

shot that shows a family seated around a dining room table. Imagine thefather on the left about to carve a roast, and the mother on the right seatedand passing a bowl of mashed potatoes to her daughter on her right who isfacing us. The establishing shot sets a visual perspective that defines theposition from which we voyeuristically view the event. We maintain thisspatial reference across shots as if creating an imaginary stage that definesthe right–left axis of action. Filmmakers follow a 180-degree rule, whichstates that after an establishing shot sets the axis of action, a camera canonly be positioned in front of that line. A camera shot that violates the 180-degree rule disrupts the moviegoer’s sense of space. For example, imaginea set of close-up shots between the father and mother as they converse witheach other. If we follow the 180-degree rule, a close-up shot of the fatherwould have his profile facing the right, as if looking at the mother. When themother speaks, we might be presented with a close-up shot of her facingto the left. These views maintain our spatial representation as defined bythe establishing shot (e.g., the father is on the left and the mother is onthe right). If the camera were to be positioned behind the daughter, thuscrossing the axis of action and violating the 180-degree rule, a close-up shotof the mother from that angle would show her facing to the right. Given ourframe of reference, it would appear as if she is talking to someone behindher and facing away from the father. This change in perspective would betantamount to suddenly viewing a theatrical play from backstage, as thingsthat were on the left would now be on the right.

From trial and error, filmmakers have learned to create a natural feel ofmovement across edits. Another continuity rule is the 30-degree rule, whichstates that successive shots should always be angled more than 30 degreesapart. This seemingly odd rule is made apparent as soon as one sees aviolation of it. Consider the worst-case scenario in which there is no changein the camera angle: For example, after taking a medium shot of our father,we turn off the camera, move directly toward him, and shoot from thiscloser position. If we were to edit these two shots together, the father wouldsuddenly appear to have expanded or to have suddenly jumped towardthe camera. That is, what we would see is apparent motion because theonly change in the image across the edit would be the scale of objects (i.e.,everything just gets bigger). Our brain interprets this change, which is calledoptical expansion, as a very quick movement toward us or a sudden changein the size of objects, as either of these two possibilities would be the case ifthis sequence occurred in real life. These jumps in movement tend to occurif successive camera shots are angled less than 30 degrees apart from eachother. Violations of this rule are called jump cuts, which are noticeable as

Page 13: 1 Psychocinematics Issues and Directions

Page 13 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

odd jerky movements or shifts of objects in a scene. These days, jump cutsare actually used as a stylistic effect often to express erratic or energeticaction.

Continuity editing rules help to maintain a smooth narrative flow. Indeed,the mark of an outstanding film editor is someone whose technique is so finethat edits are not even noticed. (How many Academy Award–winning editorsdo you know?) Of course, all rules are meant to be broken, particularlywith respect to art and aesthetics, and during the 1960s, a style of filmingemerged in which it was acceptable, indeed trendy, to make viewers awareof camera movements and editing. Some of these techniques, such asjump cuts and shaky cameras, were originated by avant-garde filmmakers,such as Jean-Luc Goddard and Francois Truffaut, and then made their wayinto Hollywood features. Others, such as a more varied use of cameramovements, angles, and zooms, came from the style of innovative featuredirectors, such as Alfred Hitchcock, Stanley Kubrick, and Martin Scorsese.

(p.10) The biggest trend in movie making since the 1960s has been theincredible increase in the number of edits in a movie (see Chapter 7). Priorto 1960, a Hollywood movie would include 300 to 700 shots, which for a110-minute movie would amount to an average shot length (ASL) of 22to 9 seconds (see Bordwell, 2002). During the 1960s, many movies wereproduced with an ASL of 4 or 5 seconds, which would put the number ofshots to over 1,000. These days an action movie may include over 2,000shots. In the James Bond movie Casino Royale (2006), an edit occurred onaverage every 2.2 seconds! Some have suggested that the introduction ofMTV music videos (not to mention Sesame Street) have created a generationaccustomed to rapid editing. Whatever the sociological forces of this culturalchange, the increase in the number of edits in a movie over the decades iswell documented.

David Bordwell (2002, 2006) has described Hollywood movies since the1960s as adopting intensified continuity editing, an apt term as basiccontinuity rules are still applied, only now camera movements and editinghave been amplified. Bordwell identified four properties of intensifiedcontinuity: (1) rapid editing, (2) extremes of lens lengths (i.e., wider wideangles, more telescopic telephotos), (3) closer framing during dialogues, and(4) free-range camera movements (e.g., weaving quickly through a crowd).With such changes, both actors and cinematographers have had to adjust.For example, with respect to closer shots during dialogues, Bordwell states:“Mouths, brows, and eyes become the principal sources of information and

Page 14: 1 Psychocinematics Issues and Directions

Page 14 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

emotion, and actors must scale their performances across varying degreesof intimate framings” (Bordwell, 2002, p. 20). Intensified continuity hasinfluenced the aesthetics of experiencing movies. In my recent freshmanseminar on the psychology of movies, some students felt that movies fromthe 1940s and 1950s were paced too slowly for their liking.

Empirical Foundations

Through careful experiments, science offers a way of understanding theregularities of our world, including the inner world of our brains. Thereare great theorists, such as Albert Einstein, who never conducted a singleexperiment, yet they conceptualized the world in new ways that couldbe tested and evaluated by others. Some theorists, such as SigmundFreud, have influenced many, though it is questionable to consider Freud’sframework as a scientific theory as it is difficult if not impossible to testempirically. Psychocinematics is grounded on experimental investigationsthat address the way mental processes are engaged during our movieexperience. To identify and characterize such processes, we must turn topsychological and biological science and consider the way the mind/braininterprets sensory information.

Psychological Science

In the 1860s, scientists such as Gustav Fechner and Hermann von Helmholtzdeveloped an empirical approach to perception and with it initiatedpsychological science. Fechner called this new science psychophysics, thuscharacterizing the way the mind interprets physical stimuli (Fechner, 1860).He approached the study of visual perception much like the way an engineermight try to understand the optics of a camera. For example, he studiedthe smallest amount of light that one could detect or the ability to perceivedifferent colors. Helmholtz (p.11) studied both vision and audition and, likeFechner, conducted behavioral experiments to determine the limits andquality of sensory processes. He developed important theories concerningthe way we perceive color, spatial depth, and motion.

Interestingly, both of these early psychological scientists had interests inaesthetics. In 1876, Fechner published Vorschule der Aesthetik (Primerof Aesthetics), in which he used his psychophysical approach to studyhedonic judgments. He showed individuals various colors or shapes, suchas rectangles of varying proportions, and asked them which ones theypreferred. From these hedonic judgments, Fechner believed he could

Page 15: 1 Psychocinematics Issues and Directions

Page 15 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

build an understanding of our aesthetic appreciation of art. Helmholtz alsoconsidered the way we appreciate paintings (Helmholtz, 1881), as well as theway music is enjoyed through tone combinations (Meulders, 2010).

Psychophysics offered a scientific method for studying perception bybreaking it down into elemental features, such as color, shape, and motion.Others, such as Hermann Ebbinghaus (1884/1964) and Wilhelm Wundt(1874/1904), furthered this approach by considering the elements ofhuman memory and emotion. One problem with these approaches wasthat the mind is not a physical entity like air or water. As a result, thesepsychologists made inferences about mental processes by studying behavior.In a highly influential paper entitled Psychology as the Behaviorist Views It,John Watson (1913) rejected the subjective notion of a mind: “Psychologyas the behaviorist views it is a purely objective experimental branchof natural science. Its theoretical goal is the prediction and control ofbehavior” (Watson, 1913, p. 158). He went on to state that psychologicalscience must proceed without ever using such terms as consciousness,mental states, or mind. Behaviorism was grounded in an analysis of theelements of behavior rather than the elements of the mind. This approachbecame the mainstay of American psychological science for the next 50years.

It was during the excitement of Watson’s behaviorism when Munsterbergwrote The Photoplay: A Psychological Study and encouraged a mindful,cognitive approach to the way we experience movies. No wonder his viewswere not well considered. Such terms as attention, imagination, and emotionwere precisely the kind that Watson wanted to expunge from his brandof psychological science. Yet as a graduate student trained under Wundt,Munsterberg proceeded to describe how mental states are driven by movies.

Gestalt psychology was somewhat more successful as an alternative tobehaviorism. Max Wertheimer, along with his two assistants, Kurt Koffkaand Wolfgang Köhler, established this wholistic approach to perceptionby suggesting that a visual scene is perceived in its entirety and cannotbe broken down into smaller elements. They argued that the perceptualworld is ambiguous or illusory, and thus it is the way viewers organize orinterpret a visual scene that determines how it is perceived. Rudolf Arnheim,who studied under Wertheimer, is known for his use of Gestalt principlesto address the psychology of art. His wrote influential books on visualaesthetics (Arnheim, 1974, 1983), which described paintings with respect to

Page 16: 1 Psychocinematics Issues and Directions

Page 16 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

the “perceptual forces” that artists impart through balance, harmony, andobject placement.

Even before his analyses of the visual arts, Arnheim wrote Film als Kunst(Film as Art) (Arnheim, 2006, originally published in 1933), in which heattempted to justify film as an art form because it alters our views of theworld and thus fails to render an exact copy of the real world. Similar toMunsterberg’s view, Arnheim considered dissolves, cross-cuts, and point-of-view editing as ways of creating an illusion of space and time that wasdifferent from the way we perceive real space and time. Arnheim wasactually disgruntled by the advent of (p.12) sound in movies as it attemptedto mimic more closely the real world (in later writings he even denigratedcolor and widescreen cinema). Arnheim offered a formalist view, whichsuggested that the way a movie is structured or formed is what makesit a work of art (Andrews, 1976). Unlike Munsterberg’s treatise, Film alsKunst was very influential among film scholars as a way of justifying filmas art. Unfortunately, among psychological scientists, Arnheim’s Gestaltistinterpretation did not lead to much empirical investigation.

The Cognitive Revolution

During the 1950s, the advent of the digital computer offered a new way tocharacterize psychological processes. Just as a computer receives inputs,stores information, performs computations, and outputs results, the mindcould be viewed as a programmable device able to recognize objects,understand language, and remember events. Cognitive psychologists thusbegan to consider perceiving, remembering, thinking, and decision makingfrom an “information processing” approach. By this view, the mind couldbe viewed as mental processes that were involved in various computationalstages of processing, such as input, storage, and output. During the 1960sand 1970s, this approach became so pervasive across all domains ofpsychological science that this era has been referred to as the cognitiverevolution.

Some cognitive psychologists developed actual computer programs thatsimulated human performance, whereas others used the computer analogymore loosely by keying on information processing terms, such as sensoryinput, short-term buffers, long-term memory, and response output. In 1967,Ulric Neisser’s Cognitive Psychology delineated the approach succinctly: “…the term ‘cognition’ refers to all the processes by which the sensory inputis transformed, reduced, elaborated, stored, recovered, and used” (Neisser,

Page 17: 1 Psychocinematics Issues and Directions

Page 17 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

1967, p. 4). In the 1980s, a multidisciplinary approach, described ascognitive science, emerged in which psychology was integrated withneuroscience, linguistics, artificial intelligence, philosophy, and anthropologyas a way of developing a deeper understanding of human cognitive function(see Gardner, 1985; Norman, 1980).

The cognitive approach evolved as a reaction against behaviorism, as itacknowledged the importance of mental processes and representations.Previous “mentalistic” traditions, such as those espoused by Wundt,William James, and the Gestaltists, failed to develop a strong scientificapproach because their mental concepts were too vague and the theoriesnot described well enough to be tested adequately. In many respectsthe cognitive approach offered a middle ground as the study of behaviorwas still paramount, only it was now best explained by referring to theway “information” is processed. With computers as evidence, it was notoutlandish to consider a physical entity that could be “programmed” toperceive, store, and perform complex calculations.

A particularly important feature of the cognitive approach is top-downprocessing, which refers to the use of knowledge (top) to direct sensoryprocesses (bottom). Cognitive psychologists adopted the term schema,which Bartlett (1932) used to describe the way prior knowledge guidesbehavior. We are constantly applying schema to drive top-down processing.For example, we have a schema for what to do in a movie theater—whichis to pay for a ticket, enter a darkened room, sit down, and enjoy a movingpicture. Top-down processing is critical in the way we interpret sensoryinput. Ernst Gombrich, the noted art historian, was the first to consider top-down processing in our aesthetic experiences (Gombrich, 1960). He arguedthat paintings are symbolic and we use personal and cultural knowledge(i.e., “schemata”) to (p.13) form expectations about what to look for in anartwork. According to Gombrich, top-down processing is the beholder’s sharein our art experience. That is, we bring to the art experience prior knowledgeand expectations. Cognitive film theorists have considered Gombrich’sviews with respect to our movie experience (see Chapters 2, 3, and 4).For example, Bordwell (1989, 1992) applied the notion of schema to ourunderstanding of film narrative. Also, the notion of modes of receptions (seeChapter 16) suggests that we regulate our movie experience through top-down processing (e.g., expectations).

Despite the acceptance of mental states as appropriate scientific constructs,few cognitive studies were directed toward our experience with movies.

Page 18: 1 Psychocinematics Issues and Directions

Page 18 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

A strong advocate for psychocinematics during the 1970s and 1980s wasJulian Hochberg, a cognitive scientist who considered the perceptual featuresof movies (Hochberg, 1986; Hochberg & Brooks, 1978). In both theoreticaland empirical analyses, Hochberg attempted to delineate the cognitiveunderpinning of our movie experience. He offered a constructionist approachin which we build a spatial representation of scenes and events as we watchmovies.

Cognitive Neuroscience

Can the complexities of our movie experience be understood by examiningbrain mechanisms? Neuroscientists have studied the workings of the brainfrom many levels, including the study of individual brain cells (neurons)and the study of global activity in the human brain. Cognitive processesare driven primarily by the workings of our two cerebral hemispheres,those large convoluted masses that embody tens of billions of neurons,each having thousands of connections to others. Neural circuits have beenidentified that pertain to various mental processes, such as vision, memory,language, emotion, and motor control. Both structural (i.e., anatomical) andfunctional (i.e., physiological) analyses have been used to investigate theneural correlates of human cognitive function. Of course, the whole brainmust work together to give us the ability to perform complex operations,such as perceiving, remembering, and decision making.

Figure 1.2 shows the lateral (outer) surface of the left cerebral cortex andmedial (inner) surface of the right cerebral cortex. Initial input of visualinformation into the cortex occurs at the most posterior region of theoccipital lobe, an area identified as V1. From this point, visual informationis processed along two major paths. The dorsal or “where” path coursesup through the parietal lobe and processes spatial information, whereasthe ventral or “what” path courses down through the temporal lobe andprocesses object information. Auditory input arrives at Heschl’s gyrus(HG) situated in the temporal lobe. From there, audition is processed asidentifiable sounds in surrounding temporal regions (Belin, Zatorre, Hoge,Evans, & Pike, 1999). As is well known, speech recognition and languageare processed predominantly in the left hemisphere in most (roughly 90%)humans.

The anterior region of the frontal lobes is called the prefrontal cortex (PFC).It receives inputs from many other brain regions and sends projections backto these regions. In this way, the PFC has the capacity to coordinate and

Page 19: 1 Psychocinematics Issues and Directions

Page 19 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

control cortical processing. In fact, patients with frontal lobe damage haveparticular problems in controlling their attention, thoughts, and feelings(Shimamura, 2008). They are distractible and in some cases have severeproblems in regulating their emotions. Neuroimaging studies have confirmedthe role of the PFC in what psychologists call executive control (Badre, 2008;D’Esposito, Postle, & Rypma, 2000; Shimamura, 2008). If one considers thecacophony of neural signals at any given moment, it (p.14)

Figure 1.2 View of left lateral (outer) surface and right medial (inner) surfaceof the cerebral cortex. Visual information enters the cerebral cortex inV1 and is then processed along two major routes, the ventral and dorsalpaths. Auditory input enters at Heschl’s gyrus (HG) and is processed inadjacent regions in the temporal lobe. The posterior parietal cortex (PPC)is a convergence zone that binds and integrates multisensory information.The prefrontal cortex (PFC) acts as an executive modulating brain activitythrough feedback loops to posterior regions. The medial (inner) surfaceshows the orbitofrontal cortex (OBF) and the anterior cingulate cortex (ACC),which are involved in emotional monitoring and evaluation. Brain imagesreprinted with permission from Digital Anatomist Interactive Atlas, Universityof Washington, Seattle, WA, copyright 1997.

becomes obvious how important it is for us to have a mechanism thatcontrols or modulates brain activity. The PFC’s role in monitoring andcontrolling neural activity forms the basis for top-down processing as itprovides a feedback mechanism in which thoughts and expectations can

Page 20: 1 Psychocinematics Issues and Directions

Page 20 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

guide and select sensory signals (see Miller & Cohen, 2001; Shimamura,2008).

Advances in neuroscience have identified another important region calledthe posterior parietal cortex, which is situated between the ventral anddorsal visual paths and is also adjacent to regions that process speechand other sounds. This region acts as a convergence zone that binds orintegrates information processing. In neuroimaging studies, it is particularlyactive when we engage our imagination, such as imagining ourselves assomeone else or imagining ourselves at a previous time and place (e.g.,reminiscing about a past event; Saxe & Wexler, 2005; Shimamura, 2011).This region also contains so-called mirror neurons, which become activewhen we imagine performing the actions of another, such as watchingsomeone bite an apple or grasp a ball (Buccino et al., 2001; Rizzolatti &Fabbri-Destro, 2010). As imagination is central to our movie experience(Currie, 1995; Munsterberg, 1916), the posterior parietal cortex must beconsidered as a critical structure for psychocinematic investigations.

On the medial surface of the PFC are the orbitofrontal cortex (OBF) andanterior cingulate cortex (ACC). These two regions are integral in monitoringand evaluating emotional responses. For example, the OBF is engaged inevaluating emotional stimuli, such as (p.15) paintings rated as beautiful(Ishizu & Zeki, 2011; Kawabata & Zeki, 2004). While watching ratherdisturbing film clips (e.g., a surgical film showing an arm amputationoperation), the OBF and ACC are engaged when subjects are instructed tosuppress their emotions (Shimamura, Marian, & Haskins, 2012). Movies havean extraordinary capacity of engaging emotional processes and empathizingwith characters. Thus, understanding the manner in which movies elicitemotions is another central goal for psychocinematics.

Since the 1990s, many cognitive scientists have turned to brain imaging asa way to map cognitive processes onto brain mechanisms. In particular, theadvent of fMRI has allowed scientists to assess on a moment-to-momentbasis brain activations in response to sensory stimuli. The technique usesthe same MRI scanners found in hospitals, though they are tuned to detectsubtle changes in blood flow, which occur when brain regions becomeactive. At any given moment, however, there are tens of thousands ofneurons active just to keep us alive, such as those involved in regulatingheart rate, respiration, or body temperature. In fMRI analyses, these steady-state activations are cancelled out by comparing one set of activations withanother. For example, one could scan individuals while they are watching

Page 21: 1 Psychocinematics Issues and Directions

Page 21 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

a movie and again while they are viewing still photographs. By subtractingthe “watching movie” scans from the “viewing photographs” scans, onecan identify brain regions that are specifically active while viewing movingpictures (see Chapters 12 and 15 for further analyses of fMRI methods andfindings).

How do our brains respond to movies? To explore this question, Hasson etal. (2008) advocated neurocinematics, in which brain activity is recordedwhile individuals watch moving pictures. Not only can such research offerintriguing findings about our movie experience, but they also can helpelucidate the neural underpinnings of natural, everyday viewing (seeHasson, Nir, Levy, Fuhrmann, & Malach, 2004; Nishimoto et al., 2011). Yetmerely recording brain activity while watching movies is not enough, as itis important to consider the psychological processes that are defined bysuch neural activity. That is, we cannot fall into a modern-day version ofphrenology where bumps on the head are replaced by bright spots in thebrain. We need to go further and develop neuropsychological theories thatdescribe the functional dynamics of brain activity and how they operate inthe service of cognition (see Shimamura, 2010).

In evolutionary terms, our survival depends on our ability to interpretthe environment, such as acknowledging a catchable prey, a threateningpredator, or a willing mate. In such heightened emotional situations, thebrain engages a set of processes intended to arouse our mind and body,thus sparking the flight-or-fight response in which adrenalin courses throughour body and induces heart palpitations, muscle twitches, butterflies in thestomach, and sweaty palms. Such physiological responses are intendedto prepare us for intense physical activity and are initiated when we arefrightened, excited, angry, or sexually aroused (see Sapolsky, 1994). Theamygdala, a cluster of neurons tucked within the medial region of thetemporal lobes, acts as an interface between cognitive and emotionalprocesses (see LeDoux, 1998). The amygdala integrates inputs from manycortical regions, and when the environmental context is interpreted asan emotional situation (e.g., angry boss, willing mate), it sends signalsto the hypothalamus, a subcortical structure, which initiates the adrenalarousal response. When we watch a movie and imagine arousing or fearfulsituations, our brains initiate these basic physiological responses.

In forming theories about brain function, scientists have viewed the brainmuch like a large business or corporation where various divisions proceedsomewhat independently toward the goal of manufacturing a final product.

Page 22: 1 Psychocinematics Issues and Directions

Page 22 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

Our eyes and ears gather raw materials (p.16) from the outside world in theform of bits of sensory information. These bits are processed and the finalproduct is meaningful information, such as the recognition of a friend atyour door or the word “hello” when greeted. Like an industrial conveyor belt,there are paths in the brain that process and build on sensory informationalong the way. From fMRI studies, it has been shown that the ventral pathsegregates the visual scene into recognizable objects (e.g., faces, chairs,and other objects; Haxby et al., 2001), whereas the dorsal path constructs aspatial layout or context into which objects are placed. Brain processes workin parallel but must be integrated to give us the ability to perform complexactivities such as understanding events or social signals.

A Conceptual Framework for Psychocinematics

Movies are rather unique as they offer a sensory experience that is similar tonatural viewing, yet they have been created for a particular purpose, whichis to instill an aesthetic response. As described earlier, psychocinematicsworks within the framework of aesthetic science, as it is essential toexamine the way movies drive our interest, understanding, and appeal (seeShimamura & Palmer, 2012). With regard to aesthetic experiences, I haveproposed the I-SKE model, which identifies the importance of the artist’sintention (I) to create an artwork and the beholder’s share in terms of theway an artwork impinges on sensations (S), knowledge (K), and emotions (E),hence the I-SKE acronym (Shimamura, 2012). Figure 1.3 characterizes thisframework with respect to the filmmaker, movie, and viewer.

The filmmaker’s intention plays a role in our movie experience. As suggestedby the so-called auteur theory of film criticism (Sarris, 1968), we oftenattribute the creative vision of a movie to a single artist, often the director,as when we say it’s a Woody Allen or Quentin Tarantino film. Certain stylesand techniques are associated with specific filmmakers, and knowledgeof their intention to communicate a certain viewpoint, idea, or feeling ispart of the viewer’s experience. In cognitive terms, we develop a schemafor certain filmmakers, and this knowledge can influence our viewing. Ascreative expressions, we expect certain things from movies that we don’texpect from viewing nature. In particular, we acknowledge that a creator hadthe intention to instill in us an aesthetic experience, which may be to offer asensory experience, a conceptual viewpoint, or a certain feeling. The I-SKEmodel offers a way to characterize these three psychological componentsof our movie experience—that is, the role of sensations, knowledge, andemotion in driving our aesthetic response to movies.

Page 23: 1 Psychocinematics Issues and Directions

Page 23 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

Sensory and Attentional Features

Our movie experience is, of course, driven by the sensory inputs thatenvelop and guide us. Through sights and sounds, the filmmaker creates aspatial environment, and quite often we engage ourselves as if we are part ofthe world portrayed. Yet even the earliest film theorists, such as Munsterberg(1916), acknowledged the fact that we do not really believe we are actuallyin the scene (see Chapters 3 and 6). As with our viewing of paintings orphotographs, we can imagine ourselves being in or “seeing in” the picture,but at the same time we know that we are outside the medium standing inan art gallery or looking at an art book (see (p.17)

Figure 1.3 The I-SKE model as it pertains to our movie experience. Adaptedfrom Shimamura, A. P. (2012). Approaching a science of aesthetics: Issuesand ideas. In A. P. Shimamura & S. E. Palmer (Eds.), Aesthetic science:Connecting minds, brains, and experience (pp. 3–28). New York, NY: OxfordUniversity Press.

Shimamura, in press; Wollheim, 1980; Chapter 6). Philosophers have thoughtdeeply about this matter, as it is not completely clear to what extent we actas if we are perceiving or imagining ourselves in a picture or movie.

Page 24: 1 Psychocinematics Issues and Directions

Page 24 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

Psychocinematics can help address such issues. For example, there arebrain regions that respond to both perceived and imagined experiences,as well as other brain regions that are only active when we are actuallyengaged in sensory processing (Kosslyn, Thompson, & Ganis, 2006). Howmuch are sensory-specific regions engaged when we watch movies? To whatextent do our brains interpret the sensory experience as being within thespatial environment portrayed by a movie? With respect to such questions,one could explore how editing rules, such as the 180-degree rule, create aspatial frame of reference (see Chapter 11). Does it take longer to adjust toa violation of the 180-degree rule as opposed to a shot that conforms to therule? Why do cuts that adhere to the 30-degree rule seem to represent amovement in the viewer’s point of view rather than a movement of objects inthe scene? Why are we so oblivious of the thousands of shot transitions thatoccur in a movie?

As mentioned earlier, one effective means of creating a seamless edit isthe match-action edit, in which a character’s movement continues acrossa cut. To create a smooth match-action edit, film editors have advocatedthat the action after the edit be repeated or overlap with the pre-edit actionby several frames (48–72 msec; see Anderson, 1998; Dymtryk, 1984).Shimamura, Cohn-Sheehy, and Shimamura (2012) confirmed this intuition byhaving participants determine the smoothest movement of a match-actionedit across a varied amount of overlap (or jump) in the action between shots.Individuals judged the smoothest action to occur when there was a three-frame repeat of the action. In another study (Smith & Henderson, 2008),individuals watched excerpts from Hollywood movies and made a keypresswhenever they detected an edit. Participants failed to detect an average of15.8% of edits, exhibiting what the authors called “edit (p.18) blindness.”Match-action edits were most often missed as participants failed to detect athird of such edits.

Movement is a potent force in driving attention. We tend to pay attentionto moving objects, particularly when we watch movies. Mital, Smith, Hill,and Henderson (2011) recorded eye movements while individuals watchedmovies. There was a strong coherence in gaze pattern among viewers asthey all tended to fixate on the same moving objects. Smith (Chapter 9)refers to this gaze attraction as attentional synchrony, and has developedthe Attentional Theory of Cinematic Continuity (AToCC) to describe itsnature. Such findings are also consistent with the notion of change blindness(Levin & Simons, 1997; Levin & Varakin, 2004; Simons & Levin, 1997), apsychological phenomenon in which individuals fail to notice alterations in a

Page 25: 1 Psychocinematics Issues and Directions

Page 25 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

scene or mislocations of objects while viewing edited scenes. By playing onthe viewer’s attentional focus, filmmakers impart a sort of magician’s sleightof hand, drawing on moments when viewers are fixated on movement as away to conceal edits. Thus, filmmakers have acquired knowledge about thecognitive demands of viewing movies and know when to insert an edit just asa magician knows when to conceal a coin or reveal a playing card. It is nowup to psychocinematics to identify more explicitly the cognitive processesrelated to such “illusions.”

In evolutionary terms, our sensory apparatus is geared primarily towardhelping us move around in space. From sensations we construct a spatialenvironment that places us within a three-dimensional setting. As mentionedearlier, within the cerebral cortex, the ventral path, which is involvedin object recognition, and the dorsal path, which is involved in spatialprocessing, help to construct this spatial environment. Filmmakers havedeveloped techniques that engage these spatial construction processes, asif we are enveloped within the 3-D world portrayed on the screen. Again, itis not as if we actually believe we are in the world projected, but we co-optthese basic cognitive processes while watching movies, and these processesdrive our experience. Psychocinematics can help link basic spatial processesto our movie experience. In a match-action edit that occurred in TheUnbearable Lightness of Being (1988, Orion Pictures), the noted film editor,Walter Murch, inserted a 10-frame overlap of the action (Murch, personalcommunication). In this scene, a camera shot of the character, Sabina (LenaOlin), bending down, cuts to a shot of her reflection in a mirror, which, ofcourse, reverses the spatial orientation and causes the same confusion asviolating the 180-degree rule. It may be that a cognitive reconstruction of thespatial scene was needed to interpret the shot transition, which took time,and thus required significant overlap in action to be perceived as a naturalmovement.

Sounds, both diegetic and nondiegetic effects, play an important role in ourmovie experience (see Boltz, 2001; Cohen, 2005). They can direct attentionto people or objects, foreshadow events, and amplify emotions. In even asimple display, a sound can alter visual perception. In one study, individualswatched two disks move toward each other, overlap, and then continue onpast. Yet when a click was presented at the moment of overlap, the two disksappeared to bounce and ricochet back in the opposite direction (Sekuler,Sekuler, & Lau, 1997). With respect to nondiegetic sounds, memory can beenhanced when mood-congruent music is played with an emotional scene.

Page 26: 1 Psychocinematics Issues and Directions

Page 26 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

For example, a happy scene is remembered better when it is accompaniedby upbeat music (Boltz, Schulkind, & Kantra, 1991).

(p.19) Knowledge, Narratives, and Events

Considering the beholder’s share of our movie experience, we mustacknowledge the role of knowledge in driving our movie experience. Asmentioned earlier, we are always applying top-down processing, using ourknowledge to develop expectations and guide sensory processing. Evenbefore we sit down to watch a movie, top-down processing is working by wayof knowledge we may have about the filmmaker, movie title, and other priorknowledge (e.g., having seen previews or hearing about the movie’s plot).From this knowledge, we develop a schema in the form of expectations ofwhat’s to come. Moreover, from the thousands of hours of watching movies,television, and YouTube videos, we are very familiar with the ways moviesare constructed.

To what extent does our movie experience depend on such familiarity?Schwan and Ildirar (2010) showed movie clips to individuals living in aremote mountain village in Turkey who did not have electricity and hadnever seen moving pictures. They showed various clips that included filmictechniques such as pans, cross-cuts, shot/reverse shots, and establishingshots. These individuals were very poor at interpreting story transitionswhen presented as shot/reverse shots, pans, and establishing shots, thoughthey could comprehend cross-cut edits. These findings demonstrated thatour understanding of the “syntax” of movies is to some extent a learnedphenomenon acquired from the countless movies and television programsviewed during our lifetime.

Through storytelling, we have developed a schema for fictional narratives(see Brewer & Lichenstein, 1981; Currie, 2012). This schema generallyconforms to Aristotle’s notion of a defined beginning, middle, and end,with various events (e.g., reversal of fortune, climax) presented along theway. Scriptwriters often use rigorously defined plot structures such thatcertain events are placed at specific moments in a movie. In Save the Cat!,a book on screenwriting, Blake Snyder (2005) suggests that scripts shouldalways introduce the hero with an action that draws us to him or her, suchas rescuing a cat. The second act, which is the longest, builds tension asthe protagonist confronts a series of challenges. Of course, we are led to aclimax, with the final act serving as a way of reducing tension and resolvingconflicts. This narrative structure makes a story an emotional roller coaster

Page 27: 1 Psychocinematics Issues and Directions

Page 27 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

ride, which begins with anticipation, creates tension, and when we reachthe top we hang on for dear life. The parallels between movies and rollercoasters are rather fitting as we actually pay for both experiences, expect anemotional thrill, and do so with the knowledge that we will survive at the end(usually seated in both cases).

An important aspect of psychocinematics is how we represent events, whichcan be described as actions encapsulated within a time and place setting.Fictional narratives can be construed as the sequence of events that makeup a story (see Chapter 12). Individual events can be segmented further,such as a murder being segmented into finding a gun, loading bullets,and confronting the victim (Carroll & Bever, 1976; Newtson, 1973; ZacksSpeer, Swallow, Braver, & Reynolds, 2007). The pacing of events adds tothe interest and enjoyment of a movie. Rhythmic pacing is exemplifiedby patterns of long and short shot durations throughout a movie (Cutting,DeLong, & Nothelfer, 2010; Salt, 2006). These waves of shot lengthsheighten and ultimately break at the final climax.

A good movie creates a series of tensions and releases, suspensions andresolutions, or cues and responses (see Chapters 7, 10, and 12). In this way,the filmmaker flirts with the viewer by concealing and revealing knowledgeas the movie proceeds. As described by Berliner (p.20) (Chapter 10), fictionalnarratives in movies can be viewed as the interplay between unity anddisunity. With respect to the visual aesthetics associated with paintings,Kreitler and Kreitler (1972) considered art as creating a visual interplay oftension and release. Carroll (1985) described a movie’s pacing as a series ofquestions and answers or what he termed the erotetic model of narrative.As events are segmented, there may be a hierarchy of unity–disunity,tension–release, or questions–answers, with the movie’s climax providingthe final resolution. With respect to psychocinematics, one could manipulatethe degree to which a narrative follows such a scheme and determine ifviolations of pacing create confusion or a lowered aesthetic experience.

Engaging Emotions and Empathy

We laugh, cry, and become frightfully scared while watching movies. Thesefeelings are driven largely by emotional engagement with the charactersportrayed. Such empathetic responses first involve imaging oneself assomeone else and, as a result, experiencing another’s feelings (Shamay-Tsoory, 2010). The ability to take on the perspective of another requiresa cognitive restructuring or what psychologists call a theory of mind

Page 28: 1 Psychocinematics Issues and Directions

Page 28 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

(see Chapters 13 and 17). As mentioned earlier, the posterior parietalcortex is particularly active when we imagine the action of others (Saxe& Wexler, 2005). Such processes have been linked to the phenomenon ofmirror neurons (Buccino et al., 2001; Rizzolatti & Fabbri-Destro, 2010). Inneuroimaging studies, this region is involved whenever we must reorientourselves in time and place, such as with theory of mind, remembering thepast, or imagining what we will be doing in the future (see Buckner & Carroll,2007; Shimamura, 2011). This cognitive engagement helps us imagine beingsomeone else.

In addition to the cognitive restructuring of taking on another’s perspective,empathy involves feeling another’s emotion. Neuroimaging studies haveshown that other regions, including the amygdala, insula, orbitofrontalcortex, and anterior cingulate cortex, are active during such emotionalresponses (Singer et al., 2004). In one study, emotion-related brain regionswere active when individuals experienced an arousing event (an electricshock to the hand) and when they watched a loved one experiencing thesame event (Singer et al., 2004). Filmmakers have developed creative useof camera movement and editing to enhance our empathetic response, andof course, good actors know how to express emotions in such a way as toencourage empathy (see Goldstein & Bloom, 2011). Quite frequently, it is ourempathetic engagement that drives our emotional experience during movies(for details, see Chapters 5, 14, and 17).

Movie sounds, both diegetic and nondiegetic, significantly impact on ouremotions. Abrupt sounds, such as a scream or explosion, induce a powerfulstartle response. Background music can amplify a variety of emotions,including sadness, fear, and joy. Simply hearing someone laugh can inducemore positive emotions, as indicated by the inclusion of “laugh” tracks in TVcomedies or hearing others laugh in a movie theater. Thayer and Levenson(1983) recorded heart rate and skin conductance (akin to measuring sweatypalms) while subjects viewed a safety film depicting industrial accidents.The clip was played with either calming music, horror music, or no music.Skin conductance (but not heart rate) was increased with horror music anddecreased with calming music. Filmmakers sometimes include incongruentmusic as a way to create an eerie or ironic mood. In Alfred Hitchcock’s TheBirds, the (p.21) sense of eerie terror is created by the playing of forebodingmusic as viewers watch an otherwise innocuous shot of birds flying.

As with music, the visual scene itself can set up our emotional response.Suspense is driven by a sense of calm with the anticipation of impending

Page 29: 1 Psychocinematics Issues and Directions

Page 29 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

terror. Indeed, anticipation or expectations play a key role in driving ouremotions. The Russian silent filmmaker Lev Kuleshov considered suchcontextual influences. He interspersed shots of an actor exhibiting a neutralexpression with shots of a child’s coffin, a woman, or a plate of soup (Levaco,1974). This same “neutral” expression was interpreted differently dependingon what image preceded it. Thus, the same expression appeared to showsorrow, lust, or hunger, depending on the context. Psychological studiesof the Kuleshov effect have confirmed the impact of the social context onemotion (Marian & Shimamura, in press; Mobbs et al., 2006). For example,if a person smiles at you and then the smile turns into a neural expression,that person will appear somewhat grumpy or disappointed. Conversely,if a person first looks angry and then the expression turns into a neuralexpression, the person looks somewhat pleasant or positive (Marian &Shimamura, in press).

Coming Attractions

The chapters to follow will elaborate and expand upon the issues introducedhere. The next four chapters lay down the philosophical foundationof psychocinematics. Thereafter, sections are divided into the threepsychological components of the I-SKE model—sensations, knowledge, andemotion—and how they drive our movie experience. The contributors focusprimarily on the ways movies present and represent fictional narratives. It isnoteworthy to point out that nonfiction films (i.e., documentaries) generallyadhere to the same cognitive and aesthetic features as will be describedfor fictional narratives. That is, continuity editing, event sequencing, andempathetic engagement are hallmark features of good documentaries. Aninteresting goal for psychocinematics will be to consider how such featureswhen applied to documentaries help disseminate knowledge in truthful(or less truthful) ways (see Chapter 8). Many students learn from movies,such as from NOVA programs, history/documentary channels, and YouTubevideos, and it is important to consider the psychological processes that drivethis potentially powerful form of learning.

Another consideration is the genre of avant-garde or experimental film.Although such films often use the kind of filmic techniques mentioned earlierto drive psychological processes, experimental films, almost by definition,attempt to extend the boundaries of what we think a movie should be.Such films often act as commentaries on the film process itself by breakingaway from our movie schema (for further analysis along these lines, seeCarroll, 2008). Some avant-garde films, such as Alain Resnais’s L’Année

Page 30: 1 Psychocinematics Issues and Directions

Page 30 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

dernièreàMarienbad (Last Year at Marienbad), offer interpretations of mentalprocesses, such as the vagaries of reminiscence, through unorthodoxnarrative techniques. These movies define and describe the movie processthrough the process itself. In order to appreciate and enjoy such meta-artcommentaries, it is necessary to have knowledge of the art of movies, itstechniques, and its history.

Digital movies have special properties (see Chapter 6) and offer interestingpossibilities for psychocinematics. One aspect is the way we experiencedigitally created movies that appear realistic though we know that theentire mise-en-scène was artificially contrived. For example, (p.22) thesedays animations through motion capture offer exquisite rendering ofperspective and shading, yet we are aware that they are still animations.Computer scientists have a term, the “uncanny valley,” which refers to adrop in familiarity (i.e., a strangeness) when a robot becomes very close tohuman likeness. For example, a prosthetic hand looks eerily strange as itbecomes very close to mimicking a real hand. A recent fMRI study (Saygin,Chaminade, Ishiguro, Driver, & Frith, 2011) suggests that the brain respondsdifferently to androidlike actions that closely resemble human actions whencompared to real human actions or actions from an obviously artificial-looking robot. When we know that what seems real is really artificial, as indigital motion capture, how does our movie experience change?

Finally, there are social factors of our movie experience that influence ourinterest and enjoyment. For example, how is our experience influenced bythe environmental context in which we view movies, as when we watch amovie with strangers in a theater, with a friend in the living room, or aloneon our laptop? What cultural knowledge is required when we experiencemovies? How much are we missing when we watch a foreign movie? From I-SKE issues concerning sensations, knowledge, and emotion to more complexsocial factors, psychocinematics has the potential to address many aspectsof our movie experience and perhaps in so doing enhance our aestheticappreciation.

Acknowledgement

I thank David Bordwell, Helen Ettlinger, Faerthen Felix, Walter Murch,Mike Roush, Gregory Shimamura, and Thomas Shimamura for helpfulconversations concerning the issues presented in this chapter.

References

Page 31: 1 Psychocinematics Issues and Directions

Page 31 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

Bibliography references:

Anderson J. (1998). The reality of illusion: An ecological approach to cognitivefilm theory. Carbondale, IL : Southern Illinois University Press.

Anderson J., & Anderson B. (1993). The myth of persistence of visionrevisited. Journal of Film and Video, 45, 3–12.

Andrews D. (1976). The major film theories: An introduction. New York, NY:Oxford University Press.

Arnheim R. (1974). Art and visual perception. Berkeley, CA : University ofCalifornia Press.

Arnheim R. (1983). The power of the center: A study of composition in thevisual arts. Berkeley, CA : University of California Press.

Arnheim R. (2006). Film as art. Berkeley, CA : University of California Press.

Badre D. (2008). Cognitive control, hierarchy, and the rostro-caudalorganization of the frontal lobes. Trends in Cognitive Science, 12, 193–200.

Bartlett F. C. (1932). Remembering: A study in experimental and socialpsychology. Cambridge, England : Cambridge University Press.

Belin P., Zatorre R. J., Hoge R., Evans A. C., & Pike B. (1999). Event-relatedfMRI of the auditory cortex. NeuroImage, 10, 417–429.

Boltz M. (2001). Musical soundtracks as schematic influence on the cognitiveprocessing of filmed events. Music Perception, 18, 427–454.

(p.23) Boltz M. G., Schulkind M., & Kantra S. (1991). Effects of backgroundmusic on the remembering of filmed events. Memory & Cognition, 19, 593–606.

Bordwell D. (1989). Making meaning: Inference and rhetoric in theinterpretation of cinema. Cambridge, MA : Harvard University Press.

Bordwell D. (1992). Cognition and comprehension: Viewing and forgetting inMildred Pierce. Journal of Dramatic Theory and Criticism, 6, 183–198.

Bordwell D. (2002). Intensified continuity: Visual style in contemporaryAmerican film. Film Quarterly, 55, 16–28.

Page 32: 1 Psychocinematics Issues and Directions

Page 32 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

Bordwell D. (2006) The way Hollywood tells it: Story and style in modernmovies. Berkeley, CA : University of California Press.

Bordwell D., Staiger J., & Thompson K. (1985). The classical Hollywoodcinema. Film style & mode of production to 1960. New York, NY: ColumbiaUniversity Press.

Brewer W. F., & Lichenstein E. H. (1981). Event schemas, story schemas, andstory grammars. In J. Long & A. Baddeley (Eds.), Attention and performance(Vol. IX, pp. 363–379). Hillsdale, NJ : Erlbaum.

Buccino G., Binkofski F., Fink G. R., Fadiga L., Fogassi L., Gallese, V.,…Freund,H.-J. (2001). Action observation activates premotor and parietal areas in asomatotopic manner: An fMRI study. European Journal of Neuroscience, 13,400–404.

Buckner R. L., & Carroll D. C. (2007). Self-projection and the brain. Trends inCognitive Science, 11, 51–57.

Carroll J. M., & Bever T. G. (1976). Segmentation in cinema perception.Science, 191, 1053–1055.

Carroll N. (1985). The power of movies. Daedalus, 114, 79–103.

Carroll N. (2008). The philosophy of motion pictures. Malden, MA : Wiley-Blackwell.

Cohen A. J. (2005). How music influences the interpretation of film andvideo: Approaches from experimental psychology. In R. A. Kendall & R. W.H. Savage (Eds.), Perspectives in systematic musicology (pp. 15–36). LosAngeles, CA : Ethnomusicology Publications.

Cook D. (2004). A history of narrative film (4th ed.). New York, NY: W. W.Norton.

Currie G. (1995). Image and mind: Film, philosophy and cognitive science.Cambridge, England : Cambridge University Press.

Currie G. (2012). Narratives and narrators: A philosophy of stories. Oxford,England : Oxford University Press.

Cutting J. E., DeLong J. E., & Nothelfer C. E. (2010). Attention and theevolution of Hollywood film. Psychological Science, 21, 440–447.

Page 33: 1 Psychocinematics Issues and Directions

Page 33 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

Deco G., & Roland P. (2010). The role of multi-area interactions for thecomputation of apparent motion. NeuroImage, 51, 1018–1026.

D’Esposito M., Postle B. R., & Rypma B. (2000). Prefrontal corticalcontributions to working memory: Evidence from event-related fMRI studies.Experimental Brain Research, 133, 3–11.

Dmytryk E. (1986). On filmmaking. London : Focal Press.

Ebbinghaus H. (1884/ 1964). Memory: A contribution to experimentalpsychology. New York, NY: Dover Publications.

Fechner G. (1876). Vorschule der Aesthetik. Leipzig, Germany : Breitkopf andHatrtel.

Fechner G. B. (1860). Elemente der Psychophysik. Leipzig, Germany :Breitkopf and Hatrtel.

Gardner H. (1985). The mind’s new science. New York, NY: Basic Books.

Goldstein T. R., & Bloom P. (2011). The mind on stage: Why cognitivescientists should study acting. Trends in Cognitive Science, 15, 141–142.

(p.24) Gombrich E. H. (1960). Art and illusion. Princeton, NJ : PrincetonUniversity Press.

Hasson U., Landesman O., Knappmeyer B., Vallines I., Rubin N., & Heeger D.J. (2008). Neurocinematics: The neuroscience of film. Projections, 2, 1–26.

Hasson U. Nir, Y., Levy I., Fuhrmann G., & Malach R. (2004). Intersubjectsynchronization of cortical activity during natural vision. Science, 303, 1634–1640.

Haxby J. V., Gobbini M. I., Furey M. L., Ishai A., Shouten J. L., & Pietrini P.(2001). Distributed and overlapping representations of faces and objects inventral temporal cortex. Science, 293, 2425–2243.

Helmholtz H. (1881). On the relation of optics to painting. Popular Lectureson Scientific Subjects (second series; pp. 73–138). New York, NY: Appleton &Company.

Hendricks G. (1975). Eadweard Muybridge: The father of the motion picture.New York, NY: Grossman Publishers.

Page 34: 1 Psychocinematics Issues and Directions

Page 34 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

Hochberg J. (1986). Representation of motion and space in video andcinematic displays. In K. R. Boff R. Kaufman, & J. P. Thomas (Eds.), Handbookof perception and human performance: Vol. 1. Sensory processing andperception (pp. 22–1 to 22–64). New York, NY: Wiley.

Hochberg J., & Brooks V. (1978). The perception of motion pictures. In E. C.Carterette & M. P. Friedman (Eds.), Handbook of perception (Vol. X). NewYork, NY : Academic Press.

Ishizu T., & Zeki S. (2011). Toward a brain-based theory of beauty. PLoS ONE,6, E21852.

Kawabata H., & Zeki S. (2004). Neural correlates of beauty. Journal ofNeurophysiology, 91, 1699–1705.

Kosslyn S. M., Thompson W. L., & Ganis G. (2006). The case for mentalimagery. New York, NY: Oxford University Press.

Kraft R. (1986). The role of cutting in the evaluation and retention of film.Journal of Experimental Psychology: Learning, Memory, and Cognition, 12,155–163.

Kreitler H., & Kreitler S. (1972). Psychology and the arts. Durham, NC : DukeUniversity Press.

LeDoux J. (1998). The emotional brain: The mysterious underpinnings ofemotional life. New York, NY: Simon & Schuster.

Levaco R. (1974). Kuleshov on film: Writings by Lev Kuleshov. Berkeley, CA:University of California Press.

Levin D. T., & Simons D. J. (1997). Failure to detect changes to attendedobjects in motion pictures. Psychonomic Bulletin and Review, 4, 501–506.

Levin D. T., & Varakin D. A. (2004). No pause for a brief disruption: Failuresof visual awareness during ongoing events. Consciousness and Cognition, 13,363–372.

Lindsay V. (1915). The art of the moving picture. New York, NY: MacMillanPress.

Marian D. E., & Shimamura A. P. (in press). Context effects following dynamicfacial expressions. American Journal of Psychology.

Page 35: 1 Psychocinematics Issues and Directions

Page 35 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

Mast G. (1984). Film/cinema/movie: A theory of experience. Chicago, IL :University of Chicago Press.

Meulders M. (2010). Helmholtz: From enlightenment to neuroscience.Cambridge, MA : MIT Press.

Miller E. K., & Cohen J. D. (2001). An integrative theory of prefrontal cortexfunction. Annual Review of Neuroscience, 24, 167–202.

Mital P. K., Smith T. J., Hill R. L., & Henderson J. M. (2011). Clustering ofgaze during dynamic scene viewing is predicted by motion. CognitiveComputation, 3, 5–24.

(p.25) Mobbs D., Weiskopf N., Lau H. C., Featherstone E., Dolan R. J., & FrithC. D. (2006). The Kuleshov effect: The influence of contextual framing onemotional attributions. Social Cognitive and Affective Neuroscience, 1, 95–106.

Munsterberg H. (1916). ThePhotoplay: A psychological study. New York, NY:D. Appleton.

Munsterberg H. (2001). Hugo Munsterberg on film: The photoplay: Apsychological study and other writings. (A. Langdale, Ed.). New York:Routledge.

Murch W. (2001). In the blink of an eye: A perspective on film editing. LosAngeles, CA : Silman-James Press.

Neisser U. (1967). Cognitive psychology. New York, NY: Prentice Hall.

Newtson D. (1973). Attribution and the unit of perception of ongoingbehavior. Journal of Personality and Social Psychology, 28, 28–38.

Nishimoto S., Vu A. T., Naselaris T., Benjamini Y., Yu B., & Gallant J. L. (2011).Reconstructing visual experiences from brain activity evoked by naturalmovies. Current Biology, 21, 1641–1646.

Norman D. A. (1980). Twelve issues for cognitive science. Cognitive Science,4, 1–32.

Perkins V. (1974). Film as film: Understanding and judging movies.Cambridge, MA : Da Capo Press.

Page 36: 1 Psychocinematics Issues and Directions

Page 36 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

Rizzolatti G., & Fabbri-Destro M. (2010). Mirror neurons: From discovery toautism. Experimental Brain Research, 200, 223–237.

Robinson J. (2005). Deeper than reason: The emotions and their role inliterature, music and art. Oxford, England : Oxford University Press.

Rossell D. (1998). Living pictures: The origins of the movies. Albany, NY :State University of New York Press.

Salt B. (2006). Moving into pictures. London, England : Starword.

Sapolsky R. M. (1994). Why zebras don’t get ulcers. New York, NY: W. H.Freeman.

Sarris A. (1968). The American cinema: Directors and directions 1929–1968.New York, NY: Dutton Press.

Saxe R., & Wexler A. (2005). Making sense of another mind: The role of theright temporo-parietal junction. Neuropsychologia, 43, 1391–1399.

Saygin A. P., Chaminade T., Ishiguro H., Driver J., & Frith C. (2011). Thething that should not be: Predictive coding and the uncanny valley inperceiving human and humanoid robot actions. Social Cognitive andAffective Neuroscience, 7, 413–422.

Schwan S., & Ildirar S. (2010). Watching film for the first time: How adultviewers interpret perceptual discontinuities in film. Psychological Science,21, 1–7.

Sekuler R., Sekuler, A. B., & Lau R. (1997). Sound alters visual motionperception. Nature, 385, 308.

Shamay-Tsoory S. (2010). The neural bases for empathy. Neuroscientist, 17,18–24.

Shimamura A. P. (2002). Muybridge in motion: Travels in art, psychology, andneurology. History of Photography, 26, 341–350.

Shimamura A. P. (2008). A neurocognitive approach to metacognitivemonitoring and control. In J. Dunlosky & R. A. Bjork (Eds.), Handbook ofmetamemory and memory (pp. 373–390). New York, NY: Psychology Press.

Page 37: 1 Psychocinematics Issues and Directions

Page 37 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

Shimamura A. P. (2010). B ridging psychological and biological science: Thegood, bad, and ugly. Perspectives in Psychological Science, 5, 772–775.

Shimamura A. P. (2011). Episodic retrieval and the cortical binding ofrelational activity. Cognitive, Affective, and Behavioral Neuroscience, 11,277–291.

(p.26) Shimamura A. P. (2012). Approaching a science of aesthetics: Issuesand ideas. In A. P. Shimamura & S. E. Palmer (Eds.), Aesthetic science:Connecting minds, brains, and experience (pp. 3–28). New York, NY: OxfordUniversity Press.

Shimamura A. P. (in press). Experiencing art: In the brain of the beholder.New York, NY: Oxford University Press.

Shimamura A. P., Cohn-Sheehy B., Pogue B., & Shimamura T. A. (2012).Detecting targets embedded in movies: Multimodal influences on attentionacross edits. Manuscript submitted.

Shimamura A. P., Cohn-Sheehy B. I., & Shimamura T. A. (in press). Perceivingmovement across film edits: A psychocinematic analysis. Psychology ofAesthetics, Creativity, and the Arts.

Shimamura A. P., Marian, D. E., & Haskins A. L. (in press). Neural correlatesof emotional regulation while viewing films. Brain Imaging and Behavior.

Shimamura A. P., & Palmer S. E. (Eds.). (2012). Aesthetic science: Connectingminds, brains, and experience. New York, NY: Oxford University Press.

Simons D. J., & Levin D. T. (1997). Change blindness. Trends in CognitiveScience, 1, 261–267.

Singer T., Seymour B., O’Doherty J., Kaube H., Dolan R. J., & Frith C. D.(2004). Empathy for pain involves the affective but not sensory componentsof pain. Science, 303, 1157–1162.

Smith T. J., & Henderson J. M. (2008). Edit blindness: The relationshipbetween attention and global change blindness in dynamic scenes. Journal ofEye Movement Research, 2, 1–17.

Snyder B. (2005). Save the cat! The last book on screenwriting you’ll everneed. Studio City, CA : Michael Wiese Productions.

Page 38: 1 Psychocinematics Issues and Directions

Page 38 of 38 Psychocinematics: Issues and Directions

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2013.All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of amonograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: VrijeUniversiteit Amsterdam library; date: 26 July 2013

Solnit R. (2004). River of shadows: Eadweard Muybridge and thetechnological wild west. New York, NY: Penguin.

Thayer J., & Levenson R. (1983). Effects of music on psychophysiologicalresponses to a stressful film. Psychomusicology, 3, 44–54.

Watson J. B. (1913). Psychology as the behaviorist views it. PsychologicalReview, 20, 158–177.

Wertheimer M. (1912). Experimentelle Studien über das Sehen vonBewegung. Zeitschrift für Psychologie, 61, 161–265.

Wollheim R. (1980). Art and its objects (2nd ed.). Cambridge, England :Cambridge University Press.

Wundt W. (1874 / 1904). Principles of physiological psychology. (E. B.Tichener, Trans.). London, England : Swan Sonnenschein & Co.

Zacks J. M., Speer, N. K., Swallow, K. M., Braver, T. S., & Reynolds, J. R.(2007). Event perception: A mind/brain perspective. Psychological Bulletin,133, 273–293.

Notes:

(1) Rope was actually filmed using 10 shots lasting roughly 10 minutes each,though the transitions between these shots are hardly noticeable as eachcontinues the sequence in real time.