Pedro Rebelo 2006

URL: http://www.sarc.qub.ac.uk/~prebelo/teaching/sd

Sound Design

Sound in Cinema

We gestate in Sound, and are born into SightCinema gestated in Sight and was born into SoundWalter Murch (in Michel Chions Audio Vision)

From sonic acuteness to the explosion of senses Cinema: Mechanistic Reversal

Sound in Cinema

Silent Cinema (1892 - 1927) Sonic environment composed of live music (piano, organ), audience,

theatre sounds etc...

1927: Synchresis (Synchronism + Synthesis) (Chion) The spontaneous and irresistible mental fusion, completely free of any

logic, that happens between a sound and a visual when these occur at exactly the same time (Chion, 1994 p. xviii).

Sound in Cinema

Fabricating reality: actress on screen with different voices

From dubbed voice to hyper-real sound

Rendered Sound : more real than real (Chion)

Sound sources often do not correspond to actions on screen

Sound in Cinema

Both Marlene Dietrich and Greta Garbo did not speak a word of English when they began to work in Hollywood

Separation (by virtue of film practice) between what we see and what we hear

A layer of sound fertilises the image layer (not always by mimicking it)

Sound communicates physical qualities (material, weight, mass, volume, tension, force...)

Sound Design

Complex set of relationships in a synchronised audio-visual environment:

Sound Design

Visual stimuli focusing listening

Audio stimuli defining visual time

Discrepancy between audio and visual stimuli providing dual modality perception

Visual stimuli focusing listening

The panning of a video camera is able to focus our listening to a particular layer (e.g. orchestral concert broadcast)

Sound Design

Example: Elis Regina & Tom Jobim Aguas de Maro

Visual stimuli focusing listening

The panning of a video camera is able to focus our listening to a particular layer (e.g. orchestral concert broadcast)

Sound Design

Example: Elis Regina & Tom Jobim Aguas de Maro

Audio stimuli defining visual time

Sound conveys time information more accurately than visuals (e.g. the moment the gun is shot is primarily a sonic event)

Sonic acceleration and deceleration can modulate visual rhythm

Rdn#06 Underworld (onedotzero_select dvd - 07)Visual movement/speed is perceived in two different modes by virtue of two contrasting sound environments.

Sound Design

Discrepancy between audio and visual stimuli providing dual modality perception

Non-linear interaction between audio and visuals

Sound can be used to occlude expected events (e.g. parts of a dialogue). (Jean Luc Goddards Weekend)

Sound Design

8 1/2 Women, Peter Greenaway1999Chapter 1Opening credit sequence features a shot of nigh-time urban cityscape with the sound of birdsong.

Designing in Context

Designing in Context Sound Design for screen or visual media is context specific: Design

of connections as much as design of materials

John Cage / Merce Cunningham collaboration: releasing accidental connectivity

Contextual elements such as performance situation and particularly timing (duration) are agreed, whereas micro-structure events are left to chance

Designing in Context

Excerpt from Variations VJohn Cage / Merce Cunningham

John Cage made Variations V in 1965 for the Merce Cunningham Dance Company. He and David Tudor settled on two systems for the sound to be affected by movement. For the first, Billy Klver and his colleagues set up a system of directional photocells aimed at the stage lights, so that the dancers triggered sounds as they cut the light beams with their movements. A second system used a series of antennas. When a dancer came within four feet of an antenna a sound would result. Ten photocells were wired to activate tape-recorders and short-wave radios. Cecil Coker designed a control circuit, which was built by my assistant Witt Wittnebert. Film footage by Stan VanDerBeek and Nam June Paik's manipulated television images were projected on screens behind the dancers. The score was created by flipping coins to determine each element and consisted of thirty-five remarks outlining the structure, components, and methodology. The specific sound score would change at each performance as it was created by radio antennas responding to the dancers' movements.

http://www.medienkunstnetz.de/works/variations-v/video/1/

Designing in Context

Sound design for the screen, in its most inventive moments, suggests an approach in which image and sound interact by a counterpoint of editing and a process of layering that brings forward both sound and image at different times.

The exploration of provocative and unusual audio-visual situations is a primary concern in a wide range of film and new media practice. Mimicking is not enough...

Three Listening Modes (Chion, 1990)

Understanding the relationship between sound, its source and its perception...

Causal Listening

Semantic Listening

Reduced Listening

Causal ListeningListening to a sound in order to gather information about its cause or source

The most common listening mode

We recognise a persons voice

Without it necessarily corresponding to an image (radio presenter)

CATEGORIES of sources: animal, human, mechanic

MORPHOLOGIES of sound (causal history)Scraping = acceleration, changes of pressure, speed, amplitude, timbre

We recognise scraping as a sound producing action (without knowing the precise nature of materials)

In cinema, the identification of cause is dependent on: the audio-visual contract and the phenomenon of synchresis

how much water in the bottle?

Semantic ListeningListening as referring to a code of language

The interpretation of a message

At the centre of linguistic studies

Not about the acoustic properties of an individual event (phoneme)

but rather its contribution to a system of differences and oppositions

e.g. consonants, vowels, diphthongs...

what does that mean?

Reduced ListeningFocusing on sound itself, independent of cause or meaning

Pierre Schaeffer

Rooted in fixed sounds - recordingInfinite repetition directs our attention away from cause or meaning

Ear TrainingSimilar to traditional music ear training (identifying an interval between two notes)

Pierre Schaeffer Trait des object musicaux (1966)System of classification that suggests a solfeggio for recorded sound objects

Reduced listening, as a discipline allows for the sound designer to engage with the media

and explore the perceptual and acoustic richness in any sound

how does that sound?

Reduced Listeningand Acousmatic Sound

In an acousmatic listening situation the sound source is not visible.

The Acousmatics

Pythagoras disciples promoting lectures behind the screen

Obscuring the source would focus the audience on sound - the message

without the distractions of bodily presenceSound as primordial mode of communication (also theorised by Marshall McLuhan)

Acousmatic Music is defined by composer Francois Bayle as shot and developed in the studio, and projected in a hall, like a film (Bayle)

sound composed to be played over an array of loudspeakers

how does that sound?

Phythagoras FootstepsVision is typically enclosed by a stable frame (painting, photography, cinema, computer screen...)

it directs/crops our gazeIn sound there is no frame...

Analysis of sound in cinema refers to the role of the frame:

diegetic sound - belongs to actions in the framecharacters speaking, objects being moved etc...

non-diegetic - sounds outside the frame background music

Phythagoras Footstepsdiegetic sound - belongs to actions in the frame

characters speaking, objects being moved etc...

non-diegetic - sounds outside the frame background music

http://www.stanford.edu/~gdegroat/PF/pictures.htm

Phythagoras Footsteps

Numerous examples problematise this categorisation.In particular, technological presence in film:

radio, computer voices, recorded conversations...

Chions terminology: Offscreen - not visible but part of the frame

Onscreen - visible action for sound

Non-diegetic - non visible

Michel Chion Audio Vision

Phythagoras FootstepsChions extended terminology:

Ambient Sound (Territory-Sound)sound that envelops a scene and inhabits its space without raising the question of the identification or visual embodiment of its

source (Chion, p.75)sound as a way of identifying a particular space or situation (church bells, birdsong, etc...)

Internal SoundSituated in the present action, physical and mental interior of a character

internal voices, bodily sounds...

On-the-Air SoundTransmitted electronically onto the scene

Phythagoras Footsteps

Dimensions and Oppositions in film sound:

Opposition between acousmatic and visualised

Opposition between objective and subjective or real and imagined

Difference between past, present and future

Michel Chion Audio Vision

Elements in a Soundtrack

AmbienceTriggers of motion

Evocation of space or situation through soundDependent on references (non-narrative)

e.g. voice whispers, whistling wind...

Earcons Imply an immediate response (e.g. Carl Stalling)

User interface sounds... not spatially defined, somewhat arbitrary

No Soundtrack

Sound element subsidiary to narrative (or film structure)Integrated with visuals to produce one single effect

The relationship between sound and image often changes throughout a film; they become so intertwined that it becomes impossible to separate them and still treat either element as a

coherent entity.

Forced Marriage

Exercise: Source Library

Sound Object Groups:

A: FootstepsB: Movement C: Environments (indoors and outdoors)D: Impact and CollisionE: Sound Events from Analysis Files

In preparation for the next two assignments the class will work as a group in recording and editing sound sources

1. Record as a group (Experiment!)

2. Divide up Editing Tasks

3. Catalogue all edited files as appropriate

4. Present Sound Library to Class

Exercise: Source Library

Audio-Visual Analysis

Itemize the different audio elementswhich is dominant and what point?

Characterise consistency

how do the different elements interact?balance, reverberation, masking

Points of Synchronisationpoints that are crucial for meaning and dynamics

Sound-Image Comparisonformal relationships in speed, materials, definition, distance, scale

technical comparison between camera movement and the soundtrackcompare by observing audio and visual elements separately

What do I see of what I hear?What do I hear of what I see?

Metaphor through SoundAre there any metaphorical relationships articulated specifically through sound?

Excerpts for Analysis:

Artificial Intelligence (S. Spielberg 2001)Delicatessen (Jeunet & Caro, 1991)Russian Ark (A. Sokurov, 2002)