This article was downloaded by: [University of Arizona]On: 18 December 2014, At: 22:47Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Journal of Media PracticePublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/rjmp20

From the Semiotics of Sight to theSemiotics of SoundPaul MoorePublished online: 11 Mar 2014.

To cite this article: Paul Moore (2000) From the Semiotics of Sight to the Semiotics of Sound, Journalof Media Practice, 1:3, 148-156, DOI: 10.1080/14682753.2000.10807088

To link to this article: http://dx.doi.org/10.1080/14682753.2000.10807088

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

1 For example, the issue on 'Cinema/Sound', Vol. 60, 1980, Yale French Studies

From the Semiotics of Sight to the Semiotics of Sound: Teaching the creation of Soundscapes

Paul Moore

Abstract This article argues that the study of sound has been neglected in relation to visual imagery. Despite the potential of sound and radio for creative and challenging produc­tion/art work (a potential recognised by many of the avant-garde artists of the twenti­eth century), there has been a tendency to teach sound as a complement for visual production or through the accepted radio conventions. This article uses the work of R. Murray Schafer and Thea Van Leeuwen as the underpinning theory in the develop­ment and description of practice work that encourages students to think about sound as a signifying practice. Examples of the practices involved in this process are explained and analysed. Finally, the article makes the case for using soundscape con­struction as a pedagogical tool capable of addressing the problem of bringing students to theoretical understandings through practice.

In a paper delivered to the Hearing is Believing 2 Conference held in Sunderland in 1996, Richard Thorn quoted Kahn and Whitehead when they complained that:

It remains almost unheard of to think about sound ... the deafening silence of sound-in-thought is mirrored by the absence of anything remotely resem­bling a coherent tradition of audio art'.

Thorn

Anyone involved in the teaching of media practice will be aware of this silence. Students who will take the utmost care over the selection and creation of appropriate visual images will often combine them with sounds which are obvious, cliched and, ultimately, inappropriate. Indeed, when sound has been given detailed and sustained analytical attention, it has often been in the context of cinema sound. 1 There is also a tendency for students to assume that sound must be synonymous with music, and a musical text will be used to 'enhance' a piece of visual work with scant regard to significations which the music may in itself carry and which may, at times, contradict rather than com­patibly coexist with the imagery.

Even when students decide to use radio as their medium of production, the creative aspects of their work are stifled by a drive to produce texts which are locked into the conventions of radio production, be they drama, feature or documentary. This deference to convention is fuelled by an industry that is often itself wary of experimental work, as this personal experience indicates. I work freelance as a radio presenter on BBC Radio Ulster. At a recent training day being headed by a senior BBC programme commissioner, I was asked to record two minutes of material offering an account of life in the front foyer of

148 JMP 1 (3) 148-156 ©Intellect Ltd 2000

Dow

nloa

ded

by [

Uni

vers

ity o

f A

rizo

na]

at 2

2:47

18

Dec

embe

r 20

14

the BBC building in Belfast. This foyer is inhabited by an automatic revolving door, two automatic turnstiles, a lift and one very bored security worker I doorman/Jobsworth. I recorded two minutes called Ennui. It con­sisted of the sounds of the door revolving, the lift operating with its disem­bodied 'voice', the creak of the turnstiles and the bored but polite responses of the security man to telephone calls. On hearing the piece the trainer was livid with indignation and accused me of trying to undermine his work. Ultimately, I had to explain myself to this senior BBC manager, who made it clear that work of this type would not find a home on network radio.

It is also significant that the study of sound is fragmented, conceptually and in curricular terms, with students being offered courses in Audio, Sound or Radio, the underlying message being that these elements do not easily sit alongside one another, one being related to technicist construction, one to musical creation and aural design and one to the broadcasting industries. It is also linked to the silence in contemporary debate about the history of sound as art. Many avant-garde artists in the early years of the twentieth century saw sound and radio production as an exciting opportunity for artistic expression, but the work of Marcel Duchamp, Marinetti and, more recently, William Burroughs has unfortunately been engulfed by the commercial imperative and its conventions. The work of Douglas Kahn (1992) tracing what he calls 'the figures of vibration, inscription and transmission' in the avant-garde empha­sises the need to address this silence. This fragmentation is then, I would argue, artificial, contrived and linked to industrial and employment practice rather than to any formal distinctions inherent in sound. This article grew out of a frustration with the traditional western cultural bias which privileges the visual over the aural and it tries to outline both theoretical and practical ways in which this bias can be challenged, arguing, ultimately, that the cultural meaning of sound deserves the same analytical attention as imagery.

It is with the concept of sound that many of the initial problems arise. Many students see sound in terms of discrete entities such as radio pro­grammes, musical texts (pop or otherwise), or particular effects rather than as a whole which creates meaning in similar ways to the visual landscapes which they are often adept at deconstructing. Sound is not these discrete pieces. It is the meaning made by their coexistence. One way to approach this acoustic whole is through the idea of immersion. Van Leeuwen argues that:

The trend in communication is now towards immersion rather than detach­ment, towards the interactive and the participatory rather than towards soli­tary enjoyments, towards ever-changing dynamic experiences rather than towards the fixing of meanings as objects to be collected.

Van Leeuwen, p. 197

An examination of the components that create this immersion in sound encourages an understanding of the differences between sound and visual images. To be immersed is to be part of. Visual images operate through a sharp distinction between subject and object. The image object is something that is examined and controlled from a distance by the observer, the subject, who then decides what the meanings and qualities of the image are. Foucault refers to this process as 'panopticism', the means by which people and things are

From the Semiotics of Sight to the Semiotics of Sound 149

Dow

nloa

ded

by [

Uni

vers

ity o

f A

rizo

na]

at 2

2:47

18

Dec

embe

r 20

14

controlled through the gaze of eyes they never see (Van Leeuwen, p. 195). Sound, on the other hand, envelops the subject, involving the immersed hearer in a lived experience, an experience that cannot be repeated or analysed at a distance. Thorn describes the way in which societies which value hearing over seeing- for example the Kaludi of Papua New Guinea- not only measure the extent of landscape boundaries according to the relationships between given sounds, but also use non-linguistic sounds to 'name' birds and animals encountered in their forest environment. For them, hearing is believing.

It follows that sound is much more difficult to shut out than visual images. John Cage describes his confusion, a mixture of fear and excitement, on being in an anachoiac chamber for the first time, an experience which led to the 'composing' of '4.33' - the sound of silence. This insistence, or non-eludible quality, of sound is also the basis for most ambient music. It is a form which, according to Nicholas Gebhardt, exists between music and sound:

Listen ... and so it happens: laughter, a sea breeze, the grate of metal on bone; listened to through hearing of the ear, each sound becomes the undoing of a rhythmic knot that anticipates the almost of each tiny resonance; that uncov­ers its own translucent surface.

Sound in a sense colonises the listener, making it impossible to 'look' away or ignore the meaning being created. Any active attempt not to listen will para­doxically ensure that the sound is being focused on, a strategy that underlines further the feeling of colonisation. As Murray Schafer suggests, we have no ear-lids.

With this colonisation, this 'tilling' of the landscape, comes 'depth'. Vision (despite the advent of 3D technology) only portrays the surface of things, a practice acknowledged by television through its embracing the virtual surfaces of digital imagery and graphics. The point of semiotics is to find the underpin­ning meanings, the significatory processes lurking under the surface. The semi­otician investigates, probes, and unpacks in order to arrive at an explanation and an understanding. Sound, however, has its own depth built in.

We can hear the hollowness of the wall, or the heart within. And indeed, our own body, rather than staying outside of what it perceives, itself resonates with what it hears, and hears itself, further diminishing the opposition between inside and outside, surface and depth.

Van Leeuwen, p. 196

It follows from this that sound, unlike sight, has no directionality. It does not have to be actively 'looked' at but places the hearer as part of an environment with which she/he becomes one.

Finally, and most crucially, sound cannot be frozen. It cannot be made still for examination. Technology may allow the constant repeating of a sound but on each occasion the aural landscape will have altered and the meaning will have changed, a fact often ignored by those who replay images believing the impact is the same each time it is seen (but the further arguing of which is beyond the scope of this paper).

150 Paul Moore

Dow

nloa

ded

by [

Uni

vers

ity o

f A

rizo

na]

at 2

2:47

18

Dec

embe

r 20

14

Soundscapes Having noted that at any given moment each of us is constantly exposed to sound, the next step is to construct a conceptual framework that will facilitate mapping of this process. This is offered through what has come to be known as 'Soundscape', a term first used by R. Murray Schafer to describe the way in which a variety of noises come together in context to suggest a particular meaning or set of meanings. Schafer is a music teacher who became fascinated with the way sounds of everyday life express cultural meanings and evoke the same responses in a variety of listeners in a range of contexts. For Schafer this process resembled the way in which arbitrary signs take on the same visual meaning in given societies. The problem for Schafer (and others following his example) is how to design the tools necessary to deconstruct these sound­scapes without imposing an inflexible set of rules or regulations that will undermine the dynamics of the soundscape.

A semiotics of sound should describe sound as a semiotic resource offering its users a rich array of semiotic choices, not as a rule book telling you what to do, or how to use sound 'correctly'.

Van Leeuwen, p. 6

One of Schafer's methods is to use or redeploy the student's knowledge of visual semiotics as a means of provoking thought on how these images would be produced as sound. While Schafer would maintain that these 'exercises' are not especially complex and can be labelled as sound poems, voiceprints or onomatopoeic play, this description fails to do justice to the complexity of the soundscape skills the students are being asked to exhibit. It also fails to recog­nise the contribution these analytical techniques can make to encouraging student thinking about soundscapes. One example will suffice to illustrate the process involved in this work. Schafer takes a Marinetti poem (Schafer, 1970, p. 23) and asks his students to turn it into a solo or choral composition. This is evidently a very focused musical activity. However, Schafer uses the knowl­edge gained in these exercises to confront his students with more complex analytical tasks based on the concept of the New Soundscape (Schafer, 1969). Sometimes he uses photographs, sometimes his own drawings but in each case the object of the assignment is to identify or trace meaning through an examination of the soundscape:

Such then are the emphatic leitmotives of the world symphony: aircraft, amplified guitars, the sounds of warfare and power machinery. These are the big blocks of sound, the flat lines of sound, the lethal weapons which now dominate the composition. They demonstrate the crudity of its orchestration.

Next the lesser leitmotives: the ubiquitous radios and television sets, the sounds of street traffic, the telephone (which Lawrence Durrell describes in Justine as 'a small, needle-like sound') the sounds of plumbing, of furnaces and air conditioners. These are the jabber-sounds.

Schafer, 1969, p. 61

It could be argued, however, that the one area Schafer does underestimate is

From the Semiotics of Sight to the Semiotics of Sound 151

Dow

nloa

ded

by [

Uni

vers

ity o

f A

rizo

na]

at 2

2:47

18

Dec

embe

r 20

14

2 'The World Forum for Acoustic Ecology' www.interact.uorego n.edu/MediaLit/WF AEHomePage.

the ability of technology to control the soundscape. The use of the Walkman for example (particularly if the piece being listened to has been created by the participant for this medium) leaves the participant less open to the tyranny of contemporary 'noise' in any environment, noise which Schafer is convinced is the enemy of the human voice. The new technologies of sound recording (hardware and method) can allow students to take their understanding of the new soundscape a stage further into the sphere of production, where alterna­tive (challenging!) soundscapes of their own are manufactured. Ian Chambers offers a wonderful, and perhaps ubiquitous, example of this act of soundscape creation when he writes about taking an 'aural walk'. This aural walk is possi­ble because of the Sony Walkman, the gadget based on an idea that appropri­ately came to Akio Morita as he walked through the soundscape which is New York. Chambers maintains that the meaning of the Walkman does not reside in itself but in the 'extension of perceptive potential' which it allows. Significantly, Chambers sees the Walkman as 'a semiotic shifter, the crucial digit in a particular organisation of sense' (Chambers, p. 50). The paradox of the Walkman for Chambers is the fact that the privacy of the act of listening through headphones actually reaffirms participation in a shared environment, but it is a participation dictated by the participant, dependent on a soundscape created by her/him within the surrounding external soundscape:

Each listener I player selects and rearranges the surrounding soundscape, and, in constructing a dialogue with it, leaves a trace in the network.

Chambers, p. 50

The Walkman is therefore a technical instrument and a cultural activity but, more importantly in this context, it is an illustration of the ways in which anyone can supplant the technology of space by the technology of time, with journeys measured not through the passing of city grid networks, but with the passing of sound time in a set of headphones. For it (the Walkman) permits the possibility, however fragile and

however transitory, of imposing your soundscape on the surrounding aural environment and thereby domesticating the external world: for a moment it can be brought under the 'Stop/Start', 'Fast Forward', 'Pause', 'Rewind' buttons.

Chambers, p. 51

One of the key platforms for the development of soundscape theory has been the World Forum for Acoustic Ecology. Founded in 1993, this international association of affiliated organisations and individuals attempts to advance interest and research into the scientific, social and cultural aspects of natural and manufactured sound environments. Students can access their work through a website that offers soundscape examples, online articles by leading researchers in the area and links to related archive materials.2 In relation to the United Kingdom the website offers access to the Touring Exhibition of Sound Environments, an exploration of soundscapes across England, Wales, Scotland and Northern Ireland. One of the contributions to this site encourages listeners to undertake a sound survey consisting of ten questions. These ten questions

152 Paul Moore

Dow

nloa

ded

by [

Uni

vers

ity o

f A

rizo

na]

at 2

2:47

18

Dec

embe

r 20

14

establish the idea that sounds in an environment directly or indirectly evoke meanings and this exercise can act as a basis for confronting students with the need to deconstruct sounds and their impact.

Most of these references are interested purely in the recording and exami­nation of soundscapes as environments. The key question in relation to sound production is to find a means whereby soundscape theory can be harnessed as a tool to enhance student practice.

Work of this type involves asking students to make the conceptual leap from pure soundscape production as ambience into the manipulation of soundscape theory to enrich production. The work of Van Leeuwen is impor­tant in this process since he argues that the components of the soundscape can be identified in terms of perspective and social distance, terms with which stu­dents of visual semiotics are already acquainted. These categories can be made explicit and used to construct soundscapes that can underpin a piece of sound production. Hence in relation to perspective Van Leeuwen argues that aural perspective divides sounds into groups, and places them at different distances to the listener 'so as to make the listener relate to them in different ways' (Van Leeuwen, p. 22). These groups can be labelled 'figure', 'ground' and 'field' and in a particular context with a particular structure they can be manipulated to create meaning. The group labels can be used, for instance, to instruct the recording engineer as to how the aural meaning should be placed in relation to dialogue. These instructions ensure that the appropriate sounds are positioned 'correctly' in the aural landscape.

Social distance can also be grouped across a scale according to where the sound is emanating from. This scale ranges from intimate distance to public distance and the sophistication of contemporary recording technologies allows both social distance and perspective to be independent variables, and to be mixed according to production needs. One way students can be introduced to this process is through techno music since it relies on a mixture of variables overlaid to produce specific sound meanings. William Orbital's 'Ten Pieces in a Modem Style' is one good example since it is accessible to those who have no experience of techno music.

These techniques work particularly well in recorded monologues. While the action in a monologue is conveyed by the key speaker, sounds can be used to position the listener. The sounds available, including the lead voice, are itemised and given a perspective (figure, ground, field) and a social distance (close, mid, far). The careful construction of perspective and distance patterns brings the listener into the monologue so that sounds 'illustrate' the verbally told story and challenge the conventional in/ out sound fades which are usually used only as bridging sequences.

The Production Assignment It is from this theoretical grounding that teaching strategies for soundscape production have been developed in the School of Media and Performing Arts at the University of Ulster. The process has three phases. Initially the students are asked to create a visual essay. Examples from John Berger (1972), or more recently Steve Redhead (1997) are used to explain exactly what is required. The visual essay produced by the student can be constructed from found or created

From the Semiotics of Sight to the Semiotics of Sound 153

Dow

nloa

ded

by [

Uni

vers

ity o

f A

rizo

na]

at 2

2:47

18

Dec

embe

r 20

14

images and each visual essay is accompanied by a commentary. This allows the student to articulate an understanding of visual semiotics.

Having completed the visual essay as the first part of the assessment process, the soundscape theory is taught through both lectures and seminars. The seminar time is particularly important since it creates the space for a detailed examination of figure, ground and field and the way these categories can be arranged to delineate perspective and social distance.

The students are then asked to produce an aural version of their visual essay. Two examples of student work will illustrate the different outcomes the application of soundscape theory can provoke. The first is a radio play called 'Freedomisnowhere', produced by a third year undergraduate student. (This piece won an RTE award for the best student production that year). In this play the central character has no voice, having lost it after her lover was killed in an aeroplane crash. This character communicates in two ways -by listening to the soundscapes created by those around her and by describing through her mind's voice, which only she hears, the construction of the soundscapes sur­rounding her and their impact on the environment. The aural landscape con­structed in the course of the piece is only shattered at the end when through hearing her lover's voice on an answering machine she had misplaced on the evening of his death, she reclaims her speech, but loses the capacity to live within a constantly changing soundscape. The theoretical overview, which the student provided for this piece, gave an explicit mapping of the recording structured according to figure, ground and field perspective and social dis­tance.

The second piece is an experimental soundscape produced by an MA student who had done no practical work of any kind before joining the pro­gramme. The student was interested in challenging linear narrative through sound and in producing meaning without dialogue. The piece describes through abstract sound a day in the life of a character, with only one key moment of verbal intercourse being used to pull the sound strands into an identifiable narrative. The most striking aspects of this production are the ways in which the altering of sounds through technology gives them a sinister aspect and the blending of found and manufactured sounds create meaning. The found sounds were sampled from RTE radio archives and the juxtaposi­tioning of modem and nostalgic sounds create unease since the listener is at once confused and reassured.

These short sound productions indicate three crucial points. First, in each case the student has attempted to apply the concepts of aural perspective and social distance in ways that develop meaning without explicit linguistic direction. The meaning is evoked through sound. Second, the listener is drawn in and immersed in the sounds of the production. There is not an easily identifiable point of listening. The listener is guided through a network of sounds where meaning is only evident through connection to another sound. Third, there is an understanding that, as with visual semi­otics, the meaning is dependent on a number of factors relating to environ­ment, memory, culture and personal context, and hence the action of the central players is only one of an array of interlocking, meaning-making influ­ences. The semiotics of sound, therefore, like vision is concerned with every­thing that can be taken as a sign.

154 Paul Moore

Dow

nloa

ded

by [

Uni

vers

ity o

f A

rizo

na]

at 2

2:47

18

Dec

embe

r 20

14

A sign is:

everything which can be taken as significantly substituting for something else. This something else does not necessarily have to exist or to actually be somewhere at the moment in which a sign stands for it. Thus semiotics is in principle the discipline studying everything which can be used to tell a lie.

Eco,p.7

Teaching Implications There are a number of implications, positive and negative, of advancing this approach to the soundscape in media practice teaching. The most obvious is that the study of sound will have to be given equity with the study of visual imagery. This parity does not, however, simply mean instruction in the art and science of radio production. It means, rather, finding teaching strategies which can challenge the conventions of radio production, and seek new methods of representing meaning through sound. As we have seen this will not, necessar­ily, be welcomed by the industry.

The development of soundscape-informed production also offers one strat­egy for approaching the most important challenge facing media studies today: how to stop clinging to what Tereza Batista calls the 'illusion of the compre­hensively all-embracing univocal work ... without surrendering to random fragmentation, collage, cut-and-mix assemblages that lead us nowhere' Having taught media studies for twenty years Batista comes to the conclusion that much of what we call media studies analysis is actually an overview of our media past which can only give students a foundation or cultural ground­ing. On this grounding it is necessary to build media practices which assist students to grasp the fragmentation present in daily media activities and to undertake contextual practical work which combines technical proficiency, principles of organisation and the cutting, sampling and mixing of original and found materials to produce a product that stems from a collaborative and reintegrative pedagogical practice.

One key to this practice is an acceptance that in the realm of sound the students may, in fact, have a greater understanding of the semiotic complex­ities which structure contemporary soundscapes. The work students produce, therefore, contains certain innate profundities, many of which they are not necessarily aware of producing. These emerge because the students experience a cultural resonance with the environment that produces it. The role of media practice teaching is to bring the student back from this advanced (but unknowing) cultural experience to a theoretical understand­ing of what is being produced and its significance. This, ultimately, is the real importance of media practice- the ability to link cultural and contextual cre­ativity with the theoretical grounding which underpins it, knowingly or oth­erwise.

In the final analysis this is a pedagogic practice which encourages students to take responsibility for reading, learning and representing the arbitrary, senseless fragmentation that is contemporary culture creates coherence out of chaos and links the public and the private spheres of a student's life. As the narrator in John Akomfrah's Last Angel of History says:

From the Semiotics of Sight to the Semiotics of Sound 155

Dow

nloa

ded

by [

Uni

vers

ity o

f A

rizo

na]

at 2

2:47

18

Dec

embe

r 20

14

If you find a crossroads, any crossroads, this crossroads, if you make an archaeological dig into this crossroads, you'll find fragments - technofossils­and if you can put those elements, those fragments together, you'll find the code. Crack that code and you have the key to your future.

Acknowledgements This article was developed from a paper originally delivered to the Association of Media Practice Educators in September 2000. I would like to thank all those who offered advice and encouragement at the conference.

I would particularly like to acknowledge the help and advice offered by my colleague, Dr. Dan Fleming at the University of Ulster, who underlined my awareness of the importance sound study has in the development of a coher­ent media practice pedagogy and introduced me to a number of sound resource centres.

References Berger, John. Ways of Seeing, London: Penguin, 1972.

Chambers, lain. Migrancy, Culture, Identity, London: Routledge, 1994.

Eco, Umberto. A Theory of Semiotics, London: Penguin, 1976.

Gebhardt, Nicholas. 'The Alchemy of Ambience', www. uiah.fi I bookshop I isea_proc I spacescapes I j I 19 .html

Kahn, Douglas and Whitehead, Gregory. Wireless Imagination - Sound, Radio and the Avant-garde, London, MIT Press,1992.

Redhead, Steve. The Clubcultures Reader, Oxford: Blackwell,1997.

Schafer, R. Murray. When Words Sing, Ontario: Berandol,1970.

-------The New Soundscape, Toronto: Berandol, 1969.

Thorn, Richard. 'Hearing is Believing- The Evidence', www.ukc.ac.ukl sdfval sound­journal I thom981.html

Van Leeuwen, Theo. Speech, Music, Sound, London: Macmillan, 1999.

156 Paul Moore

Dow

nloa

ded

by [

Uni

vers

ity o

f A

rizo

na]

at 2

2:47

18

Dec

embe

r 20

14