OPEN FORUM

Sonification: the element of surprise

Stuart Jones

Received: 28 October 2010 / Accepted: 9 August 2011 / Published online: 14 September 2011

� Springer-Verlag London Limited 2011

All composers work within constraints. Even such an

absolutist control freak as Wagner never wrote a top D for

a Heldentenor because he knew the outcome wouldn’t be

great. I would go further and say that most composers

(certainly since Monteverdi) actually enjoy the stimulus of

the constraints presented, for example, by the nature of

particular instruments and the way they are played, even by

particular players. The constraint is an opportunity, a

source of inspiration. At the same time music that is per-

formed is a provisional art, there is no definitive execution

of any work. Most composers embrace this lack of absolute

control, and enjoy the fact that their pieces are interpreted.

These are the conventional ‘limitations’ that arise from

the composer–notation–performer matrix. However, some

composers enjoy constraints and handing over of control in

the compositional process itself, and I am among them. I

enjoy structural and processual constraints in composition,

and I particularly enjoy handing over control of all or

aspects of the compositional process, for the simple reason

that doing so produces results which I could not have

imagined. I am also interested in how a musical piece may

relate to the environment it inhabits, and all these three

have drawn me to sonification.

I have discussed an early piece of mine, Meter, in my

essay about real-time sonification in this journal. In that

piece I used sonification in a performance context. I would

now like to talk about two other sonifications, one which

produced material for a composition, the other which used

sonification to reveal states in a building to its inhabitants.

1 Chesterfield starfield

This piece was written in 2000 to celebrate the opening of a

new city centre. Historically, Chesterfield had been a centre

of the coal mining industry, and there remains a proud

tradition of brass band music making, which comes from

that industrial tradition. The playing standard of these local

brass bands is often of a very high order. I decided early

that I would take advantage of this wonderful resource, and

wanted to use bands both marching through the town and

playing together in the new centre. I also wanted to rec-

ognise the mining tradition, and it was through this that I

came to the (perhaps apparently counter-intuitive) decision

to base the music and performance on the location of stars.

I calculated the position of the stars above Chesterfield at

the time of the performance (which was 11.00 am, so the

stars would be invisible) and used the resultant star map to

generate all aspects of the performance.

I was glad to find the small constellation Triangulum at

the centre of this map. Its shape (an elongated isosceles

triangle) echoed the symbol of Chesterfield, a spire, and it

gave me the first parameters of the piece—that it would be

for three bands, and that in the central performance space

their layout would replicate this shape. The routes for the

bands to march along were worked out by superimposing a

map of Chesterfield on the star map. The musical material

for both the marches and the central piece was derived by

superimposing graph paper aligned with the three sides of

Triangulum. Strands of rhythmed pitches were derived

using algorithms using the magnitudes and distances of the

three stars that Triangulum consists of (a Trianguli (Caput

Trianguli), b Trianguli, 6 Trianguli) as factors. These

strands were disposed among the instruments of the bands

using criteria based on various factors such as range and

transposition.

S. Jones (&)

20 Cleveleys Road, London E5 9JN, UK

e-mail: wawamoz@gmail.com

123

AI & Soc (2012) 27:297–298

DOI 10.1007/s00146-011-0352-4

There is an unanswerable question in relation to this

piece and ones based on similar principles, and it is this:

what difference does a knowledge of the roots of the piece

make to how you experience it? In this case the knowledge

was available to the audience, and it was important to me

that they had the opportunity to consider it. I wanted there

to be a reflective context; indeed, the motto of the piece is

‘‘the stars above us, the earth beneath our feet’’. But almost

certainly many in the audience didn’t know this back-

ground, and I hope and believe that didn’t impair their

enjoyment of the music. I also enjoy the fact that such

unanswerable questions exist, as I enjoyed the fact that the

marching of the bands brought different parts of the music

into different conjunctions of direction and distance that

were totally unpredictable.

2 Bop!—making sense of space

In this project, on the other hand, it was essential that

people were able to make a clear connection between what

they were hearing and the data it represented. Bop! was a

multi-disciplinary research project with participants from

computer science, engineering and art and design. It had

several aims, but intrinsically it set out to use a Wireless

Sensor Network distributed throughout a building (a design

and research centre) to sense data both environmental

(temperature, light level, atmospheric pressure etc.) and

human occupancy (presence, number of people, level of

activity etc.). These data were sent to a relational database,

from which they could be read in real time, with a little

latency. In parallel with this system was a series of inter-

active devices which gave inhabitants opportunities to

register their feelings. Thus changes in environmental and

human conditions could be tracked against people’s moods

over an extended period (4 weeks in the final test).

An important part of the project was feeding the data

back to the inhabitants, both to contextualise their feedback

through the interactives and to give them a sense both of

the reality of the (unseen) spaces around them and of their

role in the project. Because of this, lucidity in the data

representation and engagement of participants were of high

importance. The data were fed back, in real time, with data

visualisation and sonification. Creating the sonification was

one of my roles in the project.

The representations, both visual and aural, were of a

mixture of environmental data and human activity data.

These were being collected throughout the building, in

locations invisible to the participant and often remote from

them. The strategy for the data visualisation was extremely

simple: the data from each room, represented numerically

and graphically, scrolled across the screen; the participant

could intervene to view the ‘over time’ data for a particular

space.

My strategy for the sonification was based on the prin-

ciple that hearing and vision work in tandem, and that

while vision is focused on a particular object, hearing will

be attending to everything coming from every direction. I

therefore used speaker layout and sound dispersal tech-

niques to give the sense that sounds were coming from the

direction of their originating data, and seemed to be at a

distance proportional to their distance.

The aesthetic decisions about what sounds to use and

how they would change with data change were founded

more on design than artistic principles. That is, the primary

purpose was not aesthetic pleasure or ‘beauty’ but trans-

parency of representation. I wanted my aesthetic decisions

to bring the listener to a very strong sense of the data, how

it surrounded them, where it was coming from, how it was

different in different places. This was not intended to be

read quantitatively (‘it’s 23�C and rising in that room’) but

qualitatively (‘it’s warm in that room and getting warmer,

and there are lots of people in it so I suppose that’s not

surprising’). It was intended to give them a sense of the

unseen surrounding space as a living, changing presence,

and the opportunity to relate that sense to a more analytical

reading provided by the data visualisation. It also gave

them the opportunity to explore the data mapping, by

controlling and filtering what they were hearing, both in

relation to place (only listening to one space) or parameter

(only listening to temperature). This possibility of explo-

ration was intended to draw them into a closer relationship

with the data and thence the space itself. As for my own

satisfaction as a composer, apart from the challenge of

balancing complexity, legibility and ‘listenability’, there

was the pleasure of surprise—there was no way of knowing

what the conjunction of sounds would sound like at any

moment.

I would say that it is this last that draws me most to

sonification: the element of surprise. In that sense, for me

sonification is a device or strategy rather than an end in

itself, and parallels other strategies I use in other pieces to

achieve unpredictability. However, the fact that it does so

by relating directly to changing aspects of the world in

which the piece exists is of particular importance to me,

especially if, as in Bop!, the sonification is happening in

real time. I have always seen music as an art of transience,

of performance—with these sounds happening now in this

place amongst these people, never to be repeated. To do

this by tapping directly into the flux that the music is part

of is for me particularly satisfying, a closing of the circle.

298 AI & Soc (2012) 27:297–298

123