1

University of Adelaide

Elder Conservatorium of Music

Research Project

Submitted in partial fulfilment of the requirements for the degree of

Bachelor of Music with Honours

Evolutionary and Adaptive Algorithmic Composition

Submitted by

Simon Harley Koehn a1667613

Adelaide, June / 2017

2

CONTENTS

Abstract 3

Declaration 4

Acknowledgements 5

Introduction 6

Chapter 1 – Algorithms for the Generation of Musical Structure 8

Chapter 2 - Three Examples of Evolutionary and Adaptive Composition 13

Chapter 3 - Creativity, Authorship and Compositional Limitations 16

Conclusion 19

Bibliography 20

3

ABSTRACT

Algorithmic composition refers to the generation of musical content by means of a set of rules. This includes the application of two related categories of computation, the use of evolutionary, and adaptive algorithms. This paper provides an overview of the field of evolutionary and adaptive algorithmic composition. It presents a brief history and summary of algorithmic composition in general, and describes key techniques of evolutionary and adaptive computation that have been applied to composition. Three contemporary works of evolutionary and adaptive composition are examined in order to develop a sense of the field's scope. The feasibility of modelling creativity in algorithmic procedures is also discussed.

4

DECLARATION

The candidate declares that the material contained in this submission is his/her own work and that appropriate recognition has been given when referring to the work of others.

Signature:........................................

Simon Koehn

Date: 13/6/2017

5

ACKNOWLEDGEMENTS

Thanks to my supervisor, Christian Haines, for his steady guidance, as well as to Adele Sliuzas and Oliver Koehn, for invaluable editorial and moral support.

6

INTRODUCTION

Creativity is an indisputable element within the field of music. The creativity of a composer or musician is a key component of their practice. However, In the field of evolutionary and adaptive algorithmic composition, creative tasks are deferred to a complex set of rules. In order to do this, creativity must be expressible algorithmically.1 This presents a challenge to the composer, most obviously in a practical sense, but also philosophically as evolutionary and adaptive (E/A) approaches have implications for our understanding of human creativity. Composer and researcher David Cope addresses these implications directly, attempting to break down creativity into its essential elements, and integrate them into a larger system.2 In response to the assertion that a computer can not be truly creative, Cope suggests that if we can not create such a machine, then ‘humans are not in fact very creative after all’.3 Parallel to Cope’s argument, computer scientist Marvin Minsky asserts that consciousness can be explained by many smaller, simple processes interacting with each other.4 While it seems dramatic that creativity could be modeled by algorithmic processes, recent research and technological developments pave the way for E/A composition to push the boundaries of computational creativity. While there is some variation in definition of the word algorithm, algorithmic composition (AC) broadly refers to ‘composing by means of formalizable methods’.5 This includes the application of basic rules to the compositional process through to the use of specific mathematical equations to generate musical content. This paper focuses on two related categories of algorithmic composition: the use of evolutionary, and adaptive algorithms.6 Evolutionary computation refers to the simulation of natural systems to generate data, drawing inspiration from Darwinian evolution.7 Adaptive algorithms such as cellular automata and artificial intelligence systems make use of complex interactions between large number of “agents” to determine their output.8 These two categories are closely related by their ability to generate output that has not be explicitly described in the system. Examples of algorithmic composition appear well before the modern computer, notably in the works of composers such as Bach and Schönberg.9 In these cases, calculations were performed manually before notating the outcome. While mathematical developments have propelled algorithmic composers into new areas, it is the computational power of programmable computers that provides significantly greater possibilities. Millions of calculations can be made within seconds, allowing for more complex algorithms, the generation of more content, and the possibility for calculations to be made in response to streams of data in real time. It is the introduction of the computer, and increases in the processing power of current day computers, that have made the techniques of evolutionary and adaptive algorithms possible, and particularly in the case of artificial intelligence, continue to expand the potential of these processes.

1 Gerhard Nierhaus, Algorithmic Composition (Vienna: Springer, 2009), 1. 2 David Cope, Computer Models of Musical Creativity (Cambridge: The MIT Press, 2005.), viii. 3 Cope, Computer Models, 6. 4 Marvin Minsky, The Society of the Mind (New York: Simon and Schuster, 1985), 17. 5 Nierhaus, Algorithmic Composition,1. 6 Colin G. Johnson, “Towards a Prehistory of Evolutionary and Adaptive Computation in Music” in Applications of Evolutionary Computing, Colin G. Johnson, ed. (Berlin: Springer, 2003), 502. 7 Nierhaus, Algorithmic Composition, 157. 8 Johnson, “Towards a Prehistory” 502. 9 Nierhaus, Algorithmic Composition, 1.

7

Key contributors to E/A including Eduardo Miranda, John A. Biles, and researchers at Google’s DeepMind, have each contributed uniquely to the development of the field. As well as working with artificial intelligence in music, Miranda created CAMUS, an AC system using cellular automata, generating complex musical structure from a simple set of rules. Bile’s work with genetic algorithms demonstrates an approach for systems that can respond creatively to live musical input. WaveNet, developed by a team from DeepMind10, is a neural network capable of reproducing the characteristics of a sample set of digital audio files. Each of these represent very different approaches to E/A composition, and exhibit their own strengths and limitations. This paper will examine these works to help illustrate the scope of E/A composition, and point to future applications. Parallel to the specifics of musical creativity in computers, developments in artificial intelligence universally face much the same challenges. The long term goal of creating general intelligence, a truly conscious man made intelligence, appears to offer a sweeping solution to computational creativity.11 If by Minsky’s approach this can be achieved, perhaps truly creative computer music systems would be automatically feasible. In this way, E/A algorithmic composition is inherently multidisciplinary, drawing on computer science and philosophy, and potentially also contributing to those disciplines. In order to engage effectively with this artform it is necessary to acquire a broader understanding of the field, and the ways in which it relates to these disciplines. By looking at it’s history as well as more recent applications, this paper seeks to contribute to that understanding, and provide a foundation for further research.

10 DeepMind is a London based company engaged with AI research, co founded by Dr Shane Legg, Mustafa Suleyman and Dr Demis Hassabis https://deepmind.com/ 11 General Intelligence can be defined as ‘AI systems that possess a reasonable degree of self-understanding and autonomous self-control, and have the ability to solve a variety of complex problems in a variety of contexts, and to learn to solve new problems that they didn’t know about at the time of their creation.’ Ben Goertzelm and Cassio, Pennachin. Eds, Artificial General Intelligence. (Berlin: Springer, 2007) VI.

8

Chapter 1

Algorithms for the Generation of Musical Structure Algorithmic composition is the application of algorithmic processes to the generation of musical structure.12 These processes might be used to deal with very specific elements of a musical task, or for creating entire compositions. A broader understanding of AC is necessary in order to understand the context from which E/A composition arises, and how it is different from other AC approaches. Composer and academic Gerhard Nierhaus is a key theorist in the area of AC. His 2009 book Algorithmic Composition, provides insight into its history and various categories. History of Algorithmic Composition Historically, algorithmic composition is inherently rooted in the more general development of algorithmic procedures. The history of this spans numerous fields of study, as discoveries and their application led to developments in other areas. The diversity of influences present in AC’s origins continues to be a factor as the field develops, with new ‘models for generating musical structure’ potentially coming from ‘nearly every scientific discipline’.13 The history of all the sciences that have contributed to algorithmic composition is perhaps too vast to cover within this paper, but in looking at early instances of their application to music we can get a sense for the way algorithmic composition has come about, and continues to grow. As early as AD 1000, Guido of Arezzo, in his work Mircologus, developed a system to automatically generate musical structure.14 Pitches and melodic phrases would be applied systematically to the letters and syllables of a text to be sung. The notation used by Guido and others up until the early thirteenth century was limited in its ability to represent rhythm. Notation has since developed to represent notes temporally, specifically in the work of Franco of Cologne around 1280.15 This additional musical parameter, and the constraints it imposes, have significant implications for algorithmic composition. The increased specificity in notation allows for more explicit musical instructions to be derived from algorithmic processes. In the late eighteenth and early nineteenth century, numerous games were published involving the composition of music with the rolling of dice. Such games allowed people untrained in music to engage with its creation. The first of these, Der allezeit fertige Menuettenund Polonoisenkomponist published in 1757 by Johann Philipp Kirnberger, became the model for those that came after it. The publication included a large number of bar length musical fragments. Players consulted a table of numbers which would indicate, for a given bar within the composition and a roll of the dice, which fragment to transfer to their own manuscript. The first successor to Kirnberger’s game, Ludus melothedicus, 1758, operated similarly but with each roll generating an individual note. Up until 1812 many more such games were published, often attributed to composers, although there is some doubt as to the legitimacy of the most famous of these attributions, to Mozart and Haydn.16 12 Nierhaus, Algorithmic Composition, 1. 13 Nierhaus, Algorithmic Composition, 2. 14 Nierhaus, Algorithmic Composition, 21. 15 Nierhaus, Algorithmic Composition, 23. 16 Stephen A. Hedges, “Dice Music in the Eighteenth Century” in Music & Letters, Vol. 59, No. 2 (Apr, 1978), accessed 26/5/2017 http://www.jstor.org/stable/734136

9

The development of programmable computers and programming techniques in the twentieth century allowed for relatively simple automation of algorithmic calculations. The “Illiac Suite”, created by Lejaren Hiller and Leonard Isaacson between 1955 and 1956 was the first entirely computer generated composition. The system output was symbolically related to musical note values as designed by Hiller and Isaacson, to then be interpreted by a musician.17 Colleagues of Hiller applied some of the methods used to create the Illiac Suite, in algorithmic composition experiments throughout the following decade.18 Hiller, alongside Robert Baker, incorporated these methods into Musicomp, the first programming environment for computer music composition. Other such environments and languages soon followed, beginning with Groove created by Matthews/Moore, and including Midi Lisp, Patchwork, Common Music, and Open Music. Some of these applications are still in use and being developed.19 Some other applications can be used for algorithmic composition, although it is not their sole purpose. Csound by Barry Vercoe, and later PureData, MAX/MSP, and SuperCollider, allow for mathematical procedures to be applied to generation of data or to the direct control or generation of sound. Categories of Algorithmic Composition In developing an understanding of algorithmic composition as a whole it is helpful to be aware of the disparate disciplines and techniques that have been appropriated into the context of composition. It is the method, or procedure, used to create a piece that determines the specific category within algorithmic composition. Nierhaus identifies a significant number of categories, including markov models, transition networks, and chaos. The techniques in these categories are relevant to contemporary applications of E/A AC including the examples provided in chapter 2. Markov models Markov models were developed by mathematician Andrew Markov to probabilistically predict the outcome of a series of events. In such a model, any given “state” has a probabilistic relationship to possible future states, which is then calculated stochastically.20 When applied to composition, each state represents a musical element, a note or note duration. A corpus of existing music is then analysed to determine the probability of a particular musical element in relation to the previous element. These systems excel at the imitation of musical style, specifically that of the music it is trained on. To this end, a balance must be achieved between the musical detail that is stochastically generated, and the larger structure required to produce something musically satisfying. Transition networks Transition networks use graphs to represent all possible states of a system and its variables. In algorithmic composition, it can be used to map out fragments of musical structure in a manner comparable to the dice games of the eighteenth and nineteenth century. David Cope is a notable adopter of this approach in his work EMI or Experiments in Musical Intelligence.21 EMI is a software system for generating music in a particular style. After analysis of musical scores fed into the system, 17 Nierhaus, Algorithmic Composition, 63. 18 Fernández, Jose David. and Vico. Francisco “AI Methods in Algorithmic Composition: A Comprehensive Survey” in Journal of Artificial Intelligence Research, no 48 (2013) 517. 19 Nierhaus, Algorithmic Composition, 63. 20 Stochastic is a term defined as having a random probability distribution or pattern that may be analysed statistically but may not be predicted precisely. Oxford Dictionaries, “Stochastic”, Oxford University Press, 2017 https://en.oxforddictionaries.com/definition/stochastic 21 Nierhaus, Algorithmic Composition, 131.

10

EMI uses a complex transition network to recombine their musical elements in the creation of new scores. In this way EMI corroborates Cope’s assertions that musical creativity in humans operates in a similar way, recombining existing ideas and material into new works.22 Chaos Chaos as a mathematical term was first used in 1975 by Tein-Yien Li and James Yorke.Previous to this, concepts of small variation in conditions leading to vastly variable outcomes had been documented by mathematician Jules Henri Poincaré in his book Science et Méthod, and appear in Pierre-François Verhulst’s logistic equation. The most famous example is a system of equations created by Edward Lorenz to model air movement in the earth’s atmosphere. As an illustration of the implications of this model, Lorenz described the butterfly effect, which proposes that something as small as the flap of a butterfly’s wings may, through chaotic system of deterministically related events, lead to dramatic meteorological changes.23 Within the field of chaos theory, numerous approaches can be incorporated into processes for generating musical structure. Chaos systems in AC are primarily used by temporally mapping the state of the system to musical parameters. Naturally, these systems can produce vastly varied results, but do not allow for the careful structuring that might be desirable in the creation of a coherent musical composition. Evolutionary and Adaptive techniques As a subcategory of AC, evolutionary and adaptive algorithmic composition relates specifically to the application of certain evolutionary and adaptive computational processes to the generation of musical structure.24 E/A techniques include genetic algorithms, cellular automata, artificial neural Networks and artificial Intelligence.These computational processes have usually been developed and applied at first in the field of computer science, and later reapplied to music composition. They are of particular value to music generation because of their emergent qualities.25 Genetic algorithms Evolutionary algorithms are procedures that draw inspiration from the mechanisms of Darwinian evolution, relying on the selection and mutation of a population to solve a problem. The most common of these is a genetic algorithm. In this, a population of individuals, existing as a number or string of numbers, is randomly generated. Each individual is then assessed for its fitness, which is a reflection of its effectiveness at solving the given problem. At this point a new population is created stochastically from the first, those individuals with greater fitness scores being more likely to be combined to create the individuals of the new population. These iterations generally continue until the fitness goal has been reached by an individual, or all members of the population are equal. Genetic algorithms can be used in composition by mapping the data contained in the individuals to musical parameters. The resulting musical output can then be assessed, and the resulting fitness scores fed back into the system, until the system produces the desired compositional result.26 22 Alcedo Coenen, “David Cope, Experiments in Musical Intelligence.” in Organised Sound Vol. 2 no 1 (1997). 23 Nierhaus, Algorithmic Composition, 131. 24 Johnson, “Towards a Prehistory” and, Nierhaus, Algorithmic Composition, 25 Emergent qualities here refer to qualities that come about through the system operating that were not explicitly written into the system. Johnson, “Towards a Prehistory” 506. 26Nierhaus, Algorithmic Composition, 175.

11

Cellular Automata Cellular automata is a class of algorithms that can produce complex behavior resulting from a very simple set of rules. They mathematically model complex systems made up of a large number of identical components.27 Theories of cellular automata were first developed in the 1950s, and then published in 1963 by Jon von Neumann; but it was not until the 1970s that they were notably popularised by John Horton Conway’s Game of Life.28 In the Game of Life, the on or off state of squared cells within an infinite two dimensional lattice bear influence on the states of nearby cells according to two relatively simple rules which are carried out on every tick of a virtual clock. From this simple system many visually appealing patterns can emerge dependent on the initial cell states, serving as a powerful illustration of generative power of cellular automata. While cellular automata systems are less suited to style imitation, this generative power means that simple mappings to musical parameters can make for compelling musical results. Artificial Neural Networks A product of the field of machine learning, within computer science, Artificial Neural Networks (ANN) are computational models based loosely on systems of neurons within the human brain. ANNs are trained on sets of data, and are then able to recognize or generate data similar to that on which they have been trained. Originally ANNs were developed for image recognition and classification.29 In simple terms, an ANN works by connecting input data to the system outputs according to weights assigned to each data element. The accuracy of the output can then be assessed, and the weights adjusted to produce a more accurate result. Before being adapted to algorithmic composition, ANNs were used in the 1970s and 80s in musical analysis by Peter Todd and Gareth Loy. In 1989 Todd used an ANN to generate basic melodies which reflected the notes of the different sets of data it was trained on.30 Since then ANNs have been often used to perform similar compositional tasks, but engaging with many more musical parameters. They are also often used in conjunction with other systems. In the case of genetic algorithms, ANNs have been used to assess the fitness of individuals within the algorithms population. ANNs are desirable tools of AC in part because of their potential to perform recombinance without overt reference to the data they have been trained on.31 A given sequence of notes in the resulting composition might not appear in the input data, rather being derived in an abstract or less obvious way.32 Artificial Intelligence Artificial intelligence (AI) is an umbrella term for a variety of computational systems designed to perform tasks that are considered to require intelligence. It does not represent a uniform approach, as the various methods it encompases are often focused on a specific task, and may even arise from

27 Eduardo R. Miranda. “Cellular Automata Music: An Interdisciplinary Project” in Interface, Vol. 22 no 1. (January 1993) 4. 28 Nierhaus, Algorithmic Composition, 187. 29 Nierhaus, Algorithmic Composition, 221. 30 Fernandez “AI Methods in Algorithmic Composition” 541. 31 Cope defines recombinance as a principle where ‘creativity does not originate from a vacuum, but rather synthesizes the work of others, no matter how original the results may seem.’ Cope, Computer Models, 87. 32 Nierhaus, Algorithmic Composition, 5.

12

varying disciplines.33 The scope of AI is further complicated by the ambiguity of the word “intelligence”. If compositional intelligence is defined as “the process of generating coherent musical structure”, then some of the aforementioned AC techniques could easily fit within this definition.34 In order to distinguish these from AI systems, computer scientist John McCarthy defines AI as “making a machine behave in ways that would be called intelligent if a human were so behaving”.35 Other AC systems can then be considered to be approaches to problem solving, as opposed to imitating intelligent behaviour. In AC, many different AI concepts and techniques can be applied to composition, in varying capacities. Rule-based systems, logical reasoning, machine learning, and more crucially, knowledge representation make up some of these. It is not possible to provide a comprehensive explanation of their workings within this paper, but their significance in the context of AC should not go unnoted. As a continually growing computer science field, AI developments have the potential to demonstrate computational creativity far beyond what is possible with the AC techniques described so far, and this creativity will undoubtedly be applicable to compositional tasks. Randomness Many AC techniques, including evolutionary algorithms, rely on randomness to generate new musical content or determine re-combinations of material. The musical use of randomness fits neatly into the mathematical processes of AC, but also points to a more specific goal. Experimental composer John Cage suggests that innovative acts of performance are difficult to achieve by simply trying to be innovative. According to Cage, a performer should make “disciplined use of chance operations” in order to break away from their own creative influences and tastes.36 This use of randomness indicates an alternative perspective on the creative outcome. Where a musical score would typically be a specification of the desired musical outcome, random processes instead place emphasis on the action of generating music without a particular prescribed outcome. In AC, many systems make use of randomness to this end, seeking out new and interesting musical content that might not have occurred to a human composer unaided by such a system, and perhaps pursuits of this kind could lead to an expansion of musical lexicon. Although many adaptive systems do not make use of randomness, their generative nature can be a valuable tool in that same pursuit of new musical ideas. These systems, particularly those that incorporate learning, have the potential to perform complex musical recombinance on a scale that is not achievable by a human, and therefore produce something new, or even fast-track natural musical developments.37

33 Nierhaus, Algorithmic Composition, 225. 34 Nierhaus, Algorithmic Composition, 227. 35 John McCarthy in Nierhaus, Algorithmic Composition, 228. 36 Johnson “Towards a Prehistory” 503. 37 Johnson “Towards a Prehistory” 503-4.

13

Chapter 2

Three Examples of Evolutionary and Adaptive Composition

Many works of algorithmic composition have been created using evolutionary and adaptive computational methods. In order to reinforce understanding of this area of AC, It is worthwhile looking at specific examples of these that use a variety of approaches and get a sense of what is achievable musically. To this end this paper will look at three particular works in three key areas of E/A music: Eduardo Miranda’s CAMUS, which generates musical scores using cellular automata. Al Biles’ GenJam, which makes use of a genetic algorithm to produce jazz solos in response to a human performer. Finally, Wavenet by Google Deepmind, is an artificial neural network for producing digital audio files that reflect the characteristics of recordings it has been trained on. These examples illustrate the scope of the field and demonstrate the limitations of a range of E/A systems. CAMUS - Eduardo Miranda Eduardo Miranda is a brazilian born researcher and composer of chamber and elctroacoustic music.38 The focus of much of Miranda’s work is on artificial intelligence in music, and he as written extensively on this and other areas of E/A music composition. His 1993 paper Cellular Automata Music: An interdisciplinary Project, details his work on CAMUS and it’s affiliated systems, which together can compose and perform music generated by means of cellular automata algorithms.39 CAMUS is itself responsible for generating musical content, while PROCAMUS further manipulates the resulting score, and CAMPLAY plays the score. CAMUS uses two cellular automata systems, Conway’s Game of Life (GoL), and a cyclic space cellular automata system invented by David Griffeath which is similar to GoL, but adheres to a different set of rules.40 Where the square cells in Game of Life can be in either an on or off state, the number of possible states for cells in a cyclic space cellular automata system are determined by the user. The user will select a number of states, n, and each state is defined by numbering them 0 to n-1. The program is initialised with randomly selected states for each cell, and the system rules are enacted at every tick of a virtual clock. The first rule is, if a cell is adjacent to another cell whose state is numbered 1 less that its own state, the adjacent cell is “eaten” and takes on the state of its superior neighbor. A second rule allows the system to cycle continuously; if a cell’s state is 0, any neighboring cells that are in state n-1 will become 0 also.41 In order to generate musical structure, the two systems are mapped to musical parameters in a number of ways. The GoL and cyclic space systems use the same size lattice. Each cell in in the GoL lattice is affiliated with a musical triad, its cartesian coordinates representing the two intervals between notes. The cyclic space system is used to determine which instrument will be used; each possible cell state relating to a specific instrument. When a triadic sequence is generated by a GoL cell switching on, the

38 Dan Albertson and Ron Hannah, “The Living Composers Project: Eduardo Reck Miranda” Last modified 2017, accessed May 10, 2017, http://composers21.com/compdocs/mirandae.htm 39 Miranda. “Cellular Automata Music”. 40 Miranda. “Cellular Automata Music” 9. 41 Alexander K. Dewdney, “A cellular universe of debris, droplets, defects and demons” in Scientific American, (August 1989). 102.

14

corresponding cell on the cyclic space lattice indicates the instrumentation for that sequence. The duration of the notes and their temporal organisation is determined by the arrangement of on-state cells within the GoL automaton. A cell’s “genetic code” can be one of 10 different shapes, called AND, that are representative of a temporal arrangement of the notes in its triad. The states of the surrounding cells determine this shape according to a predetermined code.42 The practical use of CAMUS is limited in terms of its musical outcome. Its value is primarily in experimentation, and its creator has used it in developing musical elements within larger compositions.43 As an example of E/A composition, it demonstrates unique approaches to deriving musical content from relatively simple procedures. GenJam - Al Biles GenJam is an ongoing project developed by Professor John A. Biles at the Rochester Institute of Technology.44 The GenJam system is a computer simulation of a jazz musician that uses a genetic algorithm (GA) to determine musical responses to a human soloist. The system is able to perform in a live setting, adhering to a predetermined structure, and exchange solos with a human performer. In order to perform a piece, GenJam is provided with information such as the tempo, rhythmic style, the number and length of solos to take, as well as the chord progression. The system also receives pre-generated MIDI sequences for the accompanying instruments.45 Expanding on the GA explained in Chapter 1, the GenJam algorithm uses two populations of individuals to assemble the solos it will play: the “measure” and “phrase” populations.46 The individuals in the “measure” population are comprised of a list of numbers that are interpreted as a series of eight events which provide a notational instruction for each eighth note in a 4/4 measure. These events can be either a new note, a rest, or a hold. The numbers within these lists are represented by four binary digits, allowing for sixteen possible events numbered from 0-15. A 0 indicates a rest, and 15 indicates a hold, leaving 14 possible notes to be represented. Before the measure is performed, GenJam identifies which chord the measure falls under in the pre-established chord progression, and maps the selected notes to a scale relating to that chord. This allows GenJam to easily adapt to different harmonic contexts. A sequence of four measures is organised by the “phrase” population, whose chromosomes are comprised of four six-digit binary numbers. Each of these can represent 64 numbers which refer to specific members of the “measure” population. This approach allows for exact repetition of measures within each four bar phrase .47 The fitness of the individuals in the GenJam is assessed by a “mentor”, who indicates approval or disapproval of parts of the solo as it plays. By typing ‘g’ for good, and ‘b’ for bad, the fitness score for the measure and phrase being performed is adjusted up and down. Unlike many GAs, GenJam does not search for one specific goal, such as the highest scoring phrase or measure. Rather, it is designed to build a collection of usable musical ideas for performance.

42 Miranda. “Cellular Automata Music” 10. 43 Miranda. “Cellular Automata Music” 17. 44 John A. Biles, “GenJam: A Genetic Algorithm for Generating Jazz Solos” in ICMC Proceedings (1994) 131-137. 45 Biles “GenJam” 131-2. 46 Biles “GenJam” 133. 47 Biles “GenJam” 133-135.

15

While it is limited in its musical vocabulary, GenJam demonstrates the potential for GAs in composition. Biles has shown that they can be effective at searching for possible solutions within a specified melodic space, the parameters of which could certainly be expanded. His methods could also easily be adapted to other compositional tasks.48 Wavenet - Google Deepmind WaveNet is an artificial neural network (ANN) for generating audio waveforms developed by van den Oord et al. at Google Deepmind, London UK.49 The primary focus of WaveNet is generating speech to mimic any human voice, but it has also been applied to the generation of other audio signals, including music. The approaches used are therefore guided by WaveNet’s speech synthesis applications, but maintain relevance to generation of music. Unlike CAMUS and GenJam, WaveNet does not deal specifically with the generation of data that can be mapped to musical parameters. Instead, it uses deep learning to learn from audio file examples of music, and determines each consecutive digital audio sample in its output as it relates to those which precede it. Computer generated speech is primarily achieved by concatenative TTS (text-to-speech), in which many short speech fragments are recorded, and then recombined to form words. Once recorded, it is difficult to alter the sound or inflection of the the speech it generates. WaveNet addresses this by modeling the audio waveform one sample at a time. 50 The finer details of WaveNet’s inner workings are beyond the scope of this paper, but it is worth pointing out its relationship to the ANNs described in chapter 1. While it uses a similar architecture, between WaveNet’s data inputs and outputs several other layers of neurons are fed the input data according to similar weighting systems. The music produced by WaveNet does not exhibit larger structural organisation like that of GenJam, but it’s imitation of harmony and timbre is convincing.51 While this project is not focused on legitimate musical composition, it shows promising signs for the application of machine learning to compositional tasks.

48 Biles “GenJam” 137. 49 Legg, “Deepmind” 50 Aaron van den Oord, et al. "WaveNet: A Generative Model for Raw Audio." 2016 in “Deepmind” Legg, Dr Shane. Suleyman, Mustafa. and Hassabis, Dr Demis, . Accessed 13/5/2017 https://deepmind.com/blog/wavenet-generative-model-raw-audio/ 51 van den Oord, et al. "WaveNet”

16

Chapter 3

Creativity, Authorship and Compositional Limitations The use of computers and algorithms to produce music is a practice that tends to raise questions of authorship and creativity. The intentions of an AC practitioner may vary from using it as a tool for accessing new ideas, to creating automated systems that are creative themselves.52 The latter end of this spectrum represents an abstraction of the compositional process, the composer being responsible for broader aesthetic or idealistic decisions rather than specific musical ones. This abstraction defers certain creative decisions to the system, complicating the attribution of authorship, and is often a characteristic of E/A composition systems. As research in E/A AC progresses, this is an issue that is worth addressing in order to anticipate the trajectory of the field, and accordingly adapt compositional goals. The feasibility and ramifications of computational creativity are lines of enquiry that often appear in the work of of those engaged with E/A composition. David Cope has explored this a great deal and notoriously raised questions of computational creativity and its validity. Although much of his best known work deals with transition networks, elements of those systems bear resemblance to E/A compositional techniques. The issues of authorship that arise from this are spoken to directly by Eduardo Miranda regarding his cellular automata systems, in which he suggests we should shift the emphasis of our questioning. Contemplation of creative machines need not invoke a fear of human creativity becoming obsolete. Nierhaus states algorithms are ‘a tool and means for the creative examination of the complex aspects of musical production’ rather than ‘golem[s], usurping creativity from the human realm’.53 Computer Models of Musical Creativity Cope has created several systems, such as Experiments in Musical Intelligence (EMI) that perform compositional tasks usually considered to be creative. By assessing several definitions of creativity, he has sought to identify its key requirements. In Computer Models of Musical Creativity, Cope presents a number of models that meet those requirements and illustrates how they might be integrated into a truly creative system. One of the key definitions of creativity that Cope points to comes from Marvin Minsky’s book The Society of Mind, a seminal work in the field of AI:

What is creativity? How do people get new ideas? Most thinkers would agree that some of the secret lies in finding “new ways to look at things.” ... Why must our minds keep drawing lines to structure our reality? The answer is that unless we made those mind-constructed boundaries, we’d never see any “thing” at all!54

It seems plausible that creativity of this kind could be modeled with a computer, identifying, cataloging, and referencing boundaries, but importantly also extending and redefining them.55 Minsky’s theory of the society of mind proposes that the mind is comprised of many small processes which he calls agents. On their own, agents perform only simple tasks, but when many of them

52 Nierhaus, Algorithmic Composition, 260. 53 Nierhaus, Algorithmic Composition, 272-3. 54 Minsky, The Society of the Mind 134. 55 Cope, Computer Models, 9.

17

interact with each other in a specific way, a society of agents can demonstrate intelligence. Societal interactions of agents are flexible and conducive to the drawing and redrawing of lines to navigate the world in a human-like way.56 This is not only a building block of creativity, but also building block of human interaction with the world. In thinking of the mind in this way, creativity emerges as natural product of the way the brain works. Its ability to adapt and reorganise things leads inevitably to new ideas when dealing with artistic endeavours.57 The agents within a mind operate on different structural levels, some small tasks fitting within larger ones. Hierarchically superior agents manage lower level agents and rely on their own superiors to operate cohesively within the system. Many interactions also take place outside of the hierarchical structure, but an understanding of this hierarchy helps to illustrate the way larger tasks or ideas are expressed by the activation of smaller tasks.58 Looking at the computer models of musical creativity proposed by Cope, it is evident that they could be a part of a complex arrangement of agents as described by Minsky.59 The proposed models include recombinance, learning, inference and analogy; form and structure; and association. It is the integration of these elements into one system that Cope presents as a path to creativity in computer music. Authorship If it is possible for a machine to be creative, and to compose music in a genuinely creative way, the instinct might be to attribute some authorship to the system, or a user who did not create the system. Eduardo Miranda discusses this issue in reference to his CAMUS system. Rather than attempting to dismantle the ambiguity of authorship that has arisen, Miranda suggests an alternate line of inquiry to facilitate constructive discussion.

What sort of system are you using to make music?", "How open is this system for the musician's subjectivity", "What kind of thinking should the composer have to compose using a system like that?60

Developing fields such as E/A AC may require new thinking and language to effectively communicate new ideas. In this case, a shift of focus may be required so that new compositional techniques are examined in a way that reflects their context and intention. Limitations and the Future Individual systems of E/A composition are inevitably bound by their own practical limitations. These present themselves usually in terms of musical space, user control, or even available computational power.61 In addition to this, certain categories of E/A AC, and even the field as a whole, are subject to broader limitations. Some of these might be combated by the development of a general artificial intelligence. AC systems can be divided into two categories, that of knowledge based and non-knowledge based systems. Knowledge based systems follow set rules to generate their output, whereas non-knowledge based systems draw on a corpus of data from which they derive their own rules to be applied, such as with artificial neural networks. Broadly speaking, a knowledge based system is limited in by its

56 Minsky, The Society of the Mind 17. 57 Cope, Computer Models, 9. 58 Cope, Computer Models, vii. 59 Cope, Computer Models. and Minsky The Society of the Mind. 60 Miranda, “Cellular Automata Music” 13. 61 Nierhaus, Algorithmic Composition, 271.

18

possible output. Beholden to its inbuilt rules, even if the range of possible outcomes is broad such a system could be seen functionally as providing a ‘class of possible compositions’.62 This is more an extension of compositional technique than an system responsible for the compositions it generates.63 Non-knowledge base systems are appealing for their ability to extend beyond pre-defined compositional rules determined by the system’s user. Their limit then is in the data that is available to them. This is can be a significant limitation, and is certainly worth considering when selecting an AC technique for a specific purpose, however it is in this regard that developments in fields related to E/A present possible solutions. Global interconnectivity via the internet has made vast amounts of data easily accessible, and increasingly so. Soon this limitation will diminish in significance, as the corpus available to a given system becomes far greater than is practically necessary to perform a compositional task.64 The future of artificial intelligence also promises a loosening of limitations. If the creation of general intelligence could be achieved, the ability for machines to acquire information in a humanlike way and apply it in integrated systems like that proposed by Cope would dramatically expand possibilities. These systems would operate creatively, but with greater computational speed, and with vast amounts of information available, making the breadth of possibilities almost impossible to envision.

62 Nierhaus, Algorithmic Composition, 278. 63 Nierhaus, Algorithmic Composition, 278. 64 Nierhaus, Algorithmic Composition, 279.

19

Conclusion This paper provides an overview of the use of evolutionary and adaptive systems within the field of algorithmic composition. Developing a general understanding of AC and its origins provides valuable context to E/A AC. It demonstrates the ways in which E/A computation expands the potential of this type of composition. By looking at key works that employ different E/A compositional techniques this paper illustrates the ways different systems can be used, and the motivations for using them. E/A composition often prompts clarification of the definition of creativity, as creative tasks usually managed directly by the composer are deferred to an algorithmic procedure. Additionally, if a truly creative general intelligence were possible, many of the limitations of E/A composition systems might become irrelevant. The history of AC has been shown to go back as far as AD 1000 and concepts derived from other disciplines even earlier than that. Examples of AC have been identified throughout the following millennium, with the invention of programmable computers allowing for significant developmental leaps. It is on this foundation that E/A composition demonstrates the potential to push the boundaries of both computational creativity, and musical lexicon. The three systems CAMUS, GenJam and Wavenet each represent different E/A compositional techniques. Examining how they operate and generate musical content provides insight into the possible applications of each, and the general scope of the field. With CAMUS, Eduardo Miranda has shown how multiple musical parameters can be controlled by a simple system. GenJam is limited to composition within a defined musical space, but demonstrates an approach to generating a creative response to musical input. WaveNet is a system that can develop its own compositional rules without the direct control of the user by analysing a large corpus of data to form these rules. Each of these systems are not without their own limitations, but also show clear paths towards expanding them. E/A composition defers compositional tasks that might normally be carried out by a human, to an algorithmic process. This raises questions about authorship, and the definition of creativity. David Cope proposes models of creativity to be integrated into a single system. By adhering to what he identifies as the definition of creativity, this system represents a path to the development of a truly creative system.65 The broad categories of knowledge based and non-knowledge based AC systems are bound by unique limitations. Knowledge based systems can not extend beyond a finite set of possibilities allowed by their rules of operation. Non-knowledge based systems are limited by the corpus of data from which they derive their rules. Advances in computer science however, suggest a possible expansion of those limits, with increasingly large corpuses of data being available, and the promises of general intelligence allowing for creative uses of that data to be performed by extremely fast computation. This paper has provided an overview of E/A composition, including examples that have demonstrated the depth and breadth of contemporary E/A. These examples show the potential for E/A to dramatically expand its scope and possibly even involve the creation of truly creative systems.

65 Cope, Computer Models.

20

Bibliography Albertson, Dan. and Hannah, Ron. “The Living Composers Project: Eduardo Reck Miranda” Last modified 2017, accessed May 10, 2017, http://composers21.com/compdocs/mirandae.htm Biles, John A. “GenJam: A Genetic Algorithm for Generating Jazz Solos” in ICMC Proceedings (1994) 131-137. Coenen, Alcedo. “David Cope, Experiments in Musical Intelligence.” in Organised Sound Vol. 2 no 1 (1997). 57-60.

Cope, David. Computer Models of Musical Creativity. Cambridge: The MIT Press, 2005. Cope, David. The Algorithmic Composer. Madison: A-R Editions. 2000. Cope, David. New Directions In Music. Dubuque: WM. C. Brown Publishers. 1971. De Mantaras, Ramon L. and Arcos, Josep Lluis. “AI and music: From composition to expressive performance.” in AI Magazine vol. 3 no. 43 (2002). Dewdney, Alexander K. “A cellular universe of debris, droplets, defects and demons” in Scientific American, (August 1989). 102-5. Doornbusch, Paul. “Computer Sound Synthesis in 1951: The Music of CSIRAC.” in Computer Music Journal vol. 28 no.1 (2004) 10-25. Fernández, Jose David. and Vico. Francisco “AI Methods in Algorithmic Composition: A Comprehensive Survey” in Journal of Artificial Intelligence Research, no 48 (2013): 513-582 (Fernandez 2013, 517) Goertzelm Ben and Pennachin, Cassio, Editors, Artificial General Intelligence. Berlin: Springer, 2007. Hedges, Stephen A. “Dice Music in the Eighteenth Century” in Music & Letters, Vol. 59, No. 2 (Apr, 1978), 180-187. accessed 26/5/2017 http://www.jstor.org/stable/734136 Kirke, A. and Eduardo R. Miranda. editors. Guide to Computing for Expressive Music Performance. London: Springer, 2013. Johnson, Colin G. “Towards a Prehistory of Evolutionary and Adaptive Computation in Music” in Applications of Evolutionary Computing, Editor Colin G. Johnson, 502-509. Berlin: Springer, 2003. Legg, Dr Shane. Suleyman, Mustafa. and Hassabis, Dr Demis, “Deepmind”. Accessed 13/5/2017 https://deepmind.com/ Minsky, Marvin. The Society of the Mind. New York: Simon and Schuster, 1985 Miranda, Eduardo R. “Cellular Automata Music: An Interdisciplinary Project” in Interface, Vol. 22 no 1. (January 1993) 3-21.

21

Nierhaus, Gerhard, editor. Patterns of Intuition: Musical Creativity in the Light of Algorithmic Composition. Netherlands: Springer, 2015. Nierhaus, Gerhard. Algorithmic Composition. Vienna: Springer, 2009. Oxford Dictionaries, “Stochastic”, Oxford University Press, 2017 https://en.oxforddictionaries.com/definition/stochastic Romero, J., and Penousal Machado, editors. The Art of Artificial Evolution. Berlin: Springer-Verlag, 2008. Rowe, Robert. Machine Musicianship. Cambridge: The MIT Press, 2001. van den Oord, Aaron. Dieleman, Sander. Zen, Heiga. Simonyan, Karen. Vinyals, Oriol. Graves, Alex. Kalchbrenner, Nal. Senior, Andrew. and Kavukcuoglu, Koray. "WaveNet: A Generative Model for Raw Audio." 2016 in “Deepmind” Legg, Dr Shane. Suleyman, Mustafa. and Hassabis, Dr Demis, . Accessed 13/5/2017 https://deepmind.com/blog/wavenet-generative-model-raw-audio/