interactive, evolutionary textured sound composition sidney fels 1 and jônatas manzolli 2 (1) dept....

Interactive, Evolutionary Textured Interactive, Evolutionary Textured Sound CompositionSound Composition

Sidney Fels1 and Jônatas Manzolli 2

(1) Dept. of Electrical and Computer Engineering

University of British Columbia - Canada

ssfels@ece.ubc.ca

(2) Interdisciplinary Nucleus for Sound Studies (NICS)

University of Campinas (UNICAMP) - Brazil

jonatas@nics.unicamp.br

SummarySummary

• Starting Points– Evolutionary music composition– Interactive control of composition

• Melody Population

• System Description– Biological and Musical Cycles

• Interaction and Visual Representation

• Results– Musical Examples

– Piano roll representation

Starting PointsStarting Points

• What is musical texture?– relationship with visual domain– superimposition of lines or voice

• Idea from Guido d’Arezzo (XI century)– Unknown melody played on the monocord

• Evolutionary Computation– Fitness Function– Mutation and Crossover

• Significant human interaction• Constrained amount of stochasticity

Melody PopulationMelody Population

• Note Genetic Structure– Taken advantage of the MIDI protocol

• MIDI note number

• MIDI Velocity number

• Duration of the note

• Hiden Melody - established a priori

• Population - 20 melodies with 12 notes each one

System DescriptionSystem Description

• Three processes run concurrently– Musical Cycle (MC)

– Genetic Cycle (GC)

– Visualization • MC select the melody to play

– Fitness Function is applied• The Best melody is close to the Hiden Melody

• GC applies Mutation and Crossover– Mutation - random variations in pitch and duration

– Crossover - notes from father and mother melodies

System DiagramSystem Diagram

Genetic OperationsGenetic Operations

• Fitness Function

– min D = ∑ ∑ (h0,j - gij)2 where i = 1..20, j = 1…3 and h0,j = Hidden Melody

• Mutation – Mutation probability

– Change note number inside 02 octaves

– Change duration to 2(± 1)

• Crossover– Crossover probability

– If a note is selected >>> replace it with a note from parents

Mutation example

Interaction and VisualizationInteraction and Visualization

• Map performer interaction semantics to evolutionary semantics– Increases intimacy with the system

• Relationship between 02 objetos– Distance >>> mutation probability– Angular deviation >>> crossover probability

• Two objetos are two people

distance

angle

Real time controllersReal time controllers

• GUI - graphic interface• Polhemus Fastrak

• Local Position System (LPS)

Visualization of 20 melodiesVisualization of 20 melodies

length = log10(duration +1)*6angle = ((note1 - note2)*10 +180)/(2.0*3.14)width = (velocity/127.0)*4.0 + 1

ResultsResults

• Hidden Melody• Mutated Melody

Textured SoundsTextured Sounds

• Textured Hiden • Textured Mutation

Textured SoundsTextured Sounds

Future WorkFuture Work

• to experiment with Alternative Visualization• to allow additional control of MIDI• to input the Hiden Melody using MIDI controller or

a keyboard• to further experiment with mapping of two and

three object relationship semantics• to increase the complexity of the melody and

improve the ryhthm representation• to evaluate the various mappings between the

interaction space

Jônatas Manzollijonatas@nics.unicamp.brUniversity of Campinas - UnicampBrazil

Examples of resultshttp://www.ece.ubc.ca/~hct/IGGM

Address for Contact

Sidney Felsssfels@ece.ubc.caUniversity of British ColumbiaCanada