Colliding:aSuperColliderenvironmentforsynthesis-orientedlivecoding

GerardRoma

CVSSP,UniversityofSurreyGuildford,UnitedKingdomg.roma@surrey.ac.uk

Abstract.Oneofthemotivationsforlivecodingisthefreedomandflexibilitythataprogramminglanguageputsinthehandsoftheperformer.Atthesametime,systemswithexplicitconstraintsfacilitatelearningandoftenboostcreativityinunexpectedways.Somesimplifiedlanguagesandenvironmentsformusiclivecodinghavebeendevelopedduringthelastfewyears,mostoftenfocusingonmusicalevents,patternsandsequences.Thispaperdescribesaconstrainedenvironmentaimedatexploringthecreationandmodificationofsoundsynthesisandprocessingnetworksinrealtime,usingasubsetoftheSuperColliderprogramminglanguage.Thesystemhasbeenusedineducationalandconcertsettings,twocommonapplicationsoflivecodingthatbenefitfromthelowercognitiveload.

Keywords:Livecoding,soundsynthesis,liveinterfaces

IntroductionTheworldofspecializedmusiccreationprogramminglanguageshasbeengenerallydominatedbytheMusic-NparadigmpioneeredbyMaxMathews(Mathews1963).ProgramminglanguageslikeCSoundorSuperColliderembracethedivisionofthemusicproductiontaskintwoseparatelevels:asignalprocessinglevel,whichisusedtodefineinstrumentsasnetworksofunitgenerators,andacompositionallevelthatisusedtoassembleandcontroltheinstruments.AnexceptiontothisisFaust,whichisexclusivelyconcernedwithsignalprocessing.MaxandPureDatauseadifferenttypeofconnectionforsignalsandevents,althoughunderasimilarinteractionparadigm.Similarly,Chuckhasspecificfeaturesforconnectingunitgeneratorsandcontrollingthemalongtime.Theavailabilityoflanguagescapableofgeneratingmusicinrealtimehasfosteredthedevelopmentoflivecoding(Collinsetal.2003),whichhastheadvantageofgivingtheaudiencethepossibilitytoreadcomputermusicperformancesinawaythatiscomparabletoimprovisationwithphysicalinstruments.Livecodingisalsohelpfulinclassroomenvironments,allowingstudentstograspthementalprocessinvolvedinusingaprogramminglanguageorcommandline.Onefeaturethathasinspiredthelivecodingpracticeisthefreedomandpowerthataprogramminglanguagegivestotheperformer.Ontheotherhand,restrictedenvironmentsoftenresultinunexpectedcreativeoutcomes.Facingtheinfinitepossibilitiesofferedbycomputers,musiciansandartistscommonlydesigntheirsystemsonthebasisofconstraints(Magnusson2010).Constraintscanbeseenastherulesthatdefineagame,andthusareconsideredbymanytoplayanessentialroleincreativity(Boden2004;Merker2006).

Someconstrainedlanguagesareavailableforlivecoding,mostoftenwithastrongfocusonmusicalevents(Magnusson2011;McLean2014).IntheMusic-Nparadigm,thismeansnotcreatingnewinstrumentsonstage,butimprovisingnewcontrolsequencesforpre-definedinstruments.Thesystempresentedinthispaperexplorestheotherside.Inthissense,theconceptof"synthesis-oriented"livecodingcanbeopposedto"event-oriented"livecoding.Thereis,asamatteroffact,alongtraditioninchallengingthedistinctionbetweencompositionandtimbre,preciselyonthebasisofthepossibilitiesofferedbycomputers(Döbereiner2011).Underthispointofview,musicaleventscanbeseenassignals,andmusiccanbecreatedusingexclusivelysignalprocessingnetworks.

Withthereleaseofversion3(McCartney2002),theSuperColliderlanguagewassplitintotwoseparateprograms:thesynthesisserver(scsynth)andthelanguageinterpreter(sclang).Asubsetofthelanguageisusedtospecifysynthdefinitions,whichthescsynthservercanexecute.ItisnotuncommontofinddiscussionsamongtheSuperCollidercommunityoncreatingmusicstructurespurelyintheserverside.Amongotherreasons,creatingsynthdefinitionscanbelessdemandingwithrespecttodealingwiththefulllanguageandthedistributedarchitecture.Giventheamountofunitgeneratorsavailable,focusingonthesynthesissideofSuperColliderisbothsimpleandpowerful.Intermsofuser

interface,theevolutionofSuperColliderasageneralpurposelanguageledtoastrugglebetweentheneedofa"properIDE"forobject-orientedprogramming,andtheinterestinthedocument-orientedrichtexteditorthatwasavailableonOSX.

Inthiscontext,CollidingwasdesignedasaconstrainedinterfaceforcreatingSuperCollidersynthdefinitions.Apartfrommusiccreationandperformance,thefocusonsynthesisprovidesacompellingenvironmentformusicandsignalprocessingeducationsettings,allowingeasyexperimentationwithawidevarietyofsynthesistechniques.Theideaofcreatingasimplifiedenvironmentemergedwhenobservingengineeringstudentstryingtocreateproceduralprogramswithalltheasynchronouscallsrequiredtostarttheserver,createasynthdefinitionandinstantiateit.Loadingaudiobuffersandnetworkresourcesrequireasynchronouscallsaswell,whichassumeanunderstandingofanonymousfunctions.Boththeclassroomandtheconcertenvironmentsbenefitfromthereducedcomplexityandfocus.Inthecaseofmusicperformance,theconstraintisinpartaesthetic,butstillmanydifferentstylesofmusiccanbeplayed.Therestofthepaperdescribesthefunctionalitiesimplementedsofarandtheuseoftheprograminbotheducationandperformance.

RelatedworkConstrainedenvironmentsforlivecodingarecommoninclassroom-orientedapplications.Twowell-knownexamplesareEarsketch(Freemanetal.2014)andSonicPi(AaronandBlackwell2013).ThefirstisbasedonanApplicationProgrammingInterface(API)whichcanbeusedinJavascriptorPythoninawebenvironment.ThisAPIiscomplementedwithalibraryofaudioloopsthatcanbemanipulatedandpositionedinanaudiosequencertimelinethroughtheprogrammingAPI.ThesecondexposesalsoabasicAPI,inthiscaseasaRubydomain-specificlanguagethatcontrolsSuperCollidersynths.TheprogrammingenvironmentisdesignedtorunonaRaspberry-Piembeddedcomputer.Bothenvironmentsofferlimitedcapabilitiesintermsofsynthesis.Themainideabehindlivecodingmusicenvironmentsintheclassroomistoengagestudentsintoprogrammingbydoingsomethingfunandcreative.

Suchenvironmentsallowexperimentationwithgenerativemusic,butarenotsowellsuitedforsynthesis-orientedmusicorlearning.Whilethesesystemsgenerallyfocusonlearningprogrammingskills,Collidingemphasizessignalprocessing,requiringonlybasicprogrammingconcepts.Inthissense,perhapsamoresimilarapproachwouldbeusingthecompiledlanguageFaustinaninteractivesetting.FaustLive(Denouxetal.2014)isajust-in-timecompileraimedatfacilitatingthiskindofset-up,however,itdoesnotprovideaninterfaceforcoding.FaustisusedlivebyJuliusO.SmithforteachingsignalprocessingusingEmacs1.

Asthenamesuggests,CollidingismainlyinfluencedbyProcessing(ReasandFry2006)anditscousintheArduinoIDE.Bothhavesucceededincreatingsimplifieddevelopmentenvironmentsforactivitiesthattraditionallyrequiredspecializedtraining.ProcessingwasoriginallypresentedasasubsetoftheJavalanguagethatcanbeembeddedinJavaprograms.ArduinooffersasimpleCAPIforembeddedsystems.Similarly,CollidingusesasubsetoftheSuperColliderlanguage.

AnothermajorinfluenceisixiLang(Magnusson2011).Whileheavilyfocusedonmusicalevents,ixiLangstressestheimportanceofreducingcomplexityformusiclivecoding.LikeixiLang,Collidingcanbeseenasa"SuperColliderparasite",inthiscaseforsynthesis-orientedlivecoding.

�Interface

Overview

TheprogramfollowstheinteractionparadigmofprototypingeditorslikeProcessing.Theinterface(Figure1)allowstheusertocreateupto8tabs,eachwithacodeeditingwindowconfiguredwithalargefontsize.Thisencouragesshortsnippetsandfacilitatesreadability.Thenumberoftabs(andalsoofbuffers,asdescribedbelow)isnotcompletely

1https://www.youtube.com/watch?v=2lEt7dsziO0

arbitrary.Itisgenerallyagreedthatsimilarnumbersofelementsarerelatedwithworkingmemorycapacity(Miller1956).Itiscommontofind8-channellimitsinmusicproductionhardware(smallmixers,oldtaperecorders)orsoftware.Themainactionsconsistincompilingthecode(forerrorchecking),runningit(whichresultsinapotentiallyinfinitesoundstream),andstoppingthesound.Theseactionscanberunthroughkeyboardshort-cutsorusingasetofbuttonsbelowthecodewindow.Anothersetofbuttonsinthetoprightcornerexposesproject-leveloperations(addingtabs,gettinghelpforthecurrentlyselectedtext,loadingandsavingprojects,andapanicbuttonthatstopsallrunningprocesses).Textcolorinthecodeeditingwindowisusedtoindicatecompilationstate.Whitemeansthecodehasbeencompiled,greymeansitisbeingedited,andredmeansthereissomeerror.

Figure1:TheCollidinginterface

Collidingofferstwomodesofoperation:"synthediting"and"advanced".Thesyntheditingmodeaffordsthemetaphorofeditingapatchinahardwaresynthesizer,butitcanalsobeusedforlivecoding.Inthiscase,thecodeisassumedtobeasynthdefinition,andtheenvironmentprovidessomepredefinedvariablesandthemeanstotriggernotes,ifdesired.Inadvancedmode,thecodecanbeanything"playable".InSuperColliderthisisachievedbycompilingthecodeandmakingitthesourceofaNodeProxy,whichallowsusingtheinterfacewithJITLib(Rohrhuber,deCampo,andWieser2005)asaback-endforlivecoding.Thisincludesalsoeventpatterns,althoughthispossibilityhasnotbeenexploredorspecificallysupported.OneimportantdifferenceisthatinadvancedmodetheaudiooutputchanneliscontrolledbyJITLib,whileinsynthesiseditingmodeitisspecifiedexplicitly.Thusin"synthediting"modeitissimplertoaddressmultipleoutputs.Forsynthesisorientedlivecoding,i.e.,inSuperCollidertalk,ifonlyservercodeisevaluated,bothmodescanbeused,althoughtheadvancedmodeallowstersercode.

Afeedbackpanelbelowthecodewindowandbuttonsisusedforerrorreporting.Longstacktracescanbeoverwhelmingforbeginners,andarealsoundesirableinlivesituations.Whenthecoderesultsinastacktrace,thesystemselectstherelevantmessageandhighlightstheoffendingline(Figure2).Inordertoprovidesuchfeedbackwithoutdisruptingthecurrentinterpreter,itwasnecessarytoimplementcompilationasaseparateprocess,bywritingthesnippettofileandcallingsclangtoparseit.Allofthishappensunderthehood.

AllcodeisruninSuperCollider’sinternalserver,whichfacilitatesvisualizationoftheoutput.Thebottomendcontainsanoscilloscopethatprovidesvisualfeedbackabouteverythingthatishappeningintheserver.

Synthediting

Additionalfeaturesareprovidedinthesyntheditingmode.Inthismode,thecodeiswrappedtobuildandinstantiateasynthdefinition,withseveralpre-definedparameters.Awindowrepresentingthecomputerkeyboardallowstriggeringnotesusingthesynththathasbeendefinedinthecurrenttab.Thekeyboardusesanisomorphicmapping.Aslidernexttothecodewindowallowscontrollingthegain.Thisresultsinthefollowingpre-definedvariables.Itisuptothecodertomakeuseofthem.

• key:TheMIDInumbercorrespondingtothekeypressedinthecomputerkeyboard.4• freq:ThefrequencyinHzcorrespondingtothekey.• gate:Agateinputforenvelopes.• amp:Thevaluedefinedbytheslider.

Itistrivialtoextendthissystemwithotherwidgets,forexampleadditionalsliderscouldbeattachedtothekeyboard.Forthemoment,thesystemprovidestheminimumtoleveragetheinputdevicesalreadyavailableineverycomputer.ThemouseisaccessedusingthetraditionalSuperColliderunitgenerators.ThisisaconvenientsetupforclassroomPCsandlaptopperformances.ItisalsoeasytoextendtheconcepttoMIDIcontrollers.

Figure2:Errorreporting

Buffersandfiles

Besidessynthdefinitions,anotherareathatrequiresasynchronouscallsisdealingwithaudiofiles.ThisisoftenasourceofconfusioninSuperCollider.Graphicalinterfacesaregenerallyappreciatedforbrowsingthefilesystemandaccommodatingthetimeforfileloading.Collidingprovides8slotsforbuffermanagement.Buffersaresimplyaccessedviatheirbuffernumber(0-7).Aseparatewindowisusedforloading,visualizingandpre-listeningeachbuffer.Empty

bufferscanbeallocatedforrecordingsignals,andsmallbufferscanbeconvertedtowavetableformatforwavetablesynthesis.Inadditiontoloadingsoundfilesfromdisk,thesystemincludesaninterfacefortheFreesoundquark2,sothattheusercanmakebasictextqueriestotheFreesounddatabase(Akkermansetal.2011),anddownloadsounds.Forthemoment,thesearchisrestrictedtowavfiles.However,ifoggsupportiscompiledwiththeSuperColliderbinaries,itshouldbepossibletoaccessthewholeFreesounddatabaseviahigh-qualityoggpreviews3.

Finally,somebasicprojectmanagementfacilitiesareprovided.Aprojectconsistsofthetextineachtabandallthesoundfilescorrespondingtothebufferslots.Thisisespeciallyusefulforlearningenvironments,butcanalsobeusefultopreservethestateofalivecodingsession.Inthefutureitmightbeinterestingtoaddsomesimpleinterfacetoversioncontrol(e.g.git)whichisalsousuallyhardforbeginners.

InitialExperiencesCollidingwasdevelopedwhiletheauthorwasattheSonologydepartmentatEscolaSuperiordeMúsicadeCatalunya(ESMUC).Whilenoformalevaluationwasconducted,itwasusedin4overlappingsemestercoursesbetween2013and2015,coveringbasicandadvancedsynthesistechniques.Duringthepreviousyear,theSuperColliderlanguageandIDEwereused.ThedifferencewithrespecttousingplainSuperColliderwasdramatic.ByfreeingstudentsfromlearningtousetheSuperCollidereditorandlanguage,itwaspossibletofocusonsynthesisandprocessingasopposedtoprogramming.Atthesametime,thankstothelargenumberofunitgeneratorsavailableandthegeneralpowerofSuperCollideritwaspossibletoexploremanydifferentalgorithmsandconcepts,frombasicsubtractiveandmodulationtechniquestophysicalandspectralsynthesis.CollidingwasusedasakindoflabnotebookeditorthatcouldruninclassroomPCsandstudentlaptops.AssignmentsconsistedinCollidingprojectswheredifferentvariantswereexploredineachtab.Asmallcodesnippetwasusuallyseeded.

Sincestudentsweregenerallybeginnersincomputerprogramming,averysimplegrammarwasadoptedwhichworkedforallassignments(ascanbeseeninthefigures):eachline,exceptthelastone,isavariabledefinitionandassignment,wherepartofthesignalprocessinggraphisrepresented.Thissuggeststhat,forthepurposeofdefiningsynthdefinitions,thelanguagecouldbefurthersimplified.

Theonlylimitationwastheprojectmanagementfeature,whichwasnotalwaystrustedbythestudentsbecauseofitssimplicity.Howeverincaseofdoubttheprojectstructurewassimpleenoughandtextsnippetscouldbeopenwithastandardtexteditor.Somesynthesistechniquesthatrequireadditionalprocessing,suchasinthecaseofwavetablesynthesis,weremoredifficulttoaccommodate.Foradvancedtechniques,suchasspectralmodelingorvocoding,itwasgenerallyconvenienttouse"black-box"unitgeneratorsavailablefromsc3-plugins4.

Collidinghasalsobeenusedinseverallivecodingperformancesbytheauthor.Upto8synthesisprocessesmayruninparallelwhileoneofthemisbeingedited.Thebigfont,oscilloscope,andtheconstrainttosynthesis-basedcodingcontributetothereadabilityoftheperformance.Inlivecoding,thesimplifiedgrammarisnotneeded,andinsteadatersesyntaxismorecommon,inlinewiththeuseofTwitterforsharingsmallSuperColliderprograms.

ConclusionsTheSuperCollidersyntaxforbuildingsynthesizersallowsforcreatingsophisticatedpatcheswithveryfewlinesofcode.Suchconcisedescriptionsoftencontrastwiththecomplicationsoneendsdealingwithwhentryingthesamethingswithgraphicalpatchingsystemsandmodularsynthesizermetaphors.However,learninganobject-orientedlanguage

2https://github.com/g-roma/Freesound.sc3Thisdependsontheversionoflibsndfile,whichforhistoricalreasonswasfixedtoanolderversiononOSXbinaries.AtthetimeofthiswritingtheproblemisfixedinSuperCollider3.7binaries,sotherestrictiontowavforFreesoundsearcheswillberemoved.4https://github.com/supercollider/sc3-plugins

enrichedwithfunctionalprogrammingconstructscaneasilyscaretinkerersandmusicianswithlittleprogrammingbackground.TheCollidingenvironmentprovidesasimplifiedinterfacethatisparticularlyusefulwhenfocusingonsoundsynthesisandprocessing,asopposedtoevent-basedmusiccompositionandperformance.Thisfocuscanbeseenasadesignconstraintthatisusefulineducationalenvironmentsandinsynthesis-orientedlivecoding.Whilethecurrentimplementationmainlystressestheconceptandinterface,theseideascanbefurtherextendedwithoutleavingtheSuperColliderlanguage,bydefiningmorehelpervariablesandfunctions.Also,thesystemisamenabletoencapsulationofsynthesisprocessesasunitgenerators,apracticeusedbysomelivecoderswhichallowsgrowingapersonalsonicpalettewithoutlosingsimplicity.Addingsupportforthiskindofencapsulationwillbeinvestigatedinthefuture.ThecodeisavailableasaSuperColliderquark,andcanbedownloadedfromthegithubrepository5.

AcknowledgementsTheauthorwassupportedbyEscolaSuperiordeMusicadeCatalunya(ESMUC)andtheMusicTechnologyGroup(MTG)atUniversitatPompeuFabra(UPF)duringthedevelopmentofthiswork.

ReferencesAaron,Samuel,andAlanFBlackwell.2013.“FromsonicPitoovertone:creativemusicalexperienceswithdomain-specificandfunctionallanguages.”ProceedingsofthefirstACMSIGPLANworkshoponFunctionalart,music,modeling&design.ACM,35–46.

Akkermans,Vincent,FredericFont,JordiFunollet,BramDeJong,GerardRoma,SteliosTogias,andXavierSerra.2011.“Freesound2:Animprovedplatformforsharingaudioclips.”KlapuriA,LeiderC,editors.ISMIR2011:Proceedingsofthe12thInternationalSocietyforMusicInformationRetrievalConference;2011October24-28;Miami,Florida(USA).Miami:UniversityofMiami;2011.InternationalSocietyforMusicInformationRetrieval(ISMIR).

Boden,MargaretA.2004.Thecreativemind:Mythsandmechanisms.PsychologyPress.

Collins,Nick,AlexMcLean,JulianRohrhuber,andAdrianWard.2003.“Livecodinginlaptopperformance.”

Organisedsound8(03):321–330.

Denoux,Sarah,StéphaneLetz,YannOrlarey,andDominiqueFober.2014.“FAUSTLIVE,Just-In-TimeFaust

Compiler...andmuchmore.”TechnicalReport,GRAME.

Döbereiner,Luc.2011.“Modelsofconstructedsound:Nonstandardsynthesisasanaestheticperspective.”ComputerMusicJournal35(3):28–39.

Freeman,Jason,BrianMagerko,TomMcKlin,MikeReilly,JustinPermar,CameronSummers,andEricFruchter.2014.“EngagingunderrepresentedgroupsinhighschoolintroductorycomputingthroughcomputationalremixingwithEarSketch.”Proceedingsofthe45thACMtechnicalsymposiumoncomputerscienceeducation.ACM,85–90.

Magnusson,Thor.2010.“Designingconstraints:Composingandperformingwithdigitalmusicalsystems.”ComputerMusicJournal34(4):62–73.

Magnusson,Thor.2011.“theIxiLang:aSupercolliderParasiteforLiveCoding.”Icmc2011,no.August:503–506.

Mathews,MaxV.1963.“Thedigitalcomputerasamusicalinstrument.”Science142(3592):553–557.

5https://github.com/g-roma/Colliding

McCartney,James.2002.“Rethinkingthecomputermusiclanguage:SuperCollider.”ComputerMusicJournal26(4):61–68.

McLean,Alex.2014.“Makingprogramminglanguagestodanceto:livecodingwithtidal.”Proceedingsofthe2ndACMSIGPLANinternationalworkshoponFunctionalart,music,modeling&design.ACM,63–70.

Merker,BjornH.2006.“Layeredconstraintsonthemultiplecreativitiesofmusic.”MusicalCreativity:MultidisciplinaryResearchinTheoryandPractice,pp.25–41.

Miller,GeorgeA.1956.“Themagicalnumberseven,plusorminustwo:somelimitsonourcapacityforprocessinginformation.”Psychologicalreview63(2):81.

Reas,Casey,andBenFry.2006.“Processing:programmingforthemediaarts.”AI&SOCIETY20(4):526–538.Rohrhuber,Julian,AlbertodeCampo,andRenateWieser.2005.“Algorithmstodaynotesonlanguagedesignforjustintimeprogramming.”InternationalComputerMusicConference.291.

�