processing, authoring and composing of spatial sounds...
TRANSCRIPT
Thibaut Carpentier & Jérémie Garcia
UMR 9912 STMS IRCAM — CNRS — UPMC!
Processing, Authoring and Composing of Spatial Sounds (using Spat and OpenMusic)!
Workshop on Interactivity in Music Composition and Performance
May 5th 2015 CIRMMT — McGill University, Montreal
ANR-13-JS02-0004
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 2
Outline!
• Spat : state of the art & processing aspects
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 3
Outline!
• Spat : state of the art & processing aspects
• Goals & Motivations
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 4
Outline!
• Spat : state of the art & processing aspects
• Goals & Motivations
• Compositional tools (ongoing work)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 5
Outline!
• Spat : state of the art & processing aspects
• Goals & Motivations
• Compositional tools (ongoing work)
• Mixing and post-production
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 6
Outline!
• Spat : state of the art & processing aspects
• Goals & Motivations
• Compositional tools (ongoing work)
• Mixing and post-production
• Limitations & Perspectives
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 7
Spat
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 8
• What? Digital processor for sound spatialization and artificial reverberation
Spat : Overview!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 9
• What? Digital processor for sound spatialization and artificial reverberation
• Why? Allow composers, performers or sound engineers to control the diffusion of sounds in a real or virtual space
Spat : Overview!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 10
• What? Digital processor for sound spatialization and artificial reverberation
• Why? Allow composers, performers or sound engineers to control the diffusion of sounds in a real or virtual space
• How? Room effect module (temporal aspects) ��� + Spatialization module (directional aspects) ��� + intuitive control interface��� = Spat ��� = Global description of the spatial effect, modular and scalable design
Spat : Overview!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 11
Spat : Architecture!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 12
DSP chain
input instrumental or synthetic sources, typically mono or stereo
Spat : Architecture!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 13
pre-processing DSP chain
input
Doppler effect, Air absorption, pre-delay, EQ, etc.
Spat : Architecture!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
14
room effect
pre-processing DSP chain
input
Doppler effect, Air absorption, pre-delay, EQ, etc.
Artificial reverberator
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
15
room effect
pre-processing DSP chain
D ER Rev
input
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
16
room effect
pre-processing DSP chain
D ER Rev
input
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
17
room effect
pre-processing DSP chain
D ER Rev
input
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
18
room effect
pre-processing DSP chain
D ER Rev
input
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
19
room effect
pre-processing DSP chain
D ER Rev
input
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
20
room effect
pre-processing DSP chain
D ER Rev
input
Doppler effect, Air absorption, pre-delay, EQ, etc.
Artificial reverberator Object-oriented internal representation (based on simplified space-time-frequency model)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
21
room effect
panning
pre-processing DSP chain
D ER Rev
input
Doppler effect, Air absorption, pre-delay, EQ, etc.
Artificial reverberator
Configurable panning module (VBAP, HOA, WFS, stereo, etc.)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
22
room effect
panning
decoding / transcoding
pre-processing DSP chain
D ER Rev
input
speakers signals
Doppler effect, Air absorption, pre-delay, EQ, etc.
Artificial reverberator
Configurable transcoding module (transaural, HOA, etc.)
Configurable panning module (VBAP, HOA, WFS, stereo, etc.)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
23
room effect
panning
decoding / transcoding
pre-processing DSP chain
D ER Rev
input
speakers signals
Perceptual control interface
— Perceptually relevant — Intuitive control, mutually independent parameters — Continuous interpolation between room acoustical qualities — Does not rely on a physical or geometrical description — Independent of the reproduction system
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Software engineering!
24
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Software engineering!
25
• Development started in the early 1990s, originally as a library of Max patches
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Software engineering!
26
• Development started in the early 1990s, originally as a library of Max patches
• Many limitations (number of input/output)
• lack of flexibility
• lack of efficiency
• difficult to maintain
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Software engineering!
27
DSP library GUI library Spat
• Completely refactored as a set of two libraries
• host-independent
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
28
MaxMSP / PureData objects
DSP library GUI library Spat
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 29
MaxMSP / PureData objects
Live / interactive
performances
DSP library GUI library Spat
Spat : Environments!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
30
MaxMSP / PureData objects
VST / AU / AAX plugins
Live / interactive
performances
DSP library GUI library Spat
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 31
MaxMSP / PureData objects
VST / AU / AAX plugins
Live / interactive
performances
DSP library GUI library
Mixing / post-production
Spat
Spat : Environments!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
32
MaxMSP / PureData objects
VST / AU / AAX plugins
Live / interactive
performances
DSP library GUI library
OpenMusic
Mixing / post-production
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
33
MaxMSP / PureData objects
VST / AU / AAX plugins
Live / interactive
performances
DSP library GUI library
OpenMusic
Mixing / post-production
(Computer Aided)
Composition
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 34
MaxMSP / PureData objects
VST / AU / AAX plugins
Live / interactive
performances
DSP library GUI library
OpenMusic
Mixing / post-production
matlab
(Computer Aided)
Composition
Spat : Environments!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
35
MaxMSP / PureData objects
VST / AU / AAX plugins
Live / interactive
performances
DSP library GUI library
OpenMusic
Mixing / post-production research /
simulations
matlab
(Computer Aided)
Composition
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
36
MaxMSP / PureData objects
VST / AU / AAX plugins
Realtime rendering Offline rendering
Live / interactive
performances
DSP library GUI library
OpenMusic
Mixing / post-production research /
simulations
matlab
(Computer Aided)
Composition
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
37
• Main advantages & drawbacks of each environments
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
38
• Main advantages & drawbacks of each environments
MaxMSP / PureData objects
Live / interactive
performances
Flexible and powerful rendering
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
39
• Main advantages & drawbacks of each environments
MaxMSP / PureData objects
Live / interactive
performances
Flexible and powerful rendering
Lack of authoring tools (essentially mapping of controllers/sensors)
inadequate for manipulation of temporal structures
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
40
• Main advantages & drawbacks of each environments
VST / AU / AAX plugins
Mixing / post-production
Robust timeline/transport, automation, edition
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
41
• Main advantages & drawbacks of each environments
VST / AU / AAX plugins
Mixing / post-production
Robust timeline/transport, automation, edition
Inappropriate for large multichannel productions
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
42
• Main advantages & drawbacks of each environments
OpenMusic
(Computer Aided)
Composition
Efficient/flexible manipulation of temporal and musical structures
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
43
• Main advantages & drawbacks of each environments
OpenMusic
(Computer Aided)
Composition
Efficient/flexible manipulation of temporal and musical structures
Need for spatial authoring tools;
Tied to an offline workflow
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Goals!
44
• Improve workflow • Inter-exchange / inter-operability
• Reduce the gap between realtime and offline approaches
MaxMSP / PureData objects
VST / AU / AAX plugins
Realtime rendering Offline rendering
Live / interactive
performances
DSP library GUI library
OpenMusic
Mixing / post-production research /
simulations
matlab
(Computer Aided)
Composition
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 45
Compositional tools!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
OM-Spat!
46 J. Bresson & M. Schumacher (ICMC 2011)!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Observational Study!
47
Interviews with 4 composers at Ircam (~ 1h30)!
!
Observe how composers explore and implement spatial sound during the composition process.!
!
• Observations!
• Critical incidents!
• Brainstorming!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Results: trajectories & geometrical contexts!
48
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Results: time organization!
49
Antescofo (P1)!Max Cues (P2)!DAW (P3)!Score (p4)!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Results: problems and suggestions!
50
Studio vs Stage adaptation (all)!Trajectories identification and speed visualization (all)!
!
Quick input for creating new trajectories (P1 – P2)!
Room models and space transformations (P2 – P3)!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Design implications!
Support gestural/quick input!• Drawing & input devices!• Generators and transformations!
!
51
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Design implications!
Support gestural/quick input!• Drawing & input devices!• Generators and transformations!!
Parameters Visualization & Geometrical context!!!
52
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Design implications!
Support gestural/quick input!• Drawing & input devices!• Generators and transformations!!
Parameters Visualization & Geometrical context!!Timeline : !
• Focus + Context!• Time manipulations!
!
53
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
OpenMusic prototypes!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Room and parameters visualization!
55
Speed scale! Anaglyph!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Gestural Input!
56
Input from external controllers/sensors (OSC)!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spatial scenes!
57
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Timelines challenges!
Handle various data!• Gestures / Computed trajectories / no time defined!!
Support time operations!• Time stretch / Synchronize / Interpolate!
58
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Timelines challenges!
Handle various data!• Gestures / Computed trajectories / no time defined!!
Support time operations!• Time stretch / Synchronize / Interpolate!
!!Goal: provide tools for interacting with temporal data!
59
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
TimedObjects in OpenMusic!
Objects with times!Different time labels!
• defined!• not defined!• master !
60
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
TimedObjects in OpenMusic!
Objects with times!Different time labels!
• defined!• not defined!• master !
Have different strategies for!“filling” missing time values!and inserting data at time!
• Constant speed!• Constant duration!!!
61
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
TimedObjects in OpenMusic!
Objects with times!Different time labels!
• defined!• not defined!• master !
Have different strategies for!“filling” missing time values!and inserting data at time!
• Constant speed!• Constant duration!!
Ongoing work: specifying time operations!
62
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Discussion & Perspectives!
Support quick input!Ø Generators and transformations!
!
63
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Discussion & Perspectives!
Support quick input!Ø Generators and transformations!
Parameters visualization & geometrical context!Ø Non color based mappings, 3d models!
64
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Discussion & Perspectives!
Support quick input!Ø Generators and transformations!
Parameters visualization & geometrical context!Ø Non color based mappings, 3d models!
!Timelines and timed objects!Ø Rhythm!Ø Post performance alignment!
65
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Discussion & Perspectives!
Support quick input!Ø Generators and transformations!
Parameters visualization & geometrical context!Ø Non color based mappings, 3d models!
!Timelines and timed objects!Ø Rhythm!Ø Post performance alignment!
Full process in OpenMusic!Ø Real-time audio feedback & offline synthesis!
66
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Trajectoires: mobile application!
67
Explore how mobile devices can support quick/gestural input, edition and comparison of trajectories.
With Xavier Favory, intern at Ircam
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Trajectoires: Interviews!
68
4 composers!
• 3 students!
• 1 experienced!
!
Focus on spatialization control!
Brainstorm with their examples!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 69
Trajectoires: Findings!
Composers think about spatialization early but:!• Need sounds / musicians to explore it!• Usually one of the last aspects they do!!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 70
Trajectoires: Findings!
Composers think about spatialization early but:!• Need sounds / musicians to explore it!• Usually one of the last aspects they do!
!Different strategies:!!
• Recording improvisations!• Writing automations!• Movements in Max (line)!
!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 71
Trajectoires: Findings!
Composers think about spatialization early but:!• Need sounds / musicians to explore it!• Usually one of the last aspects they do!
!Different strategies:!!
• Recording improvisations!• Writing automations!• Movements in Max (line)!
!Suggestions (after trying a drawing app)!
• Real time audio feedback!• Store and replay trajectories for comparison!• Edit existing gestures!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Trajectoires!
72
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Trajectoires: Explorations with composers!
73
Technology probes approach (Hutchinson et al CHI 2003)!
Observe & understand in real world settings!
Test & assess the technology!
Generate new ideas!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Trajectoires: Results!
ü Create and store trajectories during rehearsals and use them during the concert.!
ü Move to test in different locations of the room.!ü Extensively use the replay to compare and identify best
trajectories.!!
74
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Trajectoires: Results!
ü Create and store trajectories during rehearsals and use them during the concert.!
ü Move to test in different locations of the room.!ü Extensively use the replay to compare and identify best
trajectories.!!Ø Segmented drawing!Ø Source selection for a curve!!
75
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Trajectoires: Results!
ü Create and store trajectories during rehearsals and use them during the concert.!
ü Move to test in different locations of the room.!ü Extensively use the replay to compare and identify best
trajectories.!!Ø Segmented drawing!Ø Source selection for a curve!!² Edition!² Storage (librairies) !!² Time manipulations!² Multitouch + sensors for richer input!!
76
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 77
Mixing & Post-production
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Context & Motivations!
78
Three paradigms for multichannel productions :
• Channel-based (5.1 or alike)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Context & Motivations!
79
Three paradigms for multichannel productions :
• Channel-based (5.1 or alike)
• Scene-based (Ambisonic)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Context & Motivations!
80
Three paradigms for multichannel productions :
• Channel-based (5.1 or alike)
• Scene-based (Ambisonic)
• Object-based
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Context & Motivations!
81
Three paradigms for multichannel productions :
• Channel-based (5.1 or alike)
• Scene-based (Ambisonic)
• Object-based
(also hybrid approaches e.g. Dolby Atmos)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 82
Three paradigms for multichannel productions :
• Channel-based (5.1 or alike)
• Scene-based (Ambisonic)
• Object-based
(also hybrid approaches e.g. Dolby Atmos)
Object-based approach has several advantages (adaptation to various setups, user interactivity, etc.)
Context & Motivations!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 83
Three paradigms for multichannel productions :
• Channel-based (5.1 or alike)
• Scene-based (Ambisonic)
• Object-based
(also hybrid approaches e.g. Dolby Atmos)
Object-based approach has several advantages (adaptation to various setups, user interactivity, etc.)
Current shift of content producers towards object-based spatialization (3D movies, binaural broadcast, interactive streaming, etc.)
Context & Motivations!
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Context & Motivations!
84
Object-oriented spatial processors are :
• plenty
• efficient and offer great possibilities
• mostly integrated into real-time environments (Max/MSP, PureData, etc.)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Context & Motivations!
85
Object-oriented spatial processors are :
• plenty
• efficient and offer great possibilities
• mostly integrated into real-time environments (Max/MSP, PureData, etc.)
Constraints :
• authoring of spatial sound scenes remains a major challenge (real-time environments are ill-suited to manipulation of temporal structures such as trajectories)
• DAWs provide an efficient timeline and transport bar, but they lack flexibility for multichannel streams
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Workflow!
86
Proposed approach :
• Mixing, editing and automation within the DAW
• Spatial rendering delegated in a remote application
• OSC protocol for inter-applications communication (spatialization metadata)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Workflow!
87
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Workflow!
88
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Workflow!
89
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Workflow!
90
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Workflow!
91
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Workflow!
92
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Limitations & Perspectives!
93
• Flexible / generic plugin that eases inter-exchange
• Not (yet) a standardized format for metadata
• Still tedious to setup
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Limitations & Perspectives!
94
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 95
Conclusion
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Conclusion!
96
• Improve workflow • Inter-exchange / inter-operability
• Reduce the gap between realtime and offline approaches
MaxMSP / PureData objects
VST / AU / AAX plugins
Realtime rendering Offline rendering
Live / interactive
performances
DSP library GUI library
OpenMusic
Mixing / post-production research /
simulations
matlab
(Computer Aided)
Composition
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Future work!
97
• Improve workflow • Inter-exchange / inter-operability
• Reduce the gap between realtime and offline approaches
Ø Only trajectories but other paradigms
Ø Interoperability with other tools
Ø Evaluations in musical context
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Thibaut Carpentier & Jérémie Garcia
UMR 9912 STMS IRCAM — CNRS — UPMC!
Processing, Authoring and Composing of Spatial Sounds (using Spat and OpenMusic)!
Workshop on Interactivity in Music Composition and Performance
May 5th 2015 CIRMMT — McGill University, Montreal
ANR-13-JS02-0004