breath as an embodied connection for performer-system collaborative interaction
DESCRIPTION
Breath as an Embodied Connection for Performer-System Collaborative Interaction. Greg Corness Thecla Schiphorst Simon Fraser University. Overview. Background + Literature Review Initial Inspiration Key Interaction paradigms The Ariel System Methodology - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/1.jpg)
1
![Page 2: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/2.jpg)
Breath as an Embodied Connection for Performer-System
Collaborative Interaction
Greg Corness Thecla Schiphorst
Simon Fraser University
2
![Page 3: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/3.jpg)
3
Overview Background + Literature Review
Initial Inspiration Key Interaction paradigms
The Ariel System Methodology
Studio Session (Performers improvising with Ariel) Interview technique
Procedures for Data Analysis The Three Primary Results Conclusion + Contribution
![Page 4: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/4.jpg)
“The holy grail for me as a director is to produce a dramatic relationship between performer and
media, to grant media real agency, casting them in a role on par with the live performers” [Saltz, D.
“Live Media: Interactive Technology and Theatre”]
4
![Page 5: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/5.jpg)
5
Background – Literature ReviewPerformer- System Interaction: the interaction
between a Performer and a Computer SystemPerformer-Performer Interaction: the interaction
between Two Human Performers
![Page 6: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/6.jpg)
6
Background (Performer-System)Traditional ‘Sense- Respond’ paradigm tends to
focuses on after-the-event interaction.
Rowe, R. (1999). The Aesthetics of Interactive Music Systems.
Winkler, Todd. (1995). Making Motion Musical: Gesture Mapping Strategies for Interactive Computer Music.
![Page 7: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/7.jpg)
7
Background (Performer-System)Traditional Sense- Respond paradigm tends to
focuses on After-the-Event interaction.Multi-modal base interaction considers the
system’s ability to track and anticipate the performer by sensing more channels of information.
Camurri, Antonio et. al. EyesWeb: Toward Gesture and Affect Recognition in
Interactive Dance and Music Systems.
![Page 8: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/8.jpg)
8
Background (Performer-System)Traditional Sense- Respond paradigm tends to
focuses on After-the-Event interactionMulti-modal base interaction considers the
system’s ability to track and anticipate the performer by sensing more channels of information
The performer’s ability to anticipate the system is rarely discussed.
![Page 9: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/9.jpg)
9
Background (Performer-Performer)Performance practiceResearch in neuro-scienceSocial Cues in Embodied Conversational Agent
![Page 10: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/10.jpg)
10
Background (Performer-Performer)Performance practice
Agency, Trust, Collaboration, Negotiation Intuition related to Intention
Research in neuro-scienceSocial Cues in Embodied Conversational Agent
Lockford, L., & Pelias, R. J. (2004). Bodily Poeticizing in Theatrical Improvisation: a Typology of Performative Knowledge.
Bogart, A., & Landau, T. (2005). The Viewpoints Book: a Practical Guide to Viewpoints and Composition
![Page 11: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/11.jpg)
11
Background (Performer-Performer)Performance practiceResearch in neuro-science
Mirror neurons Cognitive latency
Social Cues in Embodied Conversational Agent
Gallese, V. et. al. (2007). Intentional Attunement: mirror neurons and the neural underpinnings of interpersonal relations.
Wagner, D. M. (2003). The Illusion of Conscious Will
![Page 12: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/12.jpg)
12
Background (Performer-Performer)Performance practiceResearch in neuro-scienceSocial Cues in Embodied Conversational Agent
Generating facial expressions Adding Breath Sounds
Foster, M. E. (2007). Enhancing Human-Computer Interaction with Embodied Conversational Agents
Whalen, D. H. (1995). The Effect of Breath Sounds on the Perception of Synthetic Speech
![Page 13: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/13.jpg)
How can the interaction between a movement based performer and an autonomous generative
music system use breath to leverage the performer’s sense of intuition?
13
![Page 14: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/14.jpg)
14
Click icon to add picture
Research Design
-Performers invited to improvise with the system over several sessions.-System designed to act as an autonomous agent generating music in the improvisational sessions. -System provided two basic conditions 1) presenting a preparation breath before it acts, 2) act without any breath.
![Page 15: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/15.jpg)
15
The Ariel SystemDesigned to be
responsive
Designed to be Autonomous
Incorporates a model of embodied/social cues for projecting intention.
![Page 16: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/16.jpg)
16
The Ariel SystemDesigned to be
responsive
Designed to be Autonomous
Incorporates a model of embodied/social cues for projecting intention.
Responds to performer’s actions through a Computer Vision Tracking Responding to motion Responding to shapes
![Page 17: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/17.jpg)
17
Shape tracking Motion tracking
![Page 18: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/18.jpg)
18
The Ariel SystemDesigned to be
responsive
Designed to be Autonomous
Incorporates a model of embodied/social cues for projecting intention.
Responds to performer’s actions through a Computer Vision Tracking Responding to motion Responding to shapes
Combine external and internal influences on the system’s response using weighted random processes
![Page 19: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/19.jpg)
19
![Page 20: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/20.jpg)
20
The Ariel SystemDesigned to be
responsive
Designed to be Autonomous
Incorporates a model of embodied/social cues for projecting intention.
Responds to performer’s actions through a Computer Vision Tracking Responding to motion Responding to shapes
Combine external and internal influences on the system’s response using weighted random processes
Incorporated a simulated breathing behaviour
![Page 21: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/21.jpg)
21
Basis for modeling relationship between the Preparation Breath and following gesture
BreathBite Tail
Phrase
Loudness
![Page 22: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/22.jpg)
22
The Ariel SystemThe system was designed and tested over three pilot
studies.
“It feels like ... It feels musical, as in…what feels good...it feels like it makes sense” (with breath)
“… it feels understandable ‘you know why there was silence” (with breath)
“… it felt like more background to me this time…like …it was more like a track of music playing..” (without breath)
“…(it) felt more chaotic” (without breath)
![Page 23: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/23.jpg)
23
Studio SessionsSystem switched between presenting/not
presenting breath to provided comparison experiences for performers.
Structured improvisations guided performers to avoiding habitual approaches in interacting with a computer system.
Interviewing technique facilitating thick description of experience
![Page 24: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/24.jpg)
24
Interview TechniqueBased on Interview of Explicitation
(Petitmengin)Re-living in an “inner way”Guiding performer’s attention of the experiencePutting into words
Petitmengin, C. (2006). Describing one’s subjective experience in the second Person:
an Interview Method for the Science of Consciousness.
![Page 25: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/25.jpg)
25
Interview TechniqueRe-living in an “inner way”
Metaphor of video to encourage reflection Focus on selected Sensory Triggers found by
performer Sensitive to Body and verbal cues of Re-living
Guiding performer’s attention of the experience
Putting into words
![Page 26: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/26.jpg)
26
Interview TechniqueRe-living in an “inner way”Guiding performer’s attention of the
experience Stabilizing Attention Move from representation to experience using how
questions Directing attention to dimensions of experience
Putting into words
![Page 27: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/27.jpg)
27
Interview TechniqueRe-living in an “inner way”
Guiding performer’s attention of the experience
Putting into words Open use of language and gestures
![Page 28: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/28.jpg)
28
Data AnalysisSeries of procedures
Close reading Open coding (developing themes) Construction of models
synchronic (independent of time) diachronic (time dependent)
Construction of a General model
Petitmengin, C. (2006). Describing one’s subjective experience in the second Person: an Interview Method for the Science of Consciousness.
![Page 29: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/29.jpg)
29
Affordance from the Simulated Breath
Related observations of experience
1 Perceiving System Intention Conscious perception of a signalEmbodied understanding of gesture quality
2 Coordinating with quality of action
Increased ability to connect with new and complex material
3 Understanding and connecting with others.
Attention to presence and social expectations
![Page 30: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/30.jpg)
30
Close Reading - (Perceiving Intention)The simulated breath is perceived as a cue The volume of the breath affects the
performer’s perception of the system’s intention.
![Page 31: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/31.jpg)
31
Perception of the breath as a cue included: A conscious acknowledgment of the breath as a signal An embodied understanding of the upcoming gesture
“I find it was easier to expect. Although it (the music) was still really fast and sporadic at least you get a sense of ‘ it’s coming…. it’s coming… it’s coming” (system with breath).
Close Reading - (Perceiving Intention)
![Page 32: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/32.jpg)
32
The volume of the breath affected the perception of the system’s intention as unnatural or demanding.
“When the breath is loud it sounds like an action being taken, but when it is soft it sounds like a physical cue as if the flute is in the room and getting ready to act.”.
“ … so it was no-longer a command or anything like that, it was like a cue… so it cues you that the music is starting but it doesn’t impede in a way…it doesn’t command you to follow…”
Close Reading - (Perceiving Intention)
![Page 33: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/33.jpg)
33
Affordance from the Simulated Breath
Related observations of experience
1 Perceiving System Intention Conscious perception of a signalEmbodied understanding of gesture quality
2 Coordinating with quality of action
Increased ability to connect with new and complex material
3 Understanding and connecting with others.
Attention to presence and social expectations
![Page 34: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/34.jpg)
34
Open Coding – (Quality of Action)Separating Participant’s CommentsPerformer-Performer communication
Belief/ AssumptionsDescription of experience
Approaches to Improvisation
Belief/AssumptionsDescription of experience
Experience with the Ariel System
Belief/ AssumptionsDescription of experience
![Page 35: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/35.jpg)
35
![Page 36: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/36.jpg)
36
“we were starting to just take silences
“it felt wrong to make big motions on complete silence
“breath was the safest time to move because you know the music is going to start
“there was more freedom with no breath…”
![Page 37: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/37.jpg)
37
Open Coding - (Quality of Action)
![Page 38: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/38.jpg)
38
Model of Expected Reciprocation“… it felt like more background to me this time…like …it was more like a track of music playing..” ( no Breath)
“I was surprised at how responsive…it felt like there was somebody…like give and take” (full Breath)
![Page 39: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/39.jpg)
39
Model of Expected Reciprocation“the flute breaths and we say go ahead and sometimes it doesn’t and we are like ‘you let us down’ (with breath)
“I found I was responding to multiple things at once so picking up elements of what M and S and the music were doing. I also felt I could pay attention to effort and energy rather then precise movement” (with breath)
![Page 40: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/40.jpg)
40
Micro and Macro-actions Macro-actions refers to large actions undertaken
with conscious intentionality . micro-actions refer to small actions performed
with little awareness or conscious intention Addresses the the affect of performer’s
attention Perceiving the system as one of the partners Social and artistic expectation
![Page 41: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/41.jpg)
41
Stages in Performers’ Dynamic knowledge
Pre-improvisation Own expectations, training affecting interpretation of rules, tasks and environment
Start of Improvisation Sense presence, attention, in the room. Assigning of roles of elements still affected by previous experience
Reflection Perceive actions and states of self and othersPerceiving connection/meaning with actions
Construct New Knowledge
Develop variations of significant gesturesDevelop new connection with performersShift attention to perceived relevant sensations
Test Act in the moment based on new knowledge
![Page 42: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/42.jpg)
42
Pre-Knowledge/
Start of Improvisati
on
Reflection
New Knowledg
e
Test
![Page 43: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/43.jpg)
43
Stages in Performers’ Dynamic knowledge
Reflection Perceive actions and states of self and othersPerceiving connection/meaning with actions
Construct New Knowledge
Develop variations of significant gesturesDevelop new connection with performersShift attention to perceived relevant sensations
“…you can anticipate what is coming next-even though it wasn’t patterned.” (with breath) “My body know the length of the gesture by learning the breathing of the system”
![Page 44: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/44.jpg)
44
Affordance from the Simulated Breath
Related observations of experience
1 Perceiving System Intention Conscious perception of a signalEmbodied understanding of gesture quality
2 Coordinating with quality of action
Increased ability to connect with new and complex material
3 Understanding and connecting with others.
Attention to presence and social expectations
![Page 45: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/45.jpg)
45
![Page 46: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/46.jpg)
46
![Page 47: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/47.jpg)
47
![Page 48: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/48.jpg)
48
![Page 49: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/49.jpg)
49
![Page 50: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/50.jpg)
Conclusion
How can the interaction between a movement based performer and an autonomous generative
music system use breath to leverage the performer’s sense of intuition?
50
![Page 51: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/51.jpg)
51
Conclusion (Contributions)Designed Implemented and tested an
interactive autonomous system which uses breath as a social cue to leverage performer’s intuition.
Developed a model of interaction between Performers and autonomous computer system that addresses social cues for intuition and anticipation.
Developed a Methodology for accessing, collecting and evaluating performer experience data in performance
![Page 52: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/52.jpg)
52
Conclusion (Contributions)Designed, Implemented and tested an interactive
autonomous system which uses breath as a social cue Demonstrated the use/ possibilities of social cues as a
parameter in Performer-System interaction Demonstrated Breath as a mechanism for the
embodied knowing of the system’s intentions New Knowledge of interaction between Performers
and autonomous computer systemDeveloped a Methodology for accessing, collecting
and evaluating performer experience data in performance
![Page 53: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/53.jpg)
53
Conclusion (Contributions) Implemented a model of social cues in interactive
performanceDeveloped a model of interaction between
Performers and autonomous computer system Illustrated the importance of social cues in the
experience of interaction Identified three primary affordances of a simulated
breath in relation to the performer’s experience interacting with a system
Developed a Methodology for accessing, collecting and evaluating performer experience data in performance
![Page 54: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/54.jpg)
54
Conclusion (Contributions) Implemented a model of social cues in
interactive performanceNew Knowledge of interaction between
Performers and autonomous computer systemDeveloped a Methodology for accessing,
collecting and evaluating performer experience data introduces a new approach to studying interactive
performance paradigms. Addressing identified issues in collecting thick
description data on performer’s experience
![Page 55: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/55.jpg)
Conclusion (Limitations)
Current model for breathing behaviour quite course
Model acknowledges only a single social cueProviding time for getting to know the system
55
![Page 56: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/56.jpg)
56
Conclusion (Future Work) Improving the breathing model Incorporation intuition into the Ariel system Investigating other body intention cues such as
vision and kinaesthetic Explore the effect of attention and association
![Page 57: Breath as an Embodied Connection for Performer-System Collaborative Interaction](https://reader036.vdocuments.site/reader036/viewer/2022081515/5681657f550346895dd817eb/html5/thumbnails/57.jpg)
Thank you
57