research report (es)sense

106
REPORT (ES)SENSE JOB HUBERTS

Upload: job-huberts

Post on 22-Mar-2016

241 views

Category:

Documents


0 download

DESCRIPTION

Outcomes of (Es)Sense research project by Job Huberts

TRANSCRIPT

Page 1: Research Report (Es)Sense

ITERATION ROUND #1

REPORT(ES)SENSE

JOB HUBERTS

Page 2: Research Report (Es)Sense

(ES)SENSE

Page 3: Research Report (Es)Sense

3The (Es)Sense project explores

the influence of perceptive qualities in objects around us. In several projects over the years, research has been done on the way people perceive their selves and the world around them. Perceptive qualities in products can make the interaction more personalized and therefor more pleasant.

SEMRE & MESRO is a research set up designed to explore the perceptual interplay between a spatial event and the intervention of a person in this. SEMRE is an environment that creates sound based on the behavior of the visitor, MESRO tries to explore the space. The behavior of the two artifacts reflects human like behavior in order to create an interaction on human level. The research outcomes can be translated to a real world context in order to create design notions.

Abbreviations and concepts used in this report are explained in the chapter “Concepts list” at the end of the report.

ABSTRACT

Page 4: Research Report (Es)Sense

INTRODUCTION

Page 5: Research Report (Es)Sense

5(Es)Sense_ Abstract

Introduction_ Index_ Project description_ Vision on project_ Process

Pressure Cooker_ Goal_ Pressure cooker_ Conclusion & Reflection

Iteration round #1_ Connection to Literature_ Context_ Research Question_ Two artifacts_ Form Design_ Goal first experiments_ Acting out experiments _ Test results_ Conclusion

Iteration round #2_ Abstracted behavior_ Design of artifacts _ Technology

_ Experiments_ Reflection

Iteration round #3_ New set up_ Perceptual crossing_ Context, Question &

hypothesis_ Design of SEMRE_ Design of MESRO_ Programming_ Programming schemes_ Final set up_ Set up test_ Experiments _ Outcomes / Results_ Conclusions_ Improving parameters

Conclusion_ Design opportunities_ Connection to literature_ Future steps_ Reflection_ Concepts list_ References

Appendix

INDEX

Page 6: Research Report (Es)Sense

6The (Es)Sense project explores

the idea of perceptive qualities in objects. It is questioned how perceptive behavior can be designed in an artifact and engage people in a meaningful reciprocal interplay with this artifact.

The project is situated in the Wearable Senses topic ‘Space’ in which we explore textile applications in our environment. Exploration and integration of textile properties with technology, through making, is highly valued in the project.

PROJECT DESCRIPTION

Page 7: Research Report (Es)Sense

7This research project was used as a

tool to support statements that I make in my vision on design and society. The project provided the matrix to do my research in phenomenology and in particular the way how people perceive perceptive behavior. I, as a designer and world beholder, see the world we live in as a nice place full of unexpected events. In my opinion, people should appreciate it more to look around and experience their surroundings. In this research project I want to investigate how people can be triggered to explore and what the role of the human body is in this.

The main method I use is Research through Design (RtD). For me there is great value in creating a physical hypothesis for a question and testing this hands on. I think this is important because people will be experiencing the hypothesis with their body, providing more sincere outcomes. This means that there will be several moments during the project where people are used as reference and input for next iterations.

Starting with the principle of perceptual crossing (perceiving while being perceived) provides an opportunity to investigate how people will react on personified interactions. Do they think it is more appealing? And can it trigger them to start exploring?

VISION ON PROJECT

Page 8: Research Report (Es)Sense

8

Decisioning

Thinking

Making

Exploring

Envisioning

Introduction

Iteration Round #1 Iteration Round #3

Pressure Cooker

Conclusion

Iteration Round #2

Project description

Vision on project

Goal

Pressure Cooker

Conclusion

Reflection

Building Bird

Literature

ExploringContext

ResearchQuestion

Question& Hypothesis

Two Artifacts

FormDesign

GoalFirst Experiments

Acting OutExperiments

Test Results

Conclusion

Abstracted Behavior

Design ofArtifacts

InteractionDiagram

TechnologyProgramming

Experiments

Reflection PerceptualCrossing

New Setup

Context

Design of SEMRE

Design of MESRO

FinalSetup

SetupTest

Experiments

Outcomes/Results

Conclusions

ImprovingParameters

DesignOpportunities

Connectionto Literature

FutureSteps

Reflection

ExploringPhenomenology

PROCESS

Page 9: Research Report (Es)Sense

9

Decisioning

Thinking

Making

Exploring

Envisioning

Introduction

Iteration Round #1 Iteration Round #3

Pressure Cooker

Conclusion

Iteration Round #2

Project description

Vision on project

Goal

Pressure Cooker

Conclusion

Reflection

Building Bird

Literature

ExploringContext

ResearchQuestion

Question& Hypothesis

Two Artifacts

FormDesign

GoalFirst Experiments

Acting OutExperiments

Test Results

Conclusion

Abstracted Behavior

Design ofArtifacts

InteractionDiagram

TechnologyProgramming

Experiments

Reflection PerceptualCrossing

New Setup

Context

Design of SEMRE

Design of MESRO

FinalSetup

SetupTest

Experiments

Outcomes/Results

Conclusions

ImprovingParameters

DesignOpportunities

Connectionto Literature

FutureSteps

Reflection

ExploringPhenomenology

Page 10: Research Report (Es)Sense

PRESSURE COOKER

Page 11: Research Report (Es)Sense

11To get acquainted to the subject

of perceptual crossing, a one-week pressure cooker was held. With a group of master students a brainstorm was done on the possible directions within the project. The emphasis was put on triggering the senses. How can we stimulate the use of certain senses by showing specific behavior? Four categories where defined; sound, light, (fleeing) movement and touch. Each category was to be elaborated on by two students. The final goal was to reflect on the outcomes and make a list of interesting research subjects in the field of perceptual crossing.

GOAL

Page 12: Research Report (Es)Sense

12The fleeing movement was based

on the behavior of birds flying around. When approached by a person, they will get scared and fly away, but eventually they might return out of curiosity. Often, this behavior is shown in flocks, transforming the movement from individual to collectivistic. There is not one mindset that decides the behavior, the behavior is a result of the whole. For this pressure cooker, the movement of one bird was taken as starting point.

A bird like object was created and mounted to the ceiling. It had the ability to go up and down. It was programmed to slowly fly around (go up and down), while scanning the environment with his sensor. When approached, the bird would fly up to get away from the approaching person. The bird would perceive whether the person was staying or leaving and adapted the behavior on this human behavior. The focus was put on the senses of the eyes, the bird had one moving “eye” to perceive the world. The involved person was also

dependent on his eyes, canceling out senses as hearing and touch. In the case the eye of the robot bird would be taken away his connection to the world would disappear and the perceptive behavior would vanish, resulting in an abortion of the interaction.

PRESSURE COOKER

Page 13: Research Report (Es)Sense

13

PICTURE

13The bird, created during the pressure cooker, hanging from the ceiling

Page 14: Research Report (Es)Sense

SubjectObject with

perceptive qualities

action

action

perceiving perceptive activity of subject

perceiving perceptive activity of object

sensory feedback

sensory feedback

1414Model of perceptual crossing between subject and artifact with perceptive qualities

Page 15: Research Report (Es)Sense

15This installation consisted of one

perceptive body. To get to know more about the behavior of flocks, more bodies would have to be added in order to create a bigger perceiving body consisting of multiple birds. In an evaluative session with the involved students and Eva Deckers it came forth that the form of the object did not match its behavior. A bird, or plane, flies horizontal, not vertical. The fact that the object did look like a bird turned this observation into a problem, the form and behavior did not match. In order to solve this it would be good to create more ambiguity, leaving room for own interpretation. Only then, there would be space to focus on triggering the senses of a person instead of triggering his mind.

The fact that the sensor was part of the actuating body made the interaction stronger, creating a standalone object that was able to perceive and to react on it. This strong connection between sensors and actuators gives the impression that the object is perceiving and

directly reacting on his environment autonomously.

Looking at the next step to be taken, defining a research question, the behavior and movement in space appealed. Having explored this during the pressure cooker it appeared that it provided opportunities to investigate the initial research goal; investigating the role of the human body in exploring space.

CONCLUSION & REFLECTION

Page 16: Research Report (Es)Sense

ITERATION ROUND #1

Page 17: Research Report (Es)Sense

17To acquire more scientific

background on the subject of perceptual crossing, papers were consulted. In the department of Industrial Design already quite a few projects have be done in the field of phenomenology and perceptual crossing. The (Es)Sense project is based on the PhD project of Eva Deckers. The first step in this project is to understand why one would design for perceptual crossing. “Design for perceptive behavior, relevant to enhance the quality in interaction in the design of intelligent artifacts” (Deckers et al. 2012). Creating perceptive behavior in an artifact can aid in the personalization of the interaction one has with the artifact. If one feels emotionally connected to an artifact, the interaction threshold will be lower.

Perception happens all the time, when one perceives another being or object. Perceptual crossing, however happens only when the perceiving one perceives that the other being or object is perceiving back.

“Perception is inherently interactive and participatory: it is a reciprocal interplay between the perceiver and the perceived” (Abram, 1996). The fact that we perceive the world is caused by the fact that our body is a thing that is able to experience. Our body can touch, smell, see, hear and taste.

CONNECTION TO LITERATURE

Page 18: Research Report (Es)Sense

18For the direction indicated in

chapter “Vision on project” the environment has to be taken into account. How do people perceive an environment? Applying the words of Merleau-Ponty “The body is the centre point of perception”(Merleau-Ponty, 1958), it appears that the environment, or space, is experienced through the body. Our body is our tool to perceive the world. To trigger people to start exploring, an intelligent triggering object has to be created. To give people the perception of sharing a space with an intelligent object, a consideration has to be made between two kinds of perception over time: “perceiving the other as part of environment, versus perceiving the activity of other perceiving me” (Lenay et al. 2007). This means that while exploring a space, the person will base his thoughts about the artifact on the movement of the artifact in space and the way the artifact reacts.

Page 19: Research Report (Es)Sense

19Perceptual crossing occurs at

many different occasions in daily life. It can be experienced while being in a conversation with other people, but also while interacting with an intelligent product or service. To delimit this wide term, a context was set to avoid that the research would get undirected. As stated before, the emphasis of the research would be put on exploring space. Previously in the (Es)Sense project, Thomas van Lankveld did a project (Deckers et al. 2012). which consisted of two autonomous artificial bodies communicating with each other. The event created by the communicating bodies shaped the environment and the way people perceived this space. Building on this project, the interest arose to focus on this artificial conversation in a dynamic space. How do people react on a conversation like this and what feeling does it trigger? An artificial conversation can trigger different reactions and changes in the behavior of people.

The idea described above is quite abstract. Example situations were gathered to make a reference to the real world:

_ Two people are in an intense conversation, the door opens, how does this change the conversation?

_ One is studying, a person enters, how does the book react?

_ Interaction with vending machine, other machine starts attracting attention.

_ A dog is playing with a toy, a cat approaches, what do the dogs do?

_ Two people are in an argument, third person enters the space, how does the conversation change?

CONTEXT

Page 20: Research Report (Es)Sense

20Based on the inspiration got from

the pressure cooker, a first research question was formed. The research subject involves the behavior of people in a space. Second to this, perceptual crossing between three artifacts will be used as a tool to trigger the motion through space. The first version of the research question was composed as follows:

“How do people react on being the third player in an ongoing interaction between two non-human artifacts?”

The reaction of people will be influenced by the mood and behavior of the two artifacts. Based on this behavior, the person could either flee, embrace or get angry. The corresponding hypothesis for the question is:

“Dependent on the shown behavior, people will adapt their reaction and become part of the conversation”.

This hypothesis would be translated into a physical hypothesis; the test installation. With this physical hypothesis an answer to the question could be searched for.

RESEARCH QUESTION

Page 21: Research Report (Es)Sense

Subject

Object #2 withperceptive qualities

Object #1 withperceptive qualities

action

action

perceiving perceptive activity of #1

perceiving perceptive activity of #2

sensory feedback

sensory feedback

action

senso

ry feedback

action

senso

ry feedback

actionsensory feedback

actionsensory feedback perceivi

ng perceptive activ

ity of #

1

21Model of perceptual crossing in case of two perceptive artifactsModel of perceptual crossing in case of two perceptive artifacts

Page 22: Research Report (Es)Sense

SubjectObject with

perceptive qualities

Space withperceptive qualities

action

action

perceiving subject

perceiving object

sensory feedback

sensory feedback

action

sensory feedbackaction

sensory feedback

perceiving space

perceiving subject

action

sensory feedbackaction

sensory feedback

perceiving space

perceiving object

22Model of perceptual crossing in case of one perceptive space and one perceptive artifact

Page 23: Research Report (Es)Sense

23Models of perceptual crossing in different situations, involving multiple subjects and/or perceptive artifacts

Page 24: Research Report (Es)Sense

24 TWO ARTIFACTSLike in the project of Thomas van

Lankveld, two artifacts would form the core of the research installation. The two artifacts would use different senses to observe each other. Also their communication means would differ, providing a ground for the objects to use different senses. One object would be limited to hearing and sending light, the other object would be limited to seeing and sending sound. In this way the objects can establish a unique conversation that allows them to perceive each other. The restrictions of the artifacts can be translated to the human body, they embody the qualities of a blind person versus the qualities of a deaf person.

For research purposes, the behavior of the artifacts has to be manageable to a certain degree. If certain parameters can be controlled, it is easier to acquire the desired research outcomes. The focus can be put on the important aspects of the research, canceling out noise data.

The main parameter for this artifact would be behavior, with a focus on mood. When changing parameters in the coding of the artifact, its behavior will change, making it able to express different moods. Comparing these moods can give insights in how people are affected by intelligent perceiving objects.

To give people the impression they are dealing with an intelligent product, certain human behavior characteristics have to be taken in mind. These notions (Deckers et al. 2012), describe certain characteristics in behavior that express human qualities. For the two artifacts, a number of these notions apply. First, “Focus the Senses”, senses and acting have to be embodied in order to look natural. Perceiving is an active activity, it is based on actions made and the feedback received. Senses will be focused in the direction of the stimuli. This note provides the opportunity to create an object that is looking around and exploring. Next notion

Page 25: Research Report (Es)Sense

25that should be taken into account is the “Reaction to an External Event” of the artifact. When an external event (e.g. a person entering) is added the opportunity arises to create a common history with the artifact. Knowing this, the interaction will become more personalized. It also provides the opportunity to perceive through another artifact or person. When the artifact reacts to an external event, a third artifact or person can notice this.

Page 26: Research Report (Es)Sense

26The form of an object closely

relates to its function. It shows the physical affordances of the object which can be translated in its options and limitations. For the two artifacts, the form would be of importance to show which object has which role in the conversation. The hearing object has to communicate that it can not see or feel, but only sense sound. For the other object goes the opposite, it has to communicate that it can only perceive by seeing.

Since both objects have an explorative nature eventually, this also has to be communicated by the form and behavior. By letting them ‘look’ around to explore this quality can be emphasized. However, this way of communicating behavior has to be grounded in tests.

FORM DESIGN

Page 27: Research Report (Es)Sense

27To prove the first statements

made, experiments with real people would be needed. The most important subject to find evidence for is the fact how people perceive an artificial conversation. Do people regard two artificial objects able to have a conversation? The output of this experiment provides information for the following iteration round.

Besides the perception of an artificial conversation, information can be gained about the behavior of people in a test environment. The way they move through the room and how they use their body can provide inspiration for the further specification of the process. When people stay static or motionless, the focus should be put on triggering people to start moving around. Or, it could be used as inspiration to make the objects more dynamic and start exploring instead of the person. People that move without paying attention to the objects can indicate that the space itself is too revulsive.

GOAL FIRST EXPERIMENTS

Page 28: Research Report (Es)Sense

28The emphasis of this (Es)Sense

research project is put on the quality of bodily skills. It is questioned how bodily skills can help people in exploring space. The body is the tool to explore the world. Based on this notion, the set up of the first experiment session was slightly changed. Instead of building two artifacts embodying human qualities, the two artifacts would be acted out by two human beings. This would provide an interesting view on the bodily characteristics of the future artificial objects. The way people behave as an artifact with limitations provides insights in how to create human like behavior in the artificial objects.

The experiment set up would approach a lab setup. Taking place in a neutral space, the focus would be pulled to the conversations going on. During the experiment, one person would be blindfolded, making him blind and only able to hear. The person would be given a flashlight to use as a communication mean. While the other person would be made deaf, by

putting noise on his ears. This person is provided with a synthesizer to send sound. The interaction possibilities that arise give room for a specified interaction. Both test persons would be allowed to turn around their axes, but not to walk, giving them the affordance of an object that is fixed to its place but with the ability to ‘look’ around.

In the first stage of the experiment the two persons have to try to establish a conversation, using their options and limitations. In the second stage, a third person would enter the test room. This person gets the assignment to interfere in the conversation that is going on between the two human artifacts. This assignment is given to trigger people to explore what is going on. After this stage, a questionnaire is used to interview the three persons as a group. The two human artifacts are asked how they did establish a conversation and what feeling this evoked. Following this, the third participant is asked to give his opinion on the conversation and the degree to which he felt involved.

FIRST EXPERIMENTS

Page 29: Research Report (Es)Sense

Subject

Blind artifactDeaf artifact

initial conversation

action

senso

ry feedback

action

senso

ry feedback

actionsensory feedback

actionsensory feedback perceivi

ng perceptive activ

ity of #

1

2929Setup of acting out sessions, with one deaf person, one blind person and a subject

Page 30: Research Report (Es)Sense

30

PICTURE

30Acting out session with three students to investigate the behavior of artifacts and subject

Page 31: Research Report (Es)Sense

31For this first round of experiments,

a total of seven groups of three people were interviewed (Appendix A & B). People were selected based on their knowledge of the subject of perceptual crossing. People knowing more about the subject were placed in the position of human artifact. People fresh to the subject were placed either in the role of human artifact or of human being.

In the behavior of the human artifacts a clear division between the deaf and blind was seen. The blind person showed a more wait-and-see approach. Without external input, the blind person did hardly ever start to explore or to communicate. The deaf person took more initiative in general, producing sound and trying to get the attention of the blind person. Most blind participants reacted on this by flashing their light. When both persons got used to their capabilities they started experimenting by creating patterns or triggering each other to look around.

The third person was asked to enter the room silently so the conversation would not roughly be distorted. After entering most participants tried to distort the communication line by standing in between the artifacts. If this did not work they started observing what was going on. What could be seen was that people started moving around, spreading their attention between the two artifacts. To interfere they tried to seduce one of the objects by imitating the communication line from the other object. This led to confusion for both objects. Eventually this could lead to a situation where the blind one perceived the third person through the deaf one. This happens when the deaf started to follow the third person and the blind person started following the change in sound production.

ACTING OUT EXPERIMENTS

Page 32: Research Report (Es)Sense

32In the experiment, a division

can be made between the people representing an artifact and the people that symbolize a real human being. Those who represented an artifact indicated that they were able to establish a conversation in which they could communicate with the other human artifact. The intensity of this conversation varied. One couple was able to create patterns based on the light and sound they both created. They indicated that they perceived each other and had the feeling that they were communicating. A next couple indicated that they felt the possibility to communicate but struggled with the way this was done. Five of seven couples indicated that the light (the blind object) followed the sound (the deaf object). The other two couples stated that the sound was following the light. This can be attributed to the fact that the flashlight gave more affordance to give direction with the light beam.

The second group, the people that were assigned to interfere, experienced diverse situations. When entering the

room two out of seven participants had difficulties in recognizing a form of conversation. The other five persons indicated that they perceived a conversation between the human artifacts. The way to interrupt was difficult to determine for participants. Three of seven participants interfered physically, pushing and trying to turn the subjects. This led to a disturbance in the conversation between the artifacts and did not benefit a possible conversation between the person and an artifact. Three of the test persons started talking to the blind object in order to start a conversation and direct the movements of the light. In all three cases the test person did succeed in this. Only in one case the deaf person started following the new movement of the light. The seventh person communicated with the deaf person by hand gestures, trying to let him direct the sound in another direction. The blind person reacted on this change of direction by starting shining around with the flashlight in an attempt to find the source of the sound.

TEST RESULTS

Page 33: Research Report (Es)Sense

33People feel uncomfortable when

entering a room where they don’t know exactly what is going on. This feeling of uncertainty withholds them of exploring the scene at first. When people get to know what is happening they become more free in their attempts to influence and explore the event. The moment people get the feeling that they can apply influence to the conversation marques the moment people become part of the conversation. This moment should take place early in the process.

Providing people the ability to act as an artifact made it possible to investigate how a non-human artifact could move in this situation. This information provides a starting point for the design of the non-human artifacts. People exhibit explorative behavior in order to establish a conversation.

CONCLUSION

Page 34: Research Report (Es)Sense

ITERATION ROUND #2

Page 35: Research Report (Es)Sense

35Based on the shown behavior of

the people taking the role of human artifact the behavior of the artifact was designed. For each artifact a default behavior derived from the acting out experiments. For this, the Embodynamics framework was used (Klooster, 2003).

The blind object embodied an exploring but reactive nature. It reacts on the surroundings, purely based on the sounds it hears. It has a wait-and-see attitude. The way it reacts asks for reaction of an opponent. The turning movements it makes are unsure and staccato. The uncontrolled motions show a way of uncertainty.

The deaf object exudes confidence in its behavior. It is the initiator of the conversation. The deaf one tries to attract the attention of the blind one and directs the conversation initially. Its movements and sound express intensity. In the course of time his patters become more uniform but fragmented.

ABSTRACTED BEHAVIOR

Page 36: Research Report (Es)Sense

36In order to create a clear

connection between the object and the behavior, the form has to be well thought through. The form has to fit the function and behavior of the artifact. This makes the interaction people can have more directed and gives people an indication of the possibilities of the artifact.

Both objects are based on a high standing form like a rounded cylinder or extracted cube. These basic starting forms give the opportunity to knead the form in a way it fits the function. In the base of the form a turning point is placed, giving the artifacts the possibility to turn around and ‘look’ around. The turning point is placed in the base so that the whole body can turn, including the sensors and actuators, allowing the artifact to focus the senses (Deckers et al. 2012).

The blind object is based on a rounded cylinder, this to emphasize on the smooth motions and shy character of the artifact. It has to have a clear light emitting point and a part

that is responsible for the hearing. The first sketches show a fluent form with a big hole for the light and a smaller on top for the microphone. For quick building purposes, this form was translated to a laser cut ready form. The round forms were preserved to keep the expression of the artifact. Since this object emits light and receives sound, it is called SELR.

The deaf object is based on a tapered extracted cube, this to emphasize the dominant role of the object. Like the blind object, is has to have a part to emit sound and a part where light is received. In the sketching stage, this was translated to a high straight form with a speaker in the upper part and a light sensor in the lower part. The MDF laser cut version replicates this and maintains the form. Since this object emits sound and receives light, it is called LESR.

DESIGN OF ARTIFACTS

Page 37: Research Report (Es)Sense

LESRSELR

sends light signals

receives light signals

perceiving perceptive activity of SELR

perceiving perceptive activity of LESR

receives sound signals

sends sound signals

37Model of interaction between SELR and LESR

Page 38: Research Report (Es)Sense

3838SELR and LESR, the two artifacts designed to communicate by sound and light

Page 39: Research Report (Es)Sense

Subject

LESRSELR

action

action

perceiving perceptive activity SELR

perceiving perceptive activity of LESR

sensory feedback

sensory feedback

action

senso

ry feedback

action

senso

ry feedback

actionsensory feedback

actionsensory feedback perceivi

ng perceptive activ

ity of #

1

39Model of interaction between SELR, LESR and subject

Page 40: Research Report (Es)Sense

40Both artifacts have an own

Arduino board to make them operate independent. They communicate analogue, based on the light and sound they emit. To get the behavior clear, flow charts were made, showing the different steps in the behavior. Based on these charts the two artifacts were programmed and different actions were assigned to different events (Appendix C & D). To give SELR and LESR extra capabilities to perceive the world, an ultra sensor was added. This made it possible for them to notice if someone was approaching. The ultra sonic sensor on LESR was not used in the end, since this object is supposed to be blind and reactive on SELR.

During the programming of the artifacts a problem occurred. The signals that both artifacts did send were not strong enough to trigger the sensor on the other artifact. Particularly the synthesized sounds were not strong enough. Due to this, the two artifacts had to stand at a short distance of each other in a completely empty and silent room. Even then, the signals were too soft and low.

In order to still be able to do tests, the artifacts were given more independent behavior. In this way it looks to unprepared participants that something is going on. The objects were still able to sense with their given sensors, but the experiment was not dependent on this anymore.

TECHNOLOGY

Page 41: Research Report (Es)Sense

LESR

SELR

initiate sound

external movementsubject

start conversation

reactionwith sound

exploringmovement

movearound

bleep

no feedback

startconversation

initiate light

external movementsubject

start conversation

reactionwith light

exploringmovement

movearound

flash

no feedback

startconversation

4141PROGRAMMING SCHEMES

Page 42: Research Report (Es)Sense

42During the theme day of Wearable

Senses, a short experiment was done. The set up was similar to the setup of the first experiments, only this time, the human artifacts were replaced by SELR and LESR. People were asked to observe the situation and try to interfere with this after a while. The fact that the two artifacts did not really react on each other distorted the experience. People were perceiving this after a while and this made it hard for them to observe an ongoing conversation. After three sessions the experiments were aborted. The conclusion was that to create a perceiving environment, the artifacts had to be able to perceive each other. Artificial independent behavior shows no perceptive qualities.

EXPERIMENTS

Page 43: Research Report (Es)Sense

Subject

LESRSELR

action

senso

ry feedback

action

senso

ry feedback

actionsensory feedback

actionsensory feedback perceivi

ng perceptive activ

ity of #

1

initial conversation

43Setup of second round of experiments, involving a conversation between SELR and LESR and a subject

Page 44: Research Report (Es)Sense

44Creating SELR and LESR gave

insights in how perceptive qualities can be translated to a artificial body. An artificial body has no direct connection to another artificial body. In order to create this connection, the objects have to relate closely, in shape and behavior. If this is established, the conversation becomes more meaningful and people can try to interfere. To establish perceptive behavior, and finally perceptual crossing, the Design Notions deriving from the PeP+ research project (Deckers, 2012) have to be used as guiding line.

For the next iteration round, the role of the two artifacts has to be evaluated. The relation they had up till now is one on one. However, the behavior of SELR provides opportunities to create a more hierarchic interaction where LESR follows the directions of SELR.

REFLECTION

Page 45: Research Report (Es)Sense

4545Performing the acting out experiment during theme day as introduction to the SELR and LESR experiment

Page 46: Research Report (Es)Sense

ITERATION ROUND #3

Page 47: Research Report (Es)Sense

47Based on the outcomes of the

second iteration the direction of the project was evaluated. When talking about exploring space, space itself is the most important parameter. The space influences how people behave and what they will perceive. “The ‘sense of space’ is constituted by multimode sensory stimulation changing according to the organism’s movement relative to its environment” (Schonhammer, 2004). To investigate this value of space, artifact SELR was transformed to a space. This space would perceive the entering person and the artifact. In reaction on this, sound would be created, creating an ever present soundscape. Within the space a reincarnation of LESR would be placed. This artifact explores space based on the soundscapes, trying to persuade the person to explore the space using his bodily skills. The space will be called SEMRE, since it emits sound and receives movement. The artifact will be called MESRO from now on, this due to the fact that it emits movement (and light) and receives sounds.

NEW SET UP

Page 48: Research Report (Es)Sense

48This new set up means that

the interaction structures changes somewhat. SEMRE creates in cooperation with MESRO an event that is ongoing. The soundscape is created based on movement and is kept going due to the fact that MESRO reacts on the sound with movement. SEMRE seduces MESRO to move and explore. Since MESRO is still blind, the artifact can only identify the entering person based on the sound it ‘hears’ created by SEMRE.

PERCEPTUAL CROSSING

Page 49: Research Report (Es)Sense

Subject

MESROSEMRE

EVENT action

action

perceiving perceptive activity of SEMRE

perceiving perceptive activity of MESRO

sensory feedback

sensory feedback

action

senso

ry feedback

action

senso

ry feedback

actionsensory feedback

actionsensory feedback perceivi

ng perceptive activ

ity of #

1

49Model showing perceptive event

Page 50: Research Report (Es)Sense

SEMRE

MESROSubject

50model of communication between SEMRE, MESRO and subject

Page 51: Research Report (Es)Sense

51The real-world context this

research set up can be seen in is slightly more specified then during the acting out experiments. To make people discover new details in their environment, a for them not completely known environment is needed. Preferably this is a public space that is often visited by people. This can be a railway station (hall versus ticket machine), public square (square versus lamp post) or the main hall of a public building (hall versus stairs). SEMRE represents the public space that is asking to be explored. MESRO represents an artifact that aids SEMRE in this and tries to seduce visitors to a personalized interaction.

The implementation gives opportunities for for example signing in public space, the position of ticket machines or the location of bus stops. The results of this research can be used to create opportunities in product-space interaction design.

The set up of space versus exploring artifact brings a new research question forth:

“Can an adaptive object trigger people to explore a reactive environment?”

The physical hypothesis for this is worked out in SEMRE and MESRO, those two artifacts form the research installation that tries to answer this question. The textual hypothesis is:

“Create a guide that adapts to the behavior of people and the environment and directs people through a space.”

CONTEXT, QUESTION & HYPOTHESIS

Page 52: Research Report (Es)Sense

52SEMRE is the evolved version

of SELR, putting it in the position of conversation trigger. SELR was the object that triggered a conversation, asking from LESR to react and follow. SEMRE embodies a larger entity. In fact SEMRE inhabits the whole space with a body of sound. It has the ability to virtually move through the entire space. It can be seen as a sounds entity floating in the air and flying around. This sound body gets attracted by the visitor and starts following him, creating a sound based ghost of the visitor. SEMRE will adapt his behavior based on the activity of the visitor. If the visitor stands still for a longer time, SEMRE will eventually lose interest and start conversing with MESRO.

The physical embodiment of SEMRE consists of four speakers in this research setup. In every corner a speaker is placed, creating a surround setup. By panning the sound between the four speakers, the virtual location of the sound changes. The four speakers on itself create a space, the space between the four speakers. This does

mean that there is no real need for a physical space consisting of four walls. It can also be a part of a public square or a part of railway platform.

The appearance of the speakers is based on the forms of SELR. The straight form represent the dominant character of SEMRE. On the other hand is the form created in a way that it can easily merge with a public space where a lot of people pass. The face like form is designed to be put on head height to merge between peoples heads.

DESIGN OF SEMRE

Page 53: Research Report (Es)Sense

5353Speaker that provides the sound for the body of SEMRE, a total of four speakers is mounted in the corners

Page 54: Research Report (Es)Sense

54MESRO, the explorative artifact is

evolved from LESR. The emphasis is put on the movement that it uses as tool of expression. MESRO is the one following the directions of SEMRE, in order to explore the environment. MESRO tries to attract the attention of the visitor by searching for it and asking to follow his movements. The artifact will adapt its behavior to the soundscapes that are produced by SEMRE based on the activity of the visitor. Since SEMRE perceives the visitor, MESRO perceives that SEMRE perceives the visitor, the visitor on his turn perceives MESRO perceiving him indirectly. This leads to an indirect way of perceptual crossing.

The way MESRO moves around is based on the Embodynamics Framework (Klooster, 2003). The movement while searching is stretching and intensive. In a matter of space this means that the movements goes through space, but not per se directed. Over time, the movement will accelerate. The movement stops when the visitor is perceived. To seduce

the visitor, the movement becomes more directed and strong, releasing force in the desired direction.

Physically, MESRO changed significantly pertaining to LESR. MESRO is a long arm that has the affordance to reach and look around. It can indicate direction and has the freedom to look around. It is placed on a pedestal, raising it from the ground, and making it more of notable player in the interaction. At the end of the arm, a light is placed. This light is used by MESRO to indicate direction and as an extra communication tool towards the visitor.

DESIGN OF MESRO

Page 55: Research Report (Es)Sense

5555MESRO moving around while exploring the environment in the test lab

Page 56: Research Report (Es)Sense

56In order to create the behavior of

the artifacts, they had to be equipped with a degree of perceptive behavior. To solve the problem that showed up between LESR and SELR, a data wire was put between the two artifacts. In this way SEMRE could directly communicate the position of the person to MESRO. The test person would not perceive this directly since he or she will expect MESRO to sense and act autonomous. This solution was discussed with Eva Deckers, she agreed on the fact that the influence of data wires on the test subject would be negligible (Appendix E & F).

SEMRE had to be able to emit sound based on the movement and position of the person. For this Max MSP was used. In Max MSP it is relatively easy to synthesize soundscapes based on input from a sensor connected with Arduino. To define the space, a base sound was created. This low pitch tone waved through the space like some kind of hearable fog. To embody the sound body, a higher pitch sound was added that moved towards the position of the person. This sound was influenced by the distance people had with respect to a speaker.

PROGRAMMING

Page 57: Research Report (Es)Sense

57MESRO had to be provided

with explorative behavior. In idle mode, MESRO had to explore the space, while communicating with SEMRE. For this, different sets of random explorative behavior were programmed on the Arduino board. In idle mode, SEMRE moves slowly in all directions or investigates one particular direction if SEMRE indicates something interesting. As soon as a person enters the room, the sound body will start following this person and send data to MESRO, telling that something entered. MESRO will adapt its behavior on this and will start exploring that part of the room. For this action, MESRO uses the same data that SEMRE used to determine the position of the person. The servos in MESRO’s body will turn it towards the right position with a bit of tolerance to make it look more natural. As long as SEMRE keeps providing data, MESRO will keep looking and moving. This influences the behavior of the person, eventually influencing SEMRE.

Page 58: Research Report (Es)Sense

58The final research setup for the

experiments was built in a separate room. This was done to cancel out the distorting influences of external events. MESRO was placed in the middle of the room on a pedestal. In the four corners, the speakers of SEMRE were placed, creating a perceptive space as big as the room. At one side of the room a camera was placed to tape the experiments.

FINAL SET UP

Page 59: Research Report (Es)Sense

Data wire

s

SEMRE

MESROSubject

Sound body

Speakers SEMRE

initial conversation

59Setup of final experiments in test lab with SEMRE and MESRO and subject

Page 60: Research Report (Es)Sense

60The experiment setup was based

on the Lab, Field, Gallery approach by Koskinen (Koskinen et al. 2008). A lab setting was created as described before to be able to condition settings. For to the point research this is preferable above a field or gallery setup. Like in previous (Es)Sense experiments, the setup was kept clean and the used physical hypotheses was designed frugal (Deckers et al. 2012). This makes it easier to notice differences in the behavior of test persons.

For evaluating purposes, a questionnaire was created (Appendix G). The information that was supposed to be retrieved from this questionnaire was qualitative, people are asked to explain their own feelings. In order to get the essential information, the questionnaire was formed according to the laddering technique. This technique enables the establishment of a hierarchy of events (Corbridge et al. 1994). With help of open questions people were triggered to give the essences of their feelings during the experiment.

The questionnaire focused on two points in the experiment. First a list of questions was used to explore the feeling people had during the experiment while they where not told what was supposed to happen. They were given the assignment to freely explore the situation and try to understand what was happening. When they considered they were ready, the questions were asked. After this list, the test person was told what the purpose of SEMRE and MESRO was. People got the opportunity to experience the research installation for a second time after which a short evaluating question list was used. This second sessions was meant to test if people did understand the events in general and to compare if their behavior did change to the situation where they did not know what was going to happen.

SET UP TEST

Page 61: Research Report (Es)Sense

61

For this experiment the following hypotheses were tested:

1. People will understand that MESRO follows the sound body created by their own behavior

2. During the second session people will understand the setup and feel more freedom to play around

Page 62: Research Report (Es)Sense

62To test the setup of the experiments,

a pilot test was conducted. Based on this test the questionnaire was specified. After that, the experiments were conducted with a total of seven participants. Participants where selected on their knowledge of the subject of perceptual crossing. This resulted in test persons mainly from the Bachelor and from outside the department.

During the experiments, participants were observed and recorded. Their movements were tracked in order to see how they moved through space and if this changed over time. Most participants walked around all the time, taking breaks for observing the situation. During the questionnaire the emphasis was put on getting to know if people did understand the situation and how they perceived it.

EXPERIMENTS

Page 63: Research Report (Es)Sense

63Pitchfork shoreditch leggings,

american apparel Austin church-key terry richardson fashion axe synth mumblecore sriracha. Cray american apparel wolf raw denim, dreamcatcher blue bottle cosby sweater mixtape butcher ethnic. Wolf gastropub fap, tumblr american apparel +1 chambray art party. Flannel chillwave odd future, literally stumptown high life iphone brunch wolf banksy lo-fi cliche vice hashtag. Twee small batch ennui, thundercats irony skateboard vegan wes anderson. Meh disrupt portland, polaroid vice ugh brunch PBR trust fund. Tumblr iphone post-ironic organic deep v fixie lomo, synth 90’s ethnic irony fashion axe hella.

Church-key letterpress leggings, bespoke VHS fap you probably haven’t heard of them craft beer. Vinyl flexitarian fap wolf ennui, banksy before they sold out hashtag viral mlkshk photo booth trust fund meh deep v. Tofu thundercats polaroid, tattooed jean shorts vegan sartorial wayfarers terry richardson cosby sweater mcsweeney’s art party

fingerstache. Squid disrupt brunch small batch, iphone wayfarers try-hard literally synth narwhal plaid truffaut. You probably haven’t heard of them master cleanse PBR, skateboard organic whatever meh stumptown leggings pickled lomo banjo before they sold out street art. Raw denim actually pitchfork blog cred. Bespoke ethical master cleanse, thundercats tumblr food truck aesthetic small batch occupy.

Wes anderson chambray authentic, tumblr fashion axe cliche viral forage literally bushwick biodiesel. Williamsburg tattooed 90’s plaid odd future. Selfies pour-over ethnic, direct trade readymade fap vice swag VHS terry richardson Austin forage echo park cardigan. Aesthetic hoodie yr swag trust fund. 3 wolf moon post-ironic four loko cliche tumblr letterpress. Chillwave mustache synth blue bottle, jean shorts forage pour-over selfies. Aesthetic mcsweeney’s next level, meh mustache sustainable high life YOLO readymade fap salvia.

CHAPTER 63Experiment with SEMRE and MESRO to validate the perceptive behavior of the installation

Page 64: Research Report (Es)Sense

64The participants that had

difficulties with understanding the situation indicated that this was due to the unclear connection between SEMRE and MESRO. This can be assigned to the fact that MESRO did not always have a one-on-one reaction on the position of people. If people moved fast through the space, MESRO got delayed and kept searching in just-left corners.

Most participants got the feeling that both SEMRE and MESRO were perceiving them eventually. People interpreted the behavior of MESRO as recognizing, even when MESRO was not following the directions of SEMRE. This shows that the suggestion that an artifact is reactive can adapt in creating a relation between person and artifact.

People that did understand the situation had a more positive feeling about it (Appendix H & I). People who did not understand the situation felt neglected and did not enjoy the situation. This indicates that a better perceptual connection provides a smoother interaction. Their experience in the second session was shorter than people who did not understand it the first time.

Those who discovered the meaning of the event on their own had a playful interaction with MESRO and used SEMRE to guide MESRO and to be guided. For them the event provided an opportunity to have a perceptive conversation with MESRO, while SEMRE being the intermediator.

OUTCOMES / RESULTS

Page 65: Research Report (Es)Sense

65The direction giving movements

MESRO made when a person was perceived were not understood by all people. The participants that did understand it had the feeling that they were send back and forth between two corners. Notable, they did perceive the moving arm as an object that could point direction.

The walking patterns of the people indicated the degree to which they said they understood the situation. Understanding people took longer rests to lure MESRO. Participants who did not understand the situation stayed longer in the room and walked around more.

Page 66: Research Report (Es)Sense

66From the SEMRE and MESRO

experiment a set of conclusions can be made. When looking at the corresponding hypotheses the following can be concluded.

1. Participants did understand that MESRO reacts on the sound created my SEMRE. Not all participants realized this sound moved like an entity through the room. To MESRO characteristics were attributed like exploring and curious. SEMRE was indicated as dominant and loud.

2. Participants indicated that they felt it was easier to work with SEMRE and MESRO when they did understand the principle. They perceived particularly the behavior of MESRO as more natural.

At the beginning of this iteration round a hypothesis for the project was composed, based on the previous iteration rounds and the research question.

“Create a guide that adapts to the behavior of people and the environment and directs people through a space.”

Concluding on this we can state that MESRO did adapt to the behavior of people, its movement was focused on the person. The longer a person stood in one corner, the more effort MESRO would undertake to lure this person to another position. The way MESRO tried to direct people and trigger their urge to explore was felt less by people. Some did understand that MESRO wanted something, but the overall meaning of this fact was missed by most people.

CONCLUSIONS

Page 67: Research Report (Es)Sense

6767Person moving through the installation while being huided by SEMRE and MESRO

Page 68: Research Report (Es)Sense

68SEMRE and MESRO can be seen as

a start in creating a setup that contains a perceptive reactive environment and a perceptive body. Regarding this test set up, several parameters could be improved in order to create a clearer interaction that will be easier to understand for people.

_ Meaning of movement. The movement of MESRO is now based on the pointing movements people made in the first acting out experiments. The meaning of this can be improved by testing this specific part of the installation. Testing with different sets of movement can lead to a more concrete way of moving. Parameters that can be changed include speed, direction, delay and smoothness.

_ Meaning of sound. In the current setup the sound is kept minimalistic and produced by only 4 speakers. To increase the perception of a body of sound, the programming of SEMRE has to be done differently. Currently the location of sound is based on the

distance a person has to a speaker. This could be improved by creating a virtual X/Y grid over the space. In this way a specific location can be assigned to the sound body and all speakers will aid in creating this sound at this place. The sound itself can also be changed from this relatively bleak high pitched sound to a more characteristic repleted sound. Eva Deckers commented during the final exhibition that this influencing sound can be interesting enough to investigate more by emphasizing on making it more extreme to have a bigger impact.

_ Integration of MESRE in

environment. Now, the speakers of MESRE are clearly visible on the wall. In questionnaires people referred to these as heads of talking boxes. To eliminate this issue, the boxes have to be integrated in the environment. This can be done by creating speakers that fit better in the environment (e.g. hid in the corners) or by changing the environment (e.g. put same shape as speakers everywhere).

IMPROVING PARAMETERS

Page 69: Research Report (Es)Sense

69_ Research lab. The room that

was now use as research lab was not completely blank. One wall was open, another wall contained blackened windows. On the floor cables were put for power and data transmission. If the room would be transformed in a completely white room (or in another way so that it fits the experiment like stated before), there would be less distraction from the truly essential objectives of the experiment.

To get a more reliable outcome of the next experiment, two things have to be changed. First a control group is needed. This group is told before hand what the idea is of the setup. The rest of the experiment will be the same as for the people that do not know what the idea of the setup is. In this way it can be compared if it is essentially to know how the installation works in order to understand it. Secondly, this time, not everything was plotted. Movements and interviews with people were recorded. Experiment specific things like sensor data and random generated directions were

not recorded and therefor impossible to evaluate.Particularly the sensor data can be of significance when judging the behavior of people.

Page 70: Research Report (Es)Sense

CONCLUSION

Page 71: Research Report (Es)Sense

71Where the project was started

as a research project, a practical translation was extracted as input to future design projects. The philosophy of phenomenology is a topic of several design research projects. The translation to the creation products seems to be missing. In my opinion, the principle of perceptual crossing can play a catalyzing role in the development of subject-artifact-space interaction. A product that perceives his user and adapts his service, appearance or function to this can provide great opportunities in (public)space design.

This project emphasized on the relation between a person and a perceptive space and the role a perceptive artifact has in this. The conclusions of this project can be translated in design notions for further projects.

_ Perceiving position. In a space, our position is essential to the way we perceive a space. Changing this position or changing our routes changes the perception of space (Huberts, 2012). To create a rich subject-artifact-space interaction, both the artifact and space have to be able to perceive the position of the subject.

_ Event based behavior. An external event will enrich the common history a person can obtain with an artifact (Deckers, 2012). The interference of a subject works as an external event, creating unique shared history between subject, artifact and space. This shared history amplifies the sense of interaction.

DESIGN OPPORTUNITIES

Page 72: Research Report (Es)Sense

72_ Explorative suggestions. In this

project I experienced that the artifact can function as a catalyst in the explorative behavior of the subject. The connection between subject, artifact and space delivers takes the behavior of artifact and space away from two separate following entities. One event is created that is subject to changes by external events.

_ Inviting behavior. Based on the directions of space and external events, the artifact displays behavior that aims at inviting for interaction. The extent to which the artifact applies this determines the feeling people get. This can vary from not appealing to embracing.

Page 73: Research Report (Es)Sense

73To explore a space, one needs his

body. “It’s a truism that the best way to experience a space is by walking” (Schonhammer, 2004). Seducing people to start moving around will show them more of the space they are in. Adding a perceptive body in order to aid, creates an event in which subject and artifact share their intentionality; explore space. “In order to realize mechanisms of shared intentionality, we should design systems which are sensitive to perceptual crossing” (Marti, 2010). If the artifact in the shared space is able to perceive both subject and space a perceptual interplay can be established. This happens only when one is able to perceive the other’s ability to perceive (Deckers et al. 2010). If this connection is not possible, the establishment of a shared intentionality is distorted.

CONNECTION TO LITERATURE

Page 74: Research Report (Es)Sense

74As stated before, perceptual

crossing provides interesting features to intensify the human-artifact-(space) relation. In my Final Master Project I want to focus on this thought. Main goal will be to create a marketable product or service based on the principles of phenomenology and perceptual crossing.

I hope to use the design notions I converged as conclusion on this research project as a starting point for my FMP. Elaborating on these, I want to create a set of tools that aids me in designing for perceptual crossing in products. This will be part of my FMP proposal.

The context for this is still very broad (e.g. public space, interactive art, domestic environment) but will be defined in the first weeks of the project. In the process the emphasis will be put on creating a product that embody a personalized interaction setup based on the perceptive qualities of the artifact and the bodily skills of the user.

FUTURE STEPS

Page 75: Research Report (Es)Sense

75I chose the (Es)Sense project since

it is a fitting opportunity to ground statements I make in my vision on design and society. Besides that, it provides a framework to learn how to use the Research through Design methodology.

A clear theoretical framework, phenomenology, was provided at the start of the project. Starting to think from the human senses proved to be a good manner to forge a research question and hypothesis. People have to be able to connect to a product with their senses but also to escape these. In my opinion, this means there has to be some room for ambiguity. When there was no room for ambiguity, people indicated the interaction was less interesting, ambiguity leaves room for own perception. To support this I learned that a form has to be inherent to its function, including its sensors and actuators. In this way an independent body is created that can interact on an equal level with a person.

My goal was to go through a highly iterative process according to the Research through Design method. I experienced that short quick iterations can mean a lot to develop certain parts of the concept. At the start of an iteration I configured a question and a corresponding hypothesis. A physical version of this hypothesis (model), is a good way to test the hypothesis and find answers on the question. This way of working taught me to start building and learn to explore while building the model.

Scientific research was something I hardly got in touch with before. In the (Es)Sense project I grasped the opportunity to use scientific papers as grounding and inspiration for decisions. I noticed this aids well to substantiate my design. It also forces you to check sources and references which can lead to new information for your project.

REFLECTION

Page 76: Research Report (Es)Sense

76In my opinion, the human factor

can not be ignored in research. Therefor I wanted to involve people in my experiments, and avoid a paper based research approach. In an early stadium I applied acting out to investigate human behavior and movement. For me this proved to be a useful technique to gain inspiration and information about the bodily skills and how these can be translated to an artificial body. In discussion with Rene Ahn we agreed that this way of human like movement is important to reach the desired level of affection between subject and artifact.

I learned to converge what I want to get from a test to make a test continue smoothly. Looking for examples and answering “why?” questions helps in this. During tests it is wise to record everything so it can be analyzed afterwards. This aids in abstracting the answer to the research question. To get to the essence of what people experienced a good questionnaire is needed. Applying the laddering technique works well

in this since people are forced to give their most elementary thought on the subject.

While prototyping I tried to combine the learning of new skills with achieving result. I learned to use Max MSP in a short time by focussing on the skills I needed. Due to this I now know the basics of the program an can easily expand my knowledge in the future. Concerning physical prototyping I learned to combine sensors and actuators in one body. This taught me to use electronics on a small scale and program in a compact way, to spare micro processors.

Page 77: Research Report (Es)Sense

77Overall, I can state that the (Es)

Sense research project made me into a designer that is able to substantiate his decisions in a scientific way. By using provided theories and the right papers, inspiration and direction can be gained. In the future I will try to use this Research through Design methodology to establish a result based on hands on research. As mentioned I want to continue with the subject of phenomenology in my M2.1 project. My goal will be to integrate the principles of phenomenology and perceptual crossing in a marketready product.

Page 78: Research Report (Es)Sense

78LESR: Light Emitting Sound

Receiving. Blind object created in second iteration round

MESRO: Movement Emitting Sound Receiving Objects. Blind object created in third iteration round

Phenomenology: Philosophy that describes the structure of the objects of awareness and of awareness itself in abstraction from any claims concerning existence

Perceptual crossing: Others are recognized as intentional subjects, I distinguish their lived-body from their body as an image and that they recognize my presence as intentional subject (Deckers et al. 2012)

Research through Design: Research methodology were a physical hypothesis is created to get answers on a research question. Reflection on action.

SELR: Sound Emitting Light Receiving. Deaf object created in second iteration round

SEMRE: Sound Emitting Movement Receiving Environment. Deaf space created in third iteration round

CONCEPTS LIST

Page 79: Research Report (Es)Sense

791. Abram, D., (1996) The spell of the Sensuous: Language and Perception in a

More Than Human World. Vintage Books, New York, NY, USA.

2. Corbridge, C., Rugg, G., Major, N.P., Shadbolt, N.R., Burton, A.M., (1994) Laddering: a technique and tool use in knowledge acquisition. Department of Psychology, University of Nottingham, University Park, Nottingham NG7 2RD.

3. Deckers, E.J.L., Levy, P.D., Wensveen, S.A.G., Ahn, R.M.C., Overbeeke, C.J., (2012) Designing for perceptual crossing: applying and evaluating design notions. Eindhoven University of Technology, Industrial Design, Eindhoven, the Netherlands.

4. Deckers, E.J.L., Levy, P.D., (2012) Designing for Perceptive Qualities: 7 showcases. Eindhoven University of Technology, Industrial Design, Eindhoven, the Netherlands.

5. Klooster, S., (2003) Embodynamics Framework. Eindhoven University of Technology, Industrial Design, Eindhoven, the Netherlands.

6. Koskinen, I., Binder, T., Redstrom, J., (2008) Lab, Field, Gallery and Beyond. Volume 2, Issue 1, pages 46-57.

7. Lenay, C., Thouvenin, I., Guenand, A., Gapenne, O., Stewart, J., Maillet, B., (2007) Designing the ground for pleasurable experience.

8. Lenay, C., (2010) “It’s So Touching”: Emotional Value in Distal Contact. UTC/COSTECH, Compiegne, France.

REFERENCES

Page 80: Research Report (Es)Sense

809. Marti, P., (2010) Perceiving while being perceived. International Journey of

Design, 4(2), 27-38.

10. Merleau-Ponty, M., (1958) Phenomenology of Perception (C. Smith) Routledge & Kegan Paul.

11. Schonhammer, R., (2004) Human ‘sense of space’, moving images and architecture. Psychology of Art and Design, Faculty of Design, Burg Giebichenstein Hochschule fur Kunst und Design Halle/Saale, International Symposium Aesthetics and Architectural Composition, Dresden, 2004.

Page 81: Research Report (Es)Sense

81

Page 82: Research Report (Es)Sense

APPENDIX

Page 83: Research Report (Es)Sense

83APPENDIX A: QUESTIONNAIRE ACTING OUTQuestionnaire acting out

What role did you play during the acting out session?deaf artifact / blind artifact / person

If you played one of the artifacts:What did you try to communicate to the other artifact?

What was your feeling when the person entered the room?

Was there some kind of emotion you tried to express when the person entered the room?

If you played the person:How did you perceive the ongoing conversation when entering the room?

How did you try to participate in the conversation?

What feeling did you get while doing this?

Page 84: Research Report (Es)Sense

84Results Acting Out ExperimentsResults Acting Out ExperimentsResults Acting Out Experiments

Session Conversation established? Way of communicating Perception subject Comments

Round 1 1 (pilot) Yes, although hard to

communicate

Sending sound and light in reaction on each other Subject feels left out and sees artifacts in bubble What are the characteristics of the artifacts?

2 (pilot) Yes, light reacting on sound Start creating patterns after while in conversation Perceives conversation, though to interfere in

conversationRound 2 3 Yes Deaf focusses on environment, lights moves instead

of flashing

Feels left out and disconnected, tries to distort

communication

Artifacts make use of turning, abilities of body

4 Yes, one way communication Blind reacts on deaf, deaf guides Hard to perceive conversation, influencing blind

through deaf

Subjects succeeds in attracting blind through deaf

Round 3 5 Yes, conversation started

without problems

Reacting on each other, trying to create evolving

patterns

Hard to interrupt, tried to interrupt physically Conversation between subject and blind was there

6 Yes Sound reacts with different pitches on light Tries to make contact by making sound Conversation faltered when one participant failed

Round 4 7 No Try to attract each others attention Subject tries to bring artifacts together, does not

succeed

Subjects functions as interpreter between artifacts

8 Yes Try to copy and morph each others patterns Artifacts ignored subject

APPENDIX B: RESULTS ACTING OUT

Page 85: Research Report (Es)Sense

85Results Acting Out ExperimentsResults Acting Out ExperimentsResults Acting Out Experiments

Session Conversation established? Way of communicating Perception subject Comments

Round 1 1 (pilot) Yes, although hard to

communicate

Sending sound and light in reaction on each other Subject feels left out and sees artifacts in bubble What are the characteristics of the artifacts?

2 (pilot) Yes, light reacting on sound Start creating patterns after while in conversation Perceives conversation, though to interfere in

conversationRound 2 3 Yes Deaf focusses on environment, lights moves instead

of flashing

Feels left out and disconnected, tries to distort

communication

Artifacts make use of turning, abilities of body

4 Yes, one way communication Blind reacts on deaf, deaf guides Hard to perceive conversation, influencing blind

through deaf

Subjects succeeds in attracting blind through deaf

Round 3 5 Yes, conversation started

without problems

Reacting on each other, trying to create evolving

patterns

Hard to interrupt, tried to interrupt physically Conversation between subject and blind was there

6 Yes Sound reacts with different pitches on light Tries to make contact by making sound Conversation faltered when one participant failed

Round 4 7 No Try to attract each others attention Subject tries to bring artifacts together, does not

succeed

Subjects functions as interpreter between artifacts

8 Yes Try to copy and morph each others patterns Artifacts ignored subject

Page 86: Research Report (Es)Sense

86//A900acj7

//include libraries#include <Servo.h> #include “pitches.h”

//int sensorPin = 0; int lightVal; int pos; //position of servoint turn; //degrees of turning servoint time1; //timer to count time in idle mode to start exploringint time2; //timer to count time in idle mode to initiate soundint sensorDelay = 29; //sensorDelay for HC-SR04 sensorsint shortTime = 60; //time to start acting after short intervalint longTime = 300; //time to start acting after long interval

Servo myservo;

//Set fixed numbers#define maxDistance 300 //max distance that HC-SR04 measures

//Define pins sensor#define sensorPin 0#define speakerPin 11 #define trigPin 8#define echoPin 6

long distanceSensor;long durationSensor;

int melody1[] = { //melody1 to communicate NOTE_D4, NOTE_D4,NOTE_A4, NOTE_A4, //NOTE_D4, NOTE_D4,NOTE_A4, NOTE_A4, NOTE_G4, NOTE_F4, NOTE_E4, NOTE_D4, NOTE_C4, NOTE_D4, NOTE_E4,NOTE_F4, NOTE_G4, NOTE_A4, NOTE_D4 NOTE_D4, NOTE_A4, NOTE_A4, NOTE_G4, NOTE_F4, NOTE_E4, NOTE_D4, NOTE_C4, NOTE_D4, NOTE_E4, NOTE_F4, NOTE_G4, NOTE_A4};int melody2[] = { NOTE_C2, NOTE_C3};int melody3[] = { NOTE_D2};

int noteDurations1[] = { //duration of notes 4,4,4,4};//4, 4, 4, 4,4,4,4,4,4,4, 4, 4, 4,2,4,4,4,4,4, 4, 4, 4,4,4,4,4,4,4,2 };

APPENDIX C: CODE SELR

Page 87: Research Report (Es)Sense

87int noteDurations2[] = { 4, 2};int noteDurations3[] = { 1};

int singleNoteDuration = 500;

void setup() { Serial.begin(9600); pinMode(sensorPin, INPUT); pinMode(trigPin, OUTPUT); pinMode(echoPin, INPUT); myservo.attach(9); }

void loop() { sensors(); //void of HC-SR04 measuring distance and light sensor measuring light intensity timer(); //timer that counts time until action has to be taken

if (time1 >= shortTime){ //initiate sound for conversation after short time for (int thisNote = 0; thisNote < 29; thisNote++) { int noteDuration = 1000/noteDurations1[thisNote]; tone(speakerPin, melody1[thisNote],noteDuration); int pauseBetweenNotes = noteDuration * 1.30; delay(pauseBetweenNotes); noTone(speakerPin); time1 = 0; } }

if (time2 >= longTime){ //initiate exploring for conversation after longer time when no reaction has come

turn = random (0, 60); //looking around for(pos = turn; pos <=2*turn; pos+=1){ myservo.write(pos); delay(15); } for (int thisNote = 0; thisNote < 2; thisNote++) { //play melody to start conversation int noteDuration = 1000/noteDurations2[thisNote]; tone(speakerPin, melody2[thisNote],noteDuration); int pauseBetweenNotes = noteDuration * 1.30;

Page 88: Research Report (Es)Sense

88 delay(pauseBetweenNotes); noTone(speakerPin); time2 = 0; } for(pos = 2*turn; pos>=0; pos-=1){ myservo.write(pos); delay(15); } for (int thisNote = 0; thisNote < 2; thisNote++) { //play melody to start conversation int noteDuration = 1000/noteDurations2[thisNote]; tone(speakerPin, melody2[thisNote],noteDuration); int pauseBetweenNotes = noteDuration * 1.30; delay(pauseBetweenNotes); noTone(speakerPin); time2 = 0; } for(pos = 0; pos<turn; pos+=1){ myservo.write(pos); delay(15); } }

if(distanceSensor < 20){ //subject approaching SELR for (int thisNote = 0; thisNote < 29; thisNote++) { //play tune when subject approaches int noteDuration = 1000/noteDurations1[thisNote]; tone(speakerPin, melody1[thisNote],noteDuration); int pauseBetweenNotes = noteDuration * 1.30; delay(pauseBetweenNotes); noTone(speakerPin); time1 = 0; }

}

if(lightVal < 550){ //receives light, reacts with sound tone(speakerPin, NOTE_E2, 100); } delay(100);

}

Page 89: Research Report (Es)Sense

89int sensors(){ digitalWrite(trigPin, LOW); delayMicroseconds(2);

digitalWrite(trigPin, HIGH); delayMicroseconds(10);

digitalWrite(trigPin, LOW);

durationSensor = pulseIn(echoPin, HIGH);

distanceSensor = (durationSensor/2) / 29.1; //calculate distance

Serial.println(“distance sensor “); Serial.print(distanceSensor); Serial.println(“cm”); lightVal = analogRead(sensorPin); //get sensordata from light sensor Serial.println(“Light intensity “); Serial.println(lightVal); delay(sensorDelay);

return (distanceSensor);}

int timer(){ if (lightVal > 550){ //count when no light is received to start exploring time1++; time2++; Serial.println(“timer 1”); Serial.println(time1); } }

Page 90: Research Report (Es)Sense

90//tty.usbserial-A900ac4W

//include libraries#include <Servo.h> #include “pitches.h”

int sensorPin = 0; int micVal; int pos; //position of servoint turn; //degrees of turning servoint time1; //timer to count time in idle mode to start exploringint time2; //timer to count time in idle mode to initiate soundint sensorDelay = 29; //sensorDelay for HC-SR04int shortTime = 100;int longTime = 450;int brightness = 0;int fadeAmount = 5;

Servo myservo;

//Set fixed numbers#define maxDistance 300 //max distance that HC-SR04 measures

//Define pins sensor#define LEDPin 10 #define trigPin 8#define echoPin 6

long distanceSensor;long durationSensor;

void setup() { Serial.begin(9600); pinMode(sensorPin, INPUT); pinMode(trigPin, OUTPUT); pinMode(echoPin, INPUT); myservo.attach(9); }

void loop() { sensors(); //void of HC-SR04 measuring distance and sound sensor measuring light intensity timer(); //timer that counts time until action has to be taken

APPENDIX D: CODE LESR

Page 91: Research Report (Es)Sense

91 if (time1 >= shortTime){ //initiate light for conversation after short time analogWrite(LEDPin, brightness);

brightness = brightness + fadeAmount;

if (brightness == 0 || brightness == 255) { fadeAmount = -fadeAmount ; } }

if (time2 >= longTime){ //initiate light for conversation after short time

turn = random (0, 60); for(pos = turn; pos <=2*turn; pos+=1){ myservo.write(pos); delay(40); } digitalWrite(LEDPin, HIGH); delay(200); digitalWrite(LEDPin, LOW); delay(200); digitalWrite(LEDPin, HIGH); delay(200); digitalWrite(LEDPin, LOW); delay(200);

for(pos = 2*turn; pos>=0; pos-=1){ myservo.write(pos); delay(40); } digitalWrite(LEDPin, HIGH); delay(200); digitalWrite(LEDPin, LOW); delay(200); digitalWrite(LEDPin, HIGH); delay(200); digitalWrite(LEDPin, LOW); delay(200); for(pos = 0; pos<turn; pos+=1){ myservo.write(pos); delay(40); } }

Page 92: Research Report (Es)Sense

92 if(distanceSensor < 20){ //subject approaching LESR for(pos = 0; pos<=60; pos+=1){ //turn towards subject myservo.write(pos); analogWrite(LEDPin, brightness);

brightness = brightness + fadeAmount;

if (brightness == 0 || brightness == 255) { fadeAmount = -fadeAmount ; } delay(40); }

}

if(micVal < 200){ //receives sound, reacts with light digitalWrite(LEDPin, HIGH); delay(200); digitalWrite(LEDPin, LOW); delay(200); digitalWrite(LEDPin, HIGH); delay(200); digitalWrite(LEDPin, LOW); delay(200); digitalWrite(LEDPin, HIGH); delay(200); digitalWrite(LEDPin, LOW); delay(200); digitalWrite(LEDPin, HIGH); delay(200); digitalWrite(LEDPin, LOW); delay(200); } delay(100);}

Page 93: Research Report (Es)Sense

93int sensors(){ digitalWrite(trigPin, LOW); delayMicroseconds(2);

digitalWrite(trigPin, HIGH); delayMicroseconds(10);

digitalWrite(trigPin, LOW);

durationSensor = pulseIn(echoPin, HIGH);

distanceSensor = (durationSensor/2) / 29.1; //calculate distance

Serial.println(“distance sensor “); Serial.print(distanceSensor); Serial.println(“cm”); lightVal = analogRead(sensorPin); //get sensordata from light sensor Serial.println(“Light intensity “); Serial.println(lightVal); delay(sensorDelay);

return (distanceSensor);}

int timer(){ if (lightVal > 550){ //count when no light is received to start exploring time1++; time2++; Serial.println(“timer 1”); Serial.println(time1); } }

Page 94: Research Report (Es)Sense

94 APPENDIX E: MAX MSP PATCH SEMRE

Page 95: Research Report (Es)Sense

95APPENDIX F: CODE MESRO// usbserial-AH00ZY2U

#include <Servo.h>

Servo myservo1; // Big servo in foot Servo myservo2; // small vertical servoServo myservo3; // small horizontal servo

int pos1 = 0; // variable to store the position of Servos int pos2 = 0;int pos3 = 0;int PinA = A0; // input pins from distance sensors of SEMREint PinB = A1;int PinC = A2;int PinD = A3;

int lightPin = 3; //Pin that connects to powerLED in arm of MESRO

int valA = 0; // input val from distance sensorsint valB = 0;int valC = 0;int valD = 0;int valA1 = 0; // mapped input val from distance sensorsint valB2 = 0;int valC3 = 0;int valD4 = 0;

int corner = 1; //1=A 2=B 3=C 4=D

int dir = 0; //variable indicating direction to explore in idle mode

int ACont = 0; //variable that states if MESRO is looking in right directionint BCont = 0;int CCont = 0;int DCont = 0;

void setup() { myservo1.attach(9); // attaches the servos to indicated pins on Arduino myservo2.attach(10); myservo3.attach(11); Serial.begin(9600);}

Page 96: Research Report (Es)Sense

96

void loop() { valA = analogRead(PinA); //reads values from distance sensors valB = analogRead(PinB); valC = analogRead(PinC); valD = analogRead(PinD); valA1 = map(valA, 0, 1255, 0, 400); //maps values from distance sensors to eliminate differences between sensors valB2 = map(valB, 0, 1255, 0, 400); valC3 = map(valC, 0, 1255, 0, 400); valD4 = map(valD, 0, 1255, 0, 400); Serial.println(“valA”); //print lines to check states of distance sensors Serial.println(valA1); Serial.println(“valB”); Serial.println(valB2); Serial.println(“valC”); Serial.println(valC3); Serial.println(“valD”); Serial.println(valD4);

if ((valA1 + valB2 + valC3 + valD4)/4 <= 100){ idleMode(); //idle mode is standard activated when there is no one in the space } else

if(valA > valB && valA > valC && valA > valD){ //check to see which distance sensor measures highest input, this is the corner where the person stands corner = 1; } if(valB > valA && valB > valC && valB > valD){ corner = 2; } if(valC > valB && valC > valA && valC > valD){ corner = 3; } if(valD > valB && valD > valC && valD > valA){ corner = 4; } Serial.println(“Corner”); //print line stating the corner MESRO is going to explore Serial.println(corner);

if (corner == 1){ //dependent on the corner, the corresponding void is called movingA();

Page 97: Research Report (Es)Sense

97 } if (corner == 2){ movingB(); } if (corner == 3){ movingC(); } if (corner == 4){ movingD(); }}

Page 98: Research Report (Es)Sense

98Questionnaire (Es)Sense SEMRE & MESRO

Test subject #Recording #

You will enter a room. In this room, a conversation is going on between the space (SEMRE) and the artifact (MESRO). The goal is to intervene in this conversation and either become part of it or distort it.

Evaluation:Can you describe the conversation you encountered?

Do you feel the space (SEMRE) is able to perceive?

What does SEMRE perceive?

Do you feel the artifact (MESRO) is able to perceive?

What does MESRO perceive?

How did you perceive the interaction between SEMRE and MESRO?

What did you perceive?

Which of the three (artificial) players did you perceive as dominant?

Did you understand the behavior of MESRO (the artifact)?

Did you recognize the behavior of MESRO (the artifact)? What did you recognize?

APPENDIX G: QUESTIONNAIRE

Page 99: Research Report (Es)Sense

99

Did you understand the behavior of SEMRE?

How did you perceive the entity of SEMRE?

Did you feel involved? Why did you get this feeling?

Did the behavior of either of MESRO or SEMRE trigger changes in your own behavior?

Validation session:With the extra information given, does this change your perception of SEMRE & MESRO?

Thinking of a translation to product, in which situation can you see the principle of SEMRE & MESRO in practice?

Page 100: Research Report (Es)Sense

100 APPENDIX H: WALKING PATTERNS= short rest = long rest = making sound

#1 time taken: 1:35 #2 time taken: 1:50 #3 time taken: 1:38

#4 time taken: 6:25 #5 time taken: 2:35 #3 time taken: 4:37

Page 101: Research Report (Es)Sense

101= short rest = long rest = making sound

#7 time taken: 3:11 #8 time taken: 3:20 #9 time taken: 1:42

Page 102: Research Report (Es)Sense

102Results ExperimentsResults ExperimentsResults Experiments

Session Perception of SEMRE Perception of MESRO Perception of conversation SEMRE and

MESRO

Dominant artifact Understanding of SEMRE Understanding of MESRO Feeling of involvement Changes in own behavior Changed perception with

more information?1 (pilot) SEMRE perceives me, produces

sound

If it is able to perceive me, it flees There is some kind of interaction, although

hard to define, is collective or individual?

MESRO - No understanding, perceives

movement as animal like

I am welcome, although not

invited

Yes, more explorative

behavior

More humane, less

mysterious2 (pilot) SEMRE measures my distance

to one of the speakers

MESRO learns to perceive, reacts on

absence of sender (SEMRE)

It is a status quo between 4 speakers and

MESRO, reacts on blocking of signal

SEMRE, they protest when

their signal is blocked

Underwater world due to

bubbling sounds

It feels not complete, has body of

a tentacle

It did react on me, was not

happy, no acceptance

You move more directed,

try to test own hypothesis

Yes, experience is clearer

3 SEMRE perceives me based on

my position in the room

MESRO is not able to perceive me, only

SEMRE can do this

SEMRE dictates the conversation, MESRO

speaks in monologue, hard to interrupt this

MESRO It has no own behavior, it is a

space

No understanding, no link

between me and MESRO

I can influence, but not on a

personal level

Search for meaning I miss direct influence on

MESRO4 First thought: SEMRE perceives

me via MESRO, second: via walls

MESRO does not perceive me directly

but via absence of sound

Sound calls MESRO, made me feel clumsy and

uncomfortable

SEMRE, determines were

listening should happen

SEMRE does not want user, is in

own conversation with 4 speakers

I am not sure, gets disturbed by

subject, bit autistic

not totally, I have influence,

but no involvement

Yes, tried to explore, faster

moving between corners

Confirms own expectations

5 SEMRE notices my position and

gives the sound a higher pitch

Does not react on me, reacts only on

the absence of sound

Conversation between SEMRE and MESRO,

distorted when I interfere

I had the most control,

SEMRE was most present

It is an entity with lots of stimuli Tries to explore, when obstructed

tries to look around subject

Not involved, I had the role of

obstructor

Yes, when I discovered

working started to test

More clear, interaction

becomes clearer6 Coupling is very direct, acts as a

mirror

Does not react directly, but uses

speakers as sensors

Does not feel like conversation, I can make

myself visible through installation

MESRO and I Can feel me, no presence of entity Behavior is too vague, looked

around with face

Was involved, searching for

answers

No, I was already here to

explore what was going on

Did match expectations

7 I think SEMRE tries to lure me It seems to look around, I would expect

it would notice me at one point

Hard to notice, what is way of

communicating? 4 corners seem to lure object

None I have no idea what is happening Clearly in search of something,

not clear for what reason

Since I don’t know what

happens I didn’t feel involved

I started to explore

environment

Makes clearer what is

supposed to happen8 SEMRE has the ability to

perceive where I am

In the begin I thought that MESRO

perceived me, changed later

Action-reaction, they react on each others

actions

Me I did not understand it completely,

why the bubble sound?

Reacts on sound, moves

according to sound, just like I do

No I was the disturbance

factor

Started to move more,

became curious

Confirms own expectations

9 SEMRE can perceive me, and

chances sound according to it

MESRO does not perceive me directly,

but reacts on sound

MESRO investigates corners where something

is happening

Me SEMRE calls when I am in the way MESRO searches for sound of 4

speakers

Not concerned, Although I

was part of interrupting

No changes Did match expectations

APPENDIX I

Page 103: Research Report (Es)Sense

103Results ExperimentsResults ExperimentsResults Experiments

Session Perception of SEMRE Perception of MESRO Perception of conversation SEMRE and

MESRO

Dominant artifact Understanding of SEMRE Understanding of MESRO Feeling of involvement Changes in own behavior Changed perception with

more information?1 (pilot) SEMRE perceives me, produces

sound

If it is able to perceive me, it flees There is some kind of interaction, although

hard to define, is collective or individual?

MESRO - No understanding, perceives

movement as animal like

I am welcome, although not

invited

Yes, more explorative

behavior

More humane, less

mysterious2 (pilot) SEMRE measures my distance

to one of the speakers

MESRO learns to perceive, reacts on

absence of sender (SEMRE)

It is a status quo between 4 speakers and

MESRO, reacts on blocking of signal

SEMRE, they protest when

their signal is blocked

Underwater world due to

bubbling sounds

It feels not complete, has body of

a tentacle

It did react on me, was not

happy, no acceptance

You move more directed,

try to test own hypothesis

Yes, experience is clearer

3 SEMRE perceives me based on

my position in the room

MESRO is not able to perceive me, only

SEMRE can do this

SEMRE dictates the conversation, MESRO

speaks in monologue, hard to interrupt this

MESRO It has no own behavior, it is a

space

No understanding, no link

between me and MESRO

I can influence, but not on a

personal level

Search for meaning I miss direct influence on

MESRO4 First thought: SEMRE perceives

me via MESRO, second: via walls

MESRO does not perceive me directly

but via absence of sound

Sound calls MESRO, made me feel clumsy and

uncomfortable

SEMRE, determines were

listening should happen

SEMRE does not want user, is in

own conversation with 4 speakers

I am not sure, gets disturbed by

subject, bit autistic

not totally, I have influence,

but no involvement

Yes, tried to explore, faster

moving between corners

Confirms own expectations

5 SEMRE notices my position and

gives the sound a higher pitch

Does not react on me, reacts only on

the absence of sound

Conversation between SEMRE and MESRO,

distorted when I interfere

I had the most control,

SEMRE was most present

It is an entity with lots of stimuli Tries to explore, when obstructed

tries to look around subject

Not involved, I had the role of

obstructor

Yes, when I discovered

working started to test

More clear, interaction

becomes clearer6 Coupling is very direct, acts as a

mirror

Does not react directly, but uses

speakers as sensors

Does not feel like conversation, I can make

myself visible through installation

MESRO and I Can feel me, no presence of entity Behavior is too vague, looked

around with face

Was involved, searching for

answers

No, I was already here to

explore what was going on

Did match expectations

7 I think SEMRE tries to lure me It seems to look around, I would expect

it would notice me at one point

Hard to notice, what is way of

communicating? 4 corners seem to lure object

None I have no idea what is happening Clearly in search of something,

not clear for what reason

Since I don’t know what

happens I didn’t feel involved

I started to explore

environment

Makes clearer what is

supposed to happen8 SEMRE has the ability to

perceive where I am

In the begin I thought that MESRO

perceived me, changed later

Action-reaction, they react on each others

actions

Me I did not understand it completely,

why the bubble sound?

Reacts on sound, moves

according to sound, just like I do

No I was the disturbance

factor

Started to move more,

became curious

Confirms own expectations

9 SEMRE can perceive me, and

chances sound according to it

MESRO does not perceive me directly,

but reacts on sound

MESRO investigates corners where something

is happening

Me SEMRE calls when I am in the way MESRO searches for sound of 4

speakers

Not concerned, Although I

was part of interrupting

No changes Did match expectations

Page 104: Research Report (Es)Sense

104Results ExperimentsResults ExperimentsResults Experiments

Session Perception of SEMRE Perception of MESRO Perception of conversation SEMRE and

MESRO

Dominant artifact Understanding of SEMRE Understanding of MESRO Feeling of involvement Changes in own behavior Changed perception with

more information?1 (pilot) SEMRE perceives me, produces

sound

If it is able to perceive me, it flees There is some kind of interaction, although

hard to define, is collective or individual?

MESRO - No understanding, perceives

movement as animal like

I am welcome, although not

invited

Yes, more explorative

behavior

More humane, less

mysterious2 (pilot) SEMRE measures my distance

to one of the speakers

MESRO learns to perceive, reacts on

absence of sender (SEMRE)

It is a status quo between 4 speakers and

MESRO, reacts on blocking of signal

SEMRE, they protest when

their signal is blocked

Underwater world due to

bubbling sounds

It feels not complete, has body of

a tentacle

It did react on me, was not

happy, no acceptance

You move more directed,

try to test own hypothesis

Yes, experience is clearer

3 SEMRE perceives me based on

my position in the room

MESRO is not able to perceive me, only

SEMRE can do this

SEMRE dictates the conversation, MESRO

speaks in monologue, hard to interrupt this

MESRO It has no own behavior, it is a

space

No understanding, no link

between me and MESRO

I can influence, but not on a

personal level

Search for meaning I miss direct influence on

MESRO4 First thought: SEMRE perceives

me via MESRO, second: via walls

MESRO does not perceive me directly

but via absence of sound

Sound calls MESRO, made me feel clumsy and

uncomfortable

SEMRE, determines were

listening should happen

SEMRE does not want user, is in

own conversation with 4 speakers

I am not sure, gets disturbed by

subject, bit autistic

not totally, I have influence,

but no involvement

Yes, tried to explore, faster

moving between corners

Confirms own expectations

5 SEMRE notices my position and

gives the sound a higher pitch

Does not react on me, reacts only on

the absence of sound

Conversation between SEMRE and MESRO,

distorted when I interfere

I had the most control,

SEMRE was most present

It is an entity with lots of stimuli Tries to explore, when obstructed

tries to look around subject

Not involved, I had the role of

obstructor

Yes, when I discovered

working started to test

More clear, interaction

becomes clearer6 Coupling is very direct, acts as a

mirror

Does not react directly, but uses

speakers as sensors

Does not feel like conversation, I can make

myself visible through installation

MESRO and I Can feel me, no presence of entity Behavior is too vague, looked

around with face

Was involved, searching for

answers

No, I was already here to

explore what was going on

Did match expectations

7 I think SEMRE tries to lure me It seems to look around, I would expect

it would notice me at one point

Hard to notice, what is way of

communicating? 4 corners seem to lure object

None I have no idea what is happening Clearly in search of something,

not clear for what reason

Since I don’t know what

happens I didn’t feel involved

I started to explore

environment

Makes clearer what is

supposed to happen8 SEMRE has the ability to

perceive where I am

In the begin I thought that MESRO

perceived me, changed later

Action-reaction, they react on each others

actions

Me I did not understand it completely,

why the bubble sound?

Reacts on sound, moves

according to sound, just like I do

No I was the disturbance

factor

Started to move more,

became curious

Confirms own expectations

9 SEMRE can perceive me, and

chances sound according to it

MESRO does not perceive me directly,

but reacts on sound

MESRO investigates corners where something

is happening

Me SEMRE calls when I am in the way MESRO searches for sound of 4

speakers

Not concerned, Although I

was part of interrupting

No changes Did match expectations

Page 105: Research Report (Es)Sense

105

Page 106: Research Report (Es)Sense

JOB [email protected]

M1.2COACH: SIMONE DE WAARTTHEME: WEARABLE SENSES