SLPLLPl i k e P l a y L o o k s L i k e P L AYS o u n d s

See what we can do with a

... and suddenly Ben Schouten came up with a group of students who were crazy enough to work with Philips

Floris Provoost

““

F E S T I V A L

I N T E R V I E W

K I N E C T58

90

WITH FRENS FRIJNS and FLORIS PROVOoST

t h e i n s a n d o u t s o f t h e

108

We created a dynamic, interactive scenery of colours which could be influenced by the movement, speed and position of pas-sersby. The visitors are (subconsciously) making a dynamic ‘painting’, and in the process creating a performance for the lounging people in the surrounding area of the playground. The ‘painting’ is created through the collaboration of the movements of visitors, either con-sciously or unconsciously involved in the game. All individual parts of the concept already existed, Luminous Textile Panels (LTP) by Philips, Kinect depth-sensor by Microsoft, Processing by Ben Fry and Casey Reas, Kinect Library by Thomas Diewalt. The combination of these parts created a whole new platform for giving shape to an unlimited number of interactive activities and visualizations. This also means there was no (real) final design, it was evolving every day it was exhibited at the STRP-festival (and still is in our minds). Since Philips already has a number of corporations using LTP in their buildings, our concept might accelerate the process of interactive environments being incorporated in everyday-life.

“ “

Written by Marieke de Rooyhttp://youtu.be/wRSV0vU4VQc

01. Introduction- Students

- Construction of the report

- Approach

- Project

- Stakeholders

02. Midterm Exhibition- First year students

- Tijmen & Marieke

- Vivian

03. Explorations- Design for festival environment

- Communication and approval of STRP

- Video capturing

- Mock-up panel

- Mock-up panel 11

- Brainstorm interaction

04. Interaction Design- Interaction requirements

- Interaction designs

05. Technical Programming- Choosing a sensor

- Kinect library

- Blob detection

- Multiple Kinects

- How to connect multiple Kinects

- Multiple Kinects connected to one PC

- Pitfalls

- Using a database

- OSC

- Setup with multiple Kinects using OSC

- ArrayList

- From received data to visual

- MinDistBlob

Table of Contents

06. Visual Programming- Testing with Kinects

- Streaming content to Philips Luminous Textile Panels

- Streaming at STRP

- Designing the interaction

- Programming interaction during STRP

07. Production of the festival- Floor plan and technical drawings

- Production of the concept & stakeholders agreements

- Making a model of the festival area

- Mounting the hardware

- Electricity and network

08. STRP Festival- STRP festival

- Research and Development

- Observations

09. Interviews- Frens Frijns

- Floris Provoost

10. Conclusion and Recommendations- Conclusions

- Recommendations

11. Acknowledgments

12. References

13. Appendix

68

10

12

12

14

2022

24

26

2830

32

34

40

42

44

4648

52

5658

60

60

61

61

61

61

62

62

62

64

64

64

108110

112

116

118

120

122

126

130

6670

72

74

76

78

9092

93

94

96

96

98100

101

102

1

9

8 | Intro

du

ction

The Researcher and producer

Vivian Welten,Master Student Industrial DesignFocus: The understanding of the interction in search for behavioral patterns.

My special interest is in

designing for music concerts

and festivals. I think the

design should go beyond

the limits of the stage area

and involve the visitors in

the performance. For that

I strive to create surprising

and memorable experiences

through interactive media

design.

Let the spectator become

participant.

The Creative Director

Marieke De Rooy,Master Student Industrial DesignFocus: Trigger creativity and curiosity through playful experiences.

I aimed this project for

designing co-experiences;

visitors create meaning and

emotion together through

a social and playful activity.

Next to this I wanted visitors

of the festival to be able

to express themselves in a

non-auditive way; through

gestures and colors.

The Technical Programmer

Tijmen Van Gurp,Bachelor 3.2 Student Indus-trial DesignFocus: Creating physicy-cal dynamic interactions with surprizing factors

In this project my focus was

on technical parts of pro-

gramming. In the ideation

phase of this project my fo-

cus was on interactions with

physical cariers that would

suply for a direct interaction.

I believe that interactive con-

cepts should be intuative

and fun. One way to do that

is by giving people a sense

of control, for example the

control of a physical object.

The Visual Programmer

Anika van der Sanden,Bachelor 1.1 Student Indus-trial DesignFocus: creating a co-crea-

tion interaction that could

elite communication

Creating a co-creation inter-

action that could elite

communication. Because

our interactive system was

going to be placed at a

festival I found it interesting

to see if it could elite com-

munication between festival

visitors.

The Allrounder

Tom van ‘ T WesteindeBachelor 1.1 Student Indus-trial DesignFocus: Creating a perfor-mance interaction.

With this project I wanted to

make an interaction with a

unclear starting moment. I

wanted an interaction that

provokes a performance for

visitors in a lounge and

create a dynamic atmos-

phere. I wanted to par-

ticipate in every part of the

process to get an idea of

every aspect.

The Artistic Director

Max Verhoef,Bachelor 1.1 Student Indus-trial DesignFocus: Creating an inter-action which makes the player an artist.

At the early stage of the

project my focus was on an

interaction implemented

with a carrier, which makes it

clear whether you are play-

ing or not. Later on I found

it more important to create

the possibility for the player

to create his own things, so

everyone can be artistic in a

fun way.

The Communica-tion Designer

Alex Schepers,Bachelor 1.2 Student Indus-trial DesignFocus: Creating an in-teraction which triggers communication.

This project my focus was

on creating a social trig-

ger. During one of the first

brainstorm I had come up

with a way to make people

communicate and share

information.

Furthermore making sure

the creative process is

depicted as it is supposed

to be. Create something to

show what has been cre-

ated.

Seven Faces of Sounds like Play looks like Play

10 11ReportConstruction of theSince we have been working as a group

for most of the time, we agreed upon creating a group-report describing the pro-cess of us working in a team. The reporting will start from the moment we started to work together. How did we combine our in-dividual ideas into one concept? After the festival we split up again, all doing our own analysis of the project presented at STRP.

10 | Intro

du

ction

I n the group-report, we will discuss our group-exploration, technical and visual

realization, production of the festival and the exhibition at the STRP-festival. All parts that were done individually are described in a separate report. The con-clusions of these individual activities are combined to formulate one overall con-clusion. This conclusion is presented at the end of the group-report.

Approach

We started out individually, all with our own approach and goals

for this project. During the midterm exhibition we showed our own

concepts. After this exhibition we teamed up and started to formu-

late an overall concept, in which every one of us would be visible as

a designer. All strengths were bundled to create, organize and build

a playful, interactive lighting installation for the STRP festival. In four

weeks we succeeded to create an interactive lighting installation that

enhanced the atmosphere of the lounge area at the STRP-festival.

The Project

This project was set up to create an experience of expressiveness such

as artists perceive expressiveness. This experience would take place

during the STRP festival in the fall of 2011 in Eindhoven.

The project aimed at creating a fun, surprising, unconstrained inter-

active activity that made for a feeling of being creative in its users.

The design of the (playful) interaction was the main challenge of the

project, since the target-group, output device and location were fixed

at the outset. (1)

12

Introduction | 13

STRP’s requirements

Interactive lighting in the lounge/café area of the festival

Playful experience

Connecting visitors

Creating a good atmosphere

14

15 | Intro

du

ction

Stakeholder STRP festival

“The STRP-festival is a yearly festival in Eindhoven and is one

of the largest art & technology festivals in Europe, that focuses

on music, art and technology. The multidisciplinary program is

a mix of a 360-degree experience with adventures which ap-

peal to a wide audience. At STRP there are projects of young

game designers next to major works from the international

art circuit and experimental live cinema next to successful

pop artists and DJs. At STRP you find interactive art, light art,

robotics, concerts, DJs, theatrical and dance performances,

experimental music, interviews discussion, live cinema, films,

lectures, video art, animation and workshops. This year it is

mainly about an overview of the last 50 years of Dutch media

art and technology. The STRP festival took place from Novem-

ber 18 to November 27 at the ‘Klokgebouw’ in Eindhoven.” (2)

The client STRP is looking for interaction between unrelated

visitors. The STRP-festival has a wide variety of visitors. The visi-

tors of the STRP festival are looking for experiences, distrac-

tion, a good time with friends and inspiration, and they go

there out of curiosity. They like to create meaning and emo-

tion together by attending different events during this festival.

16

17 | Intro

du

ction

Stakeholder PhilipsThe client Philips Large Luminous Surfaces is looking

for the possibilities in interaction with their product.

They are interested in the reaction of people and the

communication between people and the Luminous

Textile Panels (LTP). This medium uses LED’s under

textile to create an atmosphere through blurry visuals.

The panels as they are now are static objects which

change appearance in a passive way. Philips provides

eight LTPs with black fabric to be used at the festival.

Their sizes are 250 x 120 cm. (3)

Philips’ requirements Interactivity with and through the LTPs

Exhibiting new product, expanding brand with interac-

tive lighting

Exploring the technical limits of the Panels

Beta-testing for the software

18 | Introduction Introduction | 19

Stakeholder TU/e Industrial Design

The goal of the faculty of Industrial Design is:

“Educating unique opportunity creators for

societal transformation through intelligent

systems. (Systems are networked technology,

products, services and users within a societal

context, and the interaction between them).”(4)

The focus of the theme Playful Interactions is

on changing people’s lives by designing playful

systems that seduce people to activities that

contribute to their health and well-being. As

Huizinga wrote in his book “Homo Ludens”

(1938) people are inherently playful beings.

How can we design products that allow for

playfulness in daily activities?

The playfull interaction group examines the

range between designing for play, such as

play environments and designing for a play-

ful approach to all sorts of other activities, for

example supporting playful communication

between older adults. (5)

TU/e ID’s Requirements Playful interaction

Intelligent systems

Societal transformation

2

Midterm Exhibition ResultsFor the midterm exhibition every student came up

with their own idea for an interaction. We worked out

our own ideas to present different possibilities at the

exhibition.

Anika, Tom, Alex, Max

The first year students want to stimulate communica-

tion between visitors. Every visitor is related to a light

spot on the panels which tries to get their attention.

These lights want to urge you to follow them and lead

you to another person so you can start a conversation.

Because of the interaction between the lights above

you, you might be triggered to interact with the person

standing in front of you.

22

23 | Mid

term Exh

ibitio

n

TijmenTijmen created a concept where people could manipulate con-

tent by movement of a physical object. Tijmen made use of the

accelerometer in his phone to test his interaction. By changing

the angle of the phone the content on the panels would move

accordingly. By putting the phone in a chair people were able to

manipulate the content by changing the angle of the chair. More

about this concept can be read in Tijmen’s individual part in the

latter part of this report.

24

25 | Mid

term Exh

ibitio

n

Marieke created a concept where people could paint on

the LTPs: the system would use the colour of your shirt, your

skin, your hair or the expression of your face to relate to

you on the panels. By looking at the panels you will leave a

trace of the colour of your shirt. If someone is just walking

through the field without looking at the panels he will only

mess up the colours or leave a rainbow of colours.

By working together with other visitors you can create and

change the colours on the panels and thus the lighting in

space. Your swatch would change according to your speed

and direction. By gathering all people underneath one

panel the game play will change.

Marieke

M1.2Vivian came with the idea of a visual world seen from underneath.

Design proposal for content creation on Luminous Textile panels

A direct mapping of the position at the ceiling as if seen from underneath

Motivation

I find it interesting to investigate how interactive multimedia design stimulates

user experiences within the context of public events and entertainment. Within

these event settings interactive media is recently introduced for commercial or

entertainment purposes. I am not directly referring to the social media. I would like

shows and performances to evolve in interactive play and let spectators become

participants by designing such interactive physical environments. Many people are

not used to this type of interaction yet, often the scope of interaction is limited to

human computer interaction. That is why I wonder what kind of input is interesting

for creating an interaction with the environment.

Research questionWhat behaviour is relevant for understanding the interaction between human and

environment?

Aim and objectivesLeads the possibility of interaction between visitor and installation in an architec-

tural context to playful behavior among festival visitors? I would like to find out

whether interactive media in the architectural setting is being recognized and how

it adds value to the experience of visitors in a temporary event setting.

Vivian

26

27 | Mid

term Exh

ibitio

n

Screen shot of Surface Film, part of thesis project MFA Design+ Technology ‘09

What changes are being recognised in the architectural context?

By the use of multimedia systems architectural spaces evolve in dynamic, respon-

sive or interactive environments. One of our biggest concerns is that the visitors will

not recognize the ceiling to be interactive. With subtle changes in the environment

we try to catch the attention of the ones who pass by.

To what extend are the visitors aware of their influence on the output?

The interaction between human and environment is difficult to define and will not

immediately be recognized. More changes in the architectural context might ques-

tion people what their influence could be on this.

What influence has the number of visitors on the behaviour of the participants?

When more people interact with the same medium it might become unclear who

is responsible for the effects. A lot of individuals creating input for the system can

result in the same large variety of output. The individual input and its effect on the

installation is than more difficult to relate. This could be one of the causes for visi-

tors not being aware of the interaction.

What were custom behaviours of the visitors interacting with the panels?

This might be important in making a difference between functional and playful

behavior. By observations we could identify the behavior which is performed to

discover new outputs and exceeds the custom behavior in that setting.

3

Designfor the festival environment 31

30 | Explorations

For this project we had 8 panels that we could use for the interaction. Before we came up with an interaction we decided to reach a clear agreement on how the panels were going to be placed. We made a template in Google SketchUp of the space where our panels would going to be placed at the festival. In this template we all presented different ideas of the positions of the panels. The only require-ments we got from STRP and Philips was that the panels had to be fixed on a minimal height of 3 meters. This was due to the varying public that would visit the festival and we could not afford any damage to the products of Philips.

of the PanelsPositioning

The spaceThe panels were going to be placed in the lounge of the light café of the STRP-festival. The light café is an area for the festival visitors where they have the pos-sibility to eat and drink something and escape from all the impulses from the other areas. In the light café there is a continuous passage of visitors.

After discussing every idea for the posi-tions of the panels we chose four ideas and sent these proposals to STRP so they could

give feedback. Of the four proposals we preferred option

one because of the fact that this option covers a large space. This option also offers many opportunities for interaction since it is effective regardless from which side it is

approached. In option 3 the panels are po-sitioned close to each other, which lights

up one area which is like a spotlight.

The feedback we received on proposal 1 was that with a half open curtain it would

give too much light pollution on the podi-um and proposal 3 was too squared. After this feedback we made one new proposal.

The feedback we received on our new pro-posal was that it would interfere with other light sources and that large parts of the space (two blocks) we had available were not cov-ered. STRP came up with a change in our new proposal. This proposal seems to them the best option because it is a bit asymmetrical and it covered the area we had completely. They also agreed with our idea to position the middle four panels horizontally and the other four a little tilted.

After we discussed the proposal of STRP we agreed that this would be a good option for the positions of the panels.

Communication and approval

of STRP

1. 2. 32 | Explo

ration

s

3. 4.

5.

6.

Video capturing

After the first client meeting with Philips we were much

more familiar with the panels. For the first explorations we

would experiment with different kinds of content for the

panels. We started with filming all sorts of footage in the

photo studio. We all brought different materials, such as

crepe, a disco ball, a flashlight, different colored liquids. We

used different light settings to play with shadows and light

effects. We switched between a black and white background

to see what was the best effect was but we soon agreed on

the black background because this gave the best contrast-

ing effects with the black textile on the panels. We also

filmed drinking glasses that were slowly filled with a liquid

and reversed by drinking through a straw. Finally we shoot

moving objects like rolling and bouncing balls.

With the footage we went to the High Tech Campus to test

this together with material from YouTube. We noticed that

using a variety of colors has a nice effect on the panels.

Movement of visual images only works when the movement

is very recognizable, for example a silhouette of a shadow

play of hands was easily recognizable, but the footage of

quick moving balls did not work out on the panels.

Shadows did have a nice effect but the silhouettes need to

be of such a size so that they are clearly visible and recogniz-

able.

For the next appointment with Philips we made some new

footage. This time we created footage based on the brain-

storm sessions which resulted in the use of natural elements

and upside down worlds. Max made a Flash animation of

walking footprints and Vivian used an enlargement of an ex-

isting video of YouTube to create the same idea as Max. We

also shot new shadow material using cloth and spotlights.

While we were testing this footage we noticed that the foot-

prints had just the same effect as the shadows we tested;

they really had to be big enough to be noticeable.

34

35 | Explo

ration

s

http://www.youtube.com/watch?v=W17Ce9TrD7M&list

=PL825402CE33CC8B00&index=8&feature=plpp_video

http://www.youtube.com/watch?v=X26kt53N35s&list=

PL825402CE33CC8B00&index=9&feature=plpp_video

36 | Conclusions and recomandations Conclusions and recomandations | 37

36 37

36 | Co

nclu

sion

s and

recom

and

ation

s

37 | Co

nclu

sion

s and

recom

and

ation

s

Exploringmovement

Philips Content ManagerIn the last exploration phase with footage we were able to test at the

university since we were equipped with software from Philips to use on

our own laptops. The software would show a preview of how uploaded

videos would look when uploaded to the panels. We used natural ma-

terial as visuals to test with that program. We filmed moving trees, fall-

ing leaves and flowing water. We also got a new idea about the upside

down world. We placed a camera with a fish eye lens on the floor and

walked around it to get the idea of a transparent floor. When importing

these files in the Philips preview-software it seemed that the leaves and

the water were hardly visible because of the blurry effect of the panels.

The trees worked out pretty well because of the color contrast and the

adapted speed of the video file.

38

40 | Explorations Explorations | 41

Mockup Panel

Motivation/reason:At this point in the project we wanted to go from paper

towards reality. Because we did not have a panel avail-

able to test content we decided to make a test-panel so

that we could get a better feeling of the situation at the

festival.

We built a spruce frame, and a surface of white sheets.

We chose the dimension of the panels we would get in

the final setup. We used a beamer with a slightly unfo-

cussed output to simulate content on the blurry panels.

This panel was installed in the hallway for a couple of

weeks. The purpose of this was the possibility to observe

the passing students in a very early stage in the project,

and see how they reacted to different footage. What we

concluded from these observations was that people only

noticed the visuals when there was an obvious variation

in color and movement.

ExamplesAt first we started to beam YouTube content on the

panel. This was just to see what kind of effect this had on

the people who walked by. The videos we played were

mainly very abstract and colorful. Bit by bit the concept

got shape, and we started to find out which visuals we

wanted to have on the panels.

With our requirements in mind Tom made content in

Particle Illusion to simulate this. There were two particles

on the panel which moved around each other and even-

tually met. With the sitting bags underneath the panel

we could experience the effect of video animation from

above just like it would be in the lounge at STRP festival.

But it still was not interactive yet. Tijmen made a Process-

ing sketch which was interactive. An iPhone was used

as accelerometer. Simply by moving the iPhone around

different kinds of balls danced around on the panel. As

soon as it got interactive, the effect of the combination

with interactivity and a large panel above you could be

experienced.

ConclusionThe decision to make a real physical panel brought us

one step closer to the real thing and helped us to test

our ideas in context. The further developed context also

narrowed down the ideas we had towards more concrete

ideas for the setting at the STRP festival. After the panel

was finished we all went our own way in developing our

ideas further. Because we tested our ideas on the panel

during this development we were able to give each

other feedback on the results.

Mockup Panel II

Motivation/reason:When we decided to use multiple Kinects we needed to

simulate a test situation with multiple Kinects that was as

close as possible to the real situation at the STRP festival.

Because we were not able to use the LTPs until the festival

we needed a test situation to find new opportunities and

possible pitfalls.

Construction/situationUntil now we had a panel hanging at a height of 2 meters,

at the festival it would hang at 3.5 meters. Therefore we

chose a spot at the Technical university where we could

make a construction from wood, some duct-tape and wires

to simulate the situation at the STRP festival. In this setup

we mounted 2 Kinects to the wood with a space of approxi-

mately 2.5 meters in between.

ConclusionWith this setup we were able see how large the new play-

ground was. We found out that we would need high quality

USB extension cables to prevent data loss of the Kinects. Be-

cause of this setup we could also decide the direction of the

Kinects, and the amount of space needed between them.

http://www.youtube.com/watch?v=ljrGmuCznKs

43

42 | Explorations

44

45 | Explo

ration

s

Brainstorm interaction

After the midterm exhibition we started thinking about the

concept for the interaction again. The best way to do this

as a group was through several brainstorms.

With the idea that colors really work well with the devices

of Philips we wrote down everything that came to our

minds concerning movement and different colors. Of all

these ideas we chose the ones

that we thought had the best potential. Using these ideas

as elements for another brainstorm we continued to

expand the options of these elements. This brainstorm

resulted in 5 new criteria. With the idea in mind that we had

to make a vague interaction because of the low resolution

of the panels in comparison to “normal panels” we did a

third brainstorm about vagueness. eparate from the last

brainstorm

we did a fourth brainstorm about ceilings because of the

way we are fixing the panels and this resulted in another

brainstorm about an upside down world. An important

part of the content was the way we would draw people’s at-

tention to a spot in the room that you normally hardly look

at. For that we closed the brainstorm sessions with three

brainstorms about seduction and deception.

Eventually there were seven brainstorms on the

table. The themes we thought were most fascinating

were the things you see from below and the ele-

ments of nature. These were the two concepts that

we started to work with, so we started to make foot-

age of this to experience the effect on the panels.

4

In the 6th week there was a group discus-sion about the concept requirements.

Everybody mentioned his or her personal requirements, interests and ambitions.

Vivian Likes the development from invol-

untary towards conscious interac-

tions. To implement this in the

concept we should take the actions

already present as point of origin for

the interaction. This project she will

set up test cases and work on the

analysis, strong visual language and

would like to develop video animat-

ing skills.

Max

Likes interactio

n through objects. H

e

wants to tu

rn ordinary objects into

something extraordinary by designing

its interactio

n. Curio

sity is an in

teresting

human characteristic to

be used. He has

no experience in

programming, so this

project could serve as a nice in

troductio

n

by observing the softw

are experiments

and developments of the concept.

Tom Wants the interaction to be not too obvi-

ous. He thinks there should not be a clearly

defined moment where people decide to take

part in the interaction. The system should do

something unexpected. Like Max he has no

experience in programming and will observe

the software development from the sideline.

He is interested in the overall development of

the concept, the relation between the input

and output and how people will react.

Marieke

Would like a more direct form of interac-

tion and the system should at the end

give rise to an intended and conscious

interaction. The focus is on connect-

ing people and creating together. She

would like to work on the visuals, and

think about the interaction, taking the

output as point of origin, and would

like to find out how people react to the

images.

Alex Thinks that the concept should be

present in a subtle way. It does not

need a clear purpose, but serves as

entertainment; give the visitors the

opportunity to create. He has some

experience in programming and

would like to develop this. He also

would like to improve his visualiza-

tion skills, but not necessarily within

the scope of this project.

Tijmen

Thinks the concept should have an

element of surpris

e and some vari-

ation by m

aking different levels of

complexity. H

e has experience in

programming, which can be useful in

developing the concept. H

e also has

to develop his form

and senses skills,

and this could be achieved by creat-

ing a report l

ay-out.

Anika

Would like the system to react to un-

intended actions. She likes program-

ming, has some experience in PHP

but likes to develop other languages

as well. She wants to develop her visu-

alization skills next semester.

Considering an interactive system consists out of three

elements - input, system processing and output - we sepa-

rated the team in terms of these categories. We would first

explore the possible solutions, then combine the system

and improve the quality.

INPUTTom and Anika

SOFTWARETijmen (towards input) and Alex (towards output)

OUTPUTMarieke, Max and Vivian

After discussing everybody’s concept require-ments we combined them into the following requirements:

Surprising element by random factor

Variable complexity

Subtle in its presence

Turn ordinary into extraordinary

No clear moment of participation

Stimulate co-creation

Trigger intended in-teractions at the end

Sketches

Since the previous brainstorms were more about content

for the panels instead of interaction scenarios, we decided

to do another brainstorm, more focused on interaction. This

time it was based on sketches. We thought about different

scenarios and immediately drew these on paper. This gave a

good impression of what a certain interaction will look like.

The actions that had to be performed for the interaction still

ranged from drinking out of a glass to waving the arms. Also

there were some options on how to start the interaction.

Results

The most important thing that came out of this brainstorm

was the principle of having at least two hot spots or play-

grounds. Using these playgrounds the user would be aware

of whether he/she was joining the interaction or not. The

second thing we wanted was to make a creator out of the

player. He or she would be able to control, paint or influence

the panel by just walking underneath it.

By thinking of these different scenarios we set a major step

towards our definitive concept. The hot spots and painting

principle would be essential for the end result.

Conclusion

5

Choosing a sensor

When reviewing the scenarios we found most interesting,

we could clearly see that most scenarios would use the

position and movement of people to create an interaction.

Using a Kinect as sensorial input sounded very promising,

it was the solution to track the movement and position of

people in space. Regarding the other options we had, this

would be the least fragile and most reliable sensor to use.

Technical specifications of the KinectWhen we choose the Kinect as the sensor for our interac-

tion the input team started to look at the specifications

and possibilities of this sensor with all its multiple options.

While searching the internet we discovered the versatility

of this sensor. Its basic properties satisfies the exact require-

ments of what we want to do with it. The combinations of an

infrared camera, a RGB camera and the infrared laser make

it possible to capture 3D images. The depth sensor consists

of an infrared laser projector combined with a monochrome

CMOS (image)sensor. The Kinect is capable of detecting the

skeleton from one up to 6 people. Because of the motorized

tilt the sensor can vertically detect in an angle of 47 degrees

and can be tilted up to 27 degrees up or down. Horizontally

the Kinect “sees” an angle of 57 degrees which provides a

playfield of 6 square meters. Next to this, we found out that

this sensor is a popular hardware product that has already

been hacked by hundreds of creative programmers. This

59 | Techn

ical Prog

ramm

ing

Technical Programmingmeans that beside the regular software of Microsoft© we

could find a lot of other software that could help us to

realize our ideas. (6)

We wanted to detect people from above, since this

would enable us to precisely locate the position of

multiple people in space. We had to find out what the

Kinect exactly detects when it is fixed parallel with the

ground. For that reason we made a small setup in which

we placed the sensor on a height of 3 meters and with

the software that the technical team had provided we

defined the playground for the interaction. Because of

the adjustable depth sensor we had to go through these

steps twice because the first time we detected the top

of the head during the measurement. This resulted in a

small, non-profitable rectangle in which hardly any inter-

action was possible.

The second time we defined a much bigger rectangle

because we changed the range to detect people’s feet

which gave more options for an interaction. We even

tried to enlarge this playfield using a fish eye lens espe-

cially made for the Kinect. It appeared that this had no

value for the detection because this lens also changed

the accuracy of the sensor in a negative way.

Kinect Library

After a lot of research about which drivers and libraries

to use (see Appendix ) we decided to go for the library

of Thomas Diewald. The main reason for this was that his

sketches from his OpenCV(source) kit worked without

calibration of hands or skeleton. Another advantage of this

library was that Tijmen could use Thomas Diewald as an ex-

pert during the process as he responded quickly to pro-

cessing-related questions via e-mail. The processing sketch

that we eventually used made use of the depth vision of the

Kinect and blob detection to detect persons in the scene. (7)

Blob Detection

Blob detection is a method that is used to filter objects and

people out of a map of pixels detected by a camera using a

certain threshold. Because the Kinect senses depth the use

of Blob detection is much more precise than when using a

normal camera. This is because in a normal camera needs

an contrasting background if you want to detect blobs.

A depth camera makes a high contrast image even in a

chaotic background. Because of this advantage of a depth

camera blob detection is faster with a depth camera than

with a normal camera.

60 | Technical Programming Technical Programming | 61

Multiple Kinects

When we first tested one Kinect searching for blobs from

above (from the ceiling to the floor) we measured the

amount of space in which it could detect blobs. This was

175 x 135 cm for detecting the head when the Kinect was

at a height of 3 m and 245 x 185 cm when the Kinect was at

3,5 m.

The total space that the panels would cover at the STRP

festival was 1000 x 420 cm. Therefore it was logical that we

needed more than one Kinect to extend our playground.

Because there was no budget it was not possible to buy

more than 3 Kinects (3*120 euro) from our own money, this

meant that we could work with a total surface of : 735 x 185

cm.

How to connect multiple Kinects

When we had decided that we wanted to use multiple

Kinects we had to find a way to combine the data into one

visual output. To do this Tijmen thought of several options

to get this done:

Multiple Kinects connected to one PCOne option would be to use one computer for connecting

the Kinects. The advantages would be that we would only

need one pc to receive the data and make a visualization.

Together with Marieke and Anika, Tijmen tried a sketch of

Thomas Diewald made especially for multiple Kinects.

PitfallsWe would need an extremely fast computer to run 2 Kinects

at the same time, we tried this on the newest computers

with a Core i7, on this computer it worked with 2 Kinects

but there was another problem. Adding 3 Kinects to one

motherboard was not possible as the USB slots of a mother-

board can only handle 1 Kinect per USB Slot. On these fast

computers there were only two USB slots available. So this

would not work with three Kinects.

Kinect Playground

Lumalive Panel

Using a databaseWhen was decided that we needed to connect all the data

of the Kinects in a different way Tijmen thought of an option

to use a MySQL database or to update XML files. We didn’t

choose any of these options because it was not practical for

the amount of data we had to transfer.

OSCAnother solution was to use OSC (Open Sound Control

which uses the UDP (User Datagram Protocol) protocol to

send data over a Local area network). Tijmen already had

some experience with this from the midterm exhibition

where an iPhone was connected with WIFI to a PC. During

the midterm exhibition this was done using an iPhone App

Touch OSC, and a library for processing called oscp5 (8).

OSC is also able to transfer data between multiple PC’s and

therefore extremely useful when we would connect multiple

PC’s with each its own Kinect to one PC. (9)

AdvantagesIn the end of the process OSC was the right choice for us.

We had no loss of data at all using the UDP protocol. With

OSC we were able to send the data of the Kinects towards

multiple IP addresses. Because of this it was possible to test

new visualizations during the STRP festival without having

to stop the visualization on the visual PC. OSC showed us

just a glimpse of what was possible, Resolume a VJ software

also has an option to receive OSC messages and thereby

many more like VVVV, MAX MSP and touch OSC. This tool is

highly recommended when you want software to communi-

cate with each other.

Setup with multiple Kinects using OSC

When we had decided to use OSC we had the following

setup:

This setup can be divided into 2 different parts, a sending

part and a receiving part. For each part a processing sketch

was written. Without going too much into detail the follow-

ing text will explain what these 2 parts did.

Sending messagesIn the Kinect sketch we incorporated OSC to send data

about each blob. We sent the data of each blob to the visual

pc. Each computer would send its data with a unique tag so

that in the receiving part on the visual pc the data could be

ordered and mapped in the right way.

Example adding message:

blobMessage.add(bb.xSize());

Example sending message:

oscP5.send(blobMessage, myRemoteLocation);

Example receiving message:

int blob_Xsize =(theOscMessage.get(i+1).intValue());

Receiving messagesThe visual PC received all the messages of the 3 computers.

The incoming data were in pure random order. For example

sometimes we first received 5 messages of one pc until an-

other pc came through. Therefore we had to filter out all the

messages and combine them into one set of data that could

be used for the visualization.

63

62 | Technical Programming

The Kinect sent the data of each blob it

sees in the order it detects these blobs,

and this in the same way that you read

a book. This could mean that in the first

case person one had position one in the

ArrayList. When a new person walks in

the playground this new person would

get position 1 and the one who had first

position 1 would get position 2.

This meant that the visual object that

belonged to the first blob would receive

new x and y values from the second

blob. This made it look as if this first blob

had moved a lot.

With one Kinect this would not be such

a big problem but when you have a

lot of persons and 3 Kinects you would

only get chaos and all the visuals would

change position to such an extent that it

would appear as random.

Therefore we had to find a way for the

position of each visual object to change

according to what was the nearest blob

(person).

To do this we wrote a function that

would calculate every distance between

all blobs and objects and engaged the

blobs to an object which had the small-

est distance. The x and y of this blob

would be used to update the visual ob-

ject. In this way the order of the ArrayList

of blobs could change without having

to change the order in which the visual

objects where stored in their ArrayList.

ArrayLists

Before we go into the visual programming a few things need

to explained first.

During our process of programming we bumped into a lot

of, for us new things. One of these things was ArrayLists.

One reason why we chose to work with ArrayLists was that

with an ArrayList data items can be easily added or removed

from the ArrayList.

Another reason we decided to choose to work with Ar-

rayLists instead of normal arrays was that a ArrayList is a

resizable-array implementation of the Java List interface.

Because of this a ArrayList has many functions we could use

to control or search for content in the ArrayList. Some of

the functions we used in our processing code were size(),

add(), remove(), get() and shuffle(). For instance the function

size() returns the length of the ArrayList and with the get()

function you can request the variable number/object from

a chosen position. We used the function shuffle() to create a

random Mondrian. (10)

From received data to visuals

Once the data received were filtered and ready to be sent to

the visualization there were several difficulties to overcome.

What will happen when someone new walks into the vision

of the Kinect?

Per blob a new visual object had to be created. Each of these

objects was assigned their own color and stored the X and Y

values of the blob in an arraylist.

MinDistBlob:

One big problem during programming was to assign one

specific visual to one person. For the amount of blobs there

were on all the 3 Kinects there were x and y values stored

in an ArrayList. To prevent this problem the order of the

blobs in the ArrayList had to be the same as the order of the

objects in the ArrayList.

65 | Techn

ical Prog

ramm

ing

6

68 | Visual Programming Visual Programming | 69

Visual ProgrammingTesting with mouseWhile Tijmen worked on the technical part of making the Ki-

nect communicate with Processing some first tests were done

with a computer-mouse as input. Marieke adjusted an already

existing code to fit the project better, and tested this on the

Philips Luminous Panels. This first code existed of “fireflies” cir-

cling around the position of the computer mouse. The speed

of the mouse influenced the way the fireflies circled around the

mouse. Since the Panels blurred the output considerably, the

details in the original code would be faded out. A simpler ver-

sion of this code was created to save processor-capacity. This

code had the same effect on the panels.

Marieke also tested her midterm idea, “Painting” in a simplified

“mouse”-version on the LTPs. When pointing to a color on one

panel one could choose the color of their swatch. By moving

your hand around you could “paint” on the panels. Immediately

it became clear that the chosen colors were clashing, and that

the stroke was not wide enough. These factors would have to

be taken into account when continuing with this concept.

Anika started with writing a collide function for the blobs

(objects perceived by the Kinect as people). In this function

different things could happen when blobs would collide, for

example: objects would change color when they collided. What

happens in this collide function is calculating the distance

between all objects; when this distance is under a certain

value collision has occurred. When objects collide this function

triggers another function, in which is written an action on this

event.

Testing with Kinects

These relatively simple codes were rewritten to react to the

output of one Kinect. The codes reacted to one variable X

and Y value (of the mouse), but now had to react to multiple

X and Y values of different blobs. (11)Classes were needed

to keep the code from getting too extensive. Since Tijmen,

Anika en Marieke were not that familiar with this kind of

programming, they asked Wouter van der Heide (2nd years

bachelor student Mathematics) for some help. Wouter

helped them setting up a processing sketch in which differ-

ent classes worked together in a structurized way.

Wouter wrote an AssignObjects class which had the func-

tion of adding or removing objects according to the amount

of blobs. Next to this class there was an Object class, which

contained the characteristics of the objects generated in the

AssignObjects class. Bit by bit they learned to use Object

Oriented Programming, and how they might be able to put

the envisioned interaction scenarios into practice

A problem occurred when they found out the Kinect

changes names of blobs when they have a different posi-

tion. More about this problem can be found in the technical

programming part of this report under mindistBlob. This

had as result that when 2 blobs switch position they will also

switch names, and as a consequence their colors will also be

switched.

Since your object was now always assigned to your blob,

Marieke saw the opportunity to put her mid-term idea into

practice; with a small adaption to the code painting became

possible. A stroke of your color would follow you along the

playground.

Meanwhile Anika focused on creating random objects when

there are no people within the Kinect’s sight. She added

a random blob class to the processing file with some help

from Bram van der Sanden (3th year bachelor student Infor-

matics). These random blobs are just like normal blobs but

are always present. These random blobs could collide with

each other and/or with the normal, “human” blobs (detected

by Kinect). Some things that can happen when a random

blob collided with a normal blob is that the normal blob

would take over the color of the random blob or the normal

blob killed the random blob.

When connecting all three Kinects to the visual PC a new

challenge arrived: combining these values into one visual

output. Marieke altered the incoming X and Y values in such

way that every Kinect had his own part of the output. For

71 | Visu

al Prog

ramm

ing

70

example, blobs of Kinect 1 were drawn in the first one-third

of the panel, and blobs of Kinect3 were drawn in the last

one-third of the panel.

After all the LTPs were installed, Marieke could start on fin-

ishing the mapping in the processing file to a final version.

It appeared that the blobs could have higher X and Y values

than expected. This meant that the resolution of the Kinect

output was not as expected (was not 640 x 480).

Blobs directly underneath the Kinect were not visible on the

Panels. These blobs had to be replaced to a visible area.

73

72 | Visu

al Prog

ramm

ing

Philips developed software for send-

ing content wireless from your pc to the

textile panels over a local network. Philips

content manager, which was the version

available at the beginning of this project,

was able to stream pre-created video

material to multiple panels. The setup of

the panels could be adjusted as long as

they are positioned in an angle of ninety

degrees in relation to each other.

This was actually not enough functional-

ity for the concept to be presented at

STRP festival. This time wireless stream-

ing of live content to multiple panels

was required. During a meeting at Philips

Lighting a programmer from Philips wrote

this custom piece of software which we

only had to adjust to our panel setup. The

software made it possible to stream live

content, selected from our laptop screens,

to the individual textile panels.

GraphEdit connects filters to a video

capture source, like a webcam or in this

case the laptop screen output. The filter

which sets this source to the laptop screen

output is the one we developed at Philips

lighting and was called ucapture.grf. This

filter has to be opened in GraphEdit. To

startup the streaming connection ucap-

ture.avs needs to run in a media player.

For this we used Media Player Classic. In

ucapture.avs the addresses of the panels

and the total pixel mapping are defined.

Several parts of the software were up-

dated during the design process to meet

our requirements.

Streaming content to Philips Luminous Textile Panels

When the LTPs were installed at the festival-area, together with

Philips Marieke started to adept the previously created file for

the streaming of the panels to the mapping of the panels as it

was now. A video file had to be created of the whole screen.

This was done using uCapture. From this file the pixels which

were meant to be sent to the Panels had to be downscaled. Also

the blank areas between the panels had to be calculated and

taken into account.

Streaming at STRP

Video file: 1600x1024 px

Processing output: 1600x720 px

Total resolution of all Panels together: 160x72 px

Resolution of one Panel: 20x40 px

Since the panels draw

their visual input in a

vertical way, horizontal

installed panels needed

to get the information of

the pixels also in a vertical

way. The pixels meant for

these panels needed to be

rotated 90degrees before

they could be send to the

panels.

All these calculations to

prepare the processing

output for sending towards

the LTPs where done in

AviSynth. Marieke worked

together with Floris Pro-

voost (Philips) to write a file

which does all these calcula-

tions.

In the first picture on the

right you can see the output

screen of processing, and

in the second picture you

can see the outcome of the

avisith file in uCapture. This

would then be the file which

would be send to the panels.

At the end of the avisynth file

all the data of the pixels was

send towards a panel with its

respectively ip address.

75

Example code rotation:

clip1 = TurnRight(rclip1)

Example code sending:

clip1 = SimpleSample(clip1, 40,20,2,”192.168.0.186”, 6038, 2,1)

Designing the Interaction

After the streaming was finished, Marieke could start

on finishing the mapping in the processing file to a

final version. It appeared that the blobs could have

higher X and Y values than expected. This meant

that the resolution of the Kinect output was not as

expected (was not 640 x 480).

Blobs directly underneath the Kinect were not visible

on the Panels. These blobs had to be replaced to a

visible area.

When Tijmen was moving around in the lounge area

he found out that he wanted the system to react on

him spreading his arms. This could be done easily by

measuring the width of the blob. Tijmen and Marieke

added different functionalities to this variable. When

pointing your arms to other Kinects you would cre-

ate a white flash when in the playground of Kinect1,

create a random Mondriaan-painting when being in

playground 2 or emit lightning when in area 3. If you

spreaded your arms in a different direction you would

create a huge blob above you. With this big blob you

could push other blobs off the panels.

Tijmen tested his Mid-term exhibition prototype too.

Using his mobile phone with accelerometer, he could

change the gravity of the balls on top of the panels.

These balls changed colors on the sound of the beat.

76 | Visual Programming

Programming interaction during STRP

During the festival Tijmen, Anika and Marieke kept programming additions to this basic interaction.

78

Visual Programming | 79

80

Visual Programming | 81

Painting The ‘painting’ code was tested and adapted several times. Marieke made

the brush size variable to the amount of people in the playground. More

people meant a smaller brush-size. Some experiments were done with

the colors. First random light colors were assigned to the blobs, but

these pastel-colors did not show much contrast with each other. This

had as result that your collaboration in the painting was not directly no-

ticeable. More bright colors were tested, but some colors really clashed

with each other. Some ‘ugly’ colors were filtered out, and in the end the

combination of colors felt satisfactory.

http://www.youtube.com/watch?v=f9kOuoorjds&list=

PL825402CE33CC8B00&index=4&feature=plpp_video

Programming interaction during STRP

82 | Visual Programming

Programming interaction during STRP

Micheal JacksonDorien van den Hurk (STRP) gave us the feedback that visitors of the

concerts during the night needed clearer interaction. Marieke created a

code in which panels would light up when someone was standing under-

neath that panel. When walking around panels would light up above you,

always in the same color. Since this interaction method did not work that

well when there are many people walking around in the area, an idea was

created to split the panels into parts; when four people are underneath

one panel the panel would split into four sections. A start for this code

was made (a panel splitting in two sections when more than one person

was underneath it). It had not been worked out more because of the

complexity of the idea.

http://www.youtube.com/watch?v=LqdCKmohGOY&list

=PL825402CE33CC8B00&index=2&feature=plpp_video

84

85 | Visu

al Prog

ramm

ing

Programming interaction during STRP

Extra Layer Hands During STRP festival we saw that children always used their hands to interact with the panels. In our

interactive installation nothing happened when they moved their hands. That was the reason why we

decided to add an extra layer (extra blob detection area) to the interaction.

When you put your hand in the air there would appear a pulse above you on the panel; a growing el-

lipse with no fill and a white stroke. When there was more than one blob detected in the second blob

detection area, processing would draw a line (connection) between the positions of those blobs. It

would connect the position of blob 0 with blob 1, blob 1 with blob 2, etc. .

When testing this at STRP festival we noticed that this wasn’t visible enough so decided to not develop

this further. (12)

http://www.youtube.com/watch?v=y5EhWBHdNf0&list=

PL825402CE33CC8B00&index=3&feature=plpp_video

87

86 | Visual Programming

Programming interaction during STRP

Adjusted Mapping for walkingWe also noticed that because the interaction was on the

panels above you people didn’t notice that they actually

where causing a change in visuals. They were more perform-

ing for the people who were sitting in the lounge that they

actually noticed that they did something. Therefore we

wanted to add that something would happen not directly

above you but a few meters in front of you. Tijmen made a

program that calculated the speed in a certain direction and

moved the visualization accordingly to the direction of the

walking route.

http://www.youtube.com/watch?v=x8kG6yG7pUM&list

=PL825402CE33CC8B00&index=7&feature=plpp_video

89

88 | Visual Programming

Programming interaction during STRP

Sound for SaturdaynightWe all noticed that during the concert on Saturday night the

atmosphere was completely different than during the day. There

were at least 4 times as many people as normal and they were

there not only for all the art but especially for all the music. This

music was most of the time with a very loud bass also in the

area where we had our setup. Tijmen had the idea to make a

visualization on the panels that would react on the sound. The

program automatically detected the right sound level and cre-

ate content that was created dynamically on the sound. More

sound meant brighter colors and thicker lines, but it was namely

the difference in sound that measured.

http://www.youtube.com/watch?v=e0cWmtvPBOs&list=

PL825402CE33CC8B00&index=1&feature=plpp_video

7

Floor plan and Technical drawings

Max and Tom did not get full approval from STRP for our

panel-mapping that we used to make our physical model.

So we adapted a new version into the old model. The final

position of the panels was set up as a unit which had a clear

division with three hot spots. We used this division to create

three separate playfields for the interaction. In this way the

playfields were divided over the surface so we could sepa-

rate different interactions. Because we received an updated

floor plan from STRP we were able to implement this new

mapping into the 2D map of the location. After we reached

an agreement about the positioning we had to further

elaborate this mapping with technical data. We made a list

with all the supplies we needed for the realization of the in-

stallation. This list contained all the extension cords, Ethenet

cables, power cables, routers, switches, computers and

sensors that made the installation into a whole. With this

information we could edit the aforementioned map. First we

did this by sketching over the original map and after that we

digitalized after on to make it easily editable and enable it to

be sent to the other parties. For this technical drawing Max

used a program called Microsoft Visio 2010.

Production of the concept and Stakeholder agreements When you are working with tree major parties, a lot of ar-

ranging is needed to let things run smoothly. When the festi-

val approached there was still some missing equipment and

due to some miscommunication there were still some am-

biguities about the division of tasks between the different

parties. As soon as the panels from Philips were transported

from Berlin to Eindhoven they had to be delivered to STRP

as soon as possible since there was no space to store them.

This concerns four boxes with approximately the size of two

refrigerators and a weight of 200 kilos. Nor the University

nor STRP could store these boxes in such a way that they

would be safe. Philips had to store them until the first day of

assembly. For the assembly neither parties had time or staff

to spare to coordinate the construction. That is why Alex and

Tom got instructions and material from Philips to guide this

process. There was still some confusion about the exact sup-

plies. Vivian set up a list with what each stakeholder needed

to provide.

List of materials and services

Philips

Eight Luminous Textile Panels (120x250)

Material for hanging up the panels

Power- and Ethernet cables

Steel cables

Router and two switches

Workshop for mounting to two students

STRP

Pliers and clamps

Available room for a computer on the ground

Power supply

Different color carpet tiles

Possibility for testing the installation

TU/e

4 computers

Cables

2 students for the assembly

Because the installation would hang on a height of 3,5 me-

ters we had to work with a servicing platform. We checked

who was liable if something went wrong. This seemed a bit

more complicated than we thought. Since STRP said that

they could not be responsible for that we informed the

University. They referred us to the insurance agent of the

University. Finally, after a couple of e-mails it seemed that

event insurance would cost us a lot of money and still gave

us a high risk. We decided that the costs of the insurance

had not enough benefit over the amount of risk we would

take with the servicing platforms.

According to the agreement we would arrange four laptops

that could stay at the festival for ten days. Because there was

no budget available within the project we had to arrange

these computers within our own network. From Jeanette

Schoumacher, who manages promotional material of the

faculty of Industrial Design, we could borrow two laptops,

which were capable enough for what we wanted to do with

it. Another two laptops were provided by the UCE-research

group of our faculty.

It was essential for the remote control of the computers that

we had an iPad. We could borrow two iPads from ID educa-

tion to easily turn on, access and apply changes into the

computers while they were on a height of 4 meters.

We made a schedule for the festival so that there was always

at least one person present that could reboot the system

when something went wrong.

Production of the Festival | 93

Making a model of the festival areaOnce we found out the first specifications of the sensor

Tom started to place it in different ways into the available

space. He tried to cover the complete field underneath

the map of the panels so that every spot under the panels

could be used for the interaction. It appeared that the size

of the playfield could not fit within the mapping of the

panels without overlapping. Even not without a combina-

tion of normal Kinects and Kinects with an extra zoom lens.

Besides

the fact that the overlapping part would become a dif-

ficult programming problem, there was no money to buy

at least five Kinects. With this information Tom started to

build a physical model of the situation at STRP. In an earlier

stadium of the process we voted for the panel mapping

and he used this to create a miniature of the conditions.

94

95 | Prod

uctio

n o

f the Festival

http://www.youtube.com/watch?v=2UJfpmQqYcw&list

=PL825402CE33CC8B00&index=6&feature=plpp_video

Mounting the hardware

The tools we were given were clamps, cable and all the electronics.

Gloves were needed to ensure that we could handle the panels safely.

Putting the panels up had some complication because of the size and

the technique with which they were hung. More can be read in de indi-

vidual part of Alex Schepers

Electricity and networkTwo electrical groups were made to ensure that if all were turned on at

once, no overloading would take place:

1. The panels

2. The data network

The first group consisted only and entirely out of the LTPs.

The eight panels were separated again at two sockets. Because the

panels could be interconnected, meaning that the power cable from

one could continue to the next, we were able to make two groups of

four without a need for long cables. The Ethernet cables between the

panels however could not be linked and had to be fed to every panel on

its own.

The second group was a mixture of electronics:

• Three Laptops that were used to collect data from the Kinect and send

it to the visuals laptop

• A LAN-router

• Two LAN-switches

Apart from all parts in this group needing power, ethernet cables were

also needed to establish a network.

Every laptop was connected through a power source which we could

individually power on or off. This meant that if a laptop would for some

reason fail, the rest of the system could still continue without a problem.

Should one of the groups have failed, the other would simply wait for

the other group to come back online.

97

96 | Production of the Festival

8

During the STRP festival our concept is exhibited at the Light

Café. This is the perfect opportunity to collect the opinions

and reactions of the visitors and participators on the interac-

tive play.

The festival will be visited by art lovers, creators, designers,

students, creative professionals, music fanatics, and many

others who are interested in art and technology.

The settingOur ceiling concept was positioned above a lounge interior.

The play field was in the centre of the lounge. The rectangu-

lar shaped area was circumcised by half room dividers, which

created lounge booths at the sides of the play field. The play

area was crossed by all those who were looking for a place

to sit down, met friends or were on their way to the next

exhibit.

DiversityThe program of STRP festival is very divers. During day time

the exhibition is the main activity, together with workshops,

readings and lectures. Throughout the week also several

schools will visit the festival, so in the mornings the overall

audience will exist of ten year olds. During the weekends the

festival is also opened during the evenings. The planning

states performances of great music artists and DJs and these

will attract a lot of festival visitors. Because the Light Café

is opened during all activities, we can be assured of a large

variety in visitors for our installation.

STRP festival

100

Production of the Festival | 101

Research and developmentThroughout the festival we changed and

updated the program for the interaction and

visualization. The presence and behavior of

visitors gave inspiration for improvements and

adjustments.

For research purposes we conducted some in-

terviews and did observations on the behavior

of the visitors at our installation. All interviews

and observations were acted out during the

finalized version of the program during the

last weekend of the festival. The functionality is

explained at page… of this report. During the

research we sat at the lounge on the side of

the play field and took notes on our IPad.

Data gatheringTo gather information for concept improvements, research

analysis and reflections we documented our observations

during our stay at the festival. Below are some results we

got from interviews and live observations. The observations

were conducted according to the categories proposed in

the outdoor play observation scheme for evaluating head-

up play (ref ).

102 | Production of the Festival Production of the Festival | 103

There appears to be a relation between the level of physical

activity and the understanding of the interaction. The more

movements are tried, the more functionality is discovered.

Most physical activity is shown in trying out the functional-

ity of the system. Repetition of the movements appears to

create awareness on how the system works, because the

movements then show the same effect. The behavior of

most visitors is actually limited to the standard behaviors

which are appropriate in the setting.

Intensive physical activityWhat stands out is the fact that only children are show-

ing intense physical activity. When the supervisors of the

school groups have given an explanation about the panels

and interaction, the children were given the opportunity to

explore the interaction for themselves. There are always a

couple of children waving and jumping below the panels.

This was one of the reasons that we added a second layer

which could detect this activity.

Non-intensive physical activityThis applies for almost all the visitors. They are walking in the

lounge unsuspectingly that they were causing an interac-

tion. This interaction, moving lines and changing colors, was

noticeable for the audience that enjoyed the sitting area of

the lounge.

But once they saw someone causing this interaction they

were more likely to copy the behavior. In the behavior of the

visitors it was remarkable that they started to wave above

their heads once they recognized that the system was inter-

active. They expected that the panels would react on this

movement, which did not happen in the beginning of the

festival. Later on the programmers adjusted the code so the

hands could also be used in the interaction.

No physical activityIf someone is standing still below the panels, a colored

circle appears above his or her head but mostly this person

doesn’t notice.(13)

ObservationsPhysical activity

FocusObservations 105

104 | Production of the Festival

There seems to be a big difference in focus between the

visitors who are walking underneath the installation and the

ones who are sitting down. Most people who are walking

under the panels only look up once, continue walking and

leave the play field. While sitting visitors look at the panels

as well as at other players and take more time examining the

surroundings. This might have to do with the fact that the

walking people have something else on their mind at that

moment.

Looking at other playersBoth adults and children are looking at the movements

made by other visitors. Movements of others that have a

new interaction as result were copied by bystanders. When

there are quite a lot of people this creates a certain conti-

nuity between watching and performing. People who are

watching want to try the interaction. With that they are the

new performers and provoke a new performers for other

people in the lounge. People who understand the interac-

tion can explain it to somebody else but they do not make it

more clear. We think that the most people were too embar-

rassed to take the initiative to make “shameful movements”,

but when we as experts of the interaction showed was pos-

sible by some extremer moves people were more likely to

step out of their comfort zone. In this way they see that they

can create something together.

Looking at game objectsUnder the panels there are hardly any objects. Only some

Fat Boy poufs pushed into the playfield. These poufs do not

have any influence on the interaction because the Kinect

only detects blobs above the shoulder. We think that this

is something that the visitors understand because they did

not try to get an interaction with any of the above named

objects.

Looking at something out of sight, possibly part of the gameWhen the visitors walk below the panel and like the thing

that they create, they often look at other visitors for confir-

mation.

ObservationsSocial Interaction

106 | Production of the Festival Production of the Festival | 107

As expected there was a lot of social interaction within the

setting of a lounge. We focused on the social interaction

which seemed to be related to the installation.

Notable was that standing people in the play field got

instructions from other visitors sitting on the side line, which

seems to origin from the fact that the sitting ones have

a better overview of the situation. And the level of social

interaction seems to decrease when the number of visitors

increases. It might be that people feel embarrassed to share

their findings or incomprehension when they are surround-

ed by a lot of people they don’t know.

Functional, with another playerThe special movements leading to erasing or coloring

require more explanation and work better individually than

with more people in the playing area. These movements are

not movements you make in daily life which is quite funny

in itself. People make these movements often unconsciously

by looking and pointing at something. Also when people

are toasting, shaking hands or greeting by giving three

kisses something happens. The same effect is reached when

somebody with a large waste is passing underneath the

sensor.

Non-functional positive/neutral, with another playerPlayers often proudly shout to other visitors what they have

discovered. They also repeated their discovery.

Unintended physical contactBecause people are looking up very often they sometimes

bump into another person, or accidently hit someone when

spreading their arms.

The differences in physical activity, focus and social interac-

tion clearly defined two different roles in our play field; the

one of active player, and the one of spectator.

9

FRENSFRIJNS

“In whatever way you look at it, the thing that makes STRP special are still the art pieces. And with 31000 visitors we surely can’t grumble.”

To find out more about the overall impression of the stakeholders about the pro-ject, we interviewed Frens Frijns (STRP Festival) and Floris Provoost (Philips) .

What did you like about the STRP festival? “I was very happy with the number of visitors and

the feedback of the artist world. I was also very

glad to see there was an increase in the number of

in art-viewers, symposium-visitors and all the other

visitors which came during the day. In whatever

way you look at it, the thing that makes STRP spe-

cial are still the art pieces. And with 31000 visitors

we surely can’t grumble.”

How did this project come about?“I came in touch with Floris Provoost for the panels

and I was already talking to Ben Schouten at the

time. Floris preferred artists to use his panels,

mostly regarding an interaction with the panels.

Because I thought the link with the TU/e was very

interesting, especially within the playful interac-

tions part, I was the entrepreneur as a manner of

speaking.”

Did we measure up to your ex-pectations/requirements?“Your determination during the festival was super.

It was really interesting to see the cooperation

between you and Philips and how that showed

itself at the STRP festival. The way you were at work

ensured me that it would all end well. To be honest

there was some chaos and some things that were

unclear at the beginning, but it turned out great at

the finish. “

What did you think of the final interaction at STRP festival with respect to your expecta-tions?“The interaction was quite underdeveloped in

the first weekend, but I saw a distinct improve-

ment towards the second weekend. De interaction

definitely increased, you could see a lot of people

starting to interact with it more. The continu-

ous variation was at least as important as how it

looked. I thought it fun to see how people didn’t

always notice what was happening above them.

That is much more fun than when it is all crystal

clear. That is the exact reason why I was of the

opinion that your project was far more interest-

ing than the other Philips project within the same

room. At this kind of festival it is very important

that the image is much more powerful and that

had happened a lot more during the second week.”

What was your general impres-sion of the collaboration be-tween all the parties (Philips, STRP, TUE) and everyone’s share?“I can’t really evaluate that. My cooperation with

Philips was limited to delivering the items neces-

sary. I was able to make some nice agreements

and that’s it. The miscommunications, as you have

stated, were not very clear on our side. You could

notice that you were not familiar with this kind

of situations and certainly not on such a location.

Overall it was very nice to work with you, every-

thing was agreeable. “

110

111 | Interview

s

What did you like about the STRP festival?“I really liked the festival this year. Probably because I’ve

been more involved in the construction of one of the art-

works. It was very versatile in the way that they used video, a

lot of good music and there were a lot of good media artists.

I thought that the line-up was of high quality and was also

very inspiring. Normally you have 4 or 5 stars and the rest

are pretty much in the middle but at this festival I found

everything of quite an appealing level of quality. “

How did this project come about?“Ad van der Meulen, someone here at Philips, wanted a

collaboration with STRP. He wanted to combine certain

research to exhibit something at the festival. Ad linked me

to Frens Frijns who sent me to the artist/designer Edwin van

der Heijden. Suddenly Ben Schouten came up with a group

of students who were crazy enough to work with Philips.

Finally Ben Schouten presented you to me as my experi-

ment team. The idea already ran for a while. At one point we

supported the idea of Edwin van der Heiden and then your

project came in and I found those two projects so interest-

ing that I wanted to support them both. In your project I had

a little more say. The thing I liked about your project was the

interactive part. For me, playful interactions has always been

something I’ve been involved with through previous col-

laboration with Ben Schouten and Rombout Frieling so I had

enough experience with it. Your group was well suited for

an experiment to expand our knowledge around a product

like your group used. This also gave you a nice opportunity

that you normally don’t get. On the other hand I will receive

feedback that I also normally would not get.

Of course I had some expectations with this project. I want-

ed to have a report with a piece of research I could link to

experiments, play and interaction. In general I find this pro-

ject very successful especially through the insights we got in

interaction. It was our first step into unfamiliar territory and

now I want to make some next steps with that. I definitely

see opportunities for further research to extend to personal

development and perceptions of a space combined with

animation and play. There are endless possibilities of such

products on which, together with the university, research

can be done. “

What did you learn from working together with different parties?“I know that you spilled a part of your valuable time on

figuring out how to deal with some technical issues, but this

certainly led to solutions. Finally, you made a one year dur-

ing project worthless in just two weeks. “

What did you think of the final inter-action at STRP festival with respect to your expectations?“To my mind the interaction was limited. People reacted to

the system but just not enough. It did not matter for the in-

teraction if you noticed your presence or not. Besides it was

too chaotic to be clearly recognizable. People understood

that the system was reacting to their presence but to me

they did not feel in control of the system. I would have used

less content. But it was certainly beautiful, decorative, fun

and it was what I wanted, also qualitatively. The interaction

was just too scantily developed. How I see it, there is too lit-

tle documentation about interaction. “

FLORIS PROVOOST 113 | In

terviews

What was your general impression of the collaboration between all the parties (Philips, STRP, TUE) and everyone’s share?“In the first place I thought it was a little bit chaotic. There

were seven students around the table, each with a dif-

ferent objective. You are people with all different per-

sonalities and goals within the project. For the long term

I noticed that team division was not evaluating equally.

The way I saw it I thought that not everybody was equally

involved. A reason for this could be that I did not see

everybody at the HTC when there was another meeting.

Secondly, I found that in the brainstorm phase there was

too little communication about the ideas that were cre-

ated, partly due to the lack of time. Some ideas may now

have slipped through the net. Generally I am as excited

about what eventually came out as the other people in-

volved, and this is definitely a reason to keep doing things

like this. “

What would you do differently next time in relation to the collabo-ration? (pro’s/con’s)?“I thought that there was too little time to properly work

out and test the concepts. You could partly blame that on

the panels that came available in a too late stadium of the

developments. Otherwise we could do more brainstorms

to gather more valuable information. If we had more time

we could have used a better, more systematical approach.

Therefore we had quite a slow start because it took a

while to create some clarity about the requirements. We

knew what making an interaction on the ceiling was dif-

ficult but we could have paid more attention to the differ-

ent options. As to the technical programming we should

have taken a better look at what already exists, instead of

diving into those problems. The communication was also

mainly about technical problems while more attention

might have been spent on the evaluation of the idea. “

115

115 | Interview

s

01

In this project we have developed a playful interaction with

Philips Luminous Textile Panels. This light-installation was

exhibited at the STRP-festival 2011.

We have established a feeling of curiosity for interacting

with our installation through vague visuals reacting inexpli-

cably on the presence and movement of visitors. We gave

meaning to a static light medium by enhancing the feeling

of connectedness between the people and the medium in

their environment.

Conclusions

Conclusion and Recommendations | 119

The provided interactive installation was not a game, but it

was more a medium to let people freely explore interactiv-

ity. There was not more to that than exploring, so people did

not make up any games. They would only explain to others

their knowledge about the interaction. No one fully under-

stood the interaction, but this gave the whole installation a

magical feeling, and gave the visitors the curiosity to discov-

er. We were able to trigger others’ interest in our installation

by performing ourselves, because we noticed people would

copy the behavior and movements of others.

Against our expectations, this relatively simple interactive

installation was still interesting enough for the festival-visi-

tors. We did not foresee the big role of the spectator. Since

the spectators were more aware of the interactivity and the

change the performers made to the content, which gave

them an important role in initiating the explorative interac-

tions.

Despite the limitations of the medium, being blurry, 2-di-

mensional, hanging on the ceiling, we came to a suitable

interaction.

Future perspective

The impact of the performance depended on the number of

people within the playground. The more people, the more

dynamic the visuals would get. This would also influence the

clearness of who was responsible for the change. The num-

ber of people on the playground for the optimal effect was

rather low; between 3 and 7 persons. In future perspectives,

the system should be more flexible in relating the interactiv-

ity to the amount of people.

The playground was not clear enough, since it is difficult to

keep an overview of the situation on the floor and the ceil-

ing at the same time. The playground was situated exactly

in-between the panels, which resulted in no direct action

above you (since there was no panel directly above you).

People would try to influence the visuals on the panels by

standing directly underneath them, but there was no play-

ground directly underneath the panels. It was interesting to

notice the system directed the movement of people in such

a way. Although for creating awareness in the interaction

the mapping should match the direct location.

Recommendations on multiple clients

Since we were not involved in the foundation of this project,

it was not clear to us which arrangements were made and

more importantly if there was something lacking in the

agreement. During a first meeting with STRP we got their in-

formation on the project and during a meeting with Philips

we heard their part. But we never had a group meeting, with

all parties involved. This resulted for us in unclearness about

regulations in responsibilities and arrangements. For future

projects it would be preferred to have all regulations down

in black and white.

Our planning and internal task divisions were not communi-

cated to the different parties. Floris (PHILIPS) mentioned in

his interview that he would have liked to be more involved

in our process, since he was especially interested in our

ideas for the interaction with the panels. Now he had only

seen our end-result, and some test-results. We were not

aware of his expectations of the project. A next time we

would have to ask our clients to what extent they would like

to be involved in the process.

Also we communicated with the clients through one con-

tact person, Vivian for STRP and Tijmen for Philips. Especially

Floris from Philips mentioned in his last interview that this

gave him the impression of some people not being involved

as much in the process as others. He would have liked to see

more from everyone.

Recommendations

Conclusion and Recommendations | 121

Recommendations on working in a team which consist of students from all years

Working together in a group of students from all years was

very interesting, and very challenging at the same time.

The first years did not have a clue how to run a project and

which steps are required in a design process. Working in

a team with senior students worked out quite well. You

have to make sure though that the senior students do not

become the clients of the first years. Not all steps should be

executed as a group, since the first years have to find their

way in the education and acquire a proactive attitude. While

the senior students should be able to go through their own,

individual, process as well.

For the more experienced students working in a group is

especially helpful when in the need of motivation. It was

useful for them to validate the knowledge they gained over

the years. By explaining the process and argumentation for

different decisions the senior students realized how much

they actually learned over the years. In this educational

model it is difficult to point out what you learned since there

are no exams or books required.

For Tijmen, a B3.2 student, it was very useful to see what stu-

dents do in the masters. Because you get a better overview

of the education through working together with master

students than through going to information sessions. We

think it will attract more students to our masters if they all

get a glimpse like Tijmen of what is expected and done in

this final part of the study.

11

124 | Conclusions and recomandations Conclusions and recomandations | 125

124 125

124 | Co

nclu

sion

s and

recom

and

ation

s

125 | Co

nclu

sion

s and

recom

and

ation

s

We are grateful that this project ended in such a success, this would not have been possible without the help of all the people in-volved in this project. Therefore our word of thanks goes to the STRP festival, Frens Frijns, Dorien van den Hurk, Jan-Bart van der Tuuk and Marnix de Nijs, who made it possible for us to showcase our work at this fantastic festival; to the Philips Large Luminous Surfaces team who lend us their fantastic panels; to Floris Provoost for providing the panels and his help with technical implications; to Ben Schouten for making this project possible and the guidance he found time to give us; to Frans Parthesius who guided us through the process and the process of communication between all the different parties; and Bram van der Sanden, Wouter van der Heide, and Thomas Diewald for their help in programming. All these people were indispensible for this project to be a success!

“ “

Acknowledgements

12

References1. Playful Interactions. Technische Universiteit Eindhoven: Intranet. [Online] 2011. http://w3.id.tue.nl/fileadmin/id/Educa-

tion_Documentation/Project_descriptions/1112-S1-Playful_Interactions-Longs.pdf.

2. About STRP. About STPR, context, STRP festival. [Online] 2011. http://strp.nl/en/context/about-strp/ .

3. Bring spaces alive. Bring spaces alive. [Online] 2011. http://www.lighting.philips.com/pwc_li/main/application_areas/as-

sets/luminous-textile/product_leaflet.pdf.

4. Foundation ID. Technische Universiteit Eindhoven: Foundation. [Online] http://w3.id.tue.nl/nl/education/foundation/ .

5. Current Themes and their Spaces. Technische Universiteit Eindhoven: Current Themes & their Spaces. [Online] http://

w3.id.tue.nl/nl/education/foundation/themes/current_themes_their_spaces/.

6. Xbox 360 manuals and specifications. Xbox 360 manuals | Xbox 360 Spec. [Online] http://support.xbox.com/en-US/xbox-

360/manuals-specs/manual-specs.

7. Processing Library – Computer Vision – diewald_CV_kit. http://thomasdiewald.com/blog/?p=1107. [Online] September 6,

2011. http://thomasdiewald.com/blog/?p=1107.

8. UDP (User Datagram Protocol). What is UDP (User Datagram Protocol)? [Online] October 2000. http://searchsoa.techtar-

get.com/definition/UDP .

9. A implementation of the OSC protocol for processing. oscP5. [Online] December 19, 2011 . http://www.sojamo.de/librar-

ies/oscP5/index.html.

10. ArrayList. ArrayList. [Online] August 30, 2010. http://www.processing.org/reference/ArrayList.html.

11. What Is a Class? What Is a Class? [Online] http://docs.oracle.com/javase/tutorial/java/concepts/class.html .

12. Processing Layers. Nootropic design. [Online] April 22, 2011 . http://nootropicdesign.com/processing-layers/.

13. Saskia Bakker, Panos Markopoulos, Yvonne de Kort. OPOS: An Observation Scheme for Evaluating. NordiCHI 2008. Octo-

ber 20-22, 2008, p. 3.

14. Blob Detection . Blob Detection - The Lab Book. [Online] http://www.labbookpages.co.uk/software/imgProc/blobDetec-

tion.html.

129 | Referen

ces

13

http://dl.dropbox.com/u/4067465/DATA_

PERMANENT/Receving_data_visuals.zip

http://dl.dropbox.com/u/4067465/DATA_

PERMANENT/Uitgaven.project.xls

http://dl.dropbox.com/u/4067465/DATA_

PERMANENT/Sketchup.zip

http://dl.dropbox.com/u/4067465/DATA_

PERMANENT/Sending_data.zip

Receving DataProcessing

Spendings Project Sketchup

Sending DataProcessing

Scenario’s

http://dl.dropbox.com/u/4067465/DATA_

PERMANENT/scenario%27s.html

Final SLPLLP Movie

http://youtu.be/wRSV0vU4VQc