user innovation interface

Department of Computer ScienceAalborg University

INF 3

TITLE:User Interface Innovation- New Interaction Styles.

PROJECT PERIOD:INF32nd. september - 19th. december, 2008

PROJECT GROUP:i301a

GROUP MEMBERS:

Gunnar Konradsson

Jonas Urth Olsen

Michael Bønnerup

Ole Risgaard Hansen

Rahuvaran Pathmanathan

Søren Hugger Møller

Thulasika Rasenthiran

SUPERVISOR:Gitte Tjørnehøj

NUMBER OF COPIES: 10

REPORT - PAGES: 57

APPENDIX - PAGES: 54

TOTAL - PAGES: 111

Synopsis:

This project deals with idea generation meth-

ods for working alternative interactions forms.

The purpose of the project is to study which

options there are to get away from the ubiqui-

tous WIMP interaction form that is predom-

inant today, even though it is more than 20

years old.

Throughout the project, di�erent methods

have been used to study how it is possible to

use alternative interaction forms. Addition-

ally the project focuses on interaction in an

Immersive Virtual Environment, more specif-

ically the Cave Automatic Virtual Environ-

ment.

The result of these studies re�ects what we

have learned to be important considerations

in a radical innovative process.

Chapter 0 User interface innovation

Preface

This project was written during the autumn of 2008 by 5th semester informaticsgroup i301a at the Department of Computer Science at Aalborg University. Theproject period began the 2nd September and the project report was completedand handled in the 19th December 2008.

The semester subject was Innovation in system design with new interactionstyles.

On the basis of the subject, knowledge from the following courses was involved:

• The User Innovation Management (UIM).

• Design and Innovation Workshop (DIW).

We would like to thank Gitte Tjørnehøj for supervising us during the projectperiod. We would also like to thank Mikael B. Skov for introducing us to theDesign and Innovation Workshop.

The middle front page picture, is owned by VR Media Lab1.

Aalborg University, 19th December 2008

1www.vrmedialab.dk/pr/imgfaciliteter/cave_stor.jpg - The picture on the front page isedited by us, and therefore it di�ers from the link-picture.

2

Reading manual

The report is divided into several chapters, the following will describe each chap-ter.

Chapter 1 is the introducing chapter for the whole project, and starts withde�ning the main topic for this project. What is innovation and what process isneeded for an innovative result. Furthermore this chapter describes the relationbetween searching for ideas and creativity.

Chapter 2 describes what interaction is and furthermore we introduce theWIMP interacting form2 and alternative interaction forms.

Chapter 3 outlines the problem statement in the light of the opening chapters.

Chapter 4 describes the research method, how we involved theory and struc-tured our research.

Chapter 5 introduces the technology behind VR and CAVEs3 and argumentswhy these are used in the research project. The chapter also outlines the inno-vation process with focus on how to generate ideas in the speci�c context.

Chapter 6 describes the di�erent idea generating methods and assesses themfor further involvement in the research.

Chapters 7, 8, 9, and 10 describe the selected idea generating methodsfurther and how we evaluate them.

Chapter 11 is the �nal discussion on idea generation methods for alternativeuser interfaces in general.

Chapter 12 deals with the conclusion of our research and innovation process.

General information

The report contains both references and footnotes. References will in the textbe noted as [reference number ] and �gure references will be written as �gure�gure number.

The reference list of literature can be found at the end of the report underBibliography. The footnotes include a more descriptive text or informationabout the topic, and will be available on the bottom on the page. Footnotes willbe represented by numbers. Source references to appendix will be representedwith letters and numbers.

2Short for Window, Icon, Menu, Pointer. A paradigm for developing user interfaces.3Immersive virtual reality environment

Contents

Contents 4

1 Innovation 2

1.1 The approach to innovation . . . . . . . . . . . . . . . . . . . . . 2

1.2 Process for innovation . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3 Creativity - a part of innovation . . . . . . . . . . . . . . . . . . 5

1.4 The innovation pyramid . . . . . . . . . . . . . . . . . . . . . . . 6

1.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2 Alternative interaction 9

2.1 Interaction forms versus interfaces . . . . . . . . . . . . . . . . . 9

2.2 Alternative interaction forms . . . . . . . . . . . . . . . . . . . . 10

3 Problem statement 12

4 Research method 13

4.1 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4.2 Our research method . . . . . . . . . . . . . . . . . . . . . . . . . 14

4.3 Research procedure . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4.4 Data collection and processing . . . . . . . . . . . . . . . . . . . 15

4.5 Data analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

5 Immersive virtual reality 17

5.1 Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

5.2 The case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

6 Idea generation methods 22

6.1 Brainstorming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

6.2 Bodystorming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

6.3 User Innovation Management . . . . . . . . . . . . . . . . . . . . 23

6.4 Ethnographic studies . . . . . . . . . . . . . . . . . . . . . . . . . 24

6.5 Picture Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

6.6 Design and Innovation Workshop . . . . . . . . . . . . . . . . . . 25

6.7 The methods in an innovation process . . . . . . . . . . . . . . . 26

6.8 Searching for new ideas . . . . . . . . . . . . . . . . . . . . . . . 27

6.9 Structuring the process . . . . . . . . . . . . . . . . . . . . . . . 28

4

7 Brainstorm 297.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297.2 Focus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297.3 Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297.4 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297.5 Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307.6 Product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317.8 Lessons learned . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

8 Ethnographic Studies 328.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328.2 Focus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328.3 Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328.4 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328.5 Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368.6 Product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368.8 Lessons learned . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

9 Picture Scenarios 389.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389.2 Focus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389.3 Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389.4 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399.5 Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409.6 Product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409.8 Lessons learned . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

10 Design and Innovation Workshop 4210.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4210.2 Focus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4210.3 Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4210.4 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4210.5 Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4410.6 Product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4510.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4510.8 Lessons learned . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

11 Discussion 4711.1 Brainstorming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4711.2 Knowledge and idea generation . . . . . . . . . . . . . . . . . . . 4711.3 New technology - old ideas . . . . . . . . . . . . . . . . . . . . . . 4811.4 The boundaries of creativity . . . . . . . . . . . . . . . . . . . . . 4911.5 Modelling interaction problem . . . . . . . . . . . . . . . . . . . . 5111.6 The oddness factor . . . . . . . . . . . . . . . . . . . . . . . . . . 5111.7 Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

12 Conclusion 54

5

A Brainstorm 56A.1 Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56A.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

B Case proposal 58B.1 GPS game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58B.2 Head up display in a 3D interface . . . . . . . . . . . . . . . . . . 59B.3 Datamining in a CAVE . . . . . . . . . . . . . . . . . . . . . . . 59

C Ethnographic Study 61C.1 Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61C.2 Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62C.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

D Picture Scenarios 66D.1 Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66D.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67D.3 Storybords & picture scenarios . . . . . . . . . . . . . . . . . . . 68

E Design and Innovation workshop 76E.1 Learning workshop . . . . . . . . . . . . . . . . . . . . . . . . . . 76E.2 Pre-sketching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78E.3 Sketching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81E.4 Mock-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96E.5 Prototyping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

Bibliography 110

6

User interface innovation Chapter 0

1

Chapter 1

Innovation

Innovation is a popular topic in the world today. It is part of di�erent typesof products, processes and work methods etc., but how can we de�ne innova-tion, and why is it that important? The de�nition of innovation is "successfulexploitation of new ideas"[5]. This means that it is not just the invention ofnew ideas we have an interest in, but also how it is put into practice, used andbrought to the market. Furthermore, how new ideas are leading to new prod-ucts, processes, systems or any kind of services which improve something or addsome sort of value.

There is no doubt that innovation is an important part of developing and pro-cessing ideas in today's market. But what does the expression "innovation"cover?

In the following pages we will explore what innovation is and how the innovationprocess is structured. As such a process is rather comprehensive we will limitthe process to the activity of generating new ideas and not how to produce andbring them to market.

1.1 The approach to innovation

The extent of a speci�c innovation process may rely on how many resourcesand time have been reserved for completing it. Innovation processes with highnovelty tend to involve high risk as they contain several unknown factors, suchas the e�ectiveness and outcome of involved developing methods.

The amount of e�ort put in to the process is often re�ected in the importanceof the result, but this is not a one-to-one relation. Great achievements aresometime conceived with very small funds, and vice versa. Furthermore, it ispossible for small scale innovations to grow big over time[15].

Figure1.1 classi�es di�erent types of innovation in a framework used in productdevelopment.

As the �gure1.1 shows there are two approaches when talking about innovation:

2


Figure 1.1: Adopted from Vadim Kotelnikov composition of Radical versus In-cremental innovation

• Incremental innovation is when an existing product is modi�ed with knownmethods or a new product is developed with known methods.

• Radical innovation is when new ideas arise and is developed further withexperiential methods or when existing products are modi�ed with experi-ential methods.

As �gure 1.1 also illustrates, radical innovation involves a much higher risk thanincremental innovation, primarily because of the uncertainties that are involved.Therefore many of the innovation processes seen today are incremental.

1.2 Process for innovation

The innovation process is generally about getting from an idea to a successfulproduct. This idea can be either about creating something new or changingsomething existing. The process consists of several activities that are necessaryfor obtaining development and structure.

The understanding of the innovation process is constantly improved as a lotof research is being done in this scienti�c area. The description of the innova-tion process we will be exploring, is based on Frank Gertsen's de�nition of aninnovation process[16],[15].

3


1.2.1 Triggering factors

The necessity of an innovation process emerges when needs for changes in theenvironment arise or when assuming the need for changes. This need can typ-ically be categorised in some external factor. These factors serve for triggeringideas and will depend on the situation for the particular stakeholder. Figure 1.2shows some of these externally factors.

An example of an extreme externally factor could be, when the unthinkablehappened in 2004 and the tsunami destroyed thousands of homes in SoutheastAsia. This caused an instant need for help both in form of medical supplies andnew habitats. This led to the creating of a lot of both radical and incrementalideas on how to help the ones a�ected by this catastrophe in the best way.

Other factors could be when new markets and new technology arise, and therebycreate problems for the stakeholders. This will again lead to a lot of ideas onsolving these problems.

Figure 1.2: Adopted from Frank Gertsen's process for innovation

The innovation process is divided into three activities: Search, selection andimplementation.

1.2.2 Search

When a triggering factor occurs the a�ected stakeholder's motivation for comingup with new ideas or solutions for solving these newly discovered problems,increases. The �rst activity in the innovation process is all about searching and�nding these ideas.

A fact to remember here, is that the �rst idea which springs to mind, will notnecessarily be the best, and therefore this activity is meant to generate as manyideas as possible [16].

4


1.2.3 Selection

In the next activity the focus will be on organizing and selecting the best ideas.This is done by clarifying what a good solution is in the given context and thenelaborate on how to select the best ideas.

Piet Hein1 a Danish scientist, mathematician, inventor, author and poet, wrotea poem in his early inventor years, which is included here:

Kunsten er ikke at få en ide, enhver kan med lethed få to. Kunsten er den

mellem to eller �ere ganske almindelige hverdagsideer, at se hvilken der er go?

Translated into English, this means that it is not hard to get one idea andeveryone can even get two. The hardest thing is to see through the di�erentideas, and pick the good one.

Based on this poem by Piet Hein, Gertsen[16] lists three key questions to follow,for balancing the selection of ideas.

• Strategic analysis: What is realistic to create of the selected ideas?

• Strategic choice: What do we do, and do we use our resources or not?

• Strategic monitoring: Are the selected ideas, what we really want?

By using these questions to analyse the listed ideas, it is easier to decide whichideas to select for further development.

1.2.4 Implementation

After having selected the ideas or the idea, it is possible for the developers tobegin implementing the ideas or the idea. This can be done by di�erent typesof product development methods.

The last but most essential part of this activity, is to �nd out what e�ect thesolution will have on the previously discovered need for changes in the environ-ment. Here it is necessary for the solution to have a successful impact, for theinnovation process and solution to be characterised as innovative.

1.3 Creativity - a part of innovation

In any innovation process one of the primary stages is to search for one or moreideas. This phase of idea generation is, as described earlier, in�uenced by severalexternal triggering factors. But besides these factors, what is important whensearching for new ideas? One required property is creativity.

Innovation is strongly connected with creativity[1].If the solution for a desired product shall be de�ned as creative, it has to be newand thereby not seen before. Furthermore, it has to have a high usability andat last it must not be trivial. Thereby creativity can be de�ned as an ingredientof the innovative process.

1Piet Hein - Danish rational idealist 16.12.1905 - 17.04.1996

5


Figure 1.3: The three characteristic for creativity

Figure 1.3 shows that creativity depend on three human characteristics. For cre-ativity to exist all three characteristics have to be present. These characteristicscan be described as follows:

• Expertise which is achieved throughout knowledge and intellectuality.

• Creative thinking skills help to generate ideas through the process by a�exible, imaginative, and methodical approach.

• Intrinsic motivation is more e�ective than extrinsic because of the personalinterest and self-motivation.

If creativity is a quali�cation for being innovative, it is essential to understandthese characteristics and thereby give developers a better chance of generatingideas for the innovation process.

Developing a product can be an individual process, but it is also frequently seenbeing done in groups of people, who may take on di�erent aspects of the process.This way they are able to play their individual strengths, knowledge and rolesin an organization.

1.4 The innovation pyramid

The innovation pyramid is a tool for structuring and prioritising innovationprojects, thereby balancing the developer's investments and providing a betteroverview.

The innovation pyramid, shown �gure 1.4, works in three levels. The topmostlevel includes a few big bets for clearing the directions for the future. The secondlevel is a portfolio of promising mid-range ideas driven by designated teams. Atthe bottom of the pyramid are early-stage ideas or incremental innovations thatpermit continuous improvement [27].

6


Figure 1.4: Rosabeth M. Kanters innovation pyramid

Typically the in�uence �ows from top to bottom. The big bets encourage thesmaller ideas in the layers beneath and thereby focus the direction.

When looking a little closer on each level in the pyramid, �gure 1.5, it is pos-sible to see the di�erent goals of each layer, and furthermore the risks that areinvolved in an innovation process.

Figure 1.5: Levels of innovation

Small developing groups or companies that want to be innovative, tend to allo-cate more resources to projects with higher innovation novelty and risk. Thisis because they plan more new products and have fewer existing products thatcan be incrementally improved, than larger and more established companies orgroups.

Therefore, if a group reduces their resources allocate to innovation and therebythe risk of loss of these resources, they raise the risk of being surpassed by moreinnovative groups or companies.

A group should therefore allocate its resources to innovation levels in a ratio

7


that matches its corporate objectives, risk tolerance and management motiva-tion. Therefore it is very important before starting an innovation process, thateach developer related to the process discusses the ratio and the ambitiousnessof the project.

1.5 Summary

The approach for involving innovation in the development of a product can beradical or incremental depending on how novel the outcome should be. Fur-thermore, getting from emerging needs for changes in a speci�c environment toan idea to a successful product can be de�ned in an innovation process. Thisprocess consists of several activities where the searching activity depends onhow creative the involved stakeholders are.

Creativity is an attribute that is important to understand, as it involves severalcharacteristics which are all necessary for creativity itself to exist.

When working with innovation projects, the innovation pyramid can be a toolfor getting a better overview and understanding of how major objectives canconsist of several minor innovation processes, and how resources are split be-tween them.

Some of the challenges involved in being innovative, is how to get the rightidea. What can be done for generating as many ideas in a searching activityas possible? Surely creativity plays a key role. But is it possible to controlthe characteristics in creativity, so that the searching activity in an innovationprocess is enhanced?

8

Chapter 2

Alternative interaction

When objects are a�ecting each other, that event is referred to as interaction.Interaction can for instance occur between a human being and an IT system.Regardless of the type of system that is under development, it cannot be avoidedtaking interaction into account to some degree, although one may not be awareof that fact. For example when developing classic software applications for atypical graphical user interface, the design is oftentimes based on some alreadyde�ned guidelines, although perhaps unconsciously.

2.1 Interaction forms versus interfaces

original Macintosh iMac 20" comparison

date January 1984 November 2003 + 20 years

price $2500 $2200 x 0.9

CPU 68000 Motorola8 MHz0.7 MIPS

G51.26 GHz2250 MIPS

x 156x 3124

memory 128KB 256MB x 2000

storage 400KB floppy drive 80GB hard drive x 200000

monitor 9" black & white512 x 34268 dpi

20" color1680 x 1050100 dpi

x 2.2x 10x 1.5

devices mousekeyboard

mousekeyboard

samesame

GUI desktop WIMP desktop WIMP same

Figure 2.1: Comparing two personal computers 20 years apart[2]

9


2.2 Alternative interaction forms

Form can be de�ned in various ways. The word refers to something that hasa shape or a general structure. Since the early 1970's, there has been littleevolution in user interfaces for the personal computer, compared to the evolutionin other �elds, like the rate of which hardware has evolved, an example of thiscan be seen in �gure 2.1. Michel Beaudouin-Lafon[2] has pointed out, that themodern computer has been using the WIMP interaction form, along with amouse and a keyboard as input devices, since early in 1984, when the originalApple Macintosh was released. Figure 2.2 shows an example of interfaces usingWIMP.

Figure 2.2: The WIMP interaction form in use in Mac OS (1984) (top) and MacOS X (2008) (bottom)

2.2.1 Reality-Based Interaction

A di�erent approach to interaction is Reality-Based Interaction (RBI), which isde�ned by Robert J. K. Jacob[13] and focuses on interaction in virtual environ-ments. The idea behind RBI is to include some elements from the real worldin the way people interact with interfaces. The way this is accomplished, is byidentifying criteria for how people interact with their real world environmentand each other. Jacob et al. set up four of these criteria, which are as follows:

• Naïve Physics: People have common sense about the physical world

10


• Body Awareness and Skills: People have an awareness of their own physicalbody and can control and coordinate it.

• Environment Awareness and Skills: People have a sense of their surround-ings and can function within their environment.

• Social Awareness and Skills: People are generally aware of other people intheir environment and can interact with them. This for instance includesthe ability to collaborate on a task and the ability to communicate.

So RBI is about making the interaction in Virtual Worlds well-known forthe user by making it close to how the real world works. But Jacob et al. alsopoints out that it is important not to focus too much on mimicking the realworld. Sometimes it is necessary to trade some of the RBI principles againstother considerations. Among these tradeo�s that need consideration, is Practi-cality, because it is important, that the system is practical to develop. This e.g.involves technical limitations. Another tradeo� is Versatility, which refer to thedegree to which a single GUI can be used to perform multiple tasks.

So the major point of RBI is to create a �tting mix between making the systemfunction in a way close to the real world and using some computer speci�c meansto enhance the experience. Jacobs et al. makes an analogy to Superman, he isa human-being and as such function as everyone else in the real world, but healso has some "unreal" features, such as the ability to �y, which enhances theway he function.

11

Chapter 3

Problem statement

This chapter, describes our problem statement after we had an understandingof innovation and interaction.

When talking about human computer interacting (HCI), the WIMP paradigmis hard to avoid. Today most user interfaces are based on this paradigm whichcan be dated to the early 1980's although the computer industry has been sub-jected to extensive development. Thereby the WIMP paradigm has become ade facto standard on how to develop a user interface for IT-systems.

In the light of this, most research today is done on this basis, consequently thisonly provides minor incremental innovation in this speci�c area of knowledge.

For getting past the boundaries of the WIMP paradigm, we believe a moreradical innovative approach is needed. However as the research in this areais minimal, it is essential throughout experimental research to involve existingtheory about acquiring knowledge on interaction forms and evaluate them forour purpose. This evaluation is possible through comparison or by performingexperiments.

Furthermore, our case for studying alternative interacting forms will be an im-mersive virtual reality environment which is a relative new interaction and dis-play system.

Based on these considerations we want to answer the following problem state-ment:

Is it possible to use methods, which are primarily used for tra-

ditional innovative development, for developing a more radical

innovative and alternative interaction form?

12

Chapter 4

Research method

In this chapter we will describe our approach to develop and structure ourresearch, inspired from information systems research theory. We present thecriteria, which form the basis for the decisions for selecting idea generatingmethods and dictate how to evaluate these. Furthermore we describe how wehandle the collecting and processing of data from experiments.

4.1 Theory

Our approach to develop and structure our research method is inspired by Infor-mation Systems Research,[17] where the goal is to produce and develop knowl-edge that enables development of IT. There are two approaches to acquiringsuch knowledge, behavioral science and design science. As it is design sciencethat we �nd useful, we will not look further into behavioral science.

The design science research seeks to create innovative artifacts, that solve oneor more problems/challenges for which existing design knowledge is inadequate.

Inspired by the framework of information systems research[17], we created ourown research method. The design science research enables the creation of aninnovative, purposeful artifact. Since we want to acquire knowledge of ideagenerating methods in the context of alternative user interfaces, these methodswill be our artifacts. The wanted knowledge for the speci�ed problem domain,which was described in chapter 3, and will be acquired by doing innovative ex-periments with exiting idea generation methods.

The way we have structured our research method is illustrated in �gure 4.1. The�gure indicates the problem area contains the WIMP problematic, which arede�ned in chapter 2. The way we address the problem is by utilizing existingtheory about idea generating methods. This is done to evaluate their potentialfor solving such problem.

13


Figure 4.1: Adopted from Information systems research

4.2 Our research method

Our criteria for selecting the possible methods to involve in our research werethat the methods' main purposes are to generate ideas. Our knowledge ofavailable methods consisted of accumulated knowledge from courses in whichwe participated 1. Furthermore we supplement this knowledge by explorationof other possible methods from scienti�c articles.

A list of all, to us, known possible idea generation methods is listed in chapter6, where we also argue for why we did and did not choose the speci�c availablemethods. Besides from the brainstorm method all our selected methods werechosen because they met the following criteria:

• Purpose of generating ideas on a speci�c context.

• Flexible for adjustment.

• Possible to acquire knowledge on interaction forms.

4.3 Research procedure

The chosen methods are as follows:

• Brainstorming

• Ethnographic Study

1Software Innovation by Jeremy Rose and Ivan Aaen, VIT by Janne Jul Jensen, UIM byAnne Marie Kanstrup and Janne Jul Jensen, Design and Innovation Workshop by Mikael B.Skov.

14


• Picture Scenarios

• Design and Innovation Workshop (Design and Innovation Workshop)

Since three of the methods were new to us, our approach to conduct and evalu-ate them was emergent. Furthermore, it is relevant to notice that the involvedexperiments were carried out independent of each other, though they might bebased some similar subjects.

Aside from Brainstorming, the experiments were not done in a particular order.In fact the Design and Innovation Workshop experiment was broken into severalparts, and some of the other experiments were conducted between these parts.

The Brainstorming method was utilized to developing potential cases for theinnovation process. This process is described in appendix, A.

The Design and Innovation Workshop were split in several phases namely pre-sketching, sketching, mock-up, and prototyping. These phases were not con-ducted continuously, as we did not want to rush the process. Furthermore, thisdid also help the processing of obtained data. A full description of the Designand Innovation Workshop, and results are available in chapter 10.

Ethnographic studies and Picture Scenarios were conducted between the someof the Design and Innovation Workshop phases. The Ethnographic Study wasdone in a CAVE with some users. The purpose with this method was to ob-serve how the user interacts in the �eld, and from, this deduct what is neededto innovatively solve problem that occurred. Chapters 8 and 9 further describehow we used this method.

For Picture Scenarios the pictures shot in the Ethnographic Study experiementwere used to construct scenarios. These scenarios were the pivot of the discus-sion and helped us to get a better understanding of the context during ideageneration activities.

4.4 Data collection and processing

Our way of collecting data was the same for all methods except Brainstorming.In Brainstorming data was collected by using only a blackboard and a computer.A camera was used to supplement documentation of the remaining experiments.It was used to document how the process went, and as output for the individualexperiments. These pictures were used in the following discussion in the group,where the achieved results were listed for further processing of the data.

Following, one or two persons with hands-on experience from the experimentgot the responsibility to begin writing the collected data in a form where theresults and process were separated and described. When done, other groupmembers went over the data and added they own experiences, to ensure a broadperspective of the experience. An instance were after a Design and InnovationWorkshop experiment, since the group were divided into two subgroups withtheir own area within the case, each subgroup had the responsibility to write

15


out the results from their area but the whole group was involved in the discussionof the output of the experiment.

4.5 Data analysis

Our analyses were done in our group room with the participation of all groupmembers. The procedure for the analysed data after each experiment, was byre�ecting upon it. This re�ection where among other things done by comparingthe achieved results with the prepared assumptions on the result, which wasbased on theory, thereby vindicating the outcome of the experiment.

The conclusions are justi�ed by reference to our research, the acquired data,and knowledge from the experiments. We also draw conclusion in proportion towhether the data from each conducted experiment contribute with new knowl-edge to general development of new interaction forms.

From the conclusion and the observations from the conducted experiments, fac-tors to be aware of when doing any kind of �eld work in the future, are linedup in a lesson learned list for each method.

16

Chapter 5

Immersive virtual reality

In this chapter we will describe the technology we will use as the case in evalu-ating the idea generation methods. We will describe the technology and arguewhy it is suitable for our project.The technology is immersive virtual reality[33]. This type of technology is avail-able to us through VR Media Lab (VMRL) at Aalborg University whom has aCAVE[31].

5.1 Technology

We will introduce, what it means when technology actually displays the thirddimension. The way the brain perceives three dimensions has a lot of in�uenceon how it is usually simulated. The brain does this by comparing the twodi�erent views from each eye. The way it often is simulated is by forcing theeyes to see a pair of di�erent images. This has been done for a fairly long time.

Figure 5.1: An early edition of a stereoskope

17


In the past it has been done with the stereoscope as seen in �gure 5.1. Todayit is done in a lot of di�erent ways, to name a few:

1. CAVE (Cave Automatic Virtual Environment) - The VR environment aperson actual steps into. This is the VR environment used in this researchproject and will be described in further detail later

2. HMD (Head Mounted Display) - These are goggles with two miniaturedisplays, one for each eye. There is in addition to the goggles often somekind of tracking device to track the movement of the head.

3. 3D Cinema - a cinema with a projector that displays two images onto thesame screen, whilst the audience has been given glasses that will separatethe images. So each eye only sees one.

4. BOOM - much like the HMD only this one is mounted on a special armthat tracks the position of the goggles.

CAVE (Cave Automatic Virtual Environment)

The CAVE at VRML, is a cube consisting of four walls a �oor and a top allmade of canvas functioning as screens. All these screens have a projector aimedat them from behind, this rear projection makes it possible to give a personstanding in the cube a view from all angles. Each projector projects two dif-ferent images, the user wears shutter glasses which separate these images andthereby fool the brain into thinking the person is looking at a three dimensionalobject. The CAVE is equipped with a position tracker that allows tracking

Figure 5.2: Two users interacting with a virtual object in a CAVE

18


of, for instance the shutter glasses. Doing so make it possible to track wherethe user is moving to and adjust what she sees accordingly. A CAVE moreoften than not is also equipped with an 8-channel sound system, which help toenhance the experience. Available interaction equipment in a CAVE:

1. Motion tracking can be used to give the coordinates of a tracked-object,one could for instance track a whole arm by placing such trackers on threekey location on a suit (shoulder, elbow and wrist). Such motion trackingcan be done with magnetism which is why most CAVEs are build on askeleton of non magnetic steel or wood. Alternatively the tracking canbe done by infrared cameras but this introduces the risk of user or otherequipment to block the visibility the tracked objects.

2. A second tool often used and associated with a CAVE is the wand this isa device is basically a tracked remote control. A user could for instancesimply click a button on a wand whilst pointing in a direction to move inthat direction. The wand has a way of communicating the position anddirection to the program in use.

3. The Pinch glove is another piece of hardware that has some advantageswhen working in a CAVE. these are gloves that has a number of combina-tions of "pinches" which is a gesture done by putting the tip of the thumbto the tip of another �nger. While wearing the gloves, the user still hashis hands free and therefore still maintains the freedom to interact withother objects.

4. Beside these two special interaction tools almost anything can be usedin a connection with a kind of motion tracking. For instance a pencil, apool cue, or a bow. All these things from the real world might help theuser to get a more intuitive interaction with an immersive virtual realityenvironment.

5.2 The case

5.2.1 Why use VR?

VR environment is a good place for working with alternative interfaces, as itis much di�erent from a traditional computer interface used on a PC or mobilephone. There is nothing preventing developers from just creating a WIMPapplication for a CAVE, but is of course something that needs to be preventedwhen you want to break free from exactly those traditions, which is the purposeof our research project.

5.2.2 VR versus traditional interfaces

A VR environment di�ers in many ways from a traditional computer interfaces,these di�erences will be described here.

The extra dimension

A VR environment adds an extra visual dimensions, that is not uncommon forcomputer applications, for instance in applications like games and 3D modeling

19


tools. What makes VR di�erent from those is the added e�ect of stereo sight, afeeling of depth, which cannot be experienced on a normal screen. In traditionalcomputer applications the extra depth is mostly used for objects with a simu-lated physically representation, like models in productivity apps and monstersand weapons in games, not so much for things like menus and windows.

Interface details

Most common interfaces are �xed to a position, like the keys on your phone, orthe image on your computer screen. That is not the case in a VR environment,and especially in a CAVE. You have images all around you. That also makes itpossible to have objects which appear a lot bigger than object in a traditionalinterface. Objects in the CAVE at VRML, are normally larger, because of thelow resolution of the display system, each screens resolution is 1024x1024 pixels,and this makes the pixels rather large because of the physical size of the screenswhich is 2.5x2.5 meters.

Input devices

Input devices for VR do not have to be di�erent, but they normally are. In aCAVE it is unpractical to use the traditional keyboard and mouse setup, becauseyou do not sit at a table and you are in a three dimensional space. So an inputdevice for 3 dimensions is more suitable.A lot of di�erent devices are available. However, studies show these devices arenot always at �rst try used to their full potential, seeing as though a normalcomputer user is a lot more experienced with keyboard and mouse. [10].

Working in a CAVE

Working in a CAVE can be a bit limiting. The administrators of the CAVEat VR Media Lab have examples of people getting physically ill. It is mostlywhen there is a lot of movement in the scene, and the frame rate in the sceneis too low, also called lagging. The discomfort disappears as soon as the persongets out of the CAVE. Other problems noted at VRML, are people getting soa�ected by the simulation that they lose the sense of orientation. Examples ofpeople falling down have been seen. Even when you are not subject to theseproblems, working long time in a CAVE can be tiring. All these factors are newto interface designers.

Home user

A full CAVE installation a multi-million DKK project, so this is it not somethingwe will see in private homes anytime soon. However it is possible to buy cheapVR equipment for your home PC 1. These factors make this technology anunknown �eld for most interface designers. This is one of the reasons whythere is not a common paradigm for VR applications, like the WIMP for PCapplications. Of course there is a lot of interaction methods, but they are notwell organized nor well distributed.

13D Stereo makes shutter glasses for 110$

20


5.2.3 Summary

VR is a technology which posses a lot of opportunities and limitations for in-terface designers. It is a niche �eld with the potential for doing research ininnovative interface development.

21

Chapter 6

Idea generation methods

In this chapter the methods that will be studied and evaluated are described.Selection of the methods is done by studying the literature in the HCI andinnovation �elds for methods that �ts to create ideas for interaction forms.Some of these methods do not exactly �t this description but can be adjustedfor it. Other methods do not at all, they will inspite of this be discussed, andwe will argue for not looking further into them.The arguments for choosing the methods will be presented in further detail inthe following.

6.1 Brainstorming

6.1.1 Description

Brainstorming[23] is perhaps the most well known method for idea generation.It is used in all kinds of areas. Brainstorming is a method often used in thestart of a project, since it helps to free the participants from �xed ideas and tothink in broader perspectives. The setup for a Brainstorming is simple, almostno materials are used. Brainstorming can therefore be utilized without concernfor needs of resources.Brainstorming is simple and easy to understand, as well as it is a cheap way ofgenerating ideas. A worth to notice characteristic for Brainstorming is that itsprocess ends before any assessments of the ideas' values take place.

6.1.2 Why evaluate

We have a lot of experience with Brainstorming, and consider it a sort of basisfor idea generation methods. So it is a method which cannot be overlookedwhen you want to evaluate idea generation methods.

Chapter 7 describes the theory behind Brainstorming, the evaluation processand conclusion.

22


6.2 Bodystorming

6.2.1 Description

Bodystorming[8] is a method extending Brainstorming, it is presented by AnttiOulasvirta, Esko Kurvinen and Tomi Kankainen, a research group from HelsinkiInstitute for Information Technology. It moves the Brainstorming closer tothe location where the product is eventually to be developed. If a softwaredeveloping team is to develop an IT system for shopping in a mall, they are togo to a mall to form their ideas.Another aspect of bodystorming is the concept of getting the idea tested earlyin the software development process. The Helsinki research group did that bycreating a mock-up of the idea right away when the bodystorming participantsgot the idea. The mock-up was used in a usability-like test, instantly aftercreating it.The last apect of bodystorming is getting the participants closer to the mindsetof the user. This is done by letting the participants act as the user during thebodystorming session.The Helsinki research group conducted four experiments testing bodystorming.Each of the experiments combined the three aspects in di�erent ways.

6.2.2 Why not evaluate

The reason we did not choose to test and evaluate bodystorming was the Helsinkiresearch group's own conclusion on their research. Bodystorming did not gener-ate more ideas than a normal Brainstorming session. What the Helsinki researchgroup concluded was that bodystorming gave the participating developers moreownership over their ideas, and likewise a better feel of the area in which theydeveloped the product. These feelings helps further development of the product.But in our research project the focus is on the idea generation process, so thatadvantage will not be noticed.

6.3 User Innovation Management

6.3.1 Description

User Innovation Management (UIM) is a method for IT-innovation developedby Anne Marie Kanstrup[18]. UIM is, as implied by the name, a method con-cerning the involvement of users in an innovative design process.More speci�cally the purpose is to create a theoretical framework for user drivenIT-innovation, and to provide a methodical process for the central themes andtechniques with regards to creating cooperation, contextual knowledge, and con-ceptualization of user driven innovation[18].

The motivation for UIM is the importance of user innovation according toKanstrup[20]. We have to move from just knowing about users' needs to know-ing about future practices. Besides knowing the users' needs, it is importantto get innovation by the users to ensure that we can create products that willful�ll the users' needs. This is due to globalisation and new economy.

23


Kanstrup de�nes management in the context of UIM as the acknowledgementof user innovation as a discipline rather than as a craft.

UIM was presented to us in a course.

6.3.2 Why not evaluate

We chose not to use evaluate UIM. This was because we felt that several of thetechniques involved in UIM, overlapped with techniques used in the Design andInnovation Workshop (see chapter 10). An example of this overlapping is thetune-in part of UIM and the Pre-sketching phase of the workshop. These twoboth covered the process of getting the developers minds focused on the tasksahead. Because of this we decided that UIM would not add enough value to theproject compared to the time it would consume, if we chose to use it.

6.4 Ethnographic studies

6.4.1 Description

Ethnography is known as a general research method. This research method isoriginally adapted from sociology and anthropology, where it is a method ofobserving human interactions in social settings and activities. It can also bedescribed as the observation of people in their cultural context. Massey[21]de�nes culture as being "made up certain values, practices, relationships andidenti�cations". People also describe a workplace as a culture, �lled with workstandards, business practices, and relationships between co-workers and betweenworkers and managers.Ethnographic studies is an important tool for interface-designers, because itdeals with the users culture, and it is a powerful way to assess users' needs.

6.4.2 Why evaluate

We chose ethnographic studies because we found it interesting to observe theusers in the �eld, where our product in the end would end up. We already haveexperience with di�erent type of observations (usability tests) and interviewsfrom other projects, so we found it easy to acquaint ourselves with ethnographicstudies.

Another purpose with the ethnographic studies is also to �nd out if this methodis useable in making innovation to new interaction styles.

Chapter 8 describes the theory behind ethnographic studies, the evaluationprocess and conclusion.

24


6.5 Picture Scenarios

6.5.1 Description

Picture Scenarios[24] is a method to capture some situations in which human-computer interaction is present. The materials used in Picture Scenarios are aseries of comic strip like photos showing the users in use case situations. Theseare used as an inspiration for the participants in the Picture Scenarios session.Picture Scenarios are useful in order help get a better understanding and tonegotiate the use of context during design activities.

6.5.2 Why evaluate

We had an assumption, that Picture Scenarios could be useful to our projectas an idea generating method. We had no experience with the method, judgedby its purpose we found the guidelines and description of the method werean interesting way to reach a solution. In our case, it was useful to take theobservation results from the ethnographic studies and use them in making thematerials in our Picture Scenarios experiment.

Chapter 9 describes the theory behind Picture Scenarios, the evaluation process,and conclusion.

6.6 Design and Innovation Workshop

6.6.1 Description

The Design and InnovationWorkshop 1 is a method developed by Jesper Kjeldskov[28].It is a method for generating ideas and materialises them with mock-ups andprototypes and �nally testing them in a usability-like test.In DIW the participants work with sketches, mock-ups, and prototypes madefrom cardboard, foam, paper, etc.DIW is a sequential process where ideas evolve from participants' notions toone or two physical prototypes which are build and tested by a group.

DIW will give the participants a closer insight into how their ideas would lookand function as opposed to just re�ecting over these. This is useful as the ideasare competently evaluated shortly after the ideas arise, we say competently dueto the building and testing.

6.6.2 Why evaluate

We chose the Design and Innovation Workshop because we felt it could help usin evolving our ideas into something useful, for one thing because the purposeof the workshop is to go pretty far with the product produced, i.e. produce apaper prototype [28].

Chapter 10 describes the theory behind Design and Innovation Workshop, theevaluation process and conclusion.

1DIW

25


6.6.3 Overview over methods

To better get an overview over the methods we chose to evaluate, we have listedthem all in table 6.1.

Table 6.1: Overview over methods

Brainstorming Ethnography Picture Scenarios DIW

Purpose Creation of newideas.

Understanding theusers needs.

Visualizing user con-text

Creation of new inter-action forms

Focus Higher level ofcreativity in team-work.

Interaction of the usersin the �eld

Understanding thecontext.

Creation and testing ofnew ideas

Participants Developers. Two ext. users andthree developers.

Developers Developers.

Process Random listing ofideas, which after-wards were ranked.

Field study with theusers in 3 di�erenttasks.

Constructing scenariosand shooting pictures.

Sketching, mockup,prototype, testing.

Product Three di�erent cre-ative ideas.

Pictures and documen-tation of the process.

Documentation of theprocess.

New interaction forms.

6.7 The methods in an innovation process

This chapter will describe how the idea generating methods are used within theinnovation process.

6.7.1 Narrowing down the triggering factor

The �rst major step was to identify the main case, thereby narrowing down thetriggering factors for a later innovation process to a more concrete problem. As�gure 6.1 shows, this was done by using the Brainstorm method, which wasused to generate a lot of ideas in this speci�c topic2. Some of these ideas wereselected and developed further by using the tool W5H23. The result can be seenin the three case proposals in appendix B.

After proceeding with a speci�c case, it became a necessity to circumscribe ourfocus a bit more, as we were forced to take time and resources into consideration.Therefore, the main problem in this case was rewritten to our current problemstatement, which allows us to concentrate on more speci�c parts of the case.

Furthermore, as we wanted to study how a human and a system can interactwithin the CAVE we needed a tangible object to focus on. We decided to involvesomething simple, the Lego concept. One of the reasons for this is the fact thatit is a well known product and it is easy to handle.

2Our subject was de�ned by the study regulation which speci�es the requirements forgained experience within a speci�c education.

3W5H2 is a method designed to help identifying the structure of a problem based on severalpoints of view.

26


Figure 6.1: The process for developing case proposals

6.8 Searching for new ideas

The approach to involve di�erent idea generating methods in the innovationprocess has, as mentioned in chapter 4, been on an experimental basis. Thisapproach allows us to experiment with the di�erent methods and collect results.Furthermore, it allows us to compare the potential of the di�erent methods inthis speci�c context.

Figure 6.2: Focus on the idea developing process

As �gure 6.2 illustrates, the triggering factors for generating ideas, are nownarrowed down to interaction, CAVE technology, and the Lego concept.

When looking closer at the innovation process the next step, after de�ning thesetriggering factors, is searching for ideas, see chapter 1.2.2. This process is about�nding new ideas for later selection and implementation.

As �gure 6.2 also shows, the searching activity entails studying and involving

27


di�erent methods for generating ideas in this innovation process.

6.9 Structuring the process

The innovation pyramid is a tool for getting a better overview of the innovationprocess. Figure 6.3 illustrates, how the innovation process is structured in thecurrent case.

The levels in our innovation pyramid can be de�ned as follows:

• The big bet for our innovation process, is the development of a new suc-cessful interaction form for IT systems.

• The mid-range idea consists of the development of a system for data miningin a CAVE.

• The bottom consists of several minor ideas, such as a 3D menu systemand object manipulation.

Figure 6.3: The innovation pyramid

28

Chapter 7

Brainstorm

This chapter describes our use of Brainstorming, along with general Brainstorm-ing theory.

7.1 Purpose

In our case, the purpose of this method is to develop new creative ideas forinnovation or improvement. Brainstorming can be an e�cient way to generatea number of ideas on a speci�c issue, and then to determine which idea is thebest solution. Brainstorming is motivating as well, since it involves all theparticipants, and encourages teamwork.

7.2 Focus

The focus is on the generation of creative ideas, and the propagation of thosebetween the participants.

7.3 Participants

The participants were seven group members. One of them was playing the roleof facilitator.

7.4 Theory

There are few basic rules to follow in Brainstorming[12]:

• Focus on quantity.

• Unusual ideas are welcome.

• No criticism.

• Combine and improve ideas.

29


Brainstorming prioritizes quantity over quality, a greater variety of ideas is moreimportant than having few ideas with a greater depth. The participants haveto generate a large number of ideas. To achieve this, even unusual ideas, thatare not always usable alternatives, are welcome because this may encourage adi�erent way of thinking. It can also lead to new and innovative ideas ratherthan ordinary ideas[12].

No matter how impossible or how silly an idea seems to be, it must not becriticized because it hinders free thinking. Instead the participant should be en-couraged to think broad and come up with as many creative ideas as possible,and the group must list all ideas without applying any restrictions and withoutworrying about the implementation.

After the idea generating process, group members attempt to combine and mod-ify the generated ideas, the merged ideas might lead to better and more innova-tive ideas. Once the ideas are captured the group can begin to assess, evaluateand analyze the e�ects and validity of the ideas. The ideas can be developedand prioritized into a more complete list of options[11].

In Brainstorming, a facilitator is optional. The role of a facilitator is to guidethe Brainstorming process and encourage participation while writing down theideas as the participants shout them out.

7.5 Process

At �rst we spent some time thinking of ideas, mainly to get in the creative moodfor the Brainstorming process. Additionally, we decided to choose a facilitator.During the process he had the responsibility to document all the ideas on theblackboard and guide the process.We had a open Brainstorming session where all the participants were allowedto throw their ideas at will.

In order to evaluate the potential in each idea, every idea was given a rank, basedon general acceptance. Because of some disagreement we created a documentcontaining all the ideas and a short explanation of the core intent for each idea.This document can be seen in appendix B. The purpose of this document wasto let every participant to select three ideas which they preferred more thanother. Then give the three selected ideas a score of one, two, or three points.This procedure allowed us to see which ideas were of particular interest to theparticipants.

7.6 Product

The outcome of the Brainstorming was three proposals, mainly based on theselected ideas.

The proposals are as follows, and in-depth explanations are given in appendixB:

30


1. GPS game.

2. Head-up display in a 3D interface.

3. Datamining in a CAVE.

After a discussion, Datamining in a CAVE was chosen.

7.7 Conclusion

The evaluation of Brainstorming was located a bit earlier in the idea processcompared to the other idea generation methods. We used Brainstorming forcoming up with projects proposals for the case we would use to study alter-native user interfaces. So the Brainstorming was not limited to interfaces in aCAVE, but alternative interfaces in general. The process and the results aredescribed in appendix A.

Brainstorming is well known to us from previous projects. The Brainstormingsession went as planned, and we did not notice a lot di�erences between workingwith UI innovation and our past experiences with Brainstorming.The �eld of knowledge was where a di�erence was mostly noticed. The outputwas in�uenced by both knowledge about technology and currents trends in soft-ware.

One case was based on the idea of location based computing (GPS game). Lo-cation based computing is a hot topic right now, as more and more devices movecloser towards a general computing platform, and are equipped with positioningtechnology like GPS and cellular triangulation. Examples of such products arethe Apple iPhone and media players with built-in GPS. We, as a pretty tech-nology savvy group, were in�uenced by that.The two other cases were based on a technology, we had available; the CAVE im-mersive 3D environment. The cases were 'Datamining in a CAVE' and 'Head-updisplay in a 3D interface'.

7.8 Lessons learned

Based on the experience gained through Brainstorming, some signi�cant obser-vations are listed:

• Before starting a brainstorm process, it is bene�cial for the participantsto have time to generate ideas and to become tuned into innovative mood.

• Brainstorm is very useful for generating lots of ideas, but it is optimallimit the process time and avoid extending it.

• Knowledge is an important factor in Brainstorming because it may havein�uence on the output.

• Having a facilitator during the process is an advantage and makes theprocess �ow.

31

Chapter 8

Ethnographic Studies

This chapter describes our use of ethnographic studies, and the general theorythere behind.

8.1 Purpose

The purpose of the Ethnographic Study, was to get an insight into the users'interaction in the �eld and an understanding of the subject. Furthermore, wewanted to get the users' needs, which we could use for further development inour project.

8.2 Focus

The focus in this method was on the users; how they interacted in the �eld, andhow they solved the given tasks in the observation-session.

8.3 Participants

In our �eld study, we used two users; a girl and a boy. The observation of thetwo users was done by three ethnographers. One main ethnographer which con-trolled the session, from helping the users with the tasks and navigate anotherethnographer who was taking pictures. A third ethnographer wrote down notesof the users' actions and comments throughout the session.

8.4 Theory

Ethnography is certainly important for interface designers which we are, be-cause we are concerned of the users' culture. There are several reasons whyEthnographic studies are of vital importance to interface design.

An Ethnographic Study is a powerful assessment of users' needs:

32


• A crucial goal of ethnographic studies is to gain capability to view a systemthrough the eyes of the user. This is especially useful in creating userinterfaces and menus to �t the needs of the end-user.

It uncovers the true nature of the system user's job:

• A goal of an Ethnographic Study is to uncover all tasks and relationshipsthat combines to form a user's job. Often a user performs a task andcommunicates in ways that are outside of their o�cial job description.

We as ethnographers can play the role of the end-user:

• The high level of user understanding that we as ethnographers can gainthrough �eld work can be a useful bonus. For instance, when the ethnog-raphers act as end-users in participatory design.

8.4.1 Methods

There are various methods by which ethnography can be incorporated into userinterface design. The main approaches in use today are listed below[14].

• Concurrent ethnographiy

• Evaluative ethnography

• Quick and dirty ethnography

• Rapid ethnography

Concurrent ethnography

Concurrent ethnography is the most famous method of ethnography, and is usedespecially in HCI research. It is also known as ethnomethodology. The methodimplies that it is being utilised in support of, and at the same time, as the designof a new system. Usually a system prototype is developed and re�ned, basedupon the result of the study.

The advantage of this method is that it ensures focus upon the user through allstages of the development of a new system. The disadvantage is, if we compare itto the other ethnographic method, that this method requires the largest amountof cooperation, coordination, and time.

Evaluative ethnography

The goal of this method is to evaluate a new design model. Ethnographicresearch is performed within a narrow context, which is aimed to focus on theaspects of work, which would be a�ected by the new design model.

The advantage is that it is useful in helping to prove or disprove a new designmodel or theory. The restricted domain allows an evaluative EthnographicStudy to be performed within a relatively short time frame. The disadvantageis that there is a tight focus which can blind the ethnographer from noticing theimportant information that is outside the domain of the study.

33


Quick and dirty ethnography

The quick and dirty method is often used as a predecessor to other ethnographicresearch methods. It can be useful in increasing awareness of large-scale usabil-ity and acceptability issues, which are important in the design of a new system.However, it is frequently the only form of ethnography practiced due to impos-ing time and/or budget constraints[14].

The advantage is that it can yield valuable knowledge of the social organiza-tion within a work setting in a short amount of time, relative to the size of theproject[14]. The disadvantage is that the results are limited to a general under-standing of a work culture. It is possible that a quick and dirty EthnographicStudy can lead to a false sense of understanding for a working culture.

Rapid ethnography

This is the most recent one of the ethnographic methods in practice. It uses "acollection of �eld methods intended to provide a reasonable understanding ofusers and their activities given signi�cant time pressures and limited time in the�eld"[22]. Some of the key principles of this practice are to use a constrainedfocus, key informants, and multiple ethnographic observers.

The advantage is that this method uses a short time frame on par with the quickand dirty method. However, this method is more rigorous and formalized thanthe quick and dirty method. The disadvantage of the method is the process isstill undergoing re�nement.

8.4.2 General process

The process of the ethnographic methods can di�er from each speci�c �eld study,but the most general de�nition of the process, is based on four ethnographicprinciples [19]:

• Planning - The purpose of the �eld study is found. A strategy is cre-ated and selections of the speci�c ethnographic methods or techniques arechosen.

• Selection of users - Which type of and how many users are selected.

• Field approach - Permission to the �eld and dealing with the owners.

• Methods and techniques - Observations, interviews, self-reporting, andother observations.

8.4.3 Guidelines for ethnographic studies

In order to facilitate the generation of useful data, the ethnographer should un-derstand the basic steps that are used to conduct an Ethnographic Study. Thefollowing guidelines are recommended as a general framework for an Ethno-graphic Study[7].

34


Preparation

• Understand organization policies and work culture.

• Familiarize yourself with the system and its history.

• Set initial goals and prepare questions.

• Gain access and permission to observe/interview.

Field Study

• Establish rapport with managers and users.

• Observe/interview users in their workplace and collect subjective/objectivequantitative/qualitative data.

• Follow any leads that emerge from the visits.

• Record your visits.

Analysis

• Compile the collected data in numerical, textual, and multimedia databases.

• Quantify data and compile statistics.

• Reduce and interpret the data.

• Re�ne the goals and the process used.

Reporting

• Consider multiple audiences and goals.

• Prepare a report and present the �ndings.

Ethnographic Study guidelines in system development

Burk and Kirk[3] made some recommendations if you plan to work with ethno-graphic studies in system development, which are listed underneath:

• Use ethnographic methods early in the design process: The highrisk of designing a system for an unknown or misunderstood user warrantsthe time investment in ethnographic methods.

• Have a well-de�ned scope The open-ended nature of ethnography canbe a weakness as well as strength. It is important to stay focused and torecognize what users are necessary to include in an Ethnographic Study.

• Choose a proper level of Ethnographic Study: Time frame andbudget constraints will often dictate what level and type of ethnographicmethods should be used.

• Make use of previous ethnographic studies: This should be a pre-cursor to any �eld study as it requires the least amount of resource ex-penditure.

35


• Wear one hat at a time : Trying to play the role of the role of theethnographer and the designer at the same time can be hazardous. Makesure to understand the user within the context of the current system beforeattempting to make design changes.

8.4.4 Drawbacks

No matter which method is chosen, there will be some disadvantages. Below wehave listed the drawbacks in ethnographic studies[14]:

Time requirements: The results of an Ethnographic Study are directly relatedto the time investment. While external constraints often limit �eld studies to afew days, even hours, formal ethnographic studies are known to take weeks oreven months.

Presentation results: Sometimes the qualitative nature of results can makethem di�cult to present in a manner that is usable by designers.

Scale: Most ethnographic studies use a small number of participants and asmall-scale environment. When you try to increase the scale it can be extremelydi�cult as it imposes as greater amount of cost, communication and e�ort.

8.5 Process

The process of the ethnographic studies started with a planning session, wherethree ethnographers were involved. After the planning session, we introducedthe session to the users, and presented the three tasks for them.The topics of the three sessions are listed below:

• Interaction with Lego.

• Interaction with boxes.

• Control by gestures.

After the session, the three ethnographers evaluated the observation and wrotedown notes about the users' interactions which could be used in further devel-opment.

8.6 Product

The product we had, after our Ethnographic Study, was a number of picturesdepicting the users' interaction.Beside the pictures, we had some notes of theusers' actions in the tasks we gave the users while we observed them.

8.7 Conclusion

The ethnography study was done in combination with Picture Scenarios, thismeant that we could not make a precise step by step direction for the session,and therefore had a session that was more like a quick and dirty version of the

36


ethnographic studies.

The process and the results are described in appendix C.

Another thing we discovered was the importance of the dialog between the users,while performing their task. This communication was key to the teamwork.We also discovered the importance of the ethnographer and users communicationi.e. the introduction to the tasks. The reason is that every time the userstarts to question the task description, he becomes noticeably unsure. As hecommunicates with the ethnographer, two things happen. Firstly, the usergets his question clari�ed and can relax and focus on rest of the instructions.Secondly, the ethnographer can adjust his way of giving instructions, so thatthe user is less likely to raise further questions. The bottom line is that it isimportant that the user is not left with questions about the task at hand, as hestarts his construction.

We noticed the users had problems to understand the �rst task, although it wasthe task we considered to be among the easier ones. The problem was that theusers were expecting a more di�cult task, and they just could not believe howsimple the task was. Here it is important that the ethnographer can calm theusers down, and clarify the misunderstanding.

Disturbance to the users was another factor we needed to take into considerationduring the Ethnographic Study. The whole idea is to observe the users in theircontext, and they should therefore not be disturbed. We experienced that thereshould not be more than one ethnographer in the CAVE along with the users.And other helpers should stay on the outside. During the session, speech shouldbe limited to the users and the ethnographer, since we have noticed that theoutside noises were disturbing and caused the users to lose focus.

Our users expressed that they found it uncomfortable to sit on the �oor whileperforming the tasks. Therefore, location and comfort was also a factor weexperienced that played a role in the CAVE.

8.8 Lessons learned

Of the observations we had in our Ethnographic Study, we have consideredthings to be aware of, for future sessions or any kind of �eld work. Belowwe have listed the conclusions as lessons learned, for convenience and futurereference, when planning an Ethnographic Study:

• Comfort and location shall be discussed and planned before the session.

• Give the users full freedom to move and choose location within their con-text.

• Explain the task clearly to the users. Communicate with them to con�rmtheir understanding before their task begins.

• Only allow users and ethnographers on the location, and prevent inter-ruptions from happening.

37

Chapter 9

Picture Scenarios

This chapter describes our use of Picture Scenarios, and the general theorybehind.

9.1 Purpose

Our purpose with Picture Scenarios is to acquaint the developers with the users'context, and what is going on in this context. Thereby giving the developers abetter understanding of the project and a goal for the future development.

9.2 Focus

The focus of this method is on dynamic use of context. This means that thecontext changes in di�erent situations where the user interacts with the appli-cations. It is e�ective, when developing for users, to know how they function,and observing them in their own context gives a clear picture of how di�erentsituations requires di�erent interaction forms.[24].

9.3 Participants

The participants in the session are; one facilitator and �ve participants. Apicture of the process is shown on �gure 9.1.

38


Figure 9.1: Picture Scenario participants

9.4 Theory

The fundamental idea of Picture Scenarios is to capture some planned situationsin pictures, which describe the users' interaction with a mobile device.These pictures will be combined with text to form what we call a picture sce-nario, which narrates and describes a context, in a richer form than just a simpleverbal description or the likes. Picture Scenarios have a layout which remindsof sequential comic series, they have the essence of classic textual scenarios, andsome of the concepts of personas and ethnographic studies[25].

When planning the picture scenario �rstly it is necessary to create a short storyfrom a textual scenario. This is done to ensure the capture of the context. Sec-ond you need a cast list of actors in the scenario. Third is splitting the story toscenes and lastly making a storyboard with locations.An essential part during the photo session is to take a lot of pictures, amongstmany pictures there are more likely to be usable ones. Likewise whilst takinga greater number of pictures these are more likely to not look set-up. Non-setup pictures does a better job of catching the situations as they occur in thecontext. The way Picture Scenarios has become a point of interest, is by thefact that it focuses on the context for researching interaction forms.

The process of Picture Scenarios starts with an introduction from the facilita-tor, in which he introduces the participants to Picture Scenarios as a method.After the participants have read the Picture Scenarios the facilitator starts adiscussion, by asking the participants what they think of the di�erent stories.This is a teaser for a longer discussion with di�erent topics, where the goal is tofocus the participants' thoughts on generating ideas for new solutions to prob-lems previously unknown. Also how they should precede their development ofprior ideas.

39


Figure 9.2: An example of a picture scenario

9.5 Process

Our process followed the description from the theory without any deviations.

9.6 Product

The session yielded no physical product as a result, but a list of questions of howthe users would interact in the context. These questions could be a steppingstone for further development for a prototype session or the likes.

9.7 Conclusion

The results we got from the session were nothing new compared to what theparticipants already knew. Each participant already had a good understandingof the topic. From this we deduct that Picture Scenarios have less of an impacton a group who already have a good understanding of the subject.

Furthermore, our experience was that the last of the three Picture Scenarioswas almost useless. This was due to the fact that the concept behind it wasto communicate via gesturing and sign language. This is almost impossible toportray in a clear manner in pictures, even with the use of text.A possible solution to this, could be to try a variation of Picture Scenarios andmake "video scenarios" instead.

Additionally, we took advantage the possibility of combining this method withethnographic studies. This provided a lot of feedback in two di�erent areas witha minimal amount of e�ort.

40


Our experience is that for this method to work at full capacity, it is importantfor the developers to have little or no knowledge about the subject, but a goodunderstanding of the method. Furthermore, the ones whose responsibility it isto construct the scenarios, needs a good imagination and to have a clear imageof the context. Furthermore, it is important for the developers to be motivatedand thereby also able to motivate involved participants, if any, in the construc-tion of the Picture Scenarios.

The construction of the concrete scenario, was developed with little knowledgeabout the CAVE and how to interact in it. Here the developers were motivatedand used their imagination a lot, when creating the tasks. Of course the lackof knowledge could be the reason for some small missteps within the tasks, butthe larger opportunity for being creative easily makes up for this.

When performing the picture scenario session, it is our experience that the out-come of this process depends a lot on participants' background and therebytheir prerequisite for being creative, as the article also points out [25].Some knowledge about the topic is essential for a positive outcome. But toomuch knowledge will also kill the imagination and thereby the creativity in theprocess. When performing the picture scenario session, it is our experiencethat the outcome of this process depends a lot on contestants' background andthereby their prerequisite for being creative, also as the article points out [25].Some knowledge about the topic is essential for a positive outcome. But toomuch knowledge will also kill the imagination and thereby the creativity in theprocess.

9.8 Lessons learned

The experience we achieved throughout studying and working with Picture Sce-narios can be listed as:

• Picture Scenarios give a good opportunity for catching and propagatingthe user context.

• The designing and construction of the scenarios should be planed carefully.

• Picture Scenarios is useful for focusing the concept behind an idea, but isimportant to be aware of the background of involved participants. Becausethese participants experience' can in�uence the development of ideas indi�erent directions.

• When involving Picture Scenarios in a process for developing ideas, itshould be utilized early in the development process, when the developersare still acquiring their knowledge about the subject.

41

Chapter 10

Design and Innovation

Workshop

This chapter describes our use of Design and Innovation Workshop, and thegeneral theory there behind.

10.1 Purpose

The purpose of the Design and Innovation Workshop (DIW) is to create anenvironment where ideas can �ow, grow, and materialise to prototypes.

10.2 Focus

The focus of DIW is to give the designers an environment for generating ideas,and to test the ideas in a simple usability-like test.

10.3 Participants

The workshop was done by a group six participants, later on divided in twosubgroups.

10.4 Theory

The general idea of a workshop is a tool for motivation and to create a platformfor building teams. The construction of the workshop is then dependent on thegoal; it can be performance, creation of initiatives, plans, process, etc. DIW is aworkshop constructed by Jesper Kjeldskov[28]. It focused on Design and Inno-vation and was presented in the course "Design and Innovation workshop"[28].The workshop consists of four activities, of which the �rst one (Pre-sketching)is a tune-in for the rest of the workshop. Originally Kjeldskov designed it to bea short learning workshop in innovation, while we chose to use it directly as atool in our research.

42


DIW consist of the following phases.

• Pre-sketching

• Sketching

• Mock-up

• Prototyping

The next sections, will describe theses phases more speci�cally.

10.4.1 Pre-sketching

Pre-sketching gives the workshop a soft start, its purpose is twofold; �rstly, it isa tool for the developers to tune in on the subject area. Secondly, it is used tobounce the ideas between the participants in order to create common ground fortheir ideas, and open for the possibility of letting ideas evolve, whilst bouncedback and forth. The only guideline introduced to the participants, is to writea list of whatever ideas and associations that spring to mind while consideringthe context. Afterwards, the group discusses and each member presents his/herideas to the others.

10.4.2 Sketching

Sketching is a technique borrowed from architecture and industrial design. Itmakes it possible for the designers to materialise their ideas quickly. Sketches areusually done with just pen and paper. It �ts in the beginning of the workshopwhere most of the ideas will be generated. As Bill Buxton describes[4], sketchesare not a tool for prototyping, but for exploring, suggesting, and provokingdesign ideas. The designers are not required to be great illustrators. This isbecause the sketches are done individually, and presented to the rest of the groupafterwards, so the developers can verbally compensate for a hard-to-understandsketch.Before the mock-up phase, each group member needs to have some commonidea of what he or she would like to produce in this phase. All of the developershave a group discussion to see if they can agree on one or two ideas.

10.4.3 Mock-up

The mock-up process is, in contrast to the previous exercises, entirely carriedout in the group, and the group works only on one or two ideas.The mock-up is a general presentation of the idea and is often physical and madefrom paper, cardboard, foam or otherwise low-�delity materials. Mock-ups turnattention away from details of the graphic design and focus on content andfunctionality. So they are useful to visualise ideas early in the design phase andto bring the designer beyond current technical possibilities[28]. The guidelinesfor what the mock-up should re�ect are the following:

1. The di�erent components of the system.

2. What the system and its components physically look like.

43


3. What the system and each components do.

4. How the user physically interact with the system.

5. How the system and the user interact with the environment.

After each group have produced one or more mock-ups, they present these tothe other groups.

10.4.4 Prototyping

The product idea needs to be tested, that is what the prototype phase is for.Usability testing is a well known method for �nding usability problems in soft-ware. It is typically used on software prototypes, but it is possible to use it onpaper prototypes as well. Snyder[29] de�nes a usability test on paper prototypeslike this:

"Paper prototyping is a variation of usability testing where representative users

perform realistic tasks by interacting with a paper version of the interface that

is manipulated by a person 'playing computer', who does not explain how the

interface is intended to work".

The goal for the paper prototype is that this can be accomplished. So notonly the physical artefact has to be created, but also a use case for the user towalk through during the test. The prototyping session is done in an iterativemanner. During the testing, the user or the design team can suggest changes.These changes are made to the prototype, and the test is run again.

10.5 Process

As part of the introduction to the workshop, and to give the design team agood understanding of the method, we went through a workshop unrelated tothis project. This is furthermore described in appendix E.1.8.

The general guidelines for the Design and Innovation Workshop were followed,starting with a pre-sketching process, where each participant wrote down a listof ideas and topics, related to the problem area.Afterwards these topics and ideas where discussed in the group, for creatinga consensus about the main focus of for the workshop, idea exchange. Nextstage was the sketching part, where each participant individually created papersketches, which again were presented individually, and then discussed withinthe group. In the Mock-up part, the ideas were split up into two di�erent top-ics of focus areas. Menu System and Object Manipulation topics. Thereafterthe group was split up into two subgroups accordingly. Again ideas were dis-cussed and presented for the group. The products from this part were physicalartefacts built with easy-to-form materials, like paper, polystyrene and piecesof cardboard. The �nal part was the prototype session, where the two groupsagain used physical artefacts, to create a new mock-up on a larger scale. Theresulting products from the two subgroups were now presented in a session in-spired by a usability test, with the opposing group members playing the roleof the user. After this session, further discussion about the results took place,

44


involving all participants from both subgroups. A more detailed description ofthe process can be found in appendix E.2.2.

10.6 Product

The products we ended up with in our workshop, were prototypes and mock-ups, which could be used for further development. Beside the di�erent types ofphysical artefacts, we also ended up with some discussions about them, whichgave us some conclusions about the �nal result of our development of new in-teraction forms. A more detailed description of the process can be found inappendix E.2.2.

10.7 Conclusion

When developing for an alternative setting, like an immersive virtual environ-ment, it is useful to have some knowledge about the technology that is available.Both more speci�c as well as general knowledge is useful. This can be exempli-�ed by our knowledge about Lego which helped us a great deal in de�ning ourproduct, during the pre-sketching and sketching processes. Likewise, the knowl-edge we had about the CAVE, the available technology and ways to interact,helped us in in creating our sketches and developing the prototypes.

Developing to a virtual environment has some impact on the way the Designand Innovation Workshop is carried out. As an example we can look at themock-up process. The mock-up can very easily exceed the boundaries whichit was meant to be limited by, and �oat towards the prototype process. Paperprototyping is meant to be a cheap and quick way of developing prototypes.When the mock-ups exceed the limits of such it is important to be aware of howto compensate. We cannot claim to have found the best way of doing this yet,but we �nd that verbal explanations uni�ed with the mock-ups compensate agreat deal. Another way might be to build mock-ups in di�erent scales.

Additionally, it can be di�cult to create a full size paper prototype of a realisticscenario with moving elements, and this can in�uence the progress of the test.This is because the subject is to create an immersive virtual environment, andthe credibility is reduced, as the simulation of the moving elements e.g. requiresextra persons within the test �eld, operating these elements.An example of this is the virtual rack menu system, where one person wasneeded for each rack section. This might not necessarily be causing problems,but it is something to be aware of.

We feel that the workshop is innovative because it in our case, was an e�cientway of ensuring that our di�erent ideas were debated, which then led to im-provements of the current ideas or the generation of new and better ideas.Furthermore, the way the process of the workshop was put together, the se-quence in which the sub processes were ordered, helped support and evolve ourideas.

45


10.8 Lessons learned

The experience we achieved throughout working with the Design and InnovationWorkshop can be listed as lessons learned:

• Avoid giving negative comments or kill the ideas during the session.

• The extent of the work with paper prototyping, may exceed being cheapand quick.

• Consensus and a clear picture of where to start the innovation is valuable.

• Mock-up is useful for demonstrating the interaction and is not very de-manding.

• In the prototype session you can end up without any physical product.

• The di�erence between the mock-up and prototyping phases, may becomeunclear.

46

Chapter 11

Discussion

In this chapter problems and challenges with idea generation methods for alter-native user interfaces, will be discussed and generalised. Brainstorming has achapter of its own, because it was treated a bit di�erent in the evaluation, andsome discussions are only relevant for this method. The remaining discussionsare relevant to several idea generation methods.

11.1 Brainstorming

Brainstorming is a well-reputed method for idea generation in a lot of indus-tries, not just software development. Brainstorming does not require a lot ofresources neither in form of expertise nor physical resources. This makes it eas-ily adaptable to all kinds of �elds, and user interface innovation is one of them.We did not �nd a lot limiting factors when working with brainstorming.

One factor that had a lot of in�uence on the ideas created in the brainstormingprocess was knowledge and current trends in software. This a�ected the ideasin both positive and negative ways.Positively by the fact that we generated ideas for new technology that was notpossible just a few years ago, so that satis�es the new technology factor forinnovation. Negatively by the fact that we limited ourselves to technology cur-rently available. All tree case proposals were inspired by our knowledge aboutour surroundings: CAVE and location based computing. This makes it quiteclear that knowledge is a big factor.

11.2 Knowledge and idea generation

Knowledge as we have experiment has a big in�uence on idea generation pro-cesses. What makes knowledge a bit special in idea generation for alternativeinterfaces is that it possible to control the knowledge better then when workingwith traditional interface development. Alternative interfaces are by de�nitiona lesser known subject to most UI developers. When developers start workingin the �eld for a speci�c alternative interface, they need to acquire knowledgeof that speci�c �eld, as opposed to traditional interface design, where the de-velopers have a basic knowledge about such interfaces. This forms a unique

47


opportunity to control how much expertise the developers acquire in order notto harm their innovative process.It is quite tricky to �gure out how much expertise the developers should ac-quire. We have not tested any solutions for this, but a proposal for controllingknowledge in the process is listed in the following section.

11.2.1 Stop and learn

This suggested idea is to basically stop the idea generation process when exper-tise is needed. In many cases ideas will pop up, that the developers do not haveenough knowledge about they are suitable for implementation. In such cases theprocess is stopped, and the group splits out to acquire the information neededto investigate further options of the idea. When this expertise is acquired thegroup steps back into the idea generation process.

An example where this suggestion would have been usable is in the mock-upprocess in DIW. One idea that popped up was using sound to make it easier tolocate the position of the menu rack (see appendix E.40) the idea was not car-ried out because we did not know enough about how sound feedback worked onpeople. If we had stopped the process and acquired this expertise about soundfeedback, we might have ended with an idea for implementing sound feedbackin a VR application.Of course there are some factors that might cause problems in an approach likeStop and Learn. If an idea process is stopped it might harm the idea �ow, andwhen started again, the group might have trouble getting in to "idea generationmode" again. The other factor is time. A Stop and learn solution takes moretime, and as already discussed, time is critical in a radical innovative process.You might end with ideas that are unusable, so by making the process longeryou might waste even more time.

11.3 New technology - old ideas

Working with our chosen idea generation methods in combination with newtechnology, we noted that a lot of the ideas generated were moving existingideas or concepts in this new technology. Several examples show this:

• The menus system in the CAVE as described in appendix E.4.6 buildupon a rack which is a physical piece of furniture moved into the VRenvironment as a new menu system. The CAVE gave us a lot of newpossibilities and we came up with an idea based on a known concept.

• The case proposal GPS game described in appendix B.1, is another exam-ple of new technology. In this case GPS was used to bring new life intoold ideas. The idea was to bring existing game concepts outside to a moreparticipating game experience.

11.3.1 New concepts

The idea generation process did not generate a lot of idea which could be char-acterised as new concepts. New concepts are not something that pops up often

48


in the IT industry. An example of a new concept is a spreadsheet application.This is not a concept that was possible before we had computers. Why ideageneration with alternative interfaces dose not seems to bring new concepts totable is a question that is hard to answer. But we have a few theories:

• Coming up with new concepts is hard. This can be seen through thehistory of the computers, not many concepts were new concepts whenthey were brought to the world. This includes WIMP which is built onthe o�ce/desk concept with folders, desktop and windows. New innovativeidea generation methods cannot change that.

• To much is "new". When working with new technology, which might behard to grasp, existing concepts is tempting to use for participants. Inthe idea generation process these known concept supply a certain feel ofsecure familiarity.

The question is: "Are known concepts bad from a UI innovation perspective?"

As described in chapter 2.2.1, UI built on concepts from the real world, gives afamiliar feeling to the user, which makes the user interface easier to use, So inthe context new concepts are bad from a UI perspective. But if all UI innova-tion is based on known concepts, we might never see "the spreadsheet" of userinterfaces.

Our experience shows, our chosen idea generation methods do not help in thesearch for new concepts, and neither does new technology. But that's not nec-essary a bad thing. But something that needs to be noted. So if the goal is anew UI concept, these methods do not have any special strength in archivingthat.

11.4 The boundaries of creativity

When searching for new ideas in an innovation process, creativity plays a bigrole. But as we have learned creativity consist of several characteristics, seechapter 1.3, these characteristics are as follows:

• Expertise which is achieved throughout knowledge and intellectuality.

• Creative thinking skills help to generate ideas through the process by a�exible, imaginative, and methodical approach.

• Intrinsic motivation which is more e�ective than extrinsic because of thepersonal interest and self-motivation.

11.4.1 Expertise versus creative thinking skills

Especially expertise is important, as we have experienced that knowledge canbe characterised further in two types that each is essential for generating newideas. First of all it a necessary to have some knowledge about the speci�c areain which the idea generation is occurring. This we will hence forth be referredto as pointer knowledge. Secondly it is necessary to have a broader knowledge

49


about other topics; this can be de�ned as life experience and varies for all indi-viduals.

The pointer knowledge will serve to direct the idea generating activity.The life experience will serve as inspiration source for generating the ideasthrough creative thinking skills such as imagination.

As we learned throughout the involved experiments, the idea generation be-comes harder for the speci�c case as the pointer knowledge is increased; therebythis pointer knowledge actually becomes an obstacle for the �ow of ideas.

Why is this?

Continuously as the pointer knowledge is increased for the individual, the ideageneration retains a dependencies of how good the individual is to disregard thisknowledge and thereby give room for being more creative, and using his or hersimagination by relating to life experience.

Therefore an innovation process is not for everyone, as the generation of newideas depends on creativity and creativity depends on the individual's ability todisregard knowledge about the area of development and, to be imaginative.

11.4.2 Intrinsic motivation

Intrinsic motivation is another characteristic under creativity also plays a keyrole during a creative idea generation activity. As we experienced during thedi�erent experiments the motivation factor was di�erent for each participant,even through their focus on the task was the same.

Why is this?

Consequently an individual's motivation depends on di�erent factors and one ofthese factors is knowledge. In light of this it would be easy to motivate a singleparticipant.

But when assembling a team the motivation factor for each participant is hardto ful�l and even though the focus for the team is the same, their individualknowledge is not.

This is a dilemma because this previously noted facts suggest that a team as-sembled by participants with both the same focus and the same knowledge,functions optimally. We are not sure that such a team has the same capacityfor generating radical innovative ideas.

When assembling a team for innovation, the individuals' knowledge should betaken in to consideration.

One way motivating a team for further progression is by adding some elementsthat could a�ect the motivation factor. One of the elements we experienced,

50


is the element of competition. When adding this element to the idea genera-tion process it could actually a�ect some participant to be more motivated andthereby creative.

11.5 Modelling interaction problem

In two of the idea generation methods, limitations were discovered when work-ing with interaction forms. The methods were Picture Scenarios, described inchapter 9, and the Design and Innovation Workshop, described in chapter 10.We discovered that interaction is hard to modulate. In the picture scenarioswe had trouble showing how users interacted in the pictures, and in the work-shop, mock-ups and prototypes were representing the objects in the application,rather than how we interacted with the objects. This is kind of obvious, but asthe main purpose from our evaluation of the methods was how they performedin UI innovation where interaction plays a big role, it was a problem in ourcontext.

In Picture Scenarios the modelling interaction problem, almost meant that thework with the method was wasted. The pictures with still photos did not makeit possible for the participants to understand how the users interacted with thebricks. Some explaining from the facilitator solved part of the problem.

In the workshop the modelling interaction problem was not quite as obviousas in Picture Scenarios, as the mock-up/prototype was only one part of howthe idea was presented and worked on. The participants were free to verballyand physically, via body language, show interactions. But the problem with theworkshop, was that the dominating focus became how you model the physicalartefact, and in the search for new interaction forms that is not that important.The important thing is how you interact with the artefact. For example thegroup working with interaction with bricks, spend quite some time on creatingthe brick model, and less time on describing how you interact with the brick.Their presentation could have worked �ne with a less perfect brick.

What does this mean to developers of alternative interaction formsand user interfaces ?

When choosing a method for idea generation, it is important to look at how themethod makes it possible, for the developer to express and work with interactionforms. Choosing a method that makes that easy will most likely generate thesmallest amount of problems regarding interaction. When working with meth-ods that might cause problems (like the Design and Innovation Workshop),making sure to focus on the important part of the idea solves the problem.

11.6 The oddness factor

Working with alternative user interfaces brings a new factor to the table: odd-ness. In both Ethnographic studies and the Design and Innovation Workshopwe experienced that. The oddness factor is the feeling of working with some-thing that is odd, funny, and out of place. In Ethnographic studies it was the

51


users who had that feeling. They had a hard time understanding what theywere supposed to do, and were feeling discomfort in the setup. This is not anunknown problem when dealing with users. An example could be the situationin a usability test. What made the situation more uncomfortable for the userswas that nothing felt familiar to them, as they were in a new situation and theywere to use unknown technology and input systems, as opposed to usabilitytests on traditional software applications, where only the situation is new. Theapplication is unknown in functionality, not in the UI concept it builds on.

This leads to the conclusion, that with idea generation methods involving users,extra energy and time must be used in order to make the users feels more com-fortable. As investing a lot of time in radical innovation is already risky (seechapter 1.1), extending the time used on idea generation methods with usersmight cause problems. The time might be better spent using methods withoutusers.

The oddness factor for developers was felt during the workshop, in working withmock-ups and prototypes that felt unfamiliar to us as developers as opposed toa paper prototype on traditional desktop applications, in which we had experi-ence with.The developer oddness does not have the same importance, as the oddness factorfor users. Working in this unknown �eld, led to laughter and amusement amongthe developers. That of course could have the consequences. Consequences as,the development group could have problems with taking the process serious,we did not, however, experience this, but this were something that needs to beconsidered when forming groups for this kind of idea generation.

11.7 Materials

We experienced, that physical materials contributed to creativity, and playeda considerable role when the participants needed to communicate about theirideas. For instance, in the Design and Innovation Workshop, these materialswere the paper and cardboard sheets used to make mock-ups. In Picture Sce-narios it was in the form of the presented pictures. In another session, it mightbe something completely di�erent. Without having these materials, the com-munication of ideas would have been di�cult. The materials gave us the abilityto build mock-ups, which combined with explanations and discussions created abasis for processing the ideas. We discovered as well, that these same materialscan be obstacles in the mind of the developer, when it comes to the generationof ideas. Take a photo for an example, it can be quiet inspiring, informative,and a base for discussions, while at the same time, it is restricting the partic-ipants from thinking of what is absent, a movement, for an example, as thedevelopers are stuck focusing on the stationary motive. We found as well, thatwhen you have to physically build a model of your idea, it is in itself restrictingthe potential pool of ideas that one could otherwise generate, as there are thingsthat is impossible to build in the real world. As the participant is aware of thefact that he eventually will have built a physical construction of his idea, hemight unconsciously reject the idea himself prematurely. An example of sucha phenomena, is that it is hard to imagine, that ideas like 'forever-ascending

52


Figure 11.1: A forever falling waterfall

stairs' or the forever-falling waterfall, illustrated in �gure 11.1 would emerge, ifthe participant was beforehand required to build it. The bottom line is, thatwhile materials play a very important role as a communicative medium in aninnovate process, the imagination is limited by these same materials.

53

Chapter 12

Conclusion

During this semester we have been introduced to various methods supportinginnovation, we have experimented by involving these in a more radical approachfor research on our objective, development of a new interaction form.

The �rst step of this innovation process was narrowing the main objective downto more palpable challenges. This left us with a simple concept to deal with,namely Lego. This concept served as a triggering factor, for all the meth-ods in our research. The methods we explored during our research processwere Brainstorming, Design and Innovation Workshop, Picture Scenarios, andEthnographic studies. As our research process was on an experimental level andthe involvement of methods, e.g. to conduct and to evaluate, was on an emer-gent basis, the research progress evolved as we involved more and more theoryand knowledge to the process, thereby giving us a better chance for learning bydoing.

As we were in a radical innovation process, we experienced that some resourcesbecame rather fragile issues, due to the fact that even with a clear vision ofthe main objective, the way to get there were very ambiguous. This fragility isillustrated in �gure 1.1, in this framework the process is located in the upper leftquadrant, this indicates that the process has a high novelty, while unfortunatelythere are high uncertainties, and thereby a risk of missteps. This means thata radical innovation process has a very complicated structure and furthermoreallocating of resources to the concrete process can be di�cult.

To answer our main problem statement, we can say that we indeed use tra-ditional innovative methods for radical innovation, and the feedback producedfrom these methods, is indeed useful. However, we experienced that it is notthe methods themselves that contribute the radical part of the results, but thedevelopers behind them. The fact is that the methods themselves do not spec-ify which people it is developed for. In fact, when involving methods for ideageneration, assembling the team based on each member characteristics withincreativity is essential.

Whilst being innovative, we learned that knowledge is an important character-istic for creativity �rst of all knowledge about a topic serves as a focus point

54

User interface innovation Chapter

for the idea generation process. Di�erent types of general knowledge inspire theimagination and thereby in�uence the output of ideas. Nevertheless, we expe-rienced a decrease in the �ow of ideas, when we increased our knowledge onthe topic. This fact suggests that excessive knowledge about a topic will limitthe innovation process itself. Thereby the ability to be creative and moreoverinnovative depends on the developer's skill to disregard his knowledge on theresearch topic.

Another obstacle in an innovation process, is the sudden lack of knowledge whileconducting a methodical process. If the developers don't have the knowledgeabout a related topic, that potentially could be bene�cial under this process,we experienced a tendency to skip the topic, and dropping ideas altogether.

Another fact that we learned throughout the experiments, was that the involve-ment of di�erent materials to support the idea generation turned out to haveboth positive and negative in�uence. Positive by giving the developers an optionto express themselves by more means than just a verbal expression, negative inthe way that static materials, like pictures, can be limiting for the �ow of cre-ativity.

Furthermore, the process of radical innovation can at times be a very inexplica-ble, and thereby it becomes hard to involve external developers in the researchor at times even for the researching developers itself to maintain overview.

At the end of the day, a radical innovation process is all about the developersmaking the right choice at the right time, and while not knowing how and whento do so, these uncertainties and risks become the price to pay for novelty.

55

Appendix A

Brainstorm

A.1 Process

On the basis of our semester theme we wanted to have a broad Brainstormingprocess about everything that had potential to be innovative and could be ex-citing to see implemented. We carried out our brainstorm process in the grouproom, where all the participants were sitting opposite of each other around thetables. The process was as following; at �rst we spent some time thinking ofideas, mainly to get in the creative mood for the Brainstorming process. Ad-ditionally we decided to choose a facilitator and during the process he had theresponsibility to document all the ideas on the blackboard and guide the process.The reason why we chose to make use of a blackboard instead of a computer ora di�erent writing tool was because it is simple. Moreover we wanted everyoneto be able to follow what was being written.

We had a free Brainstorming session where everyone was on all the time and theparticipants could suggest ideas at random (i.e. not by turn). Some were moreactive than others, and they were able to come up with new ideas. Further-more, in concordance with the Brainstorming rules we did not discuss, criticizeor question the ideas that were written on the blackboard during the processand this lead to positive ideas which created a good and creative atmosphere inthe room. Half an hour after the process started we were running out of ideasand instead of ending the Brainstorming, we took a little break to freshen upour minds in order to �nd new ideas.

After the generating process we evaluated the ideas through discussion. On theblackboard we �rst ranked them after a general approbation in order to see howpotential they were. Because of some disagreements we created a documentwith all the ideas and a short explanation of the core intent for each idea. Thisdocument can be seen in appendix B. The purpose of this document was thatevery group member should select three ideas which they preferred more thanother ideas and give these three selected ideas a score of either one, two or threepoints. This was done to �nd the idea which they wanted to work with themost. When the scores were added up, we selected those three ideas with thehighest score to build on. This procedure allowed us to see which ideas were of

56

User interface innovation Chapter A

particular interest among the group.

A.2 Results

The outcomes of the Brainstorming were three visions we had mainly based onthe selected ideas. And as a result of our discussions, focus was primarily onan area that covered possibilities of GPS and 3D interfaces and how to interactthree-dimensionally, which we felt was something new and had the potential toturn out innovative.

The visions are as follows, and in-depth explanations of these are given/concretisedin appendix B:

1. GPS game.

2. Head up display in a 3D interface.

3. Datamining in a CAVE.

Two of the visions are as the chosen ideas but one had turned out quite di�erentthrough discussion, inspiration and by combining ideas. When reaching ourcommon goal, we had to put factors such as time and the resources we had indisposal in to consideration. It had to be innovative or at least have potentialto be. Finally, the interaction possibilities in a CAVE were chosen.

57

Appendix B

Case proposal

B.1 GPS game

B.1.1 Idé

Et udendørs computerspil, som benytter sig af spillernes position. Hvordanspillet skal spilles er (endnu) ikke de�neret. Man kunne overføre et kendt com-puterspil til konceptet. Hvis man tager det populære platformspil Pacman someksempel, så bliver GPS-spillet et udendørs multiplayer-spil hvor en af delt-agerne har rollen som Pacman og de andre agerer som spøgelser. I dette eksem-pel benyttes spillernes indbyrdes position til at stikke af fra spøgelserne ellerfange Pacman. Ude i naturen kan der på forhånd eller under spillet blive gemtgodter/point, som Pacman skal 'spise'.

Inden for de seneste år har computerspil udviklet sig til at have et mere ogmere realistisk udtryk - senest med Nintendo Wii - hvor spil kan udvikles til atbenytte input-metoder som ligner virkeligheden. F.eks. tennis hvor man slår tilbolden, som når man spiller tennis i virkeligheden. Denne tendens kunne væreinteressant at trække endnu et skridt videre, hvor computeren bliver et værktøjfor at få et spil til at virke, og ikke til at afvikle spillet selv.

Vi vil med dette udendørs koncept prøve at fremme den sociale interaktionblandt folk, ved at de er ude i det fri sammen om et spil, i stedet for at de hverisær sidder foran computeren og spiller. På den måde får de også rørt sig i entid hvor en stor del af Danmarks befolkning er overvægtige. Vi mener hervedikke at dette spil kan trylle fedmeepidemi væk (hvilket heller ikke er vores for-mål), men det skulle gerne få folk op fra stolen og gøre dem fysisk og socialtaktiverede. Hvad der gør dette spil spændende, vil være den nye og anderledesform for interaktion mellem spillerne.

B.1.2 Teknik

Hver enkelt deltager har en mobiltelefon med GPS modtager.

58

User interface innovation Chapter B

B.2 Head up display in a 3D interface

Hvordan kan et tredimensionalt alternativ til et klassisk 2D head updisplay se ud og fungere?

Oplevelsen i et 3D miljø som f.eks. en CAVE er i sig selv spændende, men imange tilfælde begrænset til få bevægelser og funktioner. Hvorimod mulighed-erne for input ved en todimensional skærm ved hjælp af mus, keyboard oglignende er meget stor. Ved hjælp af få 3D trackede sensorer bør det dog væremuligt at skabe en masse forskellige udtryk. Spørgsmålet er hvilke af disse giverintuitivt mening for brugeren, og hvilke kan med fornuft blive vane, så de kanoversættes til brugbare funktioner.

Mulige funktioner:

• "Zoom"

• Skift af miljø

• Multiple interaktionsmuligheder med samme objekt

• "Højreklik"

Noget så basalt som at gå på internettet eller tjekke sin mail bliver i førsteomgang væsentligt mere vanskeligt uden et keyboard og mus. Det interessanteville være at se om disse input devices ikke kan erstattes af andre 3D devicessom i samarbejde med den tredje visuelle dimension giver en alternativ oplevelseaf informations søgning. Studiet her kunne starte med at se hvordan man villesøge information uden et keyboard, ved hjælp af blot mus eller videre med brugaf touch screens og deslige, og drage nytte af sådanne undersøgelser i forsøg medCAVE eller deslige.

Et andet eksempel hvor sådanne funktioner ville være interessante at studere,kunne være et RPG (Role Playing Game) spil. I disse bevæger spilleren signetop rundt i et 3D miljø, og har en lang række funktioner til rådighed, oftesttilgået ved hjælp af en lige så lang række af genvejtaster. En "god" spiller er istand til at sætte disse funktioner sammen i rigtig rækkefølge i forhold til sineomgivelser og dermed yde bedre. Et interessant studie kunne være at kigge påat erstatte sådanne genvejtaster med reelle bevægelser og se hvad dette villegøre ved spiloplevelsen.

B.3 Datamining in a CAVE

Hvordan kan man udnytte de egenskaber som CAVE'en tilbyder, tilat eksplorere sit data?

Der er voksende behov for nye metoder til søgning efter mønstre og struktureri et voksende mængde data, som er ikke til at �nde rundt i.

Datamining (DM) er princippet om at �nde relevant information gennem sorter-ing af store datamængder. Datamining bruges normalt af store virksomheders

59

Chapter B User interface innovation

efterretningsorganisationer, �nansanalytikere, men bruges også i videnskabernetil at udtrække præcis information ud fra de enorme datasæt, som dannes vedbrug af moderne eksperimentelle og observationsorienterede metoder. DM erblevet beskrevet[32] som "ikke-trivielt udtræk fra implicit, førhen ukendt ogpotentielt nyttig information af data og videnskaben om at udtrække nyttig in-formation fra store datasæt eller store databaser".

Hvor man med den traditionelle statistiske metode søger at slutte fra enkel-tobservationer til udsagn om hele populationen, bruges datamining til gennemsammenstilling af egenskaber om det enkelte individ eller det enkelte objekt.

Det immersive 3D miljø, som CAVE'en giver mulighed for at udvikle, har poten-tiale til at vise mønstre i datasæt, som ikke er mulige at opdage i de eksisterendesystemer. Projektet vil udforske interaktionsmulighederne i sådanne datasæt.Interaktionsformerne i CAVE installationen er bl.a. et såkaldt wand, som er etslags 3D joystik. Desuden er display systemerne i CAVE'en i stereo (via 3Dbriller). Dette betyder at man ikke bare kan tage de traditionelle interaktions-former og overføre dem til CAVE'en.

60

Appendix C

Ethnographic Study

C.1 Planning

Our approach deviates from this method in the way that we will get two resultsinstead of one. Normally, picture scenarios is a tool that can lay the base fora discussion between di�erent people of interest for the project, and therebyfunction as a tool for combining or developing ideas further[24]. Additionallywe will use the photo session to gather information, because it provides an op-portunity to study the interacting process as it takes place in an ethnographicalway. This becomes an option because the session is continuously shot on thesite and therefore is not interrupted by breaks for shifting locations, what ismore is that this approach will not let us focus on particular planned picturesbut rather on the total run of the scenario.

From earlier studies on user interaction within a CAVE we learned the impor-tance of HCI[30]. Even the smallest decision on how the user should interactwith the interface can has signi�cant impact on how the user experiences it,and thereby on the results when using it[6]. Therefore our goal was to get abetter understanding of how the participant interacted with the chosen relatedproducts. We decided to compose three scenarios in which we wanted to studythe interaction on two di�erent products, namely Lego and cardboard boxes.

As mentioned earlier, �rst of all you need to plan a short story for the scenario.It will also serve as a base in the crafting of the storyboard. The three stories forour scenarios were: Lego in a CAVE, boxes in a CAVE and controls by hands.Central for all three scenarios was the location. We decided the Picture Sessionshould take place in the CAVE, with the assumption that it was important forthe research to get a better understanding of the dimensions that the CAVEo�ered throughout these scenarios.

In all three scenarios the concept was to let the users build a tower, �rst withLego and then with cardboard boxes. During the �rst two scenarios the userswere allowed to talk and help each other while constructing the towers, duringthe third they were not, here the idea was that one of the users guided the otherto build the tower through gesticulating.

61

Chapter C User interface innovation

Figure C.1: Storyboards for Picture Scenarios

All of the storyboards can be found in appendix D.3

C.2 Process

According to our project we have used ethnographic studies by involving usersto an observation in the CAVE. Our object, which the users had to be concernedabout during the observation, was Lego, which is popular and well known inDenmark.

The ethnographic method we used to the observation of the users in the CAVE,was a so called mix between the Quick and dirty method and the Rapid ethnog-raphy. The Quick and dirty method was chosen because of the low budget oftime, and the observation should be a precursor to another method 9. Becauseof the time pressure and there was a limited time in the CAVE, this method isalso sort of "rapid ethnography".

The observation tools, was an ethnographer making notes of the actions andconversations the users had during the tasks in the observation, and on thesame time we had another ethnographer, who had the responsibility to takepictures of the users in the CAVE. The third ethnographer was helping theusers going through the di�erent types of tasks during the observation.

The users were between 18-20 years old and both were interested in IT system,so we considered them useful for our observation. Because of their understand-

62

User interface innovation Chapter C

ing in the task, we expected them to become helpful, by describing problemsfor us as these appeared, as it are mentioned in rapid ethnography.

The �rst thing we did was to introduce the users to our project, and made themunderstand the main topic with our observation, which was to �nd out how theusers interact with the Lego. After explaining the concept and rules for theusers we conducted the tasks, which is described in the next section.

C.2.1 Interacting with Lego

The �rst step in our observation was building with Lego, this also worked asa warm-up for the users to gain better understanding of the topic. We startedby asking them about their knowledge of Lego, and their previous experienceswith Lego. Afterwards, we followed up by moving the topic from Lego, to newinteraction forms and new technologies, where we wanted them to think in thecontext of new technologies and how it is used, then we asked them about theirown experience with some of them. The next event was the building and theirinteraction with Lego itself.

C.2.2 Interacting with boxes

After the �rst task was in place, we took the observation a step further. Wereplaced the box of Lego with di�erent sizes of cardboard boxes, a stick madeof wood, tape, scissors and a knife, and told the users to build a new tower ofthese materials.

Figure C.2: Materials for interacting with boxes

63

Chapter C User interface innovation

C.2.3 Control with gestures

In the last task of the observation, we delegated two roles to the users; onewould be the navigator and guide the other to build a tower of cardboard boxesby using only hand-gestures. The other role was to function as a builder andreact on the navigator's gestures. They were not allowed to talk during thistask.

At the end of the observation we took some pictures of the users for the castlist we used in the next method, Picture Scenarios.

C.3 Results

C.3.1 Interacting with Lego

The users found the �rst task in the observation very easy and there was a goodcommunication from the start, the task was simple and the interaction betweenproduct and user was good. As one would possibly expect, the whole observationwas marked by the earlier experience of the users with Lego. One thing thatfrustrated the users was the confusion regarding what solution there was thecorrect one for the building, in a way they expected we would give them theinstruction booklet for this building and for that reason they felt unsure aboutwhat they were doing.

During this task we observed the users interacting with the product; Lego. Howthey turn the bricks around with their hands, to �t them together, and howthey use their head to look around to get a better overview.

C.3.2 Interacting with boxes

After a short introduction to the second task the users easily involved themselvesin the task. They started to discuss some di�erent solutions in building the towerby cardboard boxes, and they came up with quite acceptable and solid results.They used the di�erent tools supplied by us and thereby interacted with boththem and the materials.

As in the �rst task, we saw that the users became comfortable with objects, asthey progressed with manipulating them; again we saw the necessity to turn thematerials around, in order to get a better overview of the construction.

C.3.3 Control with gestures

The third and last task was di�cult to understand for the users, a short pre-sentation of a little example helped to get them ready. They started buildingthe tower but apparently they had di�culties in �nding the right gestures andto understand them. The navigator was signalling the builder, and was usingseveral gestures, which some of, seemed straight forward and common to us.

Right hand was used for up and down movements of the object, and both hands

64

User interface innovation Chapter C

Figure C.3: Users interacting with boxes

for the left or right movements. To pick up an object, both hands were used.Frequently one hand was used to navigate around with the objects.

65

Appendix D

Picture Scenarios

D.1 Process

After completing the Picture Scenarios, the idea was to use them as a base forstarting a discussion within the study group with the primary goal, to focusthe group thoughts on how we should precede our development of a prototype,furthermore the Picture Scenarios should also provide a chance to discover pre-viously missed HCI problems.

Figure D.1: Picture Scenarios

All of the Picture Scenarios can be found in appendix D.3

The session took place in our group room located at Cassiopeia, where the groupwent through the Picture Scenarios, discussing them one by one.

66

User interface innovation Chapter D

D.2 Results

The discussing of each picture scenario resulted in some questions on the userinteraction.

• When interacting with small objects like Lego, the users used the �oora lot, either sitting down or using it as a base for modifying their mainobjects. The fact is that we have not given this much thought in ourdevelopment process, maybe the reason is that the users needed a workingtable and since there were none in the CAVE, they used the �oor, anotheranswer could be when interacting with Lego you do it by by experiencefrom childhood while sitting down.

• Another thing that came to our attention was the fact that Lego is a verysmall object to interact with and this causes the users to move in closerto their main object during constructing it. This means that we have tobe aware of the relation between the size of the objects and the user ofthe application while the interaction takes place.

• We also observed that the users were interacting with each other whileinteracting with the objects. This is another thing we have not discussed,which possibilities do we have on more than one user interacting with theapplication at once? Lego is a toy which it is much more fun to play withwhen you play along with someone else.

• In the last picture scenario we had a lot of trouble getting the scenariomessages. The fact is that trying to show gesture and movement in apicture scenario is rather di�cult, an idea could be making a video scenarioinstead.

When we look back upon our process and how we have used the method, itis clear that our short stories could have been stronger and thereby makingthe process of making the Picture Scenarios more structured. Furthermore thepreparations to the di�erent sessions in the process could have been better,since some of the decisions were taken in last minutes, and a better organizedapproach could have prevented some of the smaller problems.

• For an example, we could have done for the cardboard boxes to look morelike Lego.

• The fact that we gave the user a wooden stick helped them constructinga stronger building out of the boxes and thereby emulating the bindingfeature that Lego have, but in contrary this might have moved the focusaway from the primary idea of users interacting with big boxes.

67

Chapter D User interface innovation

D.3 Storybords & picture scenarios

Figure D.2: Storybord for interacting with Lego

Figure D.3: Storybord for interacting with boxes

68


Figure D.4: Storybord for control with gestures

69


Figure D.5: Castlist

70


Figure D.6: Picture scenario 1

71


Figure D.7: Picture scenario 2-1

72



73



74



75

Appendix E

Design and Innovation

workshop

E.1 Learning workshop

E.1.1 Purpose

Group members have no experience within this workshop area, so a trainingworkshop is deemed necessary in order to workout misunderstandings and ob-tain some level of experience within the �eld. The experience in this learningworkshop will get the designers a familiar understanding of the workshop. Sothe scaled up project workshop will be running more smoothly. The earnedexperience from this learning workshop is intended to familiarise the designerswith the workshop procedure. After an evaluation of this �rst attempt, theresults will be used to make a list of suggested improvements before repeatingthe workshop for the project.

E.1.2 Planning and preparation

Mikael B. Skov came by the group room and provided presentation slides. Thenhe introduced each phase of the workshop as the workshop progressed; He ex-plained some of the basic ideas behind sketching, potential problems whichshould be avoided, then he left, while the group proceeded. He then repeatedthe procedure, as he came back later and explained the mock-up phase, and�nally he did the same for the prototyping phase.

E.1.3 Place setup

The sketching session took place in the group room. The mock-up session tookplace in the Usability Laboratory in the Cassiopeia building.

E.1.4 Materials

Cardboard boxes, scissors, glue, tape, paper etc. was made available.

76

User interface innovation Chapter E

E.1.5 Process

The participants were divided into two groups of three members each, and therewas one facilitator during the mock-up and prototype phases.

E.1.6 The task

The workshop task description was the following: Try to imagine, that the

Danish Supermarket or Coop have asked you to design their new innovative

and interactive shopping trolley. The new trolley shall support the task of daily

shopping in a supermarket, but otherwise you are allowed to expand yourself.

E.1.7 Results

Both groups ended up designing a box, attached to the shopping trolley with abuilt-in display and some buttons. Both had a number of features in common,such as touch display, GPS technology, maps, online access or communicationability with the store, etc.

The lowest common denominator problem

As each member's collection of ideas is large in the sketching phase, the mock-up phase requires a reduction or assembly of ideas, cut down to one or twomock-up models. When the group is forced to make a decision, the democraticway is to �nd something that the majority �nds either acceptable or worse,something that is common in the majority of member collection of ideas. It canbe argued that this works very well in some contexts, as it can e�ectively �lterout some unpractical ideas and other nonsense. Recalling that the goal here isto collect innovative ideas, it shows that this might give problems by �lteringthe innovative and unique ideas away, thus leaving the group with the lowestcommon denominator.

Tendency to normalize

This problem is related to the lowest common denominator, it has to do withthe members being reluctant to con�icts, and therefore strive to �atten out theidea until everyone accepts it, in order to avoid arguments. In a creative processlike this workshop, one should strive to present and argue for ones ideas, andstill be willing to take a solution in a totally di�erent direction.

Homogeneous groups

This sort of problem arises when a group is so homogeneous that all membersthink in a similar way. There is limited variety in the ideas, and the group doesnot need much time to come up with a solution. The decision making does notrequire much discussions or argumentation between the members. This is notexactly a problem but might limit the creativity as the discussions often helpto inspire new solutions.

Lessons learned

77

Chapter E User interface innovation

In this session, the group:

• Sharpened the collective picture of what innovation is, and what it is not.

• Found out that the most common ideas among group members are distantfrom innovation.

• An agreeableness in the group does not contribute to innovation, it maydo the opposite.

E.1.8 Evaluation and Conclusion

The conclusion from this session contains a few suggestion to be taken intoaccount in the planning for next session:

• Disagreements is good, they trigger discussion and forces the participantsto articulate their ideas which may trigger the generation of new ideas.

• Freedom of expression is important, negative comments can drive the in-novation away and shut down the idea generation.

• In order to facilitate, �nd some way to reward all innovative ideas as soonas these arrive.

E.2 Pre-sketching

E.2.1 Process

The pre-sketching process is actually part of the sketching process, but we choseto separate it as a single process to make it an even more e�cient way to getour minds focused on our subject - Lego. We felt that this would help prepareus for the sketching process which was next.

The way the pre-sketching was carried out, involved the group members individ-ually writing down what words and phases they associated with Lego and theexperience of building with Lego - as many as possible. This individual processlasted for about half an hour.

The important thing about this initial part of the sketching process was thateach group member got an even chance to formulate their own thoughts. Thismeant that it was very important that no one talked during this process, toensure that nobody got distracted or in�uenced by other members of the group.

When everyone had �nished writing down their initial thoughts, we did a roundtable presentation in which each group member presented what they had writ-ten.

All these notes were combined into a single document, which meant that weended up with a pretty long list of things that characterises Lego and thingsthat people associate with Lego. To avoid duplicates we continuously combinedoverlapping words and phrases. We also tried to articulate cues into sentences,

78


to get a more precise formulation of the ideas. As already mentioned, eachgroup member presented his or her ideas, and if one or more thoughts matchedthose on the combined list, they were not added again. In several cases similarthoughts were combined into new thoughts.

E.2.2 Results

List of characteristics of Lego

1. Primært konstruktionsleg.

2. Kræver fantasi. En computer kan fx ikke bygge en Lego-model.

3. Grænseløst - fantasi, antal klodser, budget.

4. Kan kræve teknisk viden/forståelse/erfaring. Hvis man har en ide omhvad man vil bygge, kan det være svært at vide hvor man skal begynde.

5. Kan kræve forståelse af fysikkens "regler" (fx tyngdekraft).

6. Kræver inspiration. Man kan blive inspireret af klodserne. Rod kan giveinspiration.

7. Lego er en hands-on-experience. Noget man kan have mellem hænderne -man bruger sin følesans i stor grad.

8. Nogle Lego-elementer har en top og en bund.

9. Lego er et "prototypeværktøj". Man får aldrig bygget noget "færdigt"eller "rigtigt". Man kan altid fortsætte. I modsætning til fx en racerbanesom bare er én ting.

10. Lego er godt når man er �ere. Samarbejdsstimulerende. Hvis man harbygget noget hver især kan man lege sammen. Man kan få inspiration frahinandens modeller.

11. Lego er godt når man er alene. Hvis man er �ere kræver det at man bliverenige - hvis man bygger den samme ting. Man skal deles om klodserne -hvis man bygger forskellige ting.

12. Lego er et globalt "sprog". Alle kender Lego og ved hvordan det bruges.

13. Lego er sejt/genialt/dansk.

14. Antal klodser. For mange kan skabe forvirring og irritation (fx når manfølger en vejledning og har brugt en forkert klods). For få kan skabeirritation, men også tvinge én til at tænke kreativt.

15. Modellens størrelse bestemmer hvilke klodser man skal bruge.

16. Det er svært at lave en vejledning til en model.

17. Man kommer ofte til at bygge noget �ere gange. Hvis der f.eks skal brugesen ting til hver.

18. Man kan lave noget unikt.

79


19. Flow - succesoplevelser.

20. Med Mindstorms og Technic får man mere ud af legeprocessen, når maner færdig med at bygge. Modellerne er mere avancerede og kan �ere ting.

21. Det er en oplevelse at "gå på opdagelse" i Lego og opdage nye klodser.Det kan give nye ideer at man �nder ukendte klodser.

22. Ofte udstillingsmodeller. Holdbarheden er ikke altid så god.

23. Det kan være svært at lave helt om på modeller hvis man pludselig får enny ide. Man er ofte nødt til at starte forfra.

24. Lego kan blandes med andre ting.

25. Det kan være sjovt at smadre modeller. Fordelen ved at det kan væreskrøbeligt.

26. Klodserne er begrænsede. De skal passe sammen. Skal overholde nogleguidelines.

27. Man kan have for fede �ngre til at skille noget af eller samle det. Manskal have noget værktøj til det.

28. Man kan sætte Lego så hårdt sammen at det næsten ikke er til at skilleaf.

29. Når andre ser ens model, kan de komme med helt andre og bedre metodertil at bygge noget på.

30. Det er nemt at hjælpe hinanden i byggeprocessen.

31. Der er utrolig mange slags kloder, som man som bygger ikke kan have detfulde overblik over. Dette betyder at man ikke rigtig formår at bruge demi ens byggeprojekt.

32. Hvis man bygger noget, som man ikke er tilfreds med skal det skilles afigen, før man kan gå igang med et nyt projekt.

33. Nutidens modeller kan mere end i gamle dage, f.eks. biler kan køre medelektromotorer.

E.2.3 Re�ection and conclusion

The process of creating the combined list, helped us to get our minds focusedon Lego even further than us making our own individual lists. Presenting anddiscussing our individual ideas helped generating even more ideas, because peo-ple got inspired by other people's ideas, and similar thoughts could be combinedinto new more expressive points. The individual part was very crucial too. Ithelped kick start the discussion and ensured that everybody got a chance toformulate their thoughts.

In our pre-sketching phase we set no limit on how much time we had to comeup with association for each person's list, this resulted in some of the groupmembers �nishing considerably earlier than the rest. This might seem like the

80


intuitive solution seeing as though being interrupted and/or stopped in the pro-cess might kill some of the creative �ow. However, if the group were "on theclock" one might argue that the association's emerging after for instance 15minutes are not considered to be "�rst to mind" and might therefore not be asrelevant.

As already mentioned our primary intention with the pre-sketching process wasto help us get focused on Lego and thus help us in the following parts of theDesign and Innovation Workshop (DIW). We found that the pre-sketching wassuccessful in doing that. The fact that we had done the pre-sketching helped usto an easier and quicker start of the sketching process.

Furthermore the pre-sketching was a valuable tool in preparing us for the othersubsequent parts of the Design and Innovation Workshop. It also proved veryuseful to be able to revisit the list that we made during the pre-sketching, es-pecially because we had to take a break from the workshop, and needed to getour minds back on track when being able to continue the DIW. Hence, the pre-sketching process is a useful tool if you are forced to have breaks during theentire DIW process. A point that is important to remember if you know thatyou will have to divide the process into parts.

E.3 Sketching

E.3.1 Process

The process of sketching was done as described in the theory, with all membersof the group skeetching for themselves. When the process of sketching was done,the group presented their sketced ideas to each other. The way this was done wasby a member presenting a single idea without receiving any criticism, then thenext group member would present an idea from his or her sketchings remindinghim or her mostly of the idea presented by the previously group member. Thiscontinuing until no group member had any ideas left. After this each memberexpressed what new ideas had inspired them.

This was done to ensure the ideas was covered in themes rather than just in arandom order. To start the mock-up session, the group decided to split into two,from one group of six to two groups of three. This was done by each membersaying what idea of the others they found interesting, so they would not just allpick one of their own, they where more interested in further developing. Thisgave a pretty clear picture of what people wanted to work with and a way ofensuring that people from the start had a pretty clear picture of what theirgroup most likely would start out with.

E.3.2 Results

Descriptions of the sketches

#1

1. Adjusting the size of a brick by pulling its ends. Limited by the available

81


bricks - it is not possible to resize to a non-existing brick (this is primarilyveri�ed by the number of studs on the brick).

2. Zooming with two hands. Pulling your hands apart zooms in and pushingthem apart zooms out.

3. Rotate a brick with one hand. Holding a hand below the brick and twistingyour wrist.

4. Remote controlling of bricks. Each hand is mapped to a brick a handsmovement is re�ected on the brick.

#2Generally, these sketches are based on mass manipulation. The user control is aGun-like weapon, rather large with some buttons, and the user points the gunand basically shoots, in order to throw Lego bricks at whatever he is building.The bricks are self-tuning, and adjust and add themselves to the construction.

1. Here the User is building a brick wall, from a distance of a few meters,and the wall building process is semi-automatic, since the user does nothave to be very precise on details like how the bricks are attached to eachother, but in an intuitive way, the environment "knows" what he want tobuild, and assists him accordingly.

2. This sketch is about making a challenge out of the task of selecting a brick.The user is at a Tivoli-like shoot-out desk, where he has to do some sharpshooting, and actually hit the particular brick he wants to use.

3. The idea here is that the the user can grab a "roll" of ammunition, whichis a kind of random brick supply, but still with themes. An example: Theuser just built a house, and now he needs to decorate the garden. If �owersare something for his likings, he takes a roll with some random �owers indi�erent colors, and tada... he just "sprays" or the brick �owers over thegarden, giving an even spreading and smooth variety, without much e�ort.The argumentation is that this process would otherwise become tiresomeand repetitive.

4. This sketch is simply about the color of the brick construction, if the userchanges his mind about the colors he originally chose, he can use his gunto simply repaint the construction, in a spraying like movement, ratherthat changing the bricks out for ones with di�erent color.

5. This is a menu system idea, it is inspired from the movie Matrix, wherethe warehouse kind of comes to the user... here the user can either selecta di�erent type of "weapon" or grab some Lego bricks.

6. This one addresses the question about how one disassembles his Legoconstruction, and specially the case of an user who likes to destroy hisconstruction afterwards. In such a case, the user can �nd an arsenal fromhand grenades and small bombs, up to weapons of mass destruction. Theidea is that the user can then enjoy to watch his construction being blowninto the air (luckily there is no need of a cleanup afterwards).

82


#3The sketches are

1. Block cloud. When the user need inspiration on selecting the next brickfor what she is building. She can get an "cloud" of random bricks, this isdone to simulate the feeling of digging through a pile of bricks. This alsoshows the menu system generally used in the rest of the sketches. It is a�oating window looking like a window i a WIMP interface. The window isconnected to the cloud with a line. The user select items from the menu,with her pointer, which is the is a line, sort of like a laser, pointing out ofthe wand.

2. Morph menu. On every brick there are morph options, like changingcolor and size and so on. This is an connected window menu as in sketch1. This makes it easy for the user to change the bricks after they areselected from the brick placeholder, so she is not forced to select newbricks.

3. Block snapping. Then bricks are put together they are drawn to eachother, even then they not touching, and then they are snapping togetherthey make a sound. This lowers precision needed for navigation in the 3Dspace.

4. Chopstick handling. This sketch show how to handle a brick or model.The best way to describe is imagine that there are a chopstick betweenthe model a each wand. The chopstick is connected to the model andthe wand by a hinge. When the user moves the wands indepently the themode is moved the same way. Then the wands are moved away from eachother in the horizontal axis, the model or brick are changing size.

5. Exploding/imploding model. To make it possible to change one orbricks in a model, the user has the option of a controlled explosion of hermodel, in such a way that bricks are moved slightly away from each other.Then she is done, the model can be imploded back together.

6. Model cloud. The user can save her models, ad build on them later.The model can be selected from a cloud of models.

7. Un-Lego. It is possible to Un-Lego an model, which basically meansremove the characteristic of a Lego model. That is the lines between eachbricks, and buds on top the brick. This is done to make the model more"real" like a 3D model from a 3D animation program like 3D Studio Max.

8. World rotation. Then the user has a lot of objects in the 3D space,like models, bricks and menus. It should be possible to rotate the worldaround her. So all spaces, both space around her, plus the �oor and ceilingcan be utilized for objects. So to interact with the object under the �oor,the world can be rotated, so the objects are in front of her and therebymore easily accessed.

#4

83


1. Build-and-Paint-concept to develop artistic creativity. The user can buildwhatever he likes of bricks and paint the surface after his behavior or hecan select a miniature �gure which he can build after. The user can easilyextend or reduce bricks by selecting them and pushing/pulling the handsapart.

2. The Labyrinth. The main idea behind this is to overcome obstacles andnavigate a brick through a labyrinth. You can control the brick by handmovements which is re�ected on the brick - fx. by rotating a brick with ahand and place it in a speci�ed location the other hand.

#5We have to be aware of when interacting with Lego the user relays much onthe feedback he receives through touch when interacting with it. Is it possibleto copy some of the movements from building Lego in the RW to VR?

The approach to interacting is that it should be as simple as possible, so theuser can adapt to this environment easy and quickly, this can again be donewith using a mapping close to 1:1.

• Use some physical object to emulate the virtual objects in the CAVEduring interacting.

• Use a VR-glove and a tracker combination for gaining this interacting.

The sketches

Figure E.1: Sketches #1 -1


84




Figure E.5: Sketches #1 - 5

85


Figure E.6: Sketches #1 - 6




86





87




88





89





90





91






92





93




94




E.3.3 Re�ection and conclusion

Our �rst impression of the sketching phase is that it clearly does help creatinga lot of ideas. People have di�erent ways in expressing their meanings, andtherefore the results of the sketches will therefore also di�er and none of themmight be the same type of sketches.

The process of expressing ideas by drawing them is a concept everybody cancomprehend, further more when discussing afterwards no one felt the need toexceed the twodimensional space of the pen and paper to express ideas thatwere to be implemented in three.

A thing to be noted is also the concept of quantities versus qualities. Thedetails and functions of each idea/drawing may vary a lot, so producing a lot ofdrawings is not necessarily more creative than just producing a few.

95



E.4 Mock-up

E.4.1 Establishing the subgroups

There were two subgroups created for the mock-up phase, and in order to min-imise potential con�icts, there was an agreement on that the groups should beself-established.

This was conducted by scheduling a 15 minutes discussion session, where thegroups were to discuss the emerged ideas from the sketching phase, and there-after give statements about these ideas with argumentations about what theyliked, and what not. Then the subgroups where established, based on whatideas the individuals desired to work on. This resulted in the two subgroups;one group which would focus how an interactive menu should work in an immer-sive 3D environment, and the other group which would focus on how to interactwith objects and how to manipulating them in this environment.

The mock-up session took place in the Usability Laboratory in the Cassiopeia

building.

E.4.2 Process (Menu system group)

The group work started with discussion and negotiations about which of theideas from the sketching phase should be carried on into the mock-up phase.Eventually there was decided to merge two main ideas into one. A whiteboardwas used for writing down some headwords, as well as drawing sketches to co-ordinate the idea generation. Additionally there were the sketch sheets fromthe sketching phase for further details and explanations. Early in the process,a decision was taken not to let limitations, such as the available materials tobuild from, a�ect the level of creativity.

Soon, a clear common understanding became presented in a sketch of the mock-

96


up on the whiteboard and the mock-up realisation could begin. It soon became

Figure E.36: Mock-up session: Real size interactive menu

clear that the group was facing some serious limitations. The idea of a menusystem for the immersive 3D environment required the mock-up to be two me-ters high, and it proved unpractical and di�cult to build from the materialsavailable. There was decided to skip some important details from the mock-up,like the ability to duck down and see what is in a drawer. These details werechosen based on how easily they could be explained in words, since the goalwas to become able to present the idea behind the mock-up to others. The endproduct required a signi�cant number of resources (�ve or six people) to holdit in place.

E.4.3 Re�ection and conclusion (Menu system group)

Although mock-up is meant to be quick, cheap and easy, we experienced dif-�culties. We accepted the limitations of the materials at hand, and used thewhiteboard combined with gestures and discussions to create a common visionof the physical appearance and basic functionality of the design. This workedwell, and it proved su�cient to have a simple cardboard and paper mock-ups,in order to support discussions and explanations, and thereby create consensusabout the design between the group members.

E.4.4 Process (Object manipulation group)

After being split into two teams the one concerning themselves with directlymanipulating bricks started their process by discussing whether or not to use

97


the standard equipment available in the CAVE. The choose to use the equip-ment because the concensus was that they could achieve all they wanted withthe so called wandsE.5.6.

The team only created very simple mock-ups E.4.5, because they were not de-veloping a new psysical object but an interaction method.

When faced with a relative large portion of time, the team used their time tomake up an process how the simple mock-ups would work, by doing so, theycrossed over to what essentially should be the prototyping session.The group questioned every idea in a way so all details were �rmly explainedand all shortages were located. Throughout the process there were made ane�ort to keep the focus on developing for a 3D enviroment.

E.4.5 Results (Object manipulation group)

In the mock-up session our project team was split in two, with one group lookingat how an interactive menu should work in an immersive 3D environment, theother group would look at how to interact with object and how to modulatethem in this environment.

Garry's Mod

Under our mock-up process we were inspired by an modi�cation to a well-knowncomputer 3D game called Half-life, this modi�cation "Garry's Mod" o�ers mock-up a highly developed tool to modulate the games environment with the inter-action through use of mouse and keyboard.

Figure E.37: Garry's Mod: Manipulating an object in the environment

The �rst discussion was about whether or not to use the standard interaction

98


equipment available in the CAVE. For stimulate our further progress in the ses-sion we created a big Lego brick, this Lego brick gave us more inspiration tohow we would develop the manipulating device and how to interact with thevirtual bricks.

Figure E.38: Mock-up session: One big red Lego brick

The approach to taking the decision was done by focusing one more concreteproblem in our project namely how to build with Lego in an immersive 3D en-vironment, one of the problems with this idea is the fact that Lego is a productthat relies much on the physical contact that it o�ers the user, as we also saw inthe picture scenario session9. The ability to touch, turn and interact with thedi�erent objects in Lego is crucial, therefore is necessary to develop an interact-ing device that can compensate the user for this losses. As we have learned thatthe physical contact to a physical object can be emulated through a tool[9], sothe user do not feel compelled to interact directly with the emulated object, theusers senses will be satis�ed by the feedback he receives through vision alone.

For this to work there is some rules for the environment that you will have totake in consideration, both gravitation and solid mass are important for howthe object will behave and how the user will perceive it.

With this in mind the further work was done with the focus on emulating movesfrom the real world thereby creating a 1:1 mapping, as the session proceeded itbecome clear that in this case was done by using two trackers. With the inter-action device in place the attention again turned to how one should manipulatethe actual object in the environment with the two trackers. The idea is thatyou can interact with the application through these trackers with one in eachhand, not only by pressing the buttons, but also simply by moving them aroundbecause of an build in tracking sensor.

99


Figure E.39: Mock-up session: Interacting with an object

List of functions for the trackers:

H1While pointing at an object it is possible to �re a beam by pushing H1, whileholding down H1 it is possible to move the object.

V1By pushing and holding V1 and then move the control closer to your body, theobject will be moving closer towards you. By moving the control away fromyour body the object will be moved away from you current position. The fasteryou move the control the faster you move the object.

V2By pushing and holding V2 and then tilting the left control it is possible to tiltthe object, here we have a direct mapping from the control stick in your handto the chosen object.

V3By pushing V3 the object will change color, by pushing it again the color willchange again. Therefore it is possible to cycle through all the base colours.

When an object has been moved, it is also selected when you let go of H1, theobject stay highlighted and the modi�cation dots appear on the object.

H1By moving the right control inside one of these modi�cation dots and pushingand holding H1, it is possible to alter the object size at one of the three axis.By moving the control into another dot it is possible to alter the object at thenew selected axis.

100


When an object has been placed onto another object, these objects form a group.It is possible to e�ect a group of objects, in the same way as a single object, inthe way of moving as described above.

If you move the right control close to a sub-object in a group of objects it ishighlighted, and thereby it can be removed from the grouping, if it obey someconstruction rules.

H2 and V2Zoom function can be invoked by pushing and holding H2 plus V2 and movingthe controls against each other.

H3By pushing and holding H3 at the same time, you will be drawing a circle inthe air with the right control. This enables you to mark some of the sub-objectsin a speci�ed object. All the sub-objects you see through the circle when doingthis gesture will be marked.

V4By pushing V4 you will select the current highlighted object for multiplication.

H1By pushing H1 you will create a new object.

H2By pushing H2 or H3 you will deselect the option for multiplication.

H4By pushing H4 you will enter the upper menu.

E.4.6 Results (Menu system group)

This chapter describes the ideas regarding a menu system in a 3D environment.It was based on ideas from the sketching phase, along with other inspirationand knowledge. Some of the knowledge was gathered from articles[9].

One inspiration was a gun shelf rack in The Matrix1. The problem with thattype of rack is that not all of the space is used around the user, and in orderto access more items on the rack, the user has to move, which is di�cult in theCAVE as it has a 2.5x.2.5 m �oor. The idea of the rotating world which is seenon E.40, solved that problem, but Doug A. Bowman et al.[10] conducted someexperiments showing that jumping, the act of teleporting the user to an otherplace in the 3D space, is bad for spatial awareness. The user needs to �gureout where he is, just after all of the surroundings has changed. So a menu thatpops in and surrounds the user, might be an bad idea. Another problem wasthat the user has objects outside his visual �eld, that might be of interest.

1In the movie "The Matrix", where the hero needs guns in a training session in the virtualworld, a shelf rack is called upon, rolles in behind and in front of the hero, and is �lled withguns

101


To solve these problem the "snake like" surrounding menu was suggested. The�rst sketches drawn on a whiteboard can be seen at �gure E.40.

Figure E.40: Mock-up session: Surround menusystem

102


E.4.7 Re�ection and conclusion (Object manipulation group)

Our experience has shown us that the mock-up and prototype sessions are veryclosely related to each other. It was very clearly seen during the mock-upprocess for the object manipulation group which, as mentioned earlier, endedup overlapping the prototype session.

This is not speci�c to mock-ups in virtual environment, but it is probably moredistinct in the case of a virtual environment. At least that is our experience.It was challenging to create mock-ups of how to interact in a three-dimensionalenvironment.

The fact that the mock-up and prototype processes overlapped like they did wasnot a bad thing. In a way it glued the two parts of the Design and InnovationWorkshop together to one larger part, but we still felt that this one part serveda purpose and we did not feel that it was any barrier in completing the workshopas a whole.

Trying to enforce a greater distance between these two parts of the DIW mighthave made the process more time consuming and more di�cult for us, becausewe had to have very strict boundaries on our mock-ups. Hence, we might haveused more time ensuring that we in fact did not do any prototyping during themock-up process.

E.5 Prototyping

E.5.1 Process (Menu system group)

The session started with discussions, there were mainly two issues discussed.

Details of the interactions with the system:

There was an agreement on to keep the interactions simple, and it was easy forthe group members to plan the interface for the menu system.

The size of the prototype:

There was some confusion in the group about how to plan the prototyping ses-sion, as the problem with the size emerged again, the main problem was how togive the tester a realistic experience during the prototyping, as the size of therack was about two meters high while its length about �ve meters. This prob-lem was discussed back and forth. Everyone agreed upon that the prototypingshould be focussed on the rack, since it was the most unique experience, andthis part of the system was the one considered to be the most interesting onefor a usability experiment. Eventually we agreed upon conducting the usabilityexperiment, solving the size problem by borrowing extra people from the othergroup, so that we would have the required �ve participants to operate the pro-totype.

Usability experiment:

103


The last part of the session was a usability experiment. Since the physical pro-totype lacked so many of it details and functions, and the fact that we neededsome extra personnel from the other group, we decided to start the experimentsession by doing a presentation for the other group, to both explain our ideasfor them, and prepare them for playing their roles in the model representation.Then we invited one of the other team to play the role of a user, testing thesystem. First, the controls were explained for her, and then she was given aspeci�c task of �nding a particular item in the rack system, and then retrievingthat item, using the controls. At �rst, the user unexpectedly started foolingaround with the system, which was a quite humorous, as this required a bunchof people to be running around, back and forth, responding to the user input.It proved that this was seemingly a natural way for the user to get the feel ofthe controls. After a short period of time, she started on the actual exercise,having already gained a good sense of how to operate the controls, she executedthe given task with ease.

E.5.2 Re�ection and conclusion (Menu system group)

The usability experiment went surprisingly well. The user was able to fullygrasp the idea, and seemed to be comfortable and relaxed while using the sys-tem, despite how limited the physical prototype of the rack system was. It wastherefore concluded that the verbal explanations along with whiteboard sketchesand presentations, did compensate for a simple prototype model. The fact thatthe user showed enjoyment and was playing with the system, con�rms this andsuggests as well, that the role of a game could be an important part when oneis familiarizing with a prototype.

E.5.3 Process (Object manipulation group)

The group started the prototype session with a discussion and negotiation abouthow the testing of the interaction should be held and which ideas from the mock-up phase could be carried on to the prototype session.

The conclussion of the discussion was that it would be useful for the group andhelpful for the user if the test was a tutorial-like usability test of the interaction,with the option to get help from a �gure depicting a Lego-man. The tutorialwas mainly chosen because it could be di�cult for a new user to �nd out andkeep track of the di�erent functions the buttons on the mock-up model mayhave.

For the usability test, some basic interaction ideas were selected from the mock-up phase because it with these was possible to demonstrate the interaction bymoving the Lego bricks. The interaction ideas were formed into tasks which theuser had to accomplish.

Four people were involved in the actual usability test. There was a user; a test

104


leader who had the responsibility to give a simple introduction of what was goingto be tested and to introduce the tasks one by one; a Lego-man whom as a partof the tutorial-like fashion had to come up with some information about howa task could be solved if needed; and a person who stayed in background andhad the job to give the user the right interaction feedback during the test whenthe user pressed the buttons. Since there was no actual program, it was a bitdi�cult to stimulate the interaction output. In compliance with this problem,the user was asked to think out loud when clicking on the mock-up model, sothe person could react and move the Lego brick after what was being pushed onthe buttons. It should look like the hand movements was mapped to the brick,so the interaction was re�ected on the brick.

E.5.4 Re�ection and conclusion (Object manipulation group)

We learned that trying to simulate a virtual environment in the real world isa pretty challenging task. We had a lot of ideas on how the prototype shouldfunction if it was implemented in the CAVE, and we tried to create some ofthese using humans for a lot of the tasks that would have been done by thecomputers - like moving the "virtual" Lego brick.

This is not unique to prototypes in virtual environments. Creating e.g. a paperprototype of a piece of "classic" computer software will often also involve humanscarrying out tasks of the computer, like switching what is on the screen.

The extra people might be more visible in a virtual environment. In our case wehad to use several persons to move our mock-up of a Lego brick when the usersignalled to do so, and this of course led to these person being in the users �eldof vision during much of the prototype session. One thing that is unique to, orat least more likely to have an in�uence in, a virtual environment is the factthat the people performing the tasks of the computer are often times �xed totheir positions. Often times they can not just disappear, because of the natureof the 3D environment.

So one might think that the fact that we had people wandering about in our"virtual" environment was a bad thing for our prototype session, but in ourexperience it was not. The users were not bothered by extra people present. Aprototype is not a �nal product, and hence it is not expected to function exactlylike a �nal product. In some cases it might not function at all, if it is merely avisual prototype. The point of the prototype is to improve the product before�nalising it. Our prototype did function, not like a possible �nal product would,but it functioned well enough for us to get some valuable results.

Another fact that supports us in saying that the prototype session went well, wasthat we gained results from the experiment. Results of the kind that resemblesof a more regular usability test.

105


E.5.5 Why is the Design and Innovation Workshop inno-vative?

As mentioned by Jeremy Rose in the SWI course on Aalborg University[26]there are di�erent innovative development models. The workshop used by us inthis project can be categorized as a mix of two of the models mentioned in theSWI course.

These two models are the linear (light bulb) model and the network (or com-munity) model.

The linear model is described as a sequence of stages or phases, in a way like thewaterfall model. This �ts our workshop process because it consists of a numberof subprocesses which we carried out one by one - one after each other. Eachsubprocess was dependent on the previous process(es) for it to be completed.

The other model, the network model, involves "the conjunction of people, ideasand expertise" and "co-operation and competition". This also matches ourworkshop process. Most of the subprocesses of the workshop involved multiplepeople (the group members) working on and discussing the di�erent matters ofthe di�erent subprocesses. This automatically lead to sharing of ideas and alsopushing boundaries in some ways. The pushing of boundaries mostly involvedgetting new ideas and discussing the di�erent possibilities to solve the varioustasks. With regards to "competition", it can be argued that we did some de-gree of competition during the mock-up and prototype subprocesses. We didnot compete in the traditional sense. There would be no winner, and no teamwould be better than the other. But we did split into two separate teams whichworked on separate projects simultaneously. Working in teams naturally alsoimplies co-operation.

The way in which our Design and Innovation Workshop di�ers from the net-work/community model, is that the network or community in our process wasrather limited - i.e. it was the seven group members. Traditionally examplesof the network model are open source communities, which are often times verylarge communities spread across the entire globe, and obviously involving morepeople who might come from a variety of backgroundscan help igniting thesharing of ideas and expertise.

106


E.5.6 Manipulating a brick

The prototype session provided us with some useful results. As mentioned wedecided to carry out the prototype session in a tutorial-like fashion. This turnedout to be a very useful approach. Because of the innovative nature of our in-teraction methods, it was very helpful for the user to be able to get preciseinstructions regarding the current task. The way the help was provided, i.e. the"Lego man" which was activated by the wands, was also successful, it blendedwell with the rest of the "virtual world". Also, the fact that the help was un-itrusive, you had to activate it, proved very successful.

All the members of the sub-group that worked with creating the brick manip-ulation prototype, were right-handed. Hence the interaction methods were alltargeted towards right-handed users. The user, however, turned out to be left-handed. This turned out to be a completely over-looked factor which actuallymeant that the interaction was hindered. Ways to interact which seemed naturalto a right-handed person, were reversed for the left-handed user. This problemcan rather easy be solved by allowing the wand mappings to be reversed, suchthat the left-hand wand will be the right-hand wand and vice versa.

Another problem that was found was the logic between the di�erent buttons andthe di�erent functions that were mapped to them. The user expressed confusionabout the fact that some functions that were related in some way were mappedto di�erent buttons. Speci�cally it was mentioned by the user that the moveand zoom functions with advantage could be using some of the same buttons.

E.5.7 Menu system

The idea testet was the rack menus system, previously described. The usabilitytest gave both new ideas and located some problems with the existing ones.Both during planning, testing and evaluation of the usability test.

An idea that came up during planning of the test was an sca�old menu system.Why limit the "menu" to a rack, with the in-build problems like not beingable to see trough the parts of the rack (and the draws). A solution could bedesigning the rack like a sca�old with skrewer where the blocks are placed.

Durring the test the user pointed to a problem: You can not see the constructionyou are building when you reach for other blocks in rack. When you initiate themenu, the rack can stand in way of other objects like the model, you can movethe rack, out of the way, or turn around, but you as a user has do somethingactive to achieve that. Durring the test a few new ideas came up.

The original idea for selecting blocks was to use "click and hold", but it wassuggested to use "click to attatch" instead. When pointing to a block, clickwould attach the block to the pointer, no need to hold down the key, while theblock is attatch. That would also give the possibility of using the button forother things while the block is attach to the pointer. E.g. releasing a copy ofthe block, to be used in the model.Another idea that surfaced, was for drawing the skrewer. When you "click and

107


hold" to a block attached to a skrewer i the sca�old, and pull you pull out theskrewer, making it posseble to see the other blocks attached to the skrewer.

After the test an idea of "collision fadeout" was formed. To make it easy to seeall block in the rack, you can put your head inside the rack, and there will be aninvisible sphire around your head, in which objects will fade-out. In that wayblocks close to you will disappear, so blocks further away will be visible.

Figure E.41: Prototyping session: Rack menusystem

108

Bibliography

[1] Teresa Amabile. Creativity In Context. Westview Press, 1996, ISBN: 97-808-1333-0341.

[2] Michel Beaudouin-Lafon. Designing interaction, not interfaces. 2004.

[3] Jason Burke. Ethnographic methods, 2001.

[4] Bill Buxton. What sketches (and prototypes) are and are not. 2004.

[5] Universities & Skills Department for Innovation. Innovation nation. March2008.

[6] Alistair Sutcli�e et al. Investigating interaction in cave virtual environ-ments. 2003.

[7] Anne Rose et al. Using ethnographic methods in user interface re-engineering. 1995.

[8] Antti Oulasvirta et al. Understanding contexts by being there: case studiesin bodystorming. 2002.

[9] Daniel F. Keefe et al. Cavepainting: A fully immersive 3d artistic mediumand interactive experience. 2001.

[10] Doug A. Bowman et al. Travel in immersive virtual environments: Anevaluation of viewpoint motion control techniques. 1997.

[11] Dr. KRS Murthy et al. Brainstorming techniques. December 2008.

[12] Paul A Mongeau et al. Brainstorming. Winter 1999.

[13] Robert J.K. Jacob et al. Reality-based interaction: A framework for post-wimp interfaces. 2008.

[14] John Hughes et el. Moving out from the control room: Ethnography insystem design. 1995.

[15] Poul Kyvsgaard Hansen Frank Gertsen and Harry Boer. Innovationsledelse.2006.

[16] Frank Gertsen. Introduktion til innovationsledelse - set fra en virk-somhedssynsvinkel, 2008.

[17] Alan R. Hevner. The information system research cycle. 2003.

110

User interface innovation Chapter

[18] Janne Jul Jensen and Anne Marie Kanstrup. Uim course website. http:

//cs.aau.dk/~jjj/UIM/UIM.htm, 2008.

[19] Anker Helms Jørgensen. Etnogra� og interaktionsdesign, lek-tion 6a - interaktionsdesign. http://itu.dk/people/anker/IDF2008/

Lektion-6a-Etnografi.pdf, 2008.

[20] Anne Marie Kanstrup. User innovation management chapter 1. http:

//cs.aau.dk/~jjj/UIM/kap1.doc, 2008.

[21] Alexander Massey. The way we do things around here': The cultureof ethnography. http://www.geocities.com/Tokyo/2961/waywedo.htm,1998.

[22] David Millen. Rapid ethnography: Time deepening strategies for hci �eldresearch. 2000.

[23] Alex F. Osborn. Applied Imagination: Principles and Procedures of Cre-

ative Problem-Solving 3rd Edition. Creative Education Foundation, 1993,ISBN: 978-0-930222-73-4.

[24] Sonja Pedell. Picture scenarios: An extended scenario-based method formobile appliance design. 2004.

[25] Sonja Pedell and Frank Vetere. Visualizing use context with picture sce-narios in the design process. 2005.

[26] Jeremy Rose. Three models of software innovation. http:

//cs.aau.dk/~jeremy/software_innovation/si_2008/resource%

20files/lectures/science%20of%20innovation.pdf, Spring 2008.

[27] Sean Silverthorne. Lessons not learned about innovation. 2006.

[28] Mikael B. Skov. Design & innovation, inf3, slideshow. 2008.

[29] Carolyn Snyder. Paper Prototyping. Morgan Kaufmann, 2003, ISBN:1558608702.

[30] Robert J. Teather and Wolfgang Stuerzlinger. Guidelines for 3d positioningtechniques. 2007.

[31] Wikipedia. Cave automatic virtual environment. http://en.wikipedia.org/wiki/Cave_Automatic_Virtual_Environment, 2008.

[32] Wikipedia. Data mining. http://da.wikipedia.org/wiki/Data_mining,2008.

[33] Wikipedia. Virtual reality. http://en.wikipedia.org/wiki/Virtual_

reality, 2008.

111

http://cs.aau.dk/~jjj/UIM/UIM.htm

http://cs.aau.dk/~jjj/UIM/UIM.htm

http://itu.dk/people/anker/IDF2008/Lektion-6a-Etnografi.pdf

http://itu.dk/people/anker/IDF2008/Lektion-6a-Etnografi.pdf

http://cs.aau.dk/~jjj/UIM/kap1.doc

http://cs.aau.dk/~jjj/UIM/kap1.doc

http://www.geocities.com/Tokyo/2961/waywedo.htm

http://cs.aau.dk/~jeremy/software_innovation/si_2008/resource%20files/lectures/science%20of%20innovation.pdf



http://en.wikipedia.org/wiki/Cave_Automatic_Virtual_Environment

http://en.wikipedia.org/wiki/Cave_Automatic_Virtual_Environment

http://da.wikipedia.org/wiki/Data_mining

http://en.wikipedia.org/wiki/Virtual_reality

http://en.wikipedia.org/wiki/Virtual_reality

user innovation interface

Documents

research project

project report

introducing chapter

project deals

project group

innovation workshop

report pages

user innovation management