1

Protosketching: Sketching in Experience Prototyping Ilpo Koskinen, Jussi Mikkonen, Kaj Eckoldt,

Riikka Hänninen, Jing Jiang, Benjamin Schultz School of Design, University of Art and Design

Hämeentie 135 C, 00560 Helsinki, Finland Firstname.lastname at taik.fi

Katja Battarbee, Jane Fulton Suri IDEO

100 Forest Ave, Palo Alto, CA 94301, USA firstname.lastname at ideo.com

ABSTRACT In this paper we introduce the concept of protosketching. It is a set of techniques for exploring user experience simultaneously with design solutions with empathic techniques like Make Tools, but also with technical means like Arduino and Lego Mindstorms. Protosketching is particularly suitable for designing embedded systems in which one has to simultaneously define physical prototypes and dynamic interactions in response to user behaviors. We illustrate the term with an example from IP08, an advanced design class held in Spring 2008 at UIAH, Helsinki.

Author Keywords Sketching, protosketching, experience prototyping.

ACM Classification Keywords H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous.

INTRODUCTION This paper introduces the notion of protosketching, or sketching by prototyping, situates it to an interaction design process for embedded systems, and gives examples of techniques for such sketching. Specifically, we focus on embedded systems rather than interface or Web design.

Myers et al. have recently made an important observation concerning interaction design. They point out that sketching is crucially important design activity [7]. Experienced designers sketch things extensively to understand and explore design problems before starting focus on detail design, thus avoiding making wrong and expensive commitments before ideas are mature. They also point out that even though senior designers know this fact, very little literature exists for sketching interactions. For example, they note that even though Buxton [4] has recently written

an entire book about sketching, he uses only two pages for sketching the behaviors of systems, and gives few hints about how to sketch these behaviors [7]. Sketching is important, but our conceptual understanding is wanting.

However, literature on prototyping contains many sketching-like activities that go beyond just visual sketching. For example paper prototyping and quick interactive hyperlinked demos are used to do simple evaluations of interactive behavior [11]. However, most of these examples are around sketching a software interface on a screen, big or small, and have limitations for dynamic behaviors. Other examples may include acting scenarios with mock-ups in hand [see 10]. They function far less well when designing embedded and tangible systems. Then one has to simultaneously define physical prototypes and dynamic interactions in response to user behaviors. One more workaround is producing videos to demonstrate intended, complex and dynamic behaviors - but they remain static communication pieces, and cannot be easily iterated upon.

There is one prototyping tradition that specifically surpasses the boundary of interaction, physical design, and user experience, namely experience prototyping, which aims as understanding existing user experiences and context, exploring and evaluating design ideas, and communicating ideas to an audience. [3]. As Buchenau and Fulton Suri observe, "through Experience Prototypes of these artifacts and their interactive behavior we are able to evaluate a variety of ideas — by ourselves, with design colleagues, users or clients — and through successive iterations mold the user experience." [3: p. 428] Importantly, experience prototyping proves that prototyping is not bound to paper or the computer screen, but is a complex activity. This paper calls it protosketching, and illustrates it with an example from embedded systems.

SKETCHING IN EXPERIENCE PROTOTYPING Protosketching consists of a set of techniques for exploring user experience simultaneously with design solutions. Protosketching is particularly important in designing embedded and interactive systems. In designing these systems, one has to simultaneously define the prototypes and behaviors, which requires intensive collaboration between designers and prototypers and a high degree of

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. CHI 2009, April 4–9, 2009, Boston, MA, USA. Copyright 2009 ACM 978-1-60558-246-7/08/04…$5.00

technological sophistication in the prototyping/design team.

As the name suggests, protosketching stresses sketching and aims at improving designers' understanding of interaction by making human-system interaction tangible and thereby observable and later reportable. It involves using technology that is not good enough for production or even building an actual prototype, but enables observation, bodystorming, role-playing, video analysis, and other experience prototyping techniques [3].

Protosketching thus takes experience prototyping a step towards interaction design. It involves building rough but functioning prototypes for evaluating user experience before getting into more technical prototyping. It involves three activities in an iterative cycle that takes place after concept design before actual prototyping (Fig 1):

(1) Gaining Understanding through Empathic Techniques. To gain empathic understanding of user data, fuzzy ideas are illustrated with readily available materials and tools familiar from experience prototyping and other empathic design techniques [see 3, 9];

(2) Technology sketching. The second activity involves building a tangible protosketch. This illustrates the interactive behaviors of the concept, makes it researchable, communicates it to stakeholders, and informs technical design. Technical design involves issues like flowchart, block diagram and CPU design, selection of sensors and actuators, and searching for control values for the system;

(3) User evaluation. Users and designers interacting with the protosketch to evaluate user experience before making decisions about going into actual prototyping.

Fig 1. Protosketching in design processes

The first part, gaining empathic understanding, links protosketching to user study, while technology sketching carries this understanding to subsequent technological work. Protosketching is a crucial link in turning designers from thinking in human terms into thinking in terms of human.-machine interaction.

IP08 IP08 is a 9-week design class that teaches prototyping interactive technology. In this class, held annually at the School of Design in Helsinki, MA level industrial design students go through user-centered design process over nine weeks. Students have to create a design concept, learn the basics of microcontroller (ATmega8535), elementary programming in C, and refresh the basics of electric circuits. The class assumed no knowledge beyond high school physics. The class had seven students.

The basic design problem was interaction in the car and safety while driving. Talking between the front and the back seat is a major road safety issue [8]. The class turned this problem into a positive direction with the notion of co-experience [1]: how to make this interaction fun while safe.

An old 1989 BMW was bought for the class. The car was in register, and in full driving condition, but old enough to be rebuilt in the studio devoted for the class. Being located in studio context of a design university meant that students working on the car had an easy access to variously shaped and sized fellow students for user tests whenever they wanted. (Picture 1).

Picture 1. Studio Space for IP08

The first two weeks were used for lectures, learning technology, and for a brief user study. The user study consisted of observations, in-situ interviews (i.e. interviews were done in family cars), acting out situations for the camera, and bodystorming in the studio to understand issues like ergonomics, how gaze functions, and ergonomics. Then, students processed data using standard techniques like affinity walls, personas, and scenarios [2, 5, 6]. The class spent one week in concept design and in refining concepts. The class went on to 3 weeks of protosketching before constructing final prototypes using the microcontroller and standard industrial components.

PROTOSKETCHING IN IP08 Students worked in three groups, two having of two members, one having three. Each group created three

3

concepts, but only four were chosen for further development. Sometimes these concepts were a combination of several initial ideas. Three concepts are described below.

Bugbugs. User study showed that children want to know what's happening outside the car, and remaining distance to destination. Also, children get restless in the car; it would be good to offer physical activity that does not affect driving. These observations were turned into a game that gave children in the backseat an opportunity to experience the road without the driver’s active input. The steering system was rewired, making it functions as a sensor; actuators were small plastic yoyo shaped cones with LEDs and proximity touch sensors; a game motor calculated the result; and a screen showed the results. Bug-like cones lighted up in the backseat area depending on speed direction, points came from almost touching the cones, and points were lost upon touching or missing the cone.

The Car Body as an Instrument. This idea was based on the notion that people often tap various parts of the car. The idea was to turn the car into an instrument that the whole family could play by placing various types of sensors into the seats and other interiors in the car.

Interactive mirror. In the third concept the user study showed that particularly with kids face-to-face communication is important and there is no substitute for it in the car. However, when communicating with kids on the backseat, parents lost sight contact with the street ahead. Intelligence was built into the rear view mirror, using protosketching to search for suitable ranges of behaviors for the mirror.

Gaining Empathic Understanding At this stage, protosketching consists of using low-tech techniques, including scenarios, creating personas, role-playing, Make Tools, and bodystorming in the studio environment to internalize the findings from user research. For example, in Mirror, empathic protosketching sought to answer how it feels to communicate via a screen versus a mirror? The group first connected a small LCD screen to a video camera. Secondly it built a real two-way video communication based on Skype and laptops. The experience gained from this study lead to an elimination of the screen and a focus on to the mirror. (Table 1).

Bugbugs Body Mirror

Scenarios, role playing, Make Tools, personas

Scenarios, role playing, personas

Role plays, acting out situations, bodystorming,

Table 1. Empathic sketching techniques

Technology sketching Students built interactive representations of their concepts using a variety of technology sketching techniques suitable for designing embedded systems. All these sketches were

built into the car, and were designed so to study how co-experience would take shape if such system were built into the car. All sketches were flexible and easy to change when some line of inquiry proved to lead nowhere. (Table 2).

For example, in Car Body, technology sketching involved buying cheap electric musical toys, rewiring them, searching for optimal sensors and sensor placements, and finally rewiring the stereo system of the car so that instead of listening to the radio, a family could play music together with the car. In Bugbugs, technology sketching consisted of videotaping bodystorming and role-playing sessions with Make Tools, leading students to refine their concept into a do-not-touch game.

In Mirror, LEGO® Mindstorm® NXT was used for quickly iterating embedded systems. How is the experience if the mirror starts to act autonomous, interacts via gestures or simply reacts to button commands? To investigate on the behavior we used Mindstorm® NXT as an interactive skeleton holding the mirror. It allowed iterating, communicating and evaluating with various stakeholders who took it as a sketch and could not by mistake take it for final design. The flexibility allowed an agile development process and in-situ tests.

Picture 2. Up: from user study to concepts and Mindstorms. Middle: Non-technical sketching by using a small LCD screen

and video camera as a proxy for a mirror and iterating the interactive mirror concept in-situ with Mindstorms in Mirror.

Down: hacking toys in Car Body and testing Bugbugs.

Bugbugs Body Mirror

Mindstorms, taping role play and bodystorming sessions with mock-up sensors

Hacking toys, Mindstorms, marking sensors spots with tape for bodystorming

Building concepts with Mindstorms, using video screens as a proxy for a mirror

Table 2. Technology sketching techniques in IP08

User Evaluation during Protosketching Protosketching enabled bringing fellow students and visitors to the car to get feedback. Feedback focused on user experience; sensor selection and placement; finding proper values for sensors; refining the interactive behaviors of the system; and issues like ergonomics.

DISCUSSION As Buxton [4] and others have noted, sketching is a key design activity. Through sketching, designers quickly discover ideas and explore solutions. However, literature on sketching tends to focus on visuals, either on papers or on screen [7]. In this paper, we have shown that sketching goes beyond two dimensions, and it is a crucial activity in prototyping. Thus, although Myers et al are certainly right in saying that our understanding of sketching interaction is rudimentary, they are wrong if we look at advanced design practice in more detail.

Specifically, this paper has elaborated the notion of experience prototyping and empathic design [3, 9]. The reason for coining a new term, “protosketching,” was to point out that although our work builds on experience prototyping, we give a more technical interpretation to this word. Protosketching has many functions in experience prototyping. For example, in Interactive Mirror, not even the designers themselves were able to read and anticipate the experience from first sketches on paper; protosketches helped to eliminate weak ideas. They also made user tests possible, and importantly, kept empathy gained in user research alive through the whole design process. Through protoskethincng, students learned to think in terms of what technology is able to do and what it isn’t.

Naturally, this paper has a message that goes beyond IP08. Namely, we believe that protosketching takes place in any advanced design practice, whether in universities or in industry. We call for further research on protosketching activities to enrich our understanding of one crucial phase of design, what happens after concept design before actual technical prototyping. In addition to conceptual clarification, we have thus opened up a research agenda.

The benefits of including prototyping skills in traditional design training are well known [4], but these methods fall short with the new craft of designing embedded and ubiquitous technologies. As our paper has shown, there are design schools that are pushing in this direction creating a whole new type of designer. Is there something fundamental

that changes the quality of design once you have to plan and program behaviors? Does it move design into a different domain or turn designers into programmers?

ACKNOWLEDGMENTS Our thanks to the School of Design at UIAH and Tomas Edman from Ludinno for funding and resources; Hannu Paajanen for prototyping help; Kirsikka Vaajakallio for helping with Make Tools; Petra Ahde in documenting the class and design help; Timo Suomala bringing in car design expertise; and VTI Technologies for sensors and actuators.

REFERENCES 1. Battarbee, K. Co-Experience. User Experience as

Interaction. UIAH, Helsinki, 2004. 2. Beyer, H. and Holtzblatt, K., Contextual Design:

Defining Custom-Centered Systems. Morgan Kaufmann, San Francisco, CA, 1998.

3. Buchenau, M. and Fulton Suri, J., Experience prototyping, In Proc. DIS 2000, ACM Press (2000), 424 – 433.

4. Buxton, B., Sketching User Experiences: Getting the Design Right and the Right Design. Morgan Kaufmann, San Francisco, CA, 2007.

5. Carroll, J., Making Use. Scenario-Based Design of Human-Computer Interactions. MIT Press, Cambridge, MA, 2000.

6. Cooper, Alan. The Inmates are Running the Asylum. SAMS, 1999.

7. Myers, B., Park. S., Nakano, Y., Mueller, G., Ko, A., How Designers Design and Program Interactive Behaviors, In Proc. VL/HCC'08, Sept 15-18, 2008, Herrsching am Ammersee, Germany. To appear.

8. Summala, S., Karola, J., Radun, I., Couyoumdjian, A. Head-on crashes on main road network - trends and causation. Finnish Road Administration, Helsinki, 2003.

9. Stappers, P. J, Sanders, E. B.-N. Generative tools for context mapping: tuning the tools. In The Third International Conference on Design & Emotion, Loughborough, Taylor & Francis, 2003.

10.Säde, S. Cardboard Mock-Ups and Conversations. UIAH, Helsinki, 2001.

11.Tohidi, M., Buxton, W., Baecker, R., Sellen, A. 2006. Getting the Right Design and the Design Right: Testing Many Is Better Than One. In Proc. CHI 2006, April 22-27, 2006, Montréal, Québec, Canada.

12.Virzi, R., Sokolov, J. and Karis, D. 1996. Usability problem identification using both low- and high-fidelity prototypes. In Proc. CHI 1996, Vancouver, British Columbia, Canada, ACM New York, NY.