gamicad: a gamified tutorial system for first time tovi/papers/2012 uist  · gamicad: a gamified...

Download GamiCAD: A Gamified Tutorial System For First Time  tovi/papers/2012 UIST   · GamiCAD: A Gamified Tutorial System For First Time AutoCAD Users ... users using the gamified system reported ... we present GamiCAD, a gamified tutorial system for

Post on 31-Jan-2018




0 download

Embed Size (px)


  • GamiCAD: A Gamified Tutorial System For First Time AutoCAD Users

    Wei Li, Tovi Grossman, and George Fitzmaurice Autodesk Research

    ABSTRACT We present GamiCAD, a gamified in-product, interactive tutorial system for first time AutoCAD users. We introduce a software event driven finite state machine to model a us-ers progress through a tutorial, which allows the system to provide real-time feedback and recognize success and fail-ures. GamiCAD provides extensive real-time visual and audio feedback that has not been explored before in the context of software tutorials. We perform an empirical evaluation of GamiCAD, comparing it to an equivalent in-product tutorial system without the gamified components. In an evaluation, users using the gamified system reported higher subjective engagement levels and performed a set of testing tasks faster with a higher completion ratio.

    Author Keywords: Game; tutorial; learning; gamification

    ACM Classification: H5.2 [Information interfaces and presentation]: User Interfaces. - Graphical user interfaces.

    INTRODUCTION Todays software applications can contain hundreds or thousands of features [27], making the process of learning to use the software difficult for new users [20]. A long line of HCI literature has investigated the problems associated with learning and many types of learning aids have been proposed. Common formats include online help [4, 5, 41], discussion boards [16, 39], and video assistance [19, 33].

    For new users, documentation may not be effective. Carroll suggests a user has a production bias which reduces their motivation to spend any time just learning about the sys-tem [4]. A more promising approach may be to provide interactive tutorials, where users get to learn by accom-plishing tasks in the application itself [2, 14, 17, 24].

    While many types of interactive tutorials exist, they may lack a level of engagement that will provide enough moti-vation for the user to follow through with required or rec-ommended lessons.

    One technique to increase engagement levels, which recent-ly has become commonly used in many domains, is gamifi-

    cation [10]. Gamification is commonly defined as using elements of video games in non-game situations to enhance user experience and engagement levels with a product [38, 36]. Gamification has been rapidly growing in popularity, with workshops [9] and entire conferences1 now being ded-icated to the topic.

    Given its current popularity, it is reasonable to consider the appropriateness of gamification for software tutorials. Some previous commercial systems have developed tutorial systems that contain some gamified elements2 3. However, little research literature exists on this topic [12]. In particu-lar, it is unclear if there are any empirical advantages of a gamified software tutorial compared to a non-gamified counterpart [25, 31]. Without data to refer to, software de-signers must make uninformed decisions to develop gami-fied tutorials, without an awareness of the benefits or tradeoffs.

    In this paper, we investigate the use of gamified tutorials to help new users both learn and improve their performance with a software system. Our work is composed of two main contributions:

    First, we present GamiCAD, a gamified tutorial system for AutoCAD4. GamiCAD builds upon the recent work of Dong et als Jigsaw system [12], introducing several novel features. In particular, a software event driven finite state machine is used to model a users progress through a tuto-rial, which allows the system to provide real-time feedback and recognize success and failures. In addition, GamiCAD provides extensive real-time visual and audio feedback that has not been explored before in the context of software tutorials.

    Our second core contribution is an empirical evaluation of GamiCAD, comparing it to an equivalent in-product tutori-al system without the gamified components. Our evaluation shows that not only does a gamified tutorial increase sub-jective engagement and enjoyment levels, it can also im-prove learning. In particular, users completed 10% more testing tasks and were 20%-76% faster after using the tuto-rial with the gamified components.

    1 The Gamification Summit (retrieved April 13, 2012)

    2 (retrieved April 13, 2012) 3 (retrieved April 13, 2012) 4

    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. UIST 12, October 710, 2012, Cambridge, Massachusetts, USA. Copyright 2012 ACM 978-1-4503-1580-7/12/10...$15.00.


  • RELATED WORK In this section, we discuss previous research in the areas of software learning, tutorial systems, gamification, and soft-ware gamification.

    Software Learning Software learnability [11, 30] is an important research problem that has been prevalent since the beginnings of HCI. Important early work includes Carrol et al.s investi-gations of learning within word processing tools [4, 5, 26].

    A recent survey of software learnability is provided by Grossman et al [20]. Their taxonomy of learnability distin-guished between initial learnability, the initial performance with a system and extended learnability, the change in per-formance over time. In targeting new users, GamiCAD focuses mostly on initial learnability.

    In the psychology literature, the Dreyfus model proposes that a learner passes through five distinct stages during the skill acquisition (Figure 1) [13]. This indicates that tutorial systems for novice users should provide a high level of guidance and feedback. Using game mechanics may be one way to do so.

    Figure 1. Dreyfus model of skill acquisition.

    Tutorial Systems Delivering effective help material and documentation can be a challenging task, because of the active users produc-tion bias [4]. Early efforts to improve software documenta-tion included minimalist and task-centered help [4, 5, 41].

    An alternative to static help systems is animated assistance [19, 32, 33]. Such materials typically enforce a passive learning process, or force users to work at the pace of the video [17]. Pause-and-Play is an interesting new system that aims to improve upon this limitation, by automatically pacing the video based on a user progress [33].

    Alternatively, tutorial systems can also be integrated within the software application itself, providing a rich interactive learning experience [2, 14, 24, 33, 35]. For example, Sten-cils-based tutorials [24] guide a user to perform the correct step by overlaying a stencil with a hole to click-through. The level of engagement they provide could potentially be increased even more by integrating gaming components.

    In particular, Fernquist et al. suggest that tutorial experi-ences should be as compelling as possible, so that users

    stay engaged [14]. One guideline for creating such experi-ences is Csikszentmihalyis concept of flow [8], which de-scribes a users state of optimal experience. Qualifying factors for achieving flow include: perceptions of clear goals, immediate feedback, and matched skills and chal-lenges [6]. GamiCAD is designed with these factors in mind.

    Gamification Gamification is the use of design elements, which are char-acteristic for games, in non-game contexts [10]. Gamifica-tion has been used in numerous domains, such as work en-vironments [15, 36], the design industry [23], marketing [43] and, particularly, educations [18, 34, 37].

    One main motivation for gamifying a learning experience is to make the experience more engaging for the user [38]. However, there is a lack of empirical evidence that the use of gamification is effective in learning contexts [25]. Previous studies of gamified learning environments [7, 31] have been confounded in that positive educational out-comes can be attributed to the instructional design of the educational programs and not to games as a medium [25]. Our work contributes empirical data from a controlled ex-periment, comparing a gamified and non-gamified learning experience with equivalent instructional design.

    Software Gamification There are several games designed for learning software applications. Commercial examples include Microsoft Rib-bon Hero1, and the now expired Adobe LevelUp2. In the research literature Jigsaw [12] is also related to our work.

    In these examples, users need to finish small tasks to col-lect points and badges. Jigsaw promotes discovery-based learning, where users can explore multiple strategies for completing a task. While Dong et al. found that users en-joyed the Jigsaw system [12]; none of these systems have been empirically compared to non-gamified counterparts.

    Furthermore, in these examples, the extent of the gamified components is limited. For example, Ribbon Hero provides a game like menu system for choosing levels, but once a task is chosen, the system runs like a typical in-product tutorial. In addition, these previous systems give users ex-posure to a broad range of tools, but users do not necessari-ly learn the best way to use them.

    GamiCAD builds upon these systems and increases the extent of which gamified mechanics are used during indi-vidual levels. Furthermore, GamiCAD exposes users to multiple ways to use individual tools, so users learn how to use tools in both basic and advanced ways.

    Summary A