conveying feedback in skill movement...

byUrmimala Majumdar

Conveying Feedback in Skill Movement

Acquisition

Master’s Thesis at theMedia Computing GroupProf. Dr. Jan BorchersComputer Science DepartmentRWTH Aachen University

Thesis advisor:Prof. Dr. Jan Borchers

Second examiner:Prof. Dr. Torsten Wolfgang Kuhlen

Registration date: Jul 2, 2015Submission date: Aug 11, 2015

iii

I hereby declare that I have created this work completely onmy own and used no other sources or tools than the oneslisted, and that I have marked any citations accordingly.

Hiermit versichere ich, dass ich die vorliegende Arbeitselbstandig verfasst und keine anderen als die angegebe-nen Quellen und Hilfsmittel benutzt sowie Zitate kenntlichgemacht habe.

Aachen, July2015Urmimala Majumdar

v

Contents

Abstract xiii

Acknowledgements xv

Conventions xvii

1 Introduction 1

2 Related work 5

2.1 Design Space . . . . . . . . . . . . . . . . . . . 5

2.2 Movement Guidance with Technology . . . . 7

3 Taxonomy and Dimension Space 15

3.1 Background . . . . . . . . . . . . . . . . . . . 15

3.1.1 Understanding skills . . . . . . . . . . 15

3.1.2 Understanding feedback . . . . . . . . 18

3.1.3 Understanding human movementand its patterns . . . . . . . . . . . . . 19

3.2 Dimension Space . . . . . . . . . . . . . . . . 22

vi Contents

3.2.1 Design Concerns . . . . . . . . . . . . 23

3.2.2 Axes of Dimension Space . . . . . . . 24

4 Performance Measurement Criteria and Visualiza-tion 29

4.1 Performance Measurement Criteria . . . . . . 29

4.1.1 Understanding skill characteristics . . 30

4.1.2 Skill Capture . . . . . . . . . . . . . . 31

Pilot data collection session . . . . . . 34

Participants . . . . . . . . . . . . . . . 34

General procedure . . . . . . . . . . . 35

Post-processing of data . . . . . . . . 35

Discussion . . . . . . . . . . . . . . . . 36

4.2 Feedback visualizations . . . . . . . . . . . . 37

4.2.1 Rationale . . . . . . . . . . . . . . . . . 37

4.2.2 Implementation details . . . . . . . . 41

4.3 Preliminary Study . . . . . . . . . . . . . . . . 43

5 User study 47

5.1 Task . . . . . . . . . . . . . . . . . . . . . . . . 48

5.2 Pilot study . . . . . . . . . . . . . . . . . . . . 49

5.3 Participants . . . . . . . . . . . . . . . . . . . 49

5.4 Experimental Design . . . . . . . . . . . . . . 50

Contents vii

5.5 Results . . . . . . . . . . . . . . . . . . . . . . 50

5.6 Discussion . . . . . . . . . . . . . . . . . . . . 53

5.7 Summary . . . . . . . . . . . . . . . . . . . . . 54

6 Summary and future work 57

6.1 Summary and contributions . . . . . . . . . . 57

6.2 Future work . . . . . . . . . . . . . . . . . . . 59

A Background Information Questionnaire 61

B Data Collection Questionnaire 65

C Stroke Performance Table 67

D Error Visualization Trial Review Questionnaire 71

E Error Visualization System Post PerformanceQuestionnaire 73

Bibliography 75

Index 83

ix

List of Figures

2.1 Dimension Space Axes (Graham et al. [2000a]) 6

2.2 7-axis Dimension Space for Comparing Mu-sical Instruments (Birnbaum et al. [2005]) . . 7

2.3 Experimental setup for ubiquitous learningsystem (Mitobe et al. [2012]) . . . . . . . . . . 8

2.4 Posture guide in YouMove. Errors are high-lighted with red circles (Anderson et al. [2013]) 8

2.5 Ghostman setup (Chinthammit et al. [2014]) . 9

2.6 Wedge visualization (Tang et al. [2015]) . . . 10

2.7 Performance interface (Velloso et al. [2013]) . 11

2.8 Proposed system for supporting playing aguitar (Motokawa and Saito [2006]) . . . . . . 12

2.9 3D arrow to guide user’s movement inLightGuide (Sodhi et al. [2012]) . . . . . . . . 12

3.1 Cardinal planes of movement . . . . . . . . . 20

3.2 Reference positions (Bartlett [2007]) . . . . . . 21

3.3 Manipulation Taxonomy (Bullock et al. [2013]) 22

x List of Figures

3.4 Dimension Space for conveying feedback forskill movement . . . . . . . . . . . . . . . . . 25

3.5 Dimension Space Plots . . . . . . . . . . . . . 28

4.1 Tools used for our knife honing task – Ce-ramic honing rod and Kitchen knife . . . . . 30

4.2 Reflective markers placed on both sides ofthe knife . . . . . . . . . . . . . . . . . . . . . 32

4.3 Reflective markers placed on the handle ofthe honing rod . . . . . . . . . . . . . . . . . . 32

4.4 Vicon Bonita camera setup directed towardscapture volume . . . . . . . . . . . . . . . . . 33

4.5 Notations used in QOC (MacLean et al. [1991]) 37

4.6 QOC analysis of Dimension Space - Part 1 . . 38

4.7 QOC analysis of Dimension Space - Part 2 . . 38

4.8 Dimension Space for our Feedback Visual-izations . . . . . . . . . . . . . . . . . . . . . . 41

4.9 Honing rod and knife angle visualization . . 42

4.10 Honing rod jitter and stroke trace visualization 43

4.11 Honing rod and knife angle visualization . . 44

4.12 Modified Honing rod jitter and stroke tracevisualization . . . . . . . . . . . . . . . . . . . 45

5.1 Performance Graph for User 1 - Trial 4 . . . . 51

5.2 Box plot for impact angle from two-wayANOVA results . . . . . . . . . . . . . . . . . 52

5.3 Box plot for jitter (x-axis) from two-wayANOVA results . . . . . . . . . . . . . . . . . 52

List of Figures xi

5.4 Box plot for jitter (y-axis) from two-wayANOVA results . . . . . . . . . . . . . . . . . 53

5.5 Box plot for jitter (z-axis) from two-wayANOVA results . . . . . . . . . . . . . . . . . 53

A.1 Background Information Questionnairepage 1 . . . . . . . . . . . . . . . . . . . . . . . 62

A.2 Background Information Questionnairepage 2 . . . . . . . . . . . . . . . . . . . . . . . 63

B.1 Data Collection Questionnaire . . . . . . . . . 66

C.1 Stroke Performance Table for User 1 - Page 1 68



D.1 Trial Review Questionnaire . . . . . . . . . . 72

E.1 Post Performance Questionnaire . . . . . . . 74

xiii

Abstract

Acquiring a coordinated fine motor skill involves exploring the work space of thetask that can be best achieved in the presence of an expert or teacher. The teacherhelps the learner channel the movements towards the accurate actions required forthe task by giving the learner appropriate feedback. In the absence of the teacher,this critical exchange disappears and the learner is left to overcome the gaps on hisown or with graphics such as images and videos. Recently, due to improvementin technology, research in the area of technology assisted learning and associatedfeedback has gained a lot of momentum in Human Computer Interaction. Thestudies in this field so far have brought forward new techniques or methods tohelp the learner perform a certain motor task such as dancing or rehabilitation.

We present an approach for determining the performance measurement criteriafor a skill and designing appropriate low-level corrective feedback based on theevaluation. For the purpose of this thesis we have chosen the task of knife honing.For arriving at the criteria we took a comprehensive look at the taxonomy of skills,feedback, human movement pattern and studied the expert’s movements in action.To address the design concerns and choices, and to assist in the designing of thefeedback visualizations, we also present a Dimension Space. Lastly, we conducta user study to compare the designed feedback visualizations with videos to gaininsight about their correctness and ability to convey required information to helpthe learner improve his mistakes.

xv

Acknowledgements

Firstly, I would like to thank Prof. Jan Borchers for giving me the opportunity toundertake such a wonderful project at the chair. This thesis has made me learn somuch about the fundamentals of not only Human Computer Interaction, but alsoMotor Learning and Augmented Reality.

This thesis would not become a reality without the knowledge, expertise, construc-tive advice and support of my supervisor, Nur Al-huda Hamdan. I am indebted toher for all the feedback and assistance which helped me improve this work at everystep.

Moreover, thanks to Chatchavan Wacharamanotham for his invaluable help andadvice without which it would be more difficult to complete this work.

A special thank you to Krishna Subramanian for his guidance in understanding thestatistics for the analysis of the data.

I would also like to thank my second supervisor, Prof. Wolfgang Kuhlen for hisquick response and understanding.

I also appreciate all my friends and other fellow students who participated in myuser studies. I thank them for their time and insights.

I would like to thank everyone who helped in one way or another during the thesis,especially Manjari Marambath, Sathvik Parekodi and Tanya Agarwal for patientlyproof reading the numerous versions of the thesis. I sincerely appreciate all thatyou have done.

Finally, I would like to thank my family who have kept me motivated and sup-ported me throughout the duration of the thesis. I owe everything I have achievedto them.

xvii

Conventions

Throughout this thesis we use the following conventions.

The whole thesis is written in American English

Irrespective of the real users’ gender we will use ”he” as areference to a single person.

1

Chapter 1

Introduction

Training in psycho-motor skills (related to one’s physical Importance offeedback in skillacquisition and needfor systems whichprovide feedback inthe absence of aninstructor

movement and coordination) is an area of on-going re-search in the field of Human Computer Interaction. Theprocess of traditional training for acquiring such a skill in-cludes the demonstration of the skill by the instructor, theperformance by the learner and ultimately the feedback orthe performance assessment by the instructor. Initially, thelearner will lack proficiency, but this can be improved bytraining under the guidance of an instructor. Instructor’sguidance or feedback is usually seen as the means to max-imize learning and induce long-term retention. When thelearner is not in the company of the instructor the criti-cal exchange of feedback disappears, but can be managedthrough videos and images. However, such media can notprovide feedback to the degree of accuracy which is criti-cal for improving the level of efficiency in learning. Manystudies (Weing et al. [2013], Mitobe et al. [2012], Andersonet al. [2013], Sodhi et al. [2012], Xiao and Ishii [2011]) in HCIhave explored methods of communicating movement pat-terns and appropriate feedback to help the learner acquiredifferent skills, such as playing the piano, playing the koto(Japanese harp), dancing and abstract hand movements, inthe absence of an instructor.

Apart from pyscho-motor skills, from motor learning lit- Skill classificationand the challenges inskill training

erature (Anderson et al. [2001]) we know that, there aretwo other categories of skills — cognitive (related to per-

2 1 Introduction

son’s intellect and knowledge) and affective (related to anindividual’s emotional quotient). We must understand thatany ability that can be called skilled involves combinationsof cognitive, affective, and motor processes with differentweights. Therefore, although the research in HCI in thisfield is primarily concerned with the development of skillsin the psycho-motor domain, the growth of these skills isinfluenced by the learner’s knowledge, attitude and envi-ronment. This makes the process of training a complicatedone. To further add to this difficulty a majority of the mo-tor skills involve some degree of collaboration between thehands. Our hands allow us to perform several sophisti-cated and difficult activities such as using tools, gesturing,typing and writing. Thus, this type of skill acquisition in-volves not only the learning of adaptive patterns of muscu-lar movements and timing, but also understanding of thecontext and control required while performing the move-ments.

To facilitate such learning, different research techniquesOur interest is vestedin performance

feature evaluation fordesigning feedback

have been experimented with. For instance, Anderson et al.[2013] recorded the movements of the expert and presentedit to the learner on a mirror-like display; the learner thenimitated the movements. Subsequently, when the learnermade mistakes, he was shown feedback to correct his ac-tions. On the other hand, Mitobe et al. [2012] collected theexpert’s movement data using a motion capture setup andthen presented it to the learner via a head mounted display,such that the learner could see the expert’s hand and hisown side by side, and imitate the expert’s actions. How-ever, majorly these studies focused on bringing forwardnew methods using different technologies to provide feed-back. In this thesis we are interested to understand how wecan evaluate the performance features of a skill to designan appropriate visual feedback.

This thesis will explore about performance measurementScope of workcriteria required to design a visual feedback, which is agood representative of the evaluated performance features.Further, based on the criteria found, an appropriate visualfeedback will be designed and analyzed to gauge how help-ful the feedback is to the users in performing the movementsequences.

3

The work starts with giving a taxonomy of skills and feed-back to gain some foundation for defining a performancemeasurement criteria. Since in a system communicatingskill movement the actors interact with objects both in thephysical and virtual worlds to complete a task, we definea dimension space to describe the design choices available.The dimension space is based on the taxonomy, and is de-scribed with the plots and analysis methodologies. Havingthe dimension space, we take a comprehensive look at howit can be applied to a chosen activity - knife honing. Theprocedure for defining the performance measurement crite-ria is discussed which includes discussions with the expert,capture of expert data, capture of data from novice users,annotation of the data and videos captured, and finally theanalysis of results. This will give us a concrete idea aboutthe characteristics that feedback visualization must have.Further, based on the dimension space, the work proposes afeedback visualization system for the activity of knife hon-ing. It is a kind of animation visualization which is in ac-cordance with the dimension space and can be generatedfrom the data collected from the user’s performance. Thevisualizations are used to describe sparse mathematical in-formation as the feedback and ultimately evaluate whetherit fulfills the requirements of providing critical knowledgeabout improvement and quality of performance. A userstudy run by a program is included in this work to supportand evaluate the benefit of the visualizations proposed.

This thesis has three main contributions: Thesis contributions

1. We present performance features for the selected skillalong with evidence from data collected from noviceand expert users. We also describe a dimension spacefor assisting in making design choices for a systemconveying feedback for skill movement acquisition.

2. We design feedback visualizations based on the per-formance measurement criteria to show the featuresof the chosen skill — knife honing.

3. We conduct a controlled experiment to analyze howthe proposed feedback visualization affects noviceusers, discover patterns and present results.

4 1 Introduction

In the points that follow we provide a brief summary ofThesis structureeach chapter to give an overall structure of the thesis:

Chapter 2 — “Related work” In this chapter, we providean overview of other related publicized work which dealswith creating a dimension space and design space andnovel systems about motor skill acquisition from the per-spective of Human Computer Interaction.

Chapter 2 — “Related work” In this chapter, we describethe taxonomy of skills, the different types of feedback alongwith the basics of human movement and its patterns. Fur-thermore, we present the dimension space, describe the de-sign concerns, and explain the different axes.

Chapter 4 — “Performance Measurement Criteria and Vi-sualization” In this chapter, we report the apparatus andthe procedure for collection of motion data, and the post-processing of data for deciding the performance features.To establish performance measurement criteria, we explainthe common mistakes for the task of knife honing and howwe have established thresholds for these mistakes. Lastly,we present the visualization based on the criteria and dis-cussion.

Chapter 5 — “User Study” In this chapter, we outline thepilot and main user study with the task of knife honing. Inthe subsections we describe the results and discussions.

Chapter 6 — “Summary and future work” In this chap-ter, we summarize the work done in this thesis, discuss thefindings and outline the future possibilities.

5

Chapter 2

Related work

2.1 Design Space

As yet, there has been no research work that is solely ded-icated to building a framework or design space for provid-ing feedback for bimanual skill acquisition. The researchwork done in the field of technology based motor skilllearning describes several design choices, but doesn’t pro-vide a consolidated foundation from which these designdecisions can be made; this is from where the concept of ourwork originated. There are several published works whichhave created design spaces for the study of other domainssuch as information visualization, graphical interfaces anddisplays. There are also other works which introduce thenotion of Dimension Space for describing entities whichan interactive system is composed of. From these studieswe attempt to understand the process of building a designspace for feedback visualization and developing a new de-sign using the same.

The foundations of data visualization which include the Design spaces forinformationvisualization

process of data visualization, optimal visual display andthe significance of different kinds of representations whichmay be a combination of graphical elements and verbal ex-pressions is described in detail in Ware [2012]. Further, thisbook provides a very interesting note on how animationcan enrich any visualization and can be used to powerfully

6 2 Related work

Figure 2.1: Dimension Space Axes (Graham et al. [2000a])

explain the relationships between different kinds of data.Another research which deals with understanding informa-tion visualization is Card and Mackinlay [1997]. This paperdescribes a framework for designing new visualizations fordifferent kinds of data and augmenting existing designs. Itsuggests the use of table notation to organize the mappingbetween data and presentation and explains how graphicscan be used to map data with visual vocabulary and othercharacteristics such as color and depth to provide a visual-ization that is appropriate and can be easily interpreted.

The idea of Dimension Space Graham et al. [2000a] expandsDimension Spaceon the approach of Design Space Analysis MacLean andMcKerlie [1995]. The notion of Dimension Space was pro-posed to provide a more methodical way of designing sys-tems which involves interaction between entities from thereal and virtual world. This design process represents in-teractive systems on six axes as shown in Figure 2.1. Birn-baum et al. [2005] provides an account of the applicationof Dimension space in comparing musical devices with theaim of assisting in designing novel instruments. Figure 2.2shows the Dimension Space employed by the authors forthe said comparison. The study of entities from the point

2.2 Movement Guidance with Technology 7

Figure 2.2: 7-axis Dimension Space for Comparing MusicalInstruments (Birnbaum et al. [2005])

of view of the dimension space paradigm has helped us toisolate relevant characteristics which will be discussed inChapter 3 “Taxonomy and Dimension Space”.

2.2 Movement Guidance with Technology

Recently a lot of work has been done in the the field oftechnology assisted learning in Human Computer Interac-tion. Different researchers have found motivation to pro-vide movement assistance in different fields such as re-habilitation, learning musical instruments and full-bodymovements.

Some of these studies rely on data from experts to provide Guidance derivedfrom expert datafeedback. The first among studies which depend on expert

data to provide appropriate feedback is Mitobe et al. [2012].In this study guidance is provided to the user with a vir-tual hand constructed from motion capture data from anexpert. The system as shown in Figure 2.3 is aimed at being

8 2 Related work

Figure 2.3: Experimental setup for ubiquitous learning sys-tem (Mitobe et al. [2012])

Figure 2.4: Posture guide in YouMove. Errors are high-lighted with red circles (Anderson et al. [2013])

a ubiquitous learning system for high precision movementsof the hand. The expert’s hand movement data is capturedusing Hand Motion Capture System and shown to the userusing a head mounted display as a 3D virtual hand thatthe user can overlap to learn playing the Koto (Japaneseharp). While Mitobe et al. [2012] deals with learning toplay an instrument, another study — Anderson et al. [2013]proposes an augmented mirror-like display system calledYouMove which provides training based on an expert’s


Figure 2.5: Ghostman setup (Chinthammit et al. [2014])

movements. YouMove is a full-body movement instruc-tion system which allows the expert to record and anno-tate movement sequences using Kinect; these can be usedby the trainee for gradually learning the movements usingthe augmented mirror. Feedback information is providedby comparing the trainee’s and the expert’s skeleton and itis scored based on the important joints with the maximumerror, measured by Euclidean distance. Further, YouMoveincorporates posture guidance where the errors are high-lighted using red circles as can be seen in Figure 2.4.

There are also studies which have direct involvement of Guidance from directinvolvement of expertthe expert. In Chinthammit et al. [2014], the authors have

developed an augmented reality system called Ghostman,which focuses on bringing together the therapist (expert)and the patient (user) remotely, to deliver instructions andassess performance. Ghostman comprises of two subsys-tems, as shown in Figure 2.5, which consist of augmentedreality head mounted displays and communicate over theinternet. The developers have assumed details about ori-entation and scaling to simplify their task for conductingthe pilot study. Nevertheless, the design principles fol-lowed by Ghostman for providing the navigational cue,the ghost image (therapist) and the real-world image (pa-tient), are fundamental. These allow the patient to view

10 2 Related work

Figure 2.6: Wedge visualization (Tang et al. [2015])

the task at hand from the therapist’s point of view which isregarded as ”inhabiting visual augmentation”. Another re-search study which has direct involvement of the expert isZaczynski and Whitehead [2014]. This paper investigatesabout learning in the domain of interactive gaming andprovides design recommendations about the delivery andtype of feedback. In the experiments conducted to arrive atthe design recommendations, the different conditions of vi-sual feedback were tested such as Front, Front + Wide andso on. For testing the effect of haptic and verbal feedback,the researcher’s (expert in yoga poses) touch or voice wasused. Pose accuracy during the experiments was judgedand scored by the researcher. After their experiments, theauthors conclude that providing custom, contextual andmulti-modal feedback is the best way to help users resolveerrors in their actions.

Physical rehabilitation is a common motivation for researchGuidance in physicalrehabilitation in the movement guidance techniques, for patients in the

physical absence of a physiotherapist, like in the case of


Figure 2.7: Performance interface (Velloso et al. [2013])

Physio@Home Tang et al. [2014, 2015]. In these studies,the authors have designed a system to help patients un-dergoing physiotherapy using previously recorded exer-cise sequences. Physio@Home uses Vicon motion track-ing cameras and markers on the user’s shoulders and armsto capture multiple views of an exercise movement. Theprototype guides patients by overlaying visual cues on themirror-view of the patient. The system uses arrows as feed-forward and nearest correct arm as a red stick figure anddifference in angle as feedback as shown in Figure 2.6.

Machine learning has also been leveraged for training Guidance usingmachine learningmovement in the domain of physical fitness as in Mo-

tionMA (Velloso et al. [2013]). MotionMA allows a per-former to record a repeated movement using the Kinectfrom which a model is extracted. An observer can thenuse this model to practice the movement. The observer’sactions are recorded by the Kinect, a model is extractedwhich is compared to the performer’s model and real-timefeedback is shown to the observer on a display. Tasks suchas weight lifting were chosen to test the system developed.The feedback interface is designed such that ”Informationregarding static bones is displayed on traffic lights whilstthe ranges of motion of dynamic bones are displayed ondials. The user can see the video recording of the demon-stration and his own skeleton as tracked by the Kinect witheach bone in a different color depending on its score” asshown in Figure 2.7.

12 2 Related work

Figure 2.8: Proposed system for supporting playing a gui-tar (Motokawa and Saito [2006])

Figure 2.9: 3D arrow to guide user’s movement in Light-Guide (Sodhi et al. [2012])


Studies within the HCI community has also focused on Guidance usingnovel methods for thedelivery of feedback

movement guidance using novel methods for the deliveryof feedback. Motokawa and Saito [2006] proposes a sys-tem using augmented reality to teach how to learn to playthe guitar. The developed system as shown in Figure 2.8,uses a square shaped planar marker and the guitar’s natu-ral features to track the pose and position of the guitar. Thetracking is achieved via a USB camera, which captures theuser’s images; these images are used to estimate the poseand position of the guitar using the AR toolkit and edgebased tracking algorithm. The final output is provided ona PC display in the form of an overlaid virtual hand toguide the user with regard to his hand movements. An-other study — Weing et al. [2013] describes the P.I.A.N.O.system for users who are beginners and wish to learn toplay the piano. The system provides feedback and feed-forward using light cues which are projected onto the pi-ano keys using a DLP projector and a display attached tothe keyboard. The light cues are color coded and differ insize to allow for finger switches and overview of upcom-ing notes. The preliminary evaluation in this study didn’tinvolve judging how well or fast the user has learned thesong, rather it was aimed at understanding whether the fin-gering information is enough and what visual scope is usedby the user. LightGuide (Sodhi et al. [2012]) used on-bodyprojections to guide a user’s hand through 3D space, one ofwhich is shown in Figure 2.9. The hand is tracked using aMicrosoft Kinect Depth Camera by determining regions ofcontinuous depth after removal of redundant objects fromthe scene. A wide angle projector aligned with the cam-era is used to project the hints in 1D, 2D and 3D designedto provide feedback and feed-forward information to theuser.

Due to technological advancement the research in the Need for digitizingskillfield of motor training and feedback is gaining interest.

Nonetheless, most of these systems still rely on human in-volvement to provide feedback, for instance Chinthammitet al. [2014] provides an effective method of delivering in-structions but remotely. Some systems provide the methodof extracting expert information as in Velloso et al. [2013],but they don’t address learning or explore the best way ofproviding feedback. Also, some of the systems provide

14 2 Related work

feedback in a manner which is not sufficient for technicalgestures. For example, Motokawa and Saito [2006] pro-vides visual guide from the front angle instead of multipleangles which is not enough. Another example in this regardis Sodhi et al. [2012], in which feedback is only availablefor a single hand. Furthermore the feedback is only givenwhen the hand is visible from the depth camera. Thus,there is need to digitize the skill and develop an evalua-tion model that depicts the most important features in themovements of the skill in order to provide rich feedback.We draw on the design decisions made by the differentstudies and chalk out the desired performance criteria anddimension space.

15

Chapter 3

Taxonomy andDimension Space

In Chapter 2 “Related work”, we described the feedbacktechniques which have been implemented so far. However,to arrive at a point where we can present the dimensionsof feedback we need to discuss about some concepts. Fun-damentals regarding different kinds of skills will help usunderstand how a skill may be categorized and exploringabout feedback and movement patterns will enable us todetermine what kind of technique could be employed forconveying movements for a skill. Once we have exploredthese properties, we will proceed to define the taxonomyrelated to our dimensions which explains the feedback re-quirements during skill movement acquisition. The pro-posed dimension space will be introduced after this basedon the explained taxonomy

3.1 Background

3.1.1 Understanding skills

We have already introduced that educational activities Skill classificationwhich lead to the development of skills can fall under three

16 3 Taxonomy and Dimension Space

major spheres Anderson et al. [2001] in 1 “Introduction”.These are further described below.

• Cognitive - This domain has to do with the develop-ment of a person’s intellect and knowledge.

• Affective - This domain deals with the developmentof an individual’s emotional quotient - a person’s at-titude, gratitude and other feelings.

• Psycho-motor - This domain is concerned with phys-ical movement, coordination and use of the motor-skill areas. Development in this domain requirestraining and practice, and can be measured with cri-terion such as speed and precision.

Commonly these domains are referred to as Knowledge,Compilation ofpsycho-motordomain model

Attitude and Skills by trainers. Several compilations of thepsycho-motor domain model, have been suggested overthe years Harrow [1972], Ej [1966], Anderson et al. [2001],which divide the domain further to describe the simplest tothe most complex behavior. One popular compilation Arm-strong et al. [1970], that fits our point of view, is describedbelow.

• Imitation - This is absolutely the initial phase wherethe learner observes someone’s actions and move-ments and attempts to copy them.

• Manipulation - This is the phase in which the learneris able to perform the required movement for a skillby following instructions and practicing.

• Precision - In this phase the learner’s movements be-come more accurate and the number of errors are re-duced to a great extent.

• Articulation - When this phase has been reached, thelearner can co-ordinate a series of actions and relatedmovements and perform with consistency.

• Naturalization - In this final phase, the learner candeliver a high-quality performance naturally, withouthaving to think or remember about the movements ofthe skill.

3.1 Background 17

In other words, what is required to perfect a skill is repeti- Repetition andimitation leads tolearning

tion. Research Grafton et al. [1992] has demonstrated thatlearning can occur from repetition which is supported bycorrect feedback. The recurring sensory signals from themoving limbs as they interact with the tools in the environ-ment can finally lead to learning that particular movementsequence. This is because, although initially the move-ments are imprecise, the practice leads to the developmentof an internal model which can converge to the correct so-lution given proper feedback.

Now having understood how psycho-motor skills may bedeveloped, we must delve into the domain of motor skillsand understand its categorization. Motor skills has beenclassified into several categories based on the precision ofthe movement, the environment in which the movementtakes place and the type of beginning or end a movementhas by may researchers Davis et al. [2000], Galligan et al.[2002], Knapp [1963].

From the perspective of our system we are interested in Scope of skills in ourthesisskills which satisfy the following scope:

• Fine Skill - intricate precise movements using smallmuscle groups are required and generally the skill in-volves high levels of hand-eye coordination.

• Closed Skill - skill’s movements take place in a stable,predictable environment and the performer knowsexactly what to do and when.

• Internally paced Skill - the rate at which the skill isperformed is controlled by the performer.

• Discrete Skill - the skill has a clear beginning and end.

• Individual Skill - the skill is performed in isolation,that is, the skill is not an interactive or co-active activ-ity.

The most common examples of the skills that fall underthe scope described above are playing musical instruments,knife skills, communication using sign language, conduct-ing an orchestra and juggling.


3.1.2 Understanding feedback

Feedback is defined as the information that the user re-Feedbackclassification ceives about his actions during or after the movement has

been performed.The feedback Smith and Lee [1998], Daviset al. [1991] can either be a natural outcome of performingany movement (called as inherent or intrinsic feedback) orfeedback can be provided by an external source such thatit supplements the intrinsic feedback (called as augmentedor task extrinsic feedback).

Inherent or task-intrinsic feedback can be further catego-rized as follows.

• Exteroceptive - this is the feedback received by theuser via observation of movements of objects in theenvironment or sometimes even self-reflection.

• Proprioceptive - this is the feedback that the user re-ceives via his senses, in other words the feel of themovement that the user can perceive via his body sig-nals, limb movements and joint angles.

Depending on the nature and source of inherent feedback,the performer can understand whether he has made a mis-take or not. However, a reference of correctness is necessaryin the mind model, which is usually not developed accu-rately until the late stages of learning. Therefore, inherentfeedback can not be used to detect errors.

Augmented or task-extrinsic feedback can have several di-Dimensions ofaugmented feedback mensions as described below.

• Concurrent - this kind of augmented feedback is sup-plied during the performance of the movement.

• Terminal - this kind of augmented feedback is pro-vided only after the completion of the performance ofthe movement sequences.

• Immediate - this kind of augmented feedback is de-livered immediately after the performance.

3.1 Background 19

• Delayed - this kind of augmented feedback is deliv-ered after some amount of time has passed after theperformance.

• Verbal - this is the kind of augmented feedback whichis presented to the performer in a spoken form.

• Non-verbal - this is the kind of augmented feedbackwhich is presented to the performer in a form whichnot speech.

• Accumulated - this is the kind of augmented feedbackwhich is an aggregated feedback for the past few per-formances.

• Distinct - this is the kind of augmented feedbackwhich is given for a specific performance.

• Knowledge of results (KR) - this feedback is aboutwhether or not the user was able to achieve the targeti.e. whether the user’s performance was successful inrelation to his goal.

• Knowledge of performance (KP) - this feedback isabout the user’s performance and technique. It is typ-ically conveyed to performers by verbal cues, videosof the performance, pictures and other forms of visualdata.

Studies Newell and Hancock [1981], Newell and Carlton[1987], Schmidt and Lee [1988] prove that for beginners try-ing to learn a complex skill it is better to provide KP. Giventhat the tasks which fall under our selected scope of skillscan be considered as complex tasks, for this thesis we willconcentrate on conveying KP feedback to our users.

3.1.3 Understanding human movement and its pat-terns

In order to obtain a proper description of human movement Reference positionsand cardinal planesof movement

we need to define with reference to specific position or pos-ture. Two positions as depicted in Figure 3.2 are defined –fundamental and anatomical. Furthermore, the movements


Figure 3.1: Cardinal planes of movement

can be said to be taking place along the cardinal planes – thesagittal, frontal and horizontal planes Bartlett [2007]. Theseplanes are mutually perpendicular intersecting planes andthe intersection of pairs of planes gives us three axes knownas - the sagittal, frontal and vertical (longitudinal) axes.From these planes and axes are shown in Figure 3.1, weunderstand that exercise movements such as running takesplace in the sagittal plane, and movements related to say,chopping of vegetables takes place in the frontal plane. Asdiscussed before skills and activities such as playing an in-strument falls under the scope of skills that we have cho-sen. Consider the skill of playing a piano - this skill linksthe natural hand and finger movements in a complex andwell-codified pattern. So, to understand how to providefeedback for the movement patterns which fall under ourselected scope we need to gain some basic insight aboutdexterity.

Dexterity can be defined as the skillful manipulation in-Human dextrity andManipulation

Taxonomyvolving one or more hands Wiesendanger and Serrien[2001]. Grasping and bimanual coordination form the coreof dexterity. Several studies Bullock et al. [2013], Weberet al. have researched about dexterous manipulation forthe purpose of developing a framework which can be ap-plied to robot hand engineering. As mentioned in Bullock

3.1 Background 21

Figure 3.2: Reference positions (Bartlett [2007])

et al. [2013], while interacting with an external object, thefollowing elements are important.

• Contact - hand touches the external object or the en-vironment.

• Prehensile - action requires more than one finger.

• Motion - any joint in the hand moves relative to a spe-cific body frame.

• Within Hand - fingers move relative to a specific handframe.

• Motion at Contact - touch to the object moves relativeto contain point frames.

Based on these elements a taxonomy of manipulation hasbeen defined as shown in Figure 3.3. Using this taxonomywe can define what category a particular movement willfall under and subsequently analyze the strategy to evalu-ate the manipulation.


Figure 3.3: Manipulation Taxonomy (Bullock et al. [2013])

3.2 Dimension Space

The concept of design space analysis was introduced inDesign spaceanalysis MacLean et al. [1989] as part of a framework meant to rep-

resent design decisions for designed artifacts. Since thendesign space analysis has been used to usefully study anddescribe the overall structure in several domains such asaugmented reality prototype for air traffic control (Mackayet al. [1998]), adaptive graphical user interfaces (Gajos et al.[2006]) and interactive public displays (Muller et al. [2010]).

The construction of design space of a system leads to theidentification of a set of key functional and structural pos-sibilities (Questions) while creating the system and also thechoices available for each possibility (Options). In addition

3.2 Dimension Space 23

to these possibilities and choices, rules (Criteria) which linkdifferent possibilities form an important part of the designspace. Design space analysis exposes patterns among dif-ferent dimensions and helps in designing an optimal sys-tem where the reason behind every choice is clear and un-derstood.

However, design space analysis doesn’t provide enough Problem with designspace analysismethodical support for design of systems which involve ac-

tors interacting with objects both in the physical and virtualworlds to complete a task. Since any system which intendsto teach a skill has both virtual and physical entities, wewill use the Dimension Space as described in Graham et al.[2000b] to elaborate about possible design choices.

3.2.1 Design Concerns

We begin by understanding the static design concerns of Static designconcernsour system.

• Involved entities: We must start by realizing what oursystem is composed of. The entities that make up asystem aimed at providing feedback for learning askill include the objects used to perform the skill (ifany), the user of the system, the hardware for collect-ing the user’s movement data and of course the visu-alization supplied to assist the user.

• Usage of the entities: As the next step it is important tounderstand and design how the software entities in-teract with the physical objects, in other words whatalgorithms are at work behind the scenes. These al-gorithms determine the information type and contentof the visualization. In the systems we have reviewedthus far, varying algorithms have been used. Furtherwe need to understand how the users fit into the inter-action model and what kind of interaction techniquescan be employed.

• Tradeoffs among entities: While some entities are fixed,others can be replaced. For instance, if we want to de-sign for conveying movement skills for playing gui-


tar, the guitar is a fixed entity whereas other entitiessuch as the hardware used to capture the movementdata (such as Kinect or MoCap sensors) or the feed-back visualization are changeable. We need to rec-ognize the tradeoffs among the available choices ofreplaceable entities.

Besides the static design concerns, we must consider theRuntime designconcerns runtime issues of our system.

• System use over time: The characteristics certain enti-ties is bound to changes over time. For example, aftera certain number of trials the user can no longer be re-garded as a beginner and therefore the characteristicsof other entities, such as the feedback visualization,must evolve to accommodate this change.

• Discontinuities: There may be discontinuities in the in-teraction with the system, for example when real-timevisualization is provided, the user’s attention can bedivided between a couple of entities. These need to beresolved to allow for smooth interaction experience.

3.2.2 Axes of Dimension Space

As we have already discussed in 2 “Related work”, theDefining DimensionSpace axes for our

thesisDimension Space as originally proposed represents inter-active systems on six axes. We have adapted the dimensionspace for conveying feedback for skill movement acquisi-tion. In doing so we explored several values and configu-rations of axes based on knowledge gathered from the pre-vious section 3.1 “Background”. Descriptive detail of eachof the six axes as depicted in Figure 3.4 is provided in thefollowing points.

1. Feedback timing

Concurrent

Terminal


Figure 3.4: Dimension Space for conveying feedback forskill movement

Concurrent feedback means that the feedbackwill be delivered real-time, i.e. while the user isperforming the action sequences. Many studiesdiscussed in 2 “Related work” like Sodhi et al. [2012]provide concurrent feedback.

Terminal feedback indicates that the feedback is de-livered after the user has finished the performance ofthe skill. Anderson et al. [2013] has the option to pro-vide terminal feedback to the users at the end of theirperformance.

2. Feedback information content

Accumulated

Distinct

Accumulated feedback is a way to provide feed-back about multiple performance, in other words, itprovides averages of selected performances. While indistinct feedback, feedback is provided for a specificmovement. For instance, the feedback information


provided by the Performance interface in Vellosoet al. [2013] is discrete in nature.

3. Capture perspective

Sagittal plane

Frontal plane

Horizontal plane

As discussed in 3.1.3 “Understanding humanmovement and its patterns” human movements takeplace along specific cardinal planes. In layman termssagittal plane perspective is equivalent to side-viewcapture, fontal plane perspective is equivalent tofront-view capture and horizontal plane perspectiveis equivalent to top-view capture. Data for providingfeedback must first be captured along the primaryplane in which the task is being performed as long asit is unhindered by the manipulated object(s).

4. Output capability

Auditory

Haptic

Visual

After review of current methods, we understandthat there are four possible modes of deliveringfeedback.

Feedback could be audio which qualitatively or quan-titatively conveys the review of the user’s perfor-mance. Haptic feedback can be used for tasks whichinvolve object manipulation either when a user hasperformed the correct action or when he has not. Vi-sual feedback can range from very low fidelity suchas text to a graph constructed from user’s movementdata. Feedback mode could be such that its realityis blended in with virtual images, i.e. augmented re-ality. Anderson et al. [2013], Mitobe et al. [2012] can


be considered as two examples of systems which pro-vide augmented reality feedback.

5. User experience variability

High

Medium

Low

System could be catered to providing variabilityin feedback provided depending on user’s expe-rience. High variability would indicate differentfeedback is provided for novice and experiencedusers, medium would indicate that minor changes inthe feedback are made available for the experiencedusers and low would indicate that the system doesn’tadapt to different experience levels of users.

6. Tracked unit

Objects

Body

This dimension addresses the last and most im-portant task of a feedback system, which is trackingan object with the view of collecting informationfor providing feedback to enable the user reach hisperformance goal.

Some tasks require the manipulation of external ob-jects, for example, sharpening a knife. For these tasks,feedback would be incomplete without collecting andpresenting data regarding manipulated object.

In other tasks objects are absent, for example, com-municating using sign language. For such tasks theappropriate part of the human body must be trackedto provide feedback.

We can use these axes to construct plots, which will help Dimension Spaceplots for systems inRelated Work

us in contrasting the choices and examine different designoptions. Figure 3.5 shows dimension space plots for the


Figure 3.5: Dimension Space Plots

feedback provided by Ubiquitous Tenalai-docolo (Mitobeet al. [2012]), YouMove (Anderson et al. [2013]), LightGuide(Sodhi et al. [2012]) and Physio@Home (Tang et al. [2015]).

29

Chapter 4

PerformanceMeasurement Criteriaand Visualization

4.1 Performance Measurement Criteria

To design a visual feedback which is rich in information Need to capture skilldata to defineperformancemeasurement criteria

and a good representation of all the critical features, weneed to define a performance measurement criteria for theskill under study. For arriving at this evaluation measurewe need to understand, capture and analyze the character-istics that define a unique skill.

For the purpose of this thesis we wanted to choose a skill Reasons forchoosing knifehoning

with few key gestures which had a repetitive rhythm andcould be learned in a short while. We went through sev-eral videos and web-blogs about activities such as learn-ing to play the piano (Palmer et al. [1982]), learning to playthe guitar (Manus and Manus [1992], Duncan), rehabilita-tion (Glynn and Fiddler [2009]), yoga (Mittra [2002], Longand Macivor [2009]), juggling (Beek and Lewbel [1995]),dance practices (Carroll and Carroll [2012]), card tricks(Longe [1993]) and knife skills (Hertzmann [2007], Ramaand Miller [2014], Jay and Fink [2008]). Ultimately, wechose knife honing as the skill to be learned. Honing a knife

30 4 Performance Measurement Criteria and Visualization

Figure 4.1: Tools used for our knife honing task – Ceramichoning rod and Kitchen knife

realigns the edge of the knife which consequently sharpensa dull knife (Jay and Fink [2008]). Most chefs hone theirknives before every use and this task can be performed witha rod of steel, ceramic or diamond coated steel. The skillinvolves running the blade of the knife at an approximateangle along the honing rod in an arcing motion, and mustbe repeated few times until the edge realigns. The toolsused to perform the task in our data collection procedureare shown in Figure 4.1.

We chose to capture the skill using an infrared tracking sys-Capturing skill usingVicon Nexus tem — Vicon Nexus, because of its ability to capture the

skill from multiple angles. As understood from Chapter 2“Related work”, capturing data from multiple angles willgive us information to show feedback from all required po-sitions. Further, Vicon Nexus’ performance, high resolutionand sampling rate give it an edge compared to other humanmotion capture tools (Pfister et al. [2014]).

4.1.1 Understanding skill characteristics

In order to understand the characteristics of knife honingCommon errors inknife honing we went through videos available on the internet and spoke

to a chef who has been trained in knife honing. From theavailable literature and discussions with expert, it was de-termined that any novice user can make the following er-rors during knife honing.

4.1 Performance Measurement Criteria 31

• The angle between the honing rod and the knife is notin the range of 12-20 degrees.

• The honing rod is not held straight but inclined at anangle throughout the knife honing action.

• The honing rod is not held stably and it wobbleswhile the user performs the stroke with the knife.

• The user doesn’t sharpen the entire range of theknife’s blade from the base to the hilt.

In addition to the discussions, we decided to capture the Capturing expertdata as baseline skillmodel

expert performing the task. The reason behind this de-cision is two-fold – firstly, research (Wood [2009]) has al-ready shown that there is a difference in the way an expertdemonstrates his movements and the way he talks aboutthem; secondly, capturing the expert’s movements allowedus to experiment with the position of markers on the objectsused in the skill, which will be tracked by Vicon Nexus.

Based on our observations and our understanding of hu- Placing markers onobjectsman movement patterns in 3.1.3 “Understanding human

movement and its patterns”, we can categorize knife hon-ing as C P M NW NA, and this communicates that the knifeis grasped in the hand, fingers don’t move relative to thehand frame and the hand and the knife moves as a whole;from this we can conclude that we could capture motiondata of the object alone. Thus, we placed the reflectivemarkers on the honing rod as shown in Figure 4.3, and onthe knife as shown in Figure 4.2. The arrangement of themarkers on the knife is identical on both sides, however wecan distinguish the side of the stroke based on the 3D datacollected. The markers on the honing rod are placed on thehandle, so that it doesn’t interfere with the activity of knifehoning.

4.1.2 Skill Capture

To decide on the performance measurement criteria we notonly captured data from the expert performing the skill butalso from novice users. This helped us determine which


Figure 4.2: Reflective markers placed on both sides of theknife

Figure 4.3: Reflective markers placed on the handle of thehoning rod

of the common mistakes we should focus on and how tomathematically judge the error made.

Our motion capture setup included the Vicon Nexus sys-Vicon Nexus setuptem consisting of 8 Vicon Bonita cameras operating at 100Hz, mounted on tripods and camera mounts, focused at thetable where the motion capture activity took place as can beseen in Figure 4.4. This viewable area is called as the cap-ture volume.

As mentioned before Vicon Nexus tracks motion usingmarkers. These reflective markers are made out of specialretroeflective tape. The Vicon Bonita cameras are standarddigital cameras which are surrounded by a ring of LEDs.The infrared light emitted by these LEDs is reflected bythe markers and detected by the cameras. Since the light


Figure 4.4: Vicon Bonita camera setup directed towardscapture volume

that bounces off retroeflective tape is much more than othersurfaces, the camera system manages to efficiently differ-entiate between markers and other reflecting objects. Fur-ther, for the accurate reconstruction of the 3D coordinatesof the marker, Vicon Nexus demands that the marker bevisible by at least 2 cameras. Having fixed the markers asdiscussed in the previous subsection 4.1.1 “Understandingskill characteristics”, we proceeded to prepare the systemand subject for motion capture. During data collection pro-cess we wanted that the origin remain unchanged and theusers place the honing rod at the origin. To help achievethis goal we marked a box on the table with tape and as thefinal step of system preparation we set the volume originby placing the wand in the marked box. As part of subjectpreparation, we created Vicon Skeleton (.vsk) file for boththe knife and honing rod to be used for each capture ses-sion. After this we conducted pilot session as described inthe following subsection.


Pilot data collection session

Before going ahead with the main data collection session toThe purpose of thepilot study was to

verify cameraconfiguration

evaluate the correctness of our marker placement and over-all procedure, we carried out a pilot data collection session.The purpose of the user study was to find out whether thecurrent camera configuration yielded accurate results forthe way the novice users would handle the tools of the task.So, if the results were not as expected, the camera arrange-ment would need to be changed. We conducted the studywith a setup similar to the pilot session. We recruited 2participants (1 female and 1 male) for the pilot. Both theparticipants were right handed.

Each user was shown a demonstration video (on a Mac ProPilot study and taskdescription 22 inch display) with a focus on how the tools must be held

and their purpose. The users then had to perform 3 trials,where each trial comprised of 10 strokes of knife honing;this was captured using Vicon Nexus. After every trial theuser was shown the demonstration video again. During thestudy we recorded videos of the user’s performance usinga GoPro camera at 60fps. Later we analyzed the videos andthe captured data to determine how correct the camera ar-rangement was and how often the users make the commonmistakes.

The analysis showed that the camera arrangement couldPilot study resultssuccessfully capture the user data accurately and the com-mon mistakes identified were made by both users. Thus,these results permitted us to carry on with the collection oftraining data.

Participants

For the main data collection study we recruited a total of12 participants (7 female and 5 male) – 11 novice usersand 1 expert. The participants’ age was in the range of 23and 30 (average age 26). From the novice users 10 wereright handed and 1 was left-handed, and the expert wasright-handed. During the process, the participants were re-quested to use their dominant hand to hold the knife and


the other hand to hold the honing rod. On an average thedata collection required 0.5h per user.

Before beginning the data collection the participants wereasked to answer the questions, from the data collec-tion questionnaire (A “Background Information Question-naire”), regarding their previous experience in knife hon-ing/sharpening and knife skills. After the first trial theparticipants were told about the possible mistakes theycould make and were asked to explain about the mistakesthey thought they had made (B “Data Collection Question-naire”). The questions were posed to the participants andthe answers given were noted. 3 among the novice usershad previous sharpening knives with stones, 1 had someexperience sharpening knives using honing rod, while theremaining 7 had no experience at all in knife sharpen-ing/honing.

General procedure

The general procedure of data collection was similar to theprocedure followed during the pilot with the addition ofquestionnaires. Also, before starting, the users were givenan opportunity to familiarize themselves with the tools ofthe experiment to allow any correction in the way the toolswere held.

Post-processing of data

The trials capture in Vicon Nexus were reconstructed us- Reconstructing lostdata in Vicon Nexusing the Label and Reconstruct pipeline. In general there are

five types of data issues which can be observed after recon-struction as listed below.

• Markers may disappear for some milliseconds

• Markers lose their label and then become labeledagain

• After reappearing, the marker is wrongly labeled


• Markers disappear and appear, however remain un-labeled

• Markers position is not constant and it flashes errati-cally.

For majority of the users the number of gaps or missingmarkers was minimal. The reason for difference in dataquality from one capture to another can possibly be at-tributed speed of the user’s knife movement. We addressedthe data issues by either labeling missing data manually orconducting gap filling using internal algorithms of splinefill or pattern fill.

After completing the reconstruction successfully, the datawas annotated to indicate the start (Foot Off) and end (FootStrike) of every stroke. The time bar in Vicon Nexus al-lowed us to mark the left and right strokes distinctly.

Additionally, the videos of the participants were reviewed,Reviewing capturedvideos by the expert and us, to indicate the errors made (C “Stroke

Performance Table”)

Discussion

The participants were comfortable using the tools and theNovice user can’tmake out mistake

made. Hence, greatneed for feedback

way the markers were placed didn’t interfere with theirperformance. We asked the participants to explain the mis-takes they had made after each trial; the reported answersmore often than not didn’t match the mistakes that the ex-pert observed from the performance videos. However, themistakes indicated by the expert from the video reviewwere quite consistent with the mistakes determined fromthe mathematical analysis of the collected 3D data. Thus,we can deduce that a novice user is not able to understandwhether he has made a mistake, even though the differentkinds of mistakes have been described to him in detail. Fur-thermore, our analysis and review revealed that the errorscommonly made by the users are those of knife/honing rodangle and honing rod stability. Therefore, as part of our fi-

4.2 Feedback visualizations 37

Figure 4.5: Notations used in QOC (MacLean et al. [1991])

nal evaluation we decided to design visualizations specifi-cally for these two mistakes.

4.2 Feedback visualizations

We designed feedback visualizations to transform the datafrom user’s movements into a perceptually efficient visualformat based on our Dimension Space, so that the userscould view their errors along with the correct movementsfrom the expert.

4.2.1 Rationale

Although the Dimension Space doesn’t explicit include theQOC (Questions-Options-Criteria) method from the De-sign Space Analysis paradigm, we will use it to providejustification for the choices in our feedback visualization.The notations we have employed by carrying out the QOCmethod are depicted in Figure 4.5. Most of the questionsand options are based on the dimensions as defined in 3.2.2


Figure 4.6: QOC analysis of Dimension Space - Part 1

Figure 4.7: QOC analysis of Dimension Space - Part 2

“Axes of Dimension Space”. These are depicted in 4.6 and4.7.

The first question we can pose is What is tracked?. We con-For knife honing,objects should be

tracked


sider Body and Object as two options. The criteria Bodyframe movement different from object frame and Body & ob-ject frame move as whole are derived from the principles dis-cussed in 3.1.3 “Understanding human movement and itspatterns”. It basically claims that if the movement of thehand and the object do not differ then we could only trackthe object to generate data to provide feedback, whereas ifthe hand movement differs from the object movement wewould need to track the body part involved in the move-ment. Therefore, in the case of knife honing, as explainedin 4.1.1 “Understanding skill characteristics”, we choose totrack the objects that are the honing rod and the knife.

For the question When is feedback delivered?, the criteria High Terminal feedback isbetter for long termretention

level of assistance and Very low division of attention are basedon research done in the field of motor learning and hu-man computer interaction to compare the Options – Con-current and Terminal feedback. Concurrent feedback seemsto be the common choice for providing guidance as can beseen in the studies reviewed in 2 “Related work”. It in-dicates the nature and the direction of the movement forrequired corrections needed. However, studies ( Sigristet al. [2013], Schmidt and Wulf [1997], Alikhan et al. [2011])prove that concurrent feedback facilitates the correct per-formance briefly but it proves to be a great source of mo-tivation for the users. Also, concurrent feedback can causeinteractional discontinuity because the user may be forcedto divide his attention between the guidance and the actualperformance. Terminal feedback, on the other hand, avoidsthe interactional discontinuity altogether because the feed-back is provided once the user has finished the task per-formance. Further, terminal feedback is said to change thecognitive processes involved in motor learning thereby al-lowing for more effective learning. We chose to provideterminal feedback due to the above mentioned advantagesof the same.

The next question What’s the level of feedback information con- Distinct feedbackchosen for feedbackvisualization

tent? has the criteria More number of trials and Less numberof trials for its options Accumulated and Distinct respec-tively. Although Schmidt [1991] shows that accumulatedfeedback may be used to show the user the common sys-temic errors made over time, it also indicates that distinct


feedback can help to provide independent feedback abouteach discrete action. Consequently, to provide accumulatedfeedback we need to collect more data for which more tri-als are required, while for distinct feedback the feedbackinformation can be derived from few trials. Due to timerestrictions we have chosen to opt for Distinct feedback.

The criteria for the next question What is the capture per-Knife honing must becaptured from

horizontal planespective? is obvious and it is dependent on the axis aboutwhich the user performs the movements as discussed in3.1.3 “Understanding human movement and its patterns”.Since we have chosen the task of knife honing we need tocapture from the horizontal plane perspective.

The subsequent question is How adaptable to user experience?,Low adaptabilityused for feedback

visualizationsfor which the criteria vary among Completely adaptable toexperienced & novice users, Somewhat adaptable to experiencedusers and Only for novice users. These criteria are based onthe research (Newell [1991]) which explains that the levelof skill increases with practice over time and eventually thekind of feedback requires changes. At the nascent stage, theuser needs a very descriptive feedback because the mentalmodel for the task movements is in the process of forma-tion. But as the user practices and become more proficientin the task, less descriptive feedback can help his perfor-mance. For our visualizations we have chosen to designonly for novice users because time restrictions would notallow us to evaluate any visualizations designed for inter-mediate or experienced users.

The final question is What is the output? which relates toVisual feedbackprovides highest

information contenthow the feedback is presented to the user. The criteria is es-tablished on the characteristics of the different modes thatmay be employed for providing feedback. Investigations(Avanzini et al. [2009]) regarding the role of auditory feed-back in motor learning show that it proves to be very usefulin engaging the user in the task and providing encourage-ment. Weiss et al. [2014] elaborates that ”Haptic feedbackcan be used to add the perception of contact” during theperformance of the skill, whereas visual feedback gives ushighest amount of information to understand the mecha-nism of error-correction; therefore we chose to design forproviding visual feedback. However, we need to explore


Figure 4.8: Dimension Space for our Feedback Visualiza-tions

and precisely choose the kind of visual feedback to provide.Again, the criteria is derived from the features of the differ-ent kinds of visual feedback. We have chosen the option ofgraphs and animation because it can convey sufficient in-formation and is quick in terms of development time. Thus,our choices can be represented in a Dimension Space plotas shown in Figure 4.8.

4.2.2 Implementation details

We implemented a program in Python 2.71 to be executed Visualizationsdesigned usingPython as 3D graphs

in a supervised fashion, to generate the visualizations. Thereal time data from the user’s performance was capturedusing a simple Python library for retrieving data from aVicon motion capture system called pyvicon 2. The Mat-plotlib 1.4.3 library3 , which is a portable Python plotting

1http://www.python.org2https://github.com/cfinucane/pyvicon.git3http://matplotlib.org

http://www.python.org

https://github.com/cfinucane/pyvicon.git

http://matplotlib.org

http://matplotlib.org


Figure 4.9: Honing rod and knife angle visualization

package along with Scipy library4 of scientific tools, wereused to find the peaks and troughs from the collected data.One cycle of peak to trough represents one stroke of knifehoning. Data from first cycle was used to calculate hon-ing rod and knife angle and honing rod jitter and shown tothe user as two separate visualizations to reduce the user’scognitive load.

For the first visualization regarding the angle, the length ofConstructing anglevisualization the honing rod was divided into three ranges and the an-

gle between the honing rod and knife was calculated at thehighest point in each range. The resulting computationswere compared to the expert’s computations arrived at in asimilar manner. To represent the comparison between theexpert’s and the user’s angle, we plotted the surface con-taining the markers of the knife in a 3D graph as can beseen in Figure 4.9. The final visualization is shown as ananimation which shows the angles one by one in the threeranges.

In the second visualization for calculating the jitter, we es-Constructing jittervisualization tablished threshold values in 3D space from the expert’s

data and checked whether the honing rod markers’ coor-dinates lay within the threshold range. If yes, it was drawnin a 3D graph as an animated line in blue, else it was ren-dered in red. The expert’s honing rod was drawn as a static

4http://www.scipy.org/

http://www.scipy.org/

4.3 Preliminary Study 43

Figure 4.10: Honing rod jitter and stroke trace visualization

line. In addition to programming for these errors in this vi-sualization, we added the stroke trace of the expert in greenand showed the user’s stroke in blue to give a rough ideaas to how the ideal stroke pattern would look like as shownin Figure 4.10.

4.3 Preliminary Study

We conducted a preliminary study with 10 participants (2 Prelimnary studyneeded to confirmview point of 3Dgraph

female and 7 male) to confirm whether the view point se-lection of the 3D graph conveyed enough information tothe users to understand their mistakes and to ensure thatthe procedure we intended to follow during the full-scalestudy was unambiguous. The participants had little to noexperience in sharpening/honing knives.

We ran the study with a setup similar to the pilot forthe data collection session (4.1.2 “Pilot data collection ses-sion”), and maintained the elements we had decided uponduring the data collection process – Vicon Nexus motiondetection with 8 Vicon Bonita cameras, marking the centeron the table where the honing rod needed to be placed and22 inch Mac Pro display to show the demonstration videoand the feedback visualizations to the users.


Figure 4.11: Modified honing rod and knife angle visual-ization

Users were asked to view the demonstration video andGeneral procedure ofprelimnary study then perform the knife honing task (10 strokes/5 strokes on

either side) with the tools kept in front of them. After theirperformance was over they were shown the feedback visu-alizations one after the other on the screen in front of them.The entire study was video recorded and before moving onto the next knife honing trial, the users were asked to reportwhat they understood from the visualizations. Each userperformed 4 knife honing trials in total. After the perfor-mance, we allowed the user to interact with the 3D graphand indicate if they found another view point to be moreuseful in conveying their mistakes.

The results helped us adjust the azimuth and elevation ofResults of prelimnarystudy the 3D graph as shown in Figure 4.11 and Figure 4.12.

Having achieved the purpose of the preliminary study wecontinued with the main user study.

4.3 Preliminary Study 45

Figure 4.12: Modified honing rod jitter and stroke trace vi-sualization

47

Chapter 5

User study

We performed an experiment to evaluate our visualiza- The purpose of ourstudy is evaluatedesigned feedbackvisualizations

tion’s efficiency (in terms of improvement time) and effec-tiveness (in terms of better fitting to the expert performancemodel).

The independent variables for our experiment were de- Independentvariables of ourexperiment

fined as follows.

1. Feedback Type: This independent variable indicatesthe type of feedback which is provided to the usersafter their performance. Its scale is Nominal and ithas two levels as listed below.

• Visualization: The designed visualizations withabstract meta-data showing animated expertskill execution along with the user’s meta-data.

• Video: The video composed of expert’s actionsshowing a particular movement pattern.

2. Trial: This independent variable indicates the succes-sive trials of the task performed by the user. Its scaleis ordinal and it has 4 levels for 4 trials — t1, t2, t3 andt4.

The dependent variables were derived from mistakes iden- Dependent variablesof our experiment

48 5 User study

tified when conducting the pilot for the data collection ses-sion (4.1.2 “Pilot data collection session”). For our experi-ment we defined them as follows.

1. Angle: This is the first impact angle between the knifeand honing rod, which is measured and reported indegrees. It used to determine whether the impact an-gle of the user is in the range of 10 to 20 degrees.

2. Jitter: This is the difference between the maximumdeviation between the expert’s honing rod and theuser’s honing rod in all dimensions. It indicates thestability of the honing rod as compared to the expert’sand is measured in mm.

Before the experiment we hypothesized the following out-Our hypothesescome:

• H1: The feedback type has a significant influence onthe angle and jitter (in all dimensions).

• H2: The feedback type and number of trials interactto influence the angle and jitter (in all dimensions).

5.1 Task

The participants were asked to perform the task of knifeKnife honing taskwith ten strokes honing. They were requested to remove any jewelry or

metallic accessories to allow for better Vicon Nexus ac-curacy. Then, the participants were asked to watch thedemonstration video. We had decided to recruit onlyright-handed participants for the experiment. So after thedemonstration video, every participant was asked to holdthe knife in his right hand and the honing rod with his lefthand. Further, he was requested to hold the honing rod atthe marked box on the table which indicated the origin ofthe 3D coordinate system of the Vicon Nexus. Followingthis they were told to perform 1 trial (10 strokes) of knifehoning starting from the right side. After the trial they

5.2 Pilot study 49

were shown appropriate feedback. The process of perform-ing trial and viewing the feedback was repeated 3 moretimes. It was emphasized that the participants should try tocorrect their mistakes rather than perform the action morequickly.

5.2 Pilot study

Before conducting the main user study, we conducted a pi- Pilot study conductedto endurecorrectness ofgeneral procedure

lot user study to make sure that the general procedure andtask was well-defined and not tiring to the users. We ranthe study with a setup same as the one from 4.3 “Prelimi-nary Study”.

We recruited 2 users (2 male). Both of them had no ex- Pilot study procedureperience in honing/sharpening knives. Users were in-structed to perform the knife honing task with 10 strokesas described in the previous section 5.1 “Task”, and fill outquestionnaires from A “Background Information Question-naire”, D “Error Visualization Trial Review Questionnaire”and E “Error Visualization System Post Performance Ques-tionnaire” before, between the trials and after the entireexperiment was done respectively. During the study werecorded the videos of users performing the task.

From our analysis of the videos and questionnaire answers Results of pilot studywe concluded that the general procedure of the experimentand task were appropriately defined and didn’t prove tir-ing to the user. Thus, we continued to conduct the finaluser study for this work. The setup as described 5.2 “Pilotstudy” was used for the final experiment.

5.3 Participants

A total of 9 participants (2 female and 7 male) aged between23 and 29 (average age - 25) took part in our study. All theusers were right handed. They were all requested to usetheir dominant hand for holding the knife and the other

50 5 User study

hand for holding the honing rod just as in the pilot session.The study took 0.5h to complete on an average.

Before the viewing of the demonstration video we collectedGeneral procedurefollowed during

experimentinformation from the users about their previous knowledgerelated to knife honing/sharpening and their knife skills(A “Background Information Questionnaire”). After everytrial the users were asked to rate their understanding ofthe feedback for each mistake on a 5-point Likert scale (D“Error Visualization Trial Review Questionnaire”) whichranged from strongly agree to strongly disagree. After theentire experiment was done, the users had to fill out a finalquestionnaire (E “Error Visualization System Post Perfor-mance Questionnaire”) which was also based on a similar5-point Likert scale. This questionnaire was meant for eval-uating users’ overall understanding, preference and confi-dence level. Of the 9 participants, 4 had no previous expe-rience in honing a knife, 3 had some experience with thehoning rod, while 2 were experienced in sharpening knivesusing stone.

5.4 Experimental Design

We used a Within-Subjects design for both independentvariables – feedback and trial. Feedback was counterbal-anced. We had four blocks of Trials, and for every trial wehad 2 feedbacks. One feedback was paired with one fea-ture, that is to say, we have one feedback for angle andanother for jitter. So, we had 36 data points for each De-pendent Variable (9 participants x 2 feedbacks x 4 trials =36)

5.5 Results

We designed an algorithm to calculate the peaks andPeaks and troughsused to identify

strokes and calculateangle and jitter

troughs, from the received data. This algorithm first foundall the local maxima and minima in the z-axis data. Thesemaxima and minima were then filtered by calculating high-

5.5 Results 51

Figure 5.1: Performance Graph for User 1 - Trial 4

est peaks and troughs. Each consecutive pair of peak andtrough then signified one stroke of knife honing. Our algo-rithm identified the first ten strokes from the received dataas depicted in Figure 5.1. For each stroke we then calculatedthe first impact angle between the honing rod and knife atthe peak point of every stroke. We also calculated the jit-ter (in all dimensions) for every frame of each stroke. Forboth angle and jitter (in all dimensions) for one stroke, wecomputed the median to be used in the final data analysis.Using median allowed the data to not be sensitive towardsthe outliers. The resulting data met with the main assump-tions as listed below.

• Variance between groups (Visualization and Video)was homogeneous.

• Data was normally distributed.

Thus, it was analyzed using two-way ANOVA.

For the honing-rod knife impact angle the p-value (Feed-back Type) = 0.1154 � 0.05 = ↵. And, the values for p-value (Trial) = 0.6886 � 0.05 = ↵, and p-value (interactions)= 0.4377 � 0.05 = ↵. The results are as shown in Figure 5.2

For the honing-rod knife jitter (x-axis) the p-value (Feed-back Type) = 0.07920 � 0.05 = ↵. And, the values for p-

52 5 User study

Figure 5.2: Box plot for impact angle from two-wayANOVA results

Figure 5.3: Box plot for jitter (x-axis) from two-wayANOVA results

value (Trial) = 0.5517 � 0.05 = ↵, and p-value (interactions)= 0.2710 � 0.05 = ↵. The results are as shown in Figure 5.3

For the honing-rod knife jitter (y-axis) the p-value (Feed-back Type) = 0.58215 � 0.05 = ↵. And, the values for p-value (Trial) = 0.37272 � 0.05 = ↵, and p-value (interactions)= 0.4838 � 0.05 = ↵. The results are as shown in Figure 5.4

5.6 Discussion 53

Figure 5.4: Box plot for jitter (y-axis) from two-wayANOVA results

Figure 5.5: Box plot for jitter (z-axis) from two-wayANOVA results

For the honing-rod knife jitter (z-axis) the p-value (Feed-back Type) = 0.75605 � 0.05 = ↵. And, the values for p-value (Trial) = 0.00769 0.05 = ↵, and p-value (interactions)= 0.0165 0.05 = ↵. The results are as shown in Figure 5.5

5.6 Discussion

Using two-way ANOVA both hypothesis are rejected; no Both hypothesesrejected due to lackof evidence

significance was found. However, it is important to note

54 5 User study

that our results were limited by certain factors.

First of which is that we had asked the users to place theChange in referenceorigin might be a

limitationhoning on a marked box on the table which served as theorigin of the Vicon Nexus 3D space. Although, the cali-bration was unchanged since the collection of the expert’sdata, there is a possibility that the user’s origin point wasskewed.

Further, four trials may not be sufficient to see improve-More trials areperhaps required for

witnessing anychange

ment in the results. There is need to verify what is the num-ber of trials which is enough to enable all users to learn theskill.

Also, we had missing data because 2 of the participantsMissing data fromusers might be a

limitationrecorded only 9 strokes instead of 10. Consequently in ourdata analysis we had to exclude the information regardingthe tenth stroke from the remaining participants.

Another possible reason for obtaining unexpected resultsPossibly more usersare needed for the

user studycould be that the number of users was less. While it is acommonly held belief that a minimum of 5 users is suf-ficient for user testing, the number of users generally de-pends on the scope of the study.

5.7 Summary

To conclude the user study we must say that the quantita-tive analysis didn’t provide any insights on if or how the in-dependent variables affect the dependent variables. Fromour analysis we can say that this was due to missing datapoints or limited number of users or the number of trialscould have been too little to show any change or the originreference point was changed among users.

Nonetheless, the answers of questionnaires answered byUsers’ commentscontrary to dataanalysis results

the users and the comments received after each study wereencouraging and contradictory to the results obtained fromquantitative analysis.

After the first viewing the video feedback the first time,It was not easy tojudge movements

from video

5.7 Summary 55

the users reported that they found it difficult to understandhow the knife and honing should be held properly becausethe knife in the video was larger than knife provided dur-ing the user study. The visualization feedback on the otherhand had no such issues because it was an abstract repre-sentation.

The user’s also complained that while the video did give Video provided staticfeedback, it didn’tallow for reflection

an idea of how to correct the mistakes related to angle andjitter, the video didn’t tell them anything about their per-formance other than the fact that they had a mistake. Theusers were pleased with the way the jitter visualization al-lowed them to compare their performance relative to theexpert’s performance.

Most of the users understood how to read the visualization Video didn’t provideany new informationafter two trials

feedback after the first trial and reported that they wouldnot want a higher fidelity visualization for the task at hand.The users also expressed their dissatisfaction towards thevideo feedback saying that after two trials the video didn’tprovide any additional information and certainly couldn’ttell them how the mistake should be corrected.

It is important to note that several users stated that they Real-time feedbackdesired by usersthought that the abstract visualization would be most help-

ful when provided in real-time. This contradicts our ra-tionale of When is feedback delivered? as explained in 4.2.1“Rationale”. Nevertheless, this is something that we needto test in our future work.

Post the study most of the participants felt that they were Users were confidentabout performingknife honing

quite confident about performing the knife honing task.Moreover, they appeared quite eager to learn more skillswith the assistance of visualizations such as those designedby us. The suggestions of skills provided by the partic-ipants includes shooting, playing an instrument, swordfighting, flying an airplane and rowing.

57

Chapter 6

Summary and futurework

This chapter presents a retrospection of what has been donein this thesis. It summarizes the problems addressed in thework and contributions for the field of human computerinteraction. After that, an outlook is provided for the nextsteps in the future.

6.1 Summary and contributions

Overall, this work is an analysis of suitable performance Summary of the workfeatures for a skill with the objective of designing feed-back visualizations which can provide sufficient informa-tion for correction of mistakes when learning the skill’smovements. From our review of the related work inChapter 2 “Related work”, we found that there was aneed to digitize a skill. Thus, this thesis described howa skill’s performance measurement criteria may be deter-mined on the basis of discussions with the expert, observa-tion of the expert’s performance, and insights from dimen-sion space constructed to assist in making design choices.Furthermore, this thesis presented feedback visualizationsdesigned on the basis of deduced performance measure-ment criteria and tested these visualizations as compared

58 6 Summary and future work

to videos to understand their effect on user’s performance.

Chapter 3 “Taxonomy and Dimension Space” introducedbackground information about types of skills and describedthe scope of skills which interested us the most — Fine,closed, internally paced, discrete and individual skills.From motor learning literature, basics about feedback andhuman movement patterns were presented. Then, the Di-mension Space for assisting in design decisions was ex-plained which had six axes — Feedback timing, Feedbackinformation content, Capture perspective, Output capabil-ity, User experience variability and Tracked unit. We alsoconstructed Dimension Space plots for few of the studiesdiscussed in related work and gained further understand-ing of the design choices that have been made in existingresearch.

Chapter 4 “Performance Measurement Criteria and Visual-ization” then provided a clear specification of the task tobe learned - Knife Honing. With the help of domain expertand available literature, key aspects and performance mea-sures as well as improvement criteria to evaluate a learner’sprogress were identified. We established that there are twomajor criteria for evaluating a learner’s knife honing skills;first, the angle between the honing rod and knife, and sec-ond, the stability of the honing rod. Based on available in-formation and understanding, we also finalized on the ap-propriate methodology to capture the skill movements us-ing Vicon Nexus motion capture system. Afterwards, usingthe captured data we determined thresholds for the criteriaestablished to evaluate any learner’s performance. Subse-quently, with Q-O-C notations, the Dimension Space wasanalyzed and the abstract feedback visualizations build onthe design choices were presented.

Chapter 5 “User study” described the pilot and main userstudy with the task of knife honing conducted to verify thelevel of improvement in learner’s performance with the de-signed feedback visualizations. However, the results couldnot prove the hypotheses. The unexpected results are prob-ably due to missing data and less number of trials. Majorityof the users claimed in the post-study interview that afterthe first two trials the videos didn’t provide any extra infor-

6.2 Future work 59

mation to help improve their performance while the visu-alizations gave them precise information about their move-ments.

Thus, this thesis has systematically completed the threecontributions as listed in Chapter 1 “Introduction”. We pre-sented the performance features for the task of knife hon-ing and described a Dimension Space as our first contribu-tion. The methodology followed could be adopted to deter-mine the performance criteria and key indices to evaluateany skill which falls under our selected scope. Besides thiswe designed and evaluated feedback visualizations whichwere characterized by the determined evaluation parame-ters. And because, the user study didn’t prove the validityof the designed visualization it leaves a lot of room for fu-ture work and improvement.

6.2 Future work

Our work falls under the category of systems which deal Machine learningcould be employed tostrengthen the errordetection process

with enabling acquiring of a task-oriented knowledge. Thisnaturally implies that it interacts with humans and it wouldbe great if they closely parallel human abilities of learning.In other words, it would be great if it was capable of learn-ing, memory and pattern recognition. This would not onlyallow detection of more performance indicators, but alsoimprove the method in which the mistakes of the learneris judged. This is where machine learning comes into play(Michalski et al. [2013]); to be specific, supervised learningalgorithms could be applied to build a system which is ca-pable of extracting a model of movements from the expertand assessing the performance of novice users. The data wecollected for the purpose of establishing the threshold val-ues for the performance measurement criteria can be usedas the training set for our task of knife honing; the expert’sdata as the positive samples and the data from the noviceusers as the negative samples. By applying a strong low-pass filter the repetitions of knife honing can be detectedand the extracted features can be used to train a HiddenMarkov Model. Hence, in the future we can extend thiswork to include the machine learning approach and also

60 6 Summary and future work

address the designing of higher fidelity feedback visualiza-tions.

61

Appendix A

Background InformationQuestionnaire

This appendix contains the questionnaire used to gatherthe background information about the user during both thedata collection procedure and the final user study to evalu-ate the error visualization system.

62 A Background Information Questionnaire

Figure A.1: Background Information Questionnaire page 1

63

Figure A.2: Background Information Questionnaire page 2

65

Appendix B

Data CollectionQuestionnaire

This appendix contains the questionnaire used to gather in-formation as to the mistakes that the users thought they hadmade.

66 B Data Collection Questionnaire

Figure B.1: Data Collection Questionnaire

67

Appendix C

Stroke PerformanceTable

This appendix contains the Stroke Performance Table forUser 1. The table was constructed in the similar manner forthe remaining users. The table was completed by reviewingvideos of the participants by the expert and us, to indicatethe errors. We also found certain other errors apart from theones that we were already investigating. The description ofthese errors is given as follows

• OP1 - Knife movement along the edge is incorrect:User doesn’t pull down the knife in an arching mo-tion instead the user either pulls down the knife in amanner that it is almost parallel to the table or in astraight downward manner.

• OP2 - Side is not switched while knife sharpening:User sharpens one side the required number of timesin a row rather than switching sides and performingthe knife sharpening action alternately on each side.

Since, the occurrence of these errors was limited to only avery small fraction of users, we didn’t consider them whiledesigning our visualizations.

68 C Stroke Performance Table

Figure C.1: Stroke Performance Table for User 1 - Page 1


69


71

Appendix D

Error Visualization TrialReview Questionnaire

This appendix contains the questionnaire used to gather in-formation about the understanding of the user related tothe feedback provided for the mistakes of knife-honing rodangle and honing rod jitter.

72 D Error Visualization Trial Review Questionnaire

Figure D.1: Trial Review Questionnaire

73

Appendix E

Error VisualizationSystem PostPerformanceQuestionnaire

This appendix contains the questionnaire used to collect in-formation from the user post the knife honing performancerelated to the user’s confidence.

74 E Error Visualization System Post Performance Questionnaire

Figure E.1: Post Performance Questionnaire

75

Bibliography

Mir Alikhan, Rachel M Brown, Casey Carver, CeciliaChavez, Kourtney Jones, Emily Krizmanic, T ScottMarzilli, Amanda Pike, Mark Saroni, and Janay Stradley.The effect of concurrent and terminal feedback onvo2max. In International Journal of Exercise Science: Con-ference Proceedings, volume 2, page 37, 2011.

Fraser Anderson, Tovi Grossman, Justin Matejka, andGeorge Fitzmaurice. Youmove: Enhancing movementtraining with an augmented reality mirror. In Proceed-ings of the 26th Annual ACM Symposium on User InterfaceSoftware and Technology, UIST ’13, pages 311–320, NewYork, NY, USA, 2013. ACM. ISBN 978-1-4503-2268-3.doi: 10.1145/2501988.2502045. URL http://doi.acm.

org/10.1145/2501988.2502045.

Lorin W Anderson, David R Krathwohl, and Ben-jamin Samuel Bloom. A taxonomy for learning, teaching,and assessing: A revision of Bloom’s taxonomy of educationalobjectives. Allyn & Bacon, 2001.

Robert J Armstrong et al. Developing and writing behav-ioral objectives. 1970.

Federico Avanzini, Amalia De Gotzen, Simone Spagnol,and Antonio Roda. Integrating auditory feedback inmotor rehabilitation systems. In Proceedings of Interna-tional Conference on Multimodal Interfaces for Skills Transfer(SKILLS09), volume 232, 2009.

Roger Bartlett. Introduction to sports biomechanics: Analysinghuman movement patterns. Routledge, 2007.

Peter J Beek and Arthur Lewbel. The science of juggling.Scientific American, 273(5):92–97, 1995.

http://doi.acm.org/10.1145/2501988.2502045

http://doi.acm.org/10.1145/2501988.2502045

76 Bibliography

D. Birnbaum, R. Fiebrink, J. Malloch, and M. M. Wan-derley. Towards a dimension space for musical de-vices. In Proceedings of the 2005 Conference on New In-terfaces for Musical Expression, NIME ’05, pages 192–195,Singapore, Singapore, 2005. National University of Sin-gapore. URL http://dl.acm.org/citation.cfm?

id=1085939.1085993.

Ian M Bullock, Raymond R Ma, and Aaron M Dollar. Ahand-centric classification of human and robot dexterousmanipulation. Haptics, IEEE Transactions on, 6(2):129–144,2013.

S.K. Card and J. Mackinlay. The structure of the informationvisualization design space. In Information Visualization,1997. Proceedings., IEEE Symposium on, pages 92–99, Oct1997. doi: 10.1109/INFVIS.1997.636792.

Cain Carroll and Revital Carroll. Mudras of India: a compre-hensive guide to the hand gestures of yoga and Indian dance.Singing Dragon, 2012.

Winyu Chinthammit, Troy Merritt, Scott Pedersen, AndrewWilliams, Denis Visentin, Robert Rowe, and Thomas Fur-ness. Ghostman: Augmented reality application for tel-erehabilitation and remote instruction of a novel motorskill. BioMed research international, 2014, 2014.

B. Davis, R. Bull, and J. Roscoe. Physical education and thestudy of sport. Mosby, 2000. ISBN 9780723431756.

Bob Davis, Ros Bull, Jan Roscoe, Dennis Roscoe, Mike Saiz,Rob Curran, and Rob Dean. Physical education and thestudy of sport. Wolfe, 1991.

C. Duncan. The Art of Classical Guitar Playing. TheArt of Series. Alfred Music Publishing. ISBN9781457400391. URL https://books.google.co.

in/books?id=ocwwb7WImqoC.

Simpson Ej. The classification of educational objectives inthe psycho motor domain. Rockville, ERIC, 1966.

Krzysztof Z. Gajos, Mary Czerwinski, Desney S. Tan, andDaniel S. Weld. Exploring the design space for adaptivegraphical user interfaces. In Proceedings of the Working

http://dl.acm.org/citation.cfm?id=1085939.1085993

http://dl.acm.org/citation.cfm?id=1085939.1085993

https://books.google.co.in/books?id=ocwwb7WImqoC

https://books.google.co.in/books?id=ocwwb7WImqoC

Bibliography 77

Conference on Advanced Visual Interfaces, AVI ’06, pages201–208, New York, NY, USA, 2006. ACM. ISBN 1-59593-353-0. doi: 10.1145/1133265.1133306. URL http:

//doi.acm.org/10.1145/1133265.1133306.

F. Galligan, C. Maskery, and D. Crawford. Advanced PEfor Edexcel: Teacher’s resource file. Advanced Pe forEdexcel Series. Heinemann Educational, 2002. ISBN9780435506421.

Angela Jane Glynn and Helen Fiddler. The physiotherapist’spocket guide to exercise: assessment, prescription and training.Elsevier Health Sciences, 2009.

Scott T Grafton, John C Mazziotta, Sharon Presty, Karl JFriston, RS Frackowiak, and Michael E Phelps. Func-tional anatomy of human procedural learning deter-mined with regional cerebral blood flow and pet. TheJournal of neuroscience, 12(7):2542–2548, 1992.

T. C. Nicholas Graham, Leon A. Watts, Gaelle Calvary,Joelle Coutaz, Emmanuel Dubois, and Laurence Nigay.A dimension space for the design of interactive systemswithin their physical environments. In Proceedings ofthe 3rd Conference on Designing Interactive Systems: Pro-cesses, Practices, Methods, and Techniques, DIS ’00, pages406–416, New York, NY, USA, 2000a. ACM. ISBN 1-58113-219-0. doi: 10.1145/347642.347799. URL http:

//doi.acm.org/10.1145/347642.347799.

TC Graham, Leon A Watts, Gaelle Calvary, Joelle Coutaz,Emmanuel Dubois, and Laurence Nigay. A dimensionspace for the design of interactive systems within theirphysical environments. In Proceedings of the 3rd conferenceon Designing interactive systems: processes, practices, meth-ods, and techniques, pages 406–416. ACM, 2000b.

Anita J Harrow. A taxonomy of the psychomotor domain: Aguide for developing behavioral objectives. Addison-WesleyLongman Ltd, 1972.

Peter Hertzmann. Knife skills illustrated: a user’s manual.WW Norton & Company, 2007.

Sarah Jay and Ben Fink. Knives Cooks Love: Selection. Care.Techniques. Recipes. Andrews McMeel Publishing, 2008.

http://doi.acm.org/10.1145/1133265.1133306

http://doi.acm.org/10.1145/1133265.1133306

http://doi.acm.org/10.1145/347642.347799

http://doi.acm.org/10.1145/347642.347799

78 Bibliography

B. Knapp. Skill in Sport: The Attainment of Proficiency. Rout-ledge & Kegan Paul, 1963. ISBN 9780710085627.

R. Long and C. Macivor. The Key Muscles of Yoga. Sci-entific keys. Bandha Yoga Publications, 2009. ISBN9781607432388. URL https://books.google.co.

in/books?id=LbcPwWawUWEC.

Bob Longe. 101 Amazing Card Tricks. Sterling PublishingCompany, Inc., 1993.

Wendy E. Mackay, Anne-Laure Fayard, Laurent Frobert,and Lionel Medini. Reinventing the familiar: Explor-ing an augmented reality design space for air trafficcontrol. In Proceedings of the SIGCHI Conference on Hu-man Factors in Computing Systems, CHI ’98, pages 558–565, New York, NY, USA, 1998. ACM Press/Addison-Wesley Publishing Co. ISBN 0-201-30987-4. doi: 10.1145/274644.274719. URL http://dx.doi.org/10.1145/

274644.274719.

Allan MacLean and Diane McKerlie. Design space analysisand use-representations. Scenario-based design: envision-ing work and technology in system development, pages 183–207, 1995.

Allan MacLean, Richard M Young, and Thomas P Moran.Design rationale: the argument behind the artifact. InACM SIGCHI Bulletin, volume 20, pages 247–252. ACM,1989.

Allan MacLean, Richard M Young, Victoria ME Bellotti, andThomas P Moran. Questions, options, and criteria: Ele-ments of design space analysis. Human–computer interac-tion, 6(3-4):201–250, 1991.

Morton Manus and Ron Manus. Teach yourself to play guitar.Alfred Music Publishing, 1992.

Ryszard S Michalski, Jaime G Carbonell, and Tom MMitchell. Machine learning: An artificial intelligence ap-proach. Springer Science & Business Media, 2013.

K. Mitobe, M. Tomioka, M. Saito, and M. Suzuki. De-velopment of a ubiquitous learning system for dexter-ous hand operation. In Mixed and Augmented Reality (IS-

https://books.google.co.in/books?id=LbcPwWawUWEC

https://books.google.co.in/books?id=LbcPwWawUWEC

http://dx.doi.org/10.1145/274644.274719

http://dx.doi.org/10.1145/274644.274719

Bibliography 79

MAR), 2012 IEEE International Symposium on, pages 299–300, Nov 2012. doi: 10.1109/ISMAR.2012.6402583.

Dharma Mittra. Asanas: 708 Yoga Postures. New World Li-brary, 2002.

Yoichi Motokawa and Hideo Saito. Support system for gui-tar playing using augmented reality display. In Mixed andAugmented Reality, 2006. ISMAR 2006. IEEE/ACM Interna-tional Symposium on, pages 243–244. IEEE, 2006.

Jorg Muller, Florian Alt, Daniel Michelis, and AlbrechtSchmidt. Requirements and design space for interactivepublic displays. In Proceedings of the International Con-ference on Multimedia, MM ’10, pages 1285–1294, NewYork, NY, USA, 2010. ACM. ISBN 978-1-60558-933-6.doi: 10.1145/1873951.1874203. URL http://doi.acm.

org/10.1145/1873951.1874203.

KM Newell. Motor skill acquisition. Annual review of psy-chology, 42(1):213–237, 1991.

KM Newell and Mary J Carlton. Augmented informationand the acquisition of isometric tasks. Journal of motorbehavior, 19(1):4–12, 1987.

KM Newell and PA Hancock. Kinematic and kinetic pa-rameters as information feedback in the acquisition ofman-machine skills. In Proceedings of the Human Factorsand Ergonomics Society Annual Meeting, volume 25, pages671–675. Sage Publications, 1981.

Willard A Palmer, Morton Manus, Amanda Vick Lethco,and Alfred Pub. Alfred’s Basic Piano Library. Alfred Pub-lishing Company, 1982.

Alexandra Pfister, Alexandre M West, Shaw Bronner, andJack Adam Noah. Comparative abilities of microsoftkinect and vicon 3d motion capture for gait analysis.Journal of medical engineering & technology, 38(5):274–280,2014.

M. Rama and B. Miller. Cooking Basics For Dummies.–For dummies. Wiley, 2014. ISBN 9781118922323.URL https://books.google.co.in/books?id=

i9o5BAAAQBAJ.

http://doi.acm.org/10.1145/1873951.1874203

http://doi.acm.org/10.1145/1873951.1874203

https://books.google.co.in/books?id=i9o5BAAAQBAJ

https://books.google.co.in/books?id=i9o5BAAAQBAJ

80 Bibliography

Richard A Schmidt. Frequent augmented feedback can de-grade learning: Evidence and interpretations. In Tutorialsin motor neuroscience, pages 59–75. Springer, 1991.

Richard A Schmidt and Tim Lee. Motor control and learning.Human kinetics, 1988.

Richard A Schmidt and Gabriele Wulf. Continuous concur-rent feedback degrades skill learning: Implications fortraining and simulation. Human factors, 39(4):509–525,1997.

Roland Sigrist, Georg Rauter, Robert Riener, and PeterWolf. Terminal feedback outperforms concurrent vi-sual, auditory, and haptic feedback in learning a complexrowing-type task. Journal of motor behavior, 45(6):455–472,2013.

RA Smith and TD Lee. Motor control and learning: a be-havioural emphasis, 1998.

Rajinder Sodhi, Hrvoje Benko, and Andrew Wilson. Light-guide: Projected visualizations for hand movement guid-ance. In Proceedings of the SIGCHI Conference on HumanFactors in Computing Systems, CHI ’12, pages 179–188,New York, NY, USA, 2012. ACM. ISBN 978-1-4503-1015-4. doi: 10.1145/2207676.2207702. URL http://doi.

acm.org/10.1145/2207676.2207702.

Richard Tang, Hesam Alizadeh, Anthony Tang, Scott Bate-man, and Joaquim A.P. Jorge. Physio@home: Designexplorations to support movement guidance. In CHI’14 Extended Abstracts on Human Factors in ComputingSystems, CHI EA ’14, pages 1651–1656, New York, NY,USA, 2014. ACM. ISBN 978-1-4503-2474-8. doi: 10.1145/2559206.2581197. URL http://doi.acm.org/

10.1145/2559206.2581197.

Richard Tang, Anthony Tang, Xing-Dong Yang, Scott Bate-man, and Joaquim Jorge. Physio@ home: Exploring vi-sual guidance and feedback techniques for physiother-apy exercises. In Proceedings of the 33rd Annual ACMConference on Human Factors in Computing Systems, pages4123–4132. ACM, 2015.

http://doi.acm.org/10.1145/2207676.2207702

http://doi.acm.org/10.1145/2207676.2207702

http://doi.acm.org/10.1145/2559206.2581197

http://doi.acm.org/10.1145/2559206.2581197

Bibliography 81

Eduardo Velloso, Andreas Bulling, and Hans Gellersen.Motionma: Motion modelling and analysis by demon-stration. In Proceedings of the SIGCHI Conference on HumanFactors in Computing Systems, pages 1309–1318. ACM,2013.

Colin Ware. Information visualization: perception for design.Elsevier, 2012.

Matthias Weber, Guido Heumer, and Bernhard Jung. To-wards grasp learning in virtual humans by imitation ofvirtual reality users.

Matthias Weing, Amrei Rohlig, Katja Rogers, Jan Gugen-heimer, Florian Schaub, Bastian Konings, Enrico Rukzio,and Michael Weber. Piano: enhancing instrument learn-ing via interactive projected augmentation. In Proceedingsof the 2013 ACM conference on Pervasive and ubiquitous com-puting adjunct publication, pages 75–78. ACM, 2013.

Patrice L Weiss, Emily A Keshner, and Mindy F Levin. Vir-tual Reality for Physical and Motor Rehabilitation. Springer,2014.

Mario Wiesendanger and Deborah J. Serrien. Toward aphysiological understanding of human dexterity. Phys-iology, 16(5):228–233, 2001. ISSN 1548-9213.

Nicola Wood. The new journeyman: The role of an expertlearner in eliciting and transmitting skilled knowledge.2009.

Xiao Xiao and Hiroshi Ishii. Duet for solo piano: Mir-rorfugue for single user playing with recorded perfor-mances. In CHI’11 Extended Abstracts on Human Factorsin Computing Systems, pages 1285–1290. ACM, 2011.

Monica Zaczynski and Anthony D. Whitehead. Establish-ing design guidelines in interactive exercise gaming: Pre-liminary data from two posing studies. In Proceedingsof the 32Nd Annual ACM Conference on Human Factorsin Computing Systems, CHI ’14, pages 1875–1884, NewYork, NY, USA, 2014. ACM. ISBN 978-1-4503-2473-1.doi: 10.1145/2556288.2557329. URL http://doi.acm.

org/10.1145/2556288.2557329.

http://doi.acm.org/10.1145/2556288.2557329

http://doi.acm.org/10.1145/2556288.2557329

83

Index

abbrv, see abbreviationANOVA, 50–51

design rationale, 37- QOC, 37

dimension space, 22

extrinsic feedback, 18

feedback visualizations, 37future work, 57–58

human movement pattern, 19–20

intrinsic feedback, 18

knife honing, 29knife honing errors, 30

manipulation taxonomy, 21

scope of skills, 17- closed skill, 17- discrete skill, 17- fine skill, 17- individual skill, 17- internally paced skill, 17

skills, 15–16- affective skills, 16- cognitive skills, 16- psycho-motor skills, 16

vicon nexus, 32

Typeset August 10, 2015

conveying feedback in skill movement...

Documents