evaluating touch gesture usability -- d4m 2010

Evaluating Touch

Gesture Usability

Kevin Arthur

Sr. User Experience Researcher

[email protected], twitter @karthur

1Copyright © 2010 Synaptics Inc.9/22/2010

Overview

Gestures

• Technical properties

• Usability challenges

Methods for testing gesture usability• Test framework

• Examples from TouchPad gesture studies

• Considerations for mobile


Synaptics

3

TouchPad ClearPad

+ Portable music players, computer peripherals, large touchscreens+ Gesture suites

Capacitive touch technology

Copyright © 2010 Synaptics Inc.9/22/2010

Synaptics User Experience

• Usability

• Research

• Concepts

• Design (*hiring)

• Customer partnerships

• Integrated into product development and customer projects

4

Fuse Concept (with TAT, TheAlloy, Immersion, Texas Instruments)


Gesture Properties and

Challenges


Defining Touch Gestures

• Typically a continuous sequence of touch events and a motion trajectory

• Performs an action or adjusts a parameter

• Multi-finger or single-finger• Touchscreen (direct touch)• Touchpad (indirect touch)

• Gestures don’t belong everywhere:“Everything is best for something and worst for something else.” – Bill Buxton*

Touchpad examples

6

Two-Finger Pinch Zoom

Two-Finger Rotate

Three-Finger Flick

Circular Scrolling

* http://www.billbuxton.com/multitouchOverview.html


http://www.billbuxton.com/multitouchOverview.html

Gestures: Reactions & Challenges

• Useful, usable, enjoyable“Efficient,” “Super smooth,”

“Finger-flicking fun”

• Discoverability, learnability“Did not know how to do

the gesture”

• Gesture recognition“Tuned too tightly,” “Failed

more often than not”

• Misrecognition and unintended gestures“Accidentally resize”

• Responsiveness, satisfying results“Taking an eternity to

respond”


Important Properties of Gestures

8

• Gestures must be taught, discoverable

• Testing the documentation is importantLack of Affordances

• Results are not always the sameNondeterministic

• Recognition trade-offs exist between gestures in same parameter space

• Gestures should be evaluated as a setInterdependent

• Touchpad’s primary use is still for pointing and scrolling; gestures shouldn’t interfere

Interface Overloading

• Hand size, long fingernailsUser Variation


Phases in Performing a Gesture

[Intention]

Registration

Continuation

Termination

9

“I want to enlarge this photo and I’ll use pinch zoom”

Two fingers touch down and move together or apart

Fingers continue moving

Fingers lift

Does user discover and do the gesture properly? (Q1)

Is the gesture recognized correctly and reliably? (Q2)

Is the result satisfactory? (Q3)

Has the user completed the task successfully? (Q4)

Phase Example Research Questions

Phases from Mike Wu et al., 2006.Copyright © 2010 Synaptics Inc.9/22/2010

Goals when Testing Gestures

10

Respond to stakeholder needs

• Developers want feedback – formative usability studies

• Sales and OEM partners want data – summative and competitive studies

Test in context

• Holistic evaluation; test with users in realistic scenarios with working system

• Different from offline gesture recognizer tests

Answer research questions

• Discoverability, learnability, accuracy, task success and satisfaction


Usability Test Framework and

Methods


Test Framework

12

1. Gesture Introduction

• Scripted new-user experience

2. Familiarization and Practice Task

• Structured practice; assess discovery, understanding (Q1 - Intention)

3. Accuracy Task

• Measure gesture recognition rates (Q2 – Registration)

4. Satisfaction Questionnaire and Debrief

• Assess overall ease of use and preference (Q3, Q4 – Continuation, Termination)


Example Study

• 2 devices, 5 gestures, 12 participants

• One-hour session plan


The examples on the following slides are drawn from studies like this one. Their purpose here is just to illustrate the methods and not to indicate real results.

1. Gesture Introduction Example

14

Simulate new user experience• “Out of box experience” material

• Help videos

• Try the gesture until success

• If no success, moderator assists


2. Familiarization and Practice

Task: Pinch Zoom Example

Zoom in on the South Residences, near the top of the map, and find the building called The Knoll. Zoom all the way in on The Knoll and then zoom all the way back out.

15

2000 × 2000 pixels or higher.Approximately five pinch-zooms required.


2. Familiarization and Practice:

Flick and Rotate Example

17

Scenario: sorting and tagging vacation photos


2. Familiarization and Practice Task:

Flick and Rotate Example

18

1. Use the rotate gesture to make the image upright.2. Type the image’s title into the caption field and press Enter.3. Use the flick gesture to go to the next image.


19

Users given a set of randomly oriented photos to rotate and tag.


3. Accuracy Task Example

User performs a set number of gesture attempts.

Moderator records system responses (correct / no response / misrecognized).

Example with sample data:

20

Gesture Correct No ResponseMisrecognized Other/

NotesAs Pinch As Rotate

Pinch Zoom In

7 2 - 1

Pinch Zoom Out

10 0 - 0

Rotate Clockwise

6 3 1 -“requirestoo much motion”

Rotate Counter-clockwise

7 1 2 -


3. Accuracy Task: Sample Result Chart

21

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Pinch Zoom In

Pinch Zoom Out

Rotate CW

Rotate CCW

Average

Avera

ge R

eco

gn

itio

n R

ate

Misrecognized

No Response

Correct

Unified Measure

Target~90% Correct


3. Accuracy Task: Sample Results from

a Competitive Study

22

61

82

96

83

88

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2F Pinch Zoom

2F Rotate 3F Flick LR

3F Flick UD

2F Scroll

System AAverage Correct 82%

Correct No response Incorrect

94

73

85

94 96

90

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2F Pinch Zoom

2F Rotate 3F Flick LR

3F Flick UD

3F Press 2F Scroll

System BAverage Correct 89%

Correct No response Incorrect


On rotate, the high rate of misrecognition as pinch here indicated a need to balance the gesture algorithms better.

4. Satisfaction Questionnaire: Pinch

Zoom Example


Note mix of generic questions (1-2, 5-8) and specific questions about a particular gesture’s performance (3,4).

4. Satisfaction Questionnaire: Pinch

Zoom Example


4. Satisfaction Questionnaire: Sample

Result

25

7.38

4.38

7.88

8.75

8.25

1 2 3 4 5 6 7 8 9

Pinch

Rotate

3F Flick

3F Press

2F Scroll

I would use this gesture if it were available(strongly disagree = 1, strongly agree = 9)

Strongly

Disagree

Strongly

Agree


4. Satisfaction Questionnaire,

Alternative Rating Format

How well do the gestures work on each system?

Please rate from 1 (Poor) to 5 (Excellent)

Gesture System A System B System C

Pinch Zoom

Rotate N/A

Flick Left-Right

Flick Up-Down N/A

Three-Finger Press N/A N/A

Two-Finger Scrolling


A rating format like this can be more practical for a large study when there is less time to complete questionnaires.

1

2

3

4

5

Pinch Zoom 2F Rotate 1F Rotate (Chiral)

Flick LR Flick UD 1F Scroll (Linear)

1F Scroll (Circular)

2F Scrolling

User Ratings With 95% Confidence Intervals

System A System B System C

4. Satisfaction Questionnaire: Sample

Results, Alternative Rating Format

27

Excellent

Poor


Related Tests

Measure gesture side-effects

• Unintended gestures

• Degradation on other tasks

Other usability tests for touch devices

• Pointing/selection

• Scrolling

• Text input


Mobile Considerations

Direct touch

• Occlusion

Different contexts

• One-handed, two-handed

• Thumb vs. finger

• On the move

Existing vs. new gestures

• Accuracy already determined or not


Mobile App Example

30

Steps in gesture test plan:

1. Introduction

• Game startup

2. Familiarize and practice

• Go through practice mode, play first level

• Assess success at learning gestures

3. Accuracy

• Repeat gestures using practice mode or similar scenario

• All are one-finger gestures in same parameter space

4. Assess satisfaction

MonsterKill by Origin8


An outline of how you might do a usability test focused just on the gesture components of this game.

Conclusions

Elements of successful gestures• Discovery and understanding

• Robust recognition

• Satisfying results that mesh with the entire user experience

Design usability tests to address each element

Used at Synaptics to• Guide engineering improvements

• Help marketing team choose gestures

• Improve the end user’s experience


Resources

32

References

• Mark Billinghurst and Bill Buxton, “Gesture Based Interaction” in Human Input to Computer Systems (draft), www.billbuxton.com/inputManuscript.html

• Dan Saffer, Designing Gestural Interfaces, O’Reilly 2008.

• Craig Villamor et al., Touch Gesture Reference Guide, www.lukew.com/touch/

• Jacob Wobbrock et al., “User-Defined Gestures for Surface Computing,” CHI 2009.

• Mike Wu et al., “Gesture Registration, Relaxation, and Reuse for Multi-Point Direct-Touch Surfaces,” IEEE Tabletop 2006.

• HumanCentric Gesture Research, 2010. blog.humancentric.com/gesture-research/

• Don Norman and Jakob Nielsen, “Gestural Interfaces: A Step Backwards in Usability.” Interactions Sept/Oct 2010. interactions.acm.org/content/?p=1401

Acknowledgments

• Usability colleagues at Dell, HP, Lenovo, Synaptics

Contact

• [email protected], twitter: @karthur , blog: touchusability.com


http://www.billbuxton.com/inputManuscript.html

http://www.lukew.com/touch/

http://www.lukew.com/touch/

http://blog.humancentric.com/gesture-research/





http://interactions.acm.org/content/?p=1401

http://interactions.acm.org/content/?p=1401

http://www.touchusability.com/

evaluating touch gesture usability -- d4m 2010

Design

gesture properties

parameter space gestures

parameter multifinger

testing gestures10respond

discoverable testing

movingfingers liftdoes

ill use pinch zoomtwo

customer partnerships