Evaluating Touch

Gesture Usability

Kevin Arthur

Synaptics

1

UPA 2010

2

Multi-Finger TouchPad Gestures

3

Two-Finger

Pinch ZoomTwo-Finger

RotateThree-Finger

Flick

Two-Finger

Scroll

Two-Finger

Pivot Rotate

Three-Finger

Press

Motivation for Testing Gestures

• Formative usability tests for feedback to developers

• Summative and competitive usability tests for sales, OEM partners

• Need for repeatable procedures

Audience

• Do users understand the gestures?

• Can users successfully perform the gestures?

• Are gestures satisfying to use?

Questions

• Holistic evaluation; test with users in a realistic scenario on a working system

• Different from pure data collection for testing gesture recognizer offline

Context

4

Important Properties of Gestures

5

• Gestures must be taught; testing the documentation is importantLack of Affordances

• Performance is not always predictable

• Recognition trade-offs exist between gestures in same parameter space

• Gestures should be evaluated as a set

Nondeterministic and Interdependent

• TouchPad’s primary use is still for pointing and scrolling; gestures shouldn’t interfere

Interface Overloading

• Hand size, long fingernailsUser Variation

Phases in Performing a Gesture

Exposure

Registration

Continuation

Termination

• Introduction through

documentation or prior use

• Initial touch contact and motion

for gesture recognition

• Gesture is registered and

“locked in” – mode switch

• Dynamic phase with relaxed

motion requirements

• End position, fingers lifting

6

Test Framework

7

Step Objective

1 Gesture IntroductionAssess understanding, help

materials

[Exposure phase]

Train to basic performance

[All phases]

2Familiarization and

Practice Task

3 Accuracy Task

Obtain rates of correct and incorrect

gesture recognition

[Registration phase]

4Satisfaction Questionnaire

and Debrief

Assess satisfaction, ease of use

[All phases]

Gesture Introduction

8

Simulate new user experience

• “Out of box experience”

material

• Help videos

• Try the gesture until success

• If no success, moderator

assists

Familiarization and Practice Tasks:

Pinch Zoom

Zoom in on the South Residences, near the top of the map, and find the building called The Knoll. Zoom all the way in on The Knoll and then zoom all the way back out.

9

2000 × 2000 pixels or higher.

Approximately five pinch-zooms

required.

10

Flick and Rotate

11

1. Use the rotate gesture to make the image upright.2. Type the image’s title into the caption field and press Enter.3. Use the flick gesture to go to the next image.

Flick and Rotate

12

Accuracy Task

User performs a set number of gesture attempts

Moderator records table of system responses. Example:

13

Gesture CorrectNo

Response

Misrecognized Other/

NotesAs Pinch As Rotate

Pinch Zoom

In7 2 - 1

Pinch Zoom

Out8 2 - 0

Rotate

Clockwise6 3 1 -

“requires

too large a

motion”

Rotate

Counter-

clockwise

7 1 2 -

Accuracy Results

14

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Pinch Zoom In

Pinch Zoom Out

Rotate CW Rotate CCW

Average

Misrecognized

No Response

Correct

Unified Measure

Accuracy Results, Competitive Study

15

61

82

96

83

88

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2F Pinch Zoom

2F Rotate 3F Flick LR

3F Flick UD

2F Scroll

System AAverage Correct 82%

Correct No response Incorrect

94

73

85

94 96

90

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2F Pinch Zoom

2F Rotate 3F Flick LR

3F Flick UD

3F Press 2F Scroll

System BAverage Correct 89%

Correct No response Incorrect

Accuracy Results, Competitive Study

16

98 62 92 97100 77 86 9797 68 93 1000

20

40

60

80

100

120

Pinch Rotate Flick Three-Finger Press

Co

rre

ct

(%)

Correct Gesture Recognition Rates

System A

System B

System C

Satisfaction Questionnaire

17

Satisfaction Questionnaire

18

Questionnaire Result

19

7.38

4.38

7.88

8.75

8.25

1 2 3 4 5 6 7 8 9

Pinch

Rotate

3F Flick

3F Press

2F Scroll

I would use this gesture if it were available(strongly disagree = 1, strongly agree = 9)

Strongly

Disagree

Strongly

Agree

Questionnaire,

Competitive Rating Format

How well do the gestures work on each system?

Please rate from 1 (Poor) to 5 (Excellent)

Gesture System A System B System C

Pinch Zoom

Rotate N/A

Flick Left-Right

Flick Up-Down N/A

Three-Finger Press N/A N/A

Two-Finger Scrolling

20

1

2

3

4

5

Pinch Zoom 2F Rotate 1F Rotate (Chiral)

Flick LR Flick UD 1F Scroll (Linear)

1F Scroll (Circular)

2F Scrolling

User Ratings With 95% Confidence Intervals

System A System B System C

Ratings Results

21

Excellent

Poor

Related Tests & Gesture Side-Effects

Other usability tests for notebook input devices

• Pointing (target acquisition)

• Drag and drop

• Scrolling

• Typing and accidental TouchPad input

Assess gesture side-effects

• Unintended gestures

• Other issues

22

Conclusions

Properties of gestures call for careful testing

• Gesture introduction and documentation is key

• Gestures are nondeterministic – less predictable

• Gestures are interdependent; test as a set

Framework of gesture tests

1. Gesture Introduction

2. Familiarity and Practice Tasks

3. Accuracy Tasks

4. Satisfaction Questionnaire and Debrief

5. Related Tests & Gesture Side-Effects

23

Resources

24

References

• Sylvia Le Hong and Dan Mauney, “Cultural Differences and Similarities

in the Use of Gestures on Touchscreen User Interfaces,” UPA 2010. blog.humancentric.com/gesture-research/

• Mark Billinghurst and Bill Buxton, “Gesture Based Interaction” in Human

Input to Computer Systems (draft), www.billbuxton.com/inputManuscript.html

• Dan Saffer, Designing Gestural Interfaces, O’Reilly 2008.

• Craig Villamor, Dan Willis, Luke Wroblewski, Touch Gesture Reference

Guide, www.lukew.com/touch/

• Jacob Wobbrock , Meredith Ringel Morris, Andrew Wilson, “User-

Defined Gestures for Surface Computing,” CHI 2009.

• Mike Wu, Chia Shen, Kathy Ryall, Clifton Forlines, Ravin Balakrishnan,

“Gesture Registration, Relaxation, and Reuse for Multi-Point Direct-

Touch Surfaces,” IEEE Tabletop 2006.

Contact

• karthur@synaptics.com, touchusability.com

Acknowledgments

• Usability colleagues at Dell, HP, Lenovo, Synaptics