1. Gesture control: Wave goodbye to your remote control and say hello to the futureSean Smith UX Australia Conference 2013 Melbourne

2. The use of freehand gestures to interact with devices is on the rise 3. Pointing gestures and semantic gestures 4. Pointing gestures 5. Semantic gestures These are gestures with an associated meaning Remove need for GUI Potentially more efficient Considerations Potentially influenced by local culture o Some cultures might be more predisposed to the use of gestures than others o A gesture may already have an association 6. The UX Fellows Gesture Study 7. The UX Fellows Gesture StudyWhich semantic gestures would people from various cultures spontaneously use to control consumer electronic devices? Is there a common gestural language? How easy is it to choose gestures? How do people feel about gesture control? 8. We compared spontaneous gesture behavior in 18 countries. FICAUK BE DE NL FR ES ITUS MXRU TRCNKRINAU AR 9. Setting15 min Warm-up: everyday gestures 10. What are these people trying to tell us?A: I think I need to see a dentistB: I think I ate something badC: I like it!D: That meal was D: That meal is really tasty 11. Commands we explored Switch on device Volume upP-Confirm on-screen dialog (OK)Channel upDeny on-screen dialog (cancel)Context info for current programP+backward navigation in menusPause video-on-demand (VOD)Channel downRestart VODShare current programFast forward VODVolume downRewind VODMute volumeSkip chapter VODPause current programPrevious chapter VODHomepageStop VODEPGSwitch off device 12. Sample18 countries x 20 subjects = 360 participantsmax65 years 49%mean32min1651% 13. The Results 14. no. of countriesCommonality of gestures 18 16 14 12 10 8 6 4 2 0highno. of countries with the same main gesturemediumlow 15. Difficulty in creating gestures max 5.0 3.5difficulty3.0 2.5 2.01.5 1.0Self-reported difficulty: 1 = very easy to 5 = very difficult (n = 360) 16. Commands with high commonality Volume down Volume up Pause VOD Pause current program Rewind VOD Fast forward VOD Confirm on-screen dialog (OK) 17. Commands with high commonality Volume down Volume up Pause VOD Pause current program Rewind VOD Fast forward VOD Confirm on-screen dialog (OK)Commonality18 of 18Difficulty (1-5)1.8Q: How do you turn down the volume? 18. Commands with high commonality Volume down Volume up Pause VOD Pause current program Rewind VOD Fast forward VOD Confirm on-screen dialog (OK)Commonality17 of 18Difficulty (1-5)1.5Q: How do you pause viewing a film?NL 19. Commands with high commonality Volume down Volume up Pause VOD Pause current programAU, UKRewind VOD Fast forward VOD Confirm on-screen dialog (OK)Commonality15 of 18Difficulty (1-5)1.9Q: You are watching a boring scene, how do you fast-forward (with an image being shown)?NL 20. Commands with medium commonality Channel up Channel down Stop VOD Mute volume backward navigation in menus Deny on-screen dialog (cancel) Skip chapter VOD 21. Commands with medium commonality Channel up Channel down Stop VODCN, KRMute volume backward navigation in menus Deny on-screen dialog (cancel)FI, RUSkip chapter VODCommonality13 of 18Difficulty (1-5)2.1Q: How would you stop watching the film altogether (because you want to do something different)?NL 22. Commands with medium commonality Channel up Channel down Stop VODAU, CN, MXFI, RUMute volume backward navigation in menus Deny on-screen dialog (cancel) Skip chapter VODCommonality11 of 18Difficulty (1-5)BE2.0Q: How do you set the volume to "mute"?UK 23. Commands with medium commonality Channel up Channel down Stop VODAU, CN, RU, TRMute volume backward navigation in menus Deny on-screen dialog (cancel)AR, BE, IT, MXSkip chapter VODCommonality9 of 18Difficulty (1-5)1.8Q: How do you express that you dont want to buy it?KR 24. Commands with low commonality Switch on device Switch off device Previous chapter VOD Restart VOD Share current program EPG Info for current program Homepage 25. Commands with low commonality Switch on device Switch off device Previous chapter VODCN, FR, INAR, TRRestart VODFIShare current program EPGRUMXInfo for current program HomepageKR Commonality6 of 18Difficulty (1-5)3.2USQ: How would you share with your friends what you are currently watching or listening to? Note: No common gesture in UKNL 26. Commands with low commonality Switch on device Switch off device Previous chapter VODAU, USRestart VODCN, FI, IN, KR, RUMXShare current program EPG Info for current program Homepage CommonalityNLBE, UK5 of 18Difficulty (1-5) 2.8 Q: You'd like to see an overview of tonight's TV programs on all channels (EPG). How do you activate this function with a hand gesture? Note: No common gesture in ItalyTR 27. Commands with low commonality Switch on device Switch off device Previous chapter VODCN, IN, TR FR, KR, RURestart VODAU, FIShare current program EPG Info for current programESUSDEBEHomepage Commonality4 of 18Difficulty (1-5)2.8Q: How would you get more information for the running TV program?UKAR 28. Average gesture difficulty by country max 5.0 Everyday gesturesTV gestures3.0difficulty2.52.01.5min1.0 TR FR MX AR INFICN RU NL DE AU US BE ES CAITKR UK 29. General acceptance Overall, our 360 participants responded very positively to the idea of semantic gesture input for TVs. "fun to use", "convenient", "cool" Participants mentioned that they would like to use gestures to control other devices as well (e.g. audio). 30. Gesture type preferenceFR Semantic PointingCN TRINAR RU MX US DE AUFICABEESNL UK KRIT15 3 nations exhibited a preference for pointing gestures only 6 nations exhibited a preference for semantic gestures only associated with more advanced commandsMany users indicated that they would prefer the ability to switch between gesture types.9 31. Typical concerns of users Learning and retention of gestures Sensitivity and accuracy of devices Interference by other gestures in the room Compatibility with lean-back disposition 32. Conclusions 1. For basic CE commands a common language for semantic gestures appears to already exist across cultures. 2. More complex commands could be realised by: a. customising these gestures for each individual market b. allowing users to customise gestures individually c.proposing a full set of semantic gestures that need to be learnedd. switching to pointing gestures in combination with a GUI 33. Thumbs up to the audience! Find detailed results and report on the UX Fellows website: www.uxfellows.com Drop me a line: sean@u1group.com