getting touchy AN INTRODUCTION TO TOUCH AND POINTER EVENTS

Patrick H. Lauke / Sainté Mobile Days / Saint-Etienne / 23 November 2013

1. introduction

2. touch events

3. break

4. pointer events

5. debugging/testing

6. conclusion

patrickhlauke.github.io/touch

“how can I make my website work on touch devices?”

you don't need touch eventsbrowsers emulate regular mouse events

patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html

patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html

mouseover > mousemove* > mousedown > (focus) > mouseup > click

*only a single “sacrificial” event is fired

on first tapmouseover > mousemove > mousedown >

(focus) > mouseup > click

subsequent tapsmousemove > mousedown > (focus) >

mouseup > click

tapping awaymouseout > (blur)

focus/blur only on focusable elements in Opera Mobile and Firefoxmouseout not on iOS Safari and embedded WebView (e.g. iOS Chrome)

emulation works,but is limiting/problematic

1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track

1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track

patrickhlauke.github.io/touch/tests/event-listener_show-delay.html

patrickhlauke.github.io/touch/tests/event-listener_show-delay.html

1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track

steamcommunity.com/id/redux_splintered/screenshots/?appid=8870

vimeo.com/ondemand

patrickhlauke.github.io/touch/css-dropdown/mouse-only.html

1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track

patrickhlauke.github.io/touch/particle/2

patrickhlauke.github.io/touch/particle/2(bug in iOS7: moving finger does seem to scroll and fire multiple mousemove events?!)

“we need to go deeper...”

touch eventsintroduced in iOS 2.0, adopted in Chrome/Firefox/Opera

www.w3.org/TR/touch-events

touchstarttouchmove

touchend

touchcanceltouchentertouchleave

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html

touchstart > [touchmove]+ > touchend > mouseover > mousemove > mousedown >

(focus) > mouseup > click

on first taptouchstart > [touchmove]+ > touchend > mouseover > mousemove > mousedown > (focus) > mouseup > click

subsequent tapstouchstart > [touchmove]+ > touchend > mousemove >

mousedown > (focus) > mouseup > click

tapping awaymouseout > (blur)

too many touchmove events abort the tap (after touchend)not all browsers consistently send the touchmove

some browsers outright weird...

Browser/Android 4.3(AppleWebKit/534.30)

mouseover > mousemove > touchstart > touchend > mousedown > mouseup > click

Browser/Blackberry PlayBook 2.0(AppleWebKit/536.2)

touchstart > mouseover > mousemove > mousedown > touchend > mouseup > click

touch eventsvs

limitations/problems

1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track

1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track

patrickhlauke.github.io/touch/tests/event-listener_show-delay.html

patrickhlauke.github.io/touch/tests/event-listener_show-delay.html

why the delay?double-tap to zoom

(mostly anyway)

what if browsers didn't wait?

Puffin/Android (popular in Korea?) puffinbrowser.comdouble-tap zooms and fires mouse events + click

(also, doesn't support touch events at all)

UCWeb/Android (popular in China?) ucweb.comno delay...but only because it doesn't implement double-tap to zoom

when does the delay happen?

patrickhlauke.github.io/touch/tests/event-listener.html

patrickhlauke.github.io/touch/tests/event-listener.html

“how can we make it feel responsive like a native app?”

react to events fired before the 300ms delay...

interlude:simple feature detection for

touch events

if ('ontouchstart' in window) {

/* some clever stuff here */

}

/* common performance “trick” */

var clickEvent = ('ontouchstart' in window ? 'touchend' : 'click');

blah.addEventListener(clickEvent,function() { ... }, false);

don't make it touch-exclusive

hacks.mozilla.org/2013/04/detecting-touch...

if ('ontouchstart' in window) {

/* browser supports touch events but user is not necessarily using touch (exclusively) */

}

hybrid devicestouch + mouse + keyboard

patrickhlauke.github.io/touch/tests/event-listener_show-delay-naive-event-fix.html

bugzilla.mozilla.org/show_bug.cgi?id=888304

Android + mouse – behaves like touchtouchstart > touchend > mouseover > mousemove > mousedown > mouseup > click

Blackberry PlayBook 2.0 + mouse – like desktop mousemouseover > mousedown > mousemove > mouseup > click

Android + keyboard – like desktop keyboardfocus > click

VoiceOver enables keyboard access on iOS

iOS + keyboard – similar to touchfocus > touchstart > touchend > mouseover > mousemove > mousedown

blur > mouseup > click

iOS + VoiceOver – similar to touchfocus > touchstart > touchend > mouseover > mousemove > mousedown

blur > mouseup > click

Android + TalkBack – keyboard/mouse hybridfocus > blur > mousedown > mouseup > click > focus(?)

touch or mouse or keyboard

touch and mouse and keyboard

/* doubled-up event listeners */

foo.addEventListener('touchend', someFunction, false);

foo.addEventListener('click', someFunction, false);

/* doubled-up event listeners */

foo.addEventListener('touchend',function(e) {/* prevent delay + mouse events */e.preventDefault();someFunction();/* or even e.target.click(); */

}, false);

foo.addEventListener('click',someFunction, false);

github.com/ftlabs/fastclick

preventDefaultkills scrolling, long-press,

pinch/zoom

browsers working to remove delay when possible

patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html

patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html

patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html

Bug 922896 - Optimizations to remove 300ms touch > mouse events delay[RESOLVED - FIXED]

Issue 169642: Remove ~300ms delay on tap for mobile sites [...]

[...] no more 300ms click delay, or double-tap zoom, on mobile websites

patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

iOS/Safari designed themselves into a cornerwith “double-tap to scroll”

bugs.webkit.org/show_bug.cgi?id=122212

1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track

steamcommunity.com/id/redux_splintered/screenshots/?appid=8870

vimeo.com/ondemand

patrickhlauke.github.io/touch/css-dropdown/mouse-only.html

no isolated hover (or focus) on touch devices*

* iOS fakes it, Samsung Galaxy S4 + stylus, ...

patrickhlauke.github.io/touch/css-dropdown/mouse-only.html

patrickhlauke.github.io/touch/css-dropdown/mouse-only.html

http://developer.apple.com/library/IOS/...

Modernizr issue 869: Detecting a mouse user

www.stucox.com/blog/you-cant-detect-a-touchscreen

avoid hover-based interfaces?

complement for keyboard / touch!

patrickhlauke.github.io/touch/css-dropdown/mouse-keyboard.html

patrickhlauke.github.io/touch/css-dropdown/mouse-keyboard.html

patrickhlauke.github.io/touch/css-dropdown/mouse-keyboard-touch.html

1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track

patrickhlauke.github.io/touch/particle/2

patrickhlauke.github.io/touch/particle/2(bug in iOS7: moving finger does seem to scroll and fire multiple mousemove events?!)

touchstart > [touchmove]+ > touchend > mouseover > mousemove* > mousedown >

(focus) > mouseup > click

*mouse event emulation fires only a single mousemove

doubling up handling of mousemove and touchmove

var posX, posY;

...

function positionHandler(e) {posX = e.clientX;posY = e.clientY;

}

...

canvas.addEventListener('mousemove', positionHandler, false);

var posX, posY;

...

function positionHandler(e) {/* handle both mouse and touch? */

}

...

canvas.addEventListener('mousemove',positionHandler, false);

canvas.addEventListener('touchmove',positionHandler, false);

interface MouseEvent : UIEvent {readonly attribute long screenX;readonly attribute long screenY;readonly attribute long clientX;readonly attribute long clientY;readonly attribute boolean ctrlKey;readonly attribute boolean shiftKey;readonly attribute boolean altKey;readonly attribute boolean metaKey;readonly attribute unsigned short button;readonly attribute EventTarget relatedTarget;void initMouseEvent(...);

};

www.w3.org/TR/DOM-Level-2-Events/events.html#Events-MouseEvent

interface TouchEvent : UIEvent {readonly attribute TouchList touches;readonly attribute TouchList targetTouches;readonly attribute TouchList changedTouches;readonly attribute boolean altKey;readonly attribute boolean metaKey;readonly attribute boolean ctrlKey;readonly attribute boolean shiftKey;

};

www.w3.org/TR/touch-events/#touchevent-interface

interface Touch {readonly attribute long identifier;readonly attribute EventTarget target;readonly attribute long screenX;readonly attribute long screenY;readonly attribute long clientX;readonly attribute long clientY;readonly attribute long pageX;readonly attribute long pageY;

};

www.w3.org/TR/touch-events/#touch-interface

var posX, posY;

...

function positionHandler(e) {if ((e.clientX)&&(e.clientY)) {

posX = e.clientX;posY = e.clientY;

} else if (e.targetTouches) {posX = e.targetTouches[0].clientX;posY = e.targetTouches[0].clientY;e.preventDefault();

}}

...

canvas.addEventListener('mousemove',positionHandler, false );

canvas.addEventListener('touchmove',positionHandler, false );

patrickhlauke.github.io/touch/particle/3

patrickhlauke.github.io/touch/particle/4

www.splintered.co.uk/experiments/archives/paranoid_0.5

tracking finger movementover time ... swipe gestures

patrickhlauke.github.io/touch/swipe

patrickhlauke.github.io/touch/swipe

patrickhlauke.github.io/touch/picture-slider

don't forget mouse/keyboard !

bradfrostweb.com/demo/mobile-first

touchmove fires...a lot!

do absolute minimum on each touchmove

(usually: store new coordinates)

heavy JavaScript on requestAnimationFrame

setInterval

patrickhlauke.github.io/touch/touch-limit

why stop at a single point?multitouch support

interface TouchEvent : UIEvent {readonly attribute TouchList touches;readonly attribute TouchList targetTouches;readonly attribute TouchList changedTouches;readonly attribute boolean altKey;readonly attribute boolean metaKey;readonly attribute boolean ctrlKey;readonly attribute boolean shiftKey;

};

www.w3.org/TR/touch-events/#touchevent-interface

for (i=0; i<e.targetTouches.length; i++) {

...

posX = e.targetTouches[i].clientX;posY = e.targetTouches[i].clientY;

...

}

patrickhlauke.github.io/touch/tracker/multi-touch-tracker.html

iOS/iPad even preventDefault()can't override 4-finger gestures

iOS7/Safari even preventDefault()can't override back/forward swipe gestures

multitouch gestures

/* iOS/Safari has gesture events for size/rotation, not supported in Chrome/Firefox/Opera, not part of the W3C Touch Events spec. */

gesturestart, gesturechange, gestureend

e.scale, e.rotation

patrickhlauke.github.io/touch/iosgestures

patrickhlauke.github.io/touch/iosgestures/image.html

/* with some trigonometry we can replicate these from basic principles. */

var distance = Math.sqrt(Math.pow(...)+Math.pow(...));var angle = Math.atan2(...);

patrickhlauke.github.io/touch/pinch-zoom-rotate

shinydemos.com/picture-organizer

interlude:older devices/OS versions

and multitouch...

HTC Hero – Android 2.1

LG Optimus 2X – Android 2.3.7

touch events andInternet Explorer...

blogs.msdn.com/...

www.w3.org/TR/pointerevents

not just some “not invented here” new approach for IE10+

html5labs.interoperabilitybridges.com/prototypes/...

code.google.com/p/chromium/issues/detail?id=162757

bugzilla.mozilla.org/show_bug.cgi?id=822898

...what about Apple?

bugs.webkit.org/show_bug.cgi?id=105463(no real activity since April 2013?)

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html

pointerover > mouseover >pointerenter > mouseenter > pointerdown > mousedown > pointermove > mousemove >

focus > pointerup > mouseup > pointerout > mouseout >

pointerleave > mouseleave > click

mouse events fired “inline” with pointer events(for the primary contact, e.g. first finger on screen)

gotpointercapture

lostpointercapture

vendor-prefixed in IE10MSPointerDownMSPointerMoveMSPointerUp

navigator.msPointerEnablednavigator.msMaxTouchPoints

-ms-touch-action

unprefixed in IE11

interface MouseEvent : UIEvent {readonly attribute long screenX;readonly attribute long screenY;readonly attribute long clientX;readonly attribute long clientY;readonly attribute boolean ctrlKey;readonly attribute boolean shiftKey;readonly attribute boolean altKey;readonly attribute boolean metaKey;readonly attribute unsigned short button;readonly attribute EventTarget relatedTarget;void initMouseEvent(...);

};

www.w3.org/TR/DOM-Level-2-Events/events.html#Events-MouseEvent

/* Pointer Events extend Mouse Events vs Touch Events and their completely new objects/arrays */

interface PointerEvent : MouseEvent {readonly attribute long pointerId;readonly attribute long width;readonly attribute long height;readonly attribute float pressure;readonly attribute long tiltX;readonly attribute long tiltY;readonly attribute DOMString pointerType;readonly attribute boolean isPrimary;

};

www.w3.org/TR/pointerevents/#pointerevent-interface

simple feature detection for pointer events

if (navigator.pointerEnabled) {

/* some clever stuff here but this covers touch, stylus, mouse, etc */

}

/* still listen to click for keyboard! */

if (navigator.maxTouchPoints > 1) {

/* multitouch-capable device */

}

patrickhlauke.github.io/touch/tests/pointer-multitouch-detect.html

pointer eventsvs

limitations/problemsof mouse event emulation

1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track

1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track

patrickhlauke.github.io/touch/tests/event-listener_show-delay.html

patrickhlauke.github.io/touch/tests/event-listener_show-delay.html

patrickhlauke.github.io/touch/tests/event-listener.html

patrickhlauke.github.io/touch/tests/event-listener.html

patrickhlauke.github.io/touch/tests/event-listener.html

“how can we make it feel responsive like a native app?”

preventDefault() won't work(but you can prevent most mouse compatibility events by

cancelling pointerdown)

touch-action: auto|none|[pan-x][pan-y]

www.w3.org/TR/pointerevents/#the-touch-action-css-property

only prevents default touch action (e.g. double-tap to zoom)does not stop synthetic mouse events nor click

touch-action:nonekills scrolling, long-press,

pinch/zoom

touch-action:none

http://patrickhlauke.github.io/touch/tests/event-listener_touch-action-none.html

http://patrickhlauke.github.io/touch/tests/event-listener_touch-action-none.html

pointerover > mouseover >pointerenter > mouseenter > pointerdown > mousedown > pointermove > mousemove >

focus > pointerup > mouseup >

click > pointerout > mouseout >pointerleave > mouseleave

1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track

msdn.microsoft.com/en-us/library/ie/dn265029(v=vs.85).aspx

msdn.microsoft.com/en-us/library/ie/jj152135(v=vs.85).aspx

patrickhlauke.github.io/touch/css-dropdown/mouse-only-aria-haspopup.html

channel9.msdn.com/Blogs/IE/IE11-Using-Hover-Menus-with-Touch

1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track

patrickhlauke.github.io/touch/particle/2

touch-action:none

patrickhlauke.github.io/touch/particle/2a

what about

multitouch?

patrickhlauke.github.io/touch/tracker/mouse-tracker_touch-action-none.html

/* PointerEvents don't have the handy touch arrays, so we have to replicate something similar... */

/* PointerEvents don't have the handy touch arrays, so we have to replicate something similar... */

var points = [];

switch (e.type) {

case 'pointerdown': /* add to the array */ break;

case 'pointermove': /* update the relevant array entry's x and y */ break;

case 'pointerup': /* remove the relevant array entry */ break;

}

patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer.html

/* like iOS/Safari, IE10/Win has higher-level gestures, but these are not part of the W3C Pointer Events spec.

Replicate these from basic principles again? */

www.exploretouch.ie/behind-the-scenes

pointer events as the future?

what about backwards-compatibility?

touchstart > [touchmove]+ > touchend >[300ms delay] > mouseover > mousemove >

mousedown > mouseup > click

+pointerover > mouseover > pointerdown >

mousedown > pointermove > mousemove > pointerup > mouseup > pointerout > mouseout >

[300ms delay] > click

www.w3.org/community/touchevents

polyfills for pointer events(code for tomorrow, today)

handjs.codeplex.com

www.catuhe.com/msdn/handjs

github.com/Polymer/PointerEvents

www.polymer-project.org

utility libraries(because life is too short to hand-code gesture support etc)

eightmedia.github.io/hammer.js

jQuery Mobile? Sencha Touch? …check for support of IE10+ and “this is a touch device” assumptions

debugging/testing

Issue 181204: Emulate touch events - event order different from real devices

Bug 920956 - DevTools touch emulation: suppress regular mouse events ...

developers.google.com/chrome-developer-tools/docs/console#monitoring_events

Bug 861876 - [...] multiple mousemoves being fired

further reading...

Matt Gaunt – Touch Feedback for Mobile Siteswww.gauntface.co.uk/blog/2013/06/25/touch-feedback-for-mobile-sites

Jonathan Stark – FastActivegithub.com/jonathanstark/FastActive

Stephen Woods – HTML5 Touch Interfaceswww.slideshare.net/ysaw/html5-touch-interfaces-sxsw-extended-version

David Rousset – Unifying touch and mouse: how Pointer Events will make cross-browsers touch support easyblogs.msdn.com/b/davrous/archive/2013/02/20/handling-touch[...]

Chris Wilson + Paul Kinlan – Touch And Mouse: Together Again For The First Timewww.html5rocks.com/en/mobile/touchandmouse

Patrick H. Lauke – Webseiten zum Anfassenwebkrauts.de/artikel/2012/einfuehrung-touch-events

Ryan Fioravanti – Creating Fast Buttons for Mobile Web Applicationsdevelopers.google.com/mobile/articles/fast_buttons

Boris Smus – Multi-touch Web Developmentwww.html5rocks.com/en/mobile/touch

Boris Smus – Generalized input on the cross-device websmus.com/mouse-touch-pointer

Rick Byers + Boris Smus (Google I/O) – Point, Click, Tap, Touch - Building Multi-Device Web Interfacesdevelopers.google.com/events/io/sessions/361772634

Grant Goodale – Touch Eventswww.w3.org/conf/2011/#Touch_Events

W3C – Touch Events Extensionswww.w3.org/TR/2013/NOTE-touch-events-extensions-20131031

Mozilla Developer Network – Touch Eventsdeveloper.mozilla.org/en-US/docs/Web/Guide/API/DOM/Events/Touch_events

WebPlatform.org – Pointer Eventsdocs.webplatform.org/wiki/concepts/Pointer_Events

Rick Byers – The best way to avoid the dreaded 300ms click delay is to disable double-tap zoomplus.google.com/+RickByers/posts/ej7nsuoaaDa

Tim Kadlec – Avoiding the 300ms Click Delay, Accessiblytimkadlec.com/2013/11/Avoiding-the-300ms-click-delay-accessibly

Microsoft – Pointer events updates (differences between IE10 and IE11)msdn.microsoft.com/en-us/library/ie/dn304886(v=vs.85).aspx

youtube.com/watch?v=AZKpByV5764

merci@patrick_h_lauke

github.com/patrickhlauke/touchslideshare.net/redux

paciellogroup.comsplintered.co.uk