getting touchyAN INTRODUCTION TO TOUCH AND POINTER EVENTS

Patrick H. Lauke / DevConFu / Jūrmala, Latvia / 29 May 2014

patrickhlauke.github.io/touch

Touch/pointer events test results

“how can I make my websitework on touch devices?”

you don't need touch eventsbrowsers emulate regular

mouse events

patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html

patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html

compatibility mouse events(mouseenter) > mouseover > mousemove* > mousedown >

(focus) > mouseup > click

* only a single “sacrificial” mousemove event fired

on first tap(mouseenter) > mouseover > mousemove >

mousedown > (focus) > mouseup > click

subsequent tapsmousemove > mousedown > mouseup > click

tapping awaymouseout > (blur)

focus / blur only on focusable elements in Firefoxmouseout not on iOS Safari/WebView (e.g. iOS Chrome)

Opera Mobile and

emulation works,but is limiting/problematic

1.  delayed event dispatch2.  mousemove doesn't track

1.  delayed event dispatch2.  mousemove doesn't track

patrickhlauke.github.io/touch/tests/event-listener_show-delay.html

patrickhlauke.github.io/touch/tests/event-listener_show-delay.html

1.  delayed event dispatch2.  mousemove doesn't track

patrickhlauke.github.io/touch/particle/2

patrickhlauke.github.io/touch/particle/2

(iOS7 bug: moving finger fires mousemove on scroll)

“we need to go deeper...”

touch events

introduced by Apple, adoptedin Chrome/Firefox/Opera

www.w3.org/TR/touch-events

touchstarttouchmovetouchend

touchcancel

touchentertouchleave

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html

Bug 128534 - 'mouseenter' mouse compat event not fired...

events fired on taptouchstart > [touchmove]+ > touchend >

(mouseenter) > mouseover > mousemove > mousedown >(focus) > mouseup > click

(mouse events only fired for single-finger tap)

on first taptouchstart > [touchmove]+ > touchend >

(mouseenter) > mouseover > mousemove > mousedown >(focus) > mouseup > click

subsequent tapstouchstart > [touchmove]+ > touchend >mousemove > mousedown > mouseup > click

tapping awaymouseout > (mouseleave) > (blur)

•  too many touchmove events prevent mouse compatibility events

after touchend (not considered a "clean" tap)

•  too many touchmove events on activatable elements can lead to

touchcancel (in old Chrome/Browser versions)

•  not all browsers consistently send the touchmove

some browsers outright weird...

Browser/Android 4.3(AppleWebKit/534.30)

mouseover > mousemove > touchstart > touchend >mousedown > mouseup > click

Browser/Blackberry PlayBook 2.0(AppleWebKit/536.2)

touchstart > mouseover > mousemove > mousedown >touchend > mouseup > click

Touch/pointer events test results

touch eventsvs

limitations/problems

1.  delayed event dispatch2.  mousemove doesn't track

1.  delayed event dispatch2.  mousemove doesn't track

patrickhlauke.github.io/touch/tests/event-listener_show-delay.html

why the delay?double-tap to zoom

(mostly anyway)

what if browsers didn't wait?

Puffin/Android double-tap zooms and fires mouse events + click(also, doesn't support touch events)

Try it out in Chrome? chrome://flags/

when does the delay happen?

patrickhlauke.github.io/touch/tests/event-listener.html

touch / mouse events delaytouchstart > [touchmove]+ > touchend >

[300ms delay]

(mouseenter) > mouseover > mousemove > mousedown >(focus) > mouseup > click

“how can we make it feelresponsive like a native app?”

react to events fired before the300ms delay...

touchstart for an “immediate”control

(fire/jump button on a game)

touchend for a control thatfires after finger lifted

interlude: simple featuredetection for touch events

/* feature detection for touch events */

if ( 'ontouchstart' in window ) { /* some clever stuff here */}

/* older browsers have flaky support so more hacky tests needed...use Modernizr.touch or similar */

/* common performance “trick” */

var clickEvent = ( 'ontouchstart' in window ? 'touchend' : 'click' );

blah.addEventListener( clickEvent , function() { ... }, false);

don't make it touch-exclusive

/* common performance “trick” */

var clickEvent = ( 'ontouchstart' in window ? 'touchend' : 'click');

...

/* if touch events are supported, only listen to touchend, not click */

hybrid devicestouch + mouse + keyboard

Android + mouse – behaves like touchtouchstart > touchend > mouseover > mousemove > mousedown >

(focus) > mouseup > click

Blackberry PlayBook 2.0 + mouse - like desktopmouse mouseover > mousedown > mousemove > mouseup > click

Android + keyboard – like desktop keyboardfocus > click

iOS + VoiceOver (with/without keyboard) – similar to touchfocus > touchstart > touchend > (mouseenter) > mouseover >

mousemove > mousedown > blur > mouseup > click

Android + TalkBack – keyboard/mouse hybridfocus > blur > mousedown > mouseup > click > focus(?)

hacks.mozilla.org - Detecting touch [...]

/* feature detection for touch events */

if ('ontouchstart' in window) { /* browser supports touch events but user is not necessarily using touch (exclusively) */

/* it could be a mobile, tablet, desktop, fridge ... */}

touch or mouse or keyboard

touch and mouse and keyboard

/* doubled-up event listeners */

foo.addEventListener(' touchend ', someFunction, false);foo.addEventListener(' click ', someFunction, false);

/* prevent delay + mouse events */

foo.addEventListener(' touchstart ', function(e) { e.preventDefault();}, false);

/* doubled-up event listeners */

foo.addEventListener('touchend', someFunction, false);foo.addEventListener('click', someFunction, false);

preventDefault() killsscrolling, pinch/zoom, etc

apply preventDefault()carefully

(just on buttons/links, not entire page)

github.com/ftlabs/fastclick

browsers working to removedouble-tap to zoom delay

(when page not zoomable)

<meta name="viewport" content="user-scalable=no">patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

<meta name="viewport" content="user-scalable=no">patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

... content="minimum-scale=1, maximum-scale=1"patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html

... content="minimum-scale=1, maximum-scale=1"patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html

Bug 922896 - Optimizations to remove 300ms [...] delay[RESOLVED - FIXED]

what about accessibility?

patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

Chrome 32+ / Android: ... content="width=device-width"updates.html5rocks.com/2013/12/300ms-tap-delay-gone-away

Bug 941995 - Remove 300ms [...] on "responsive" pages

patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

iOS/Safari designed themselves into a corner: “double-tap to scroll”bugs.webkit.org/show_bug.cgi?id=122212

1.  delayed event dispatch2.  mousemove doesn't track

patrickhlauke.github.io/touch/particle/2

patrickhlauke.github.io/touch/particle/2(iOS7 bug: moving finger fires mousemove on scroll)

events fired on taptouchstart > [touchmove]+ > touchend >

(mouseenter) > mouseover >mousemove* > mousedown > (focus) >

mouseup > click

* mouse event emulation fires only a single mousemove

too many touchmove events prevent mouse compatibility events after touchend

doubling up handling ofmousemove and touchmove

var posX, posY;...function positionHandler(e) { posX = e.clientX ; posY = e.clientY ;}...canvas.addEventListener(' mousemove ', positionHandler, false);

var posX, posY;...function positionHandler(e) { /* handle both mouse and touch */}...canvas.addEventListener(' mousemove ', positionHandler, false);canvas.addEventListener(' touchmove ', positionHandler, false);

interface MouseEvent : UIEvent { readonly attribute long screenX ; readonly attribute long screenY ; readonly attribute long clientX ; readonly attribute long clientY ; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute unsigned short button; readonly attribute EventTarget relatedTarget; void initMouseEvent(...);};

www.w3.org/TR/DOM-Level-2-Events/events.html#Events-MouseEvent

partial interface MouseEvent { readonly attribute double screenX; readonly attribute double screenY; readonly attribute double pageX ; readonly attribute double pageY ; readonly attribute double clientX; readonly attribute double clientY; readonly attribute double x ; readonly attribute double y ; readonly attribute double offsetX ; readonly attribute double offsetY ;};

www.w3.org/TR/cssom-view/#extensions-to-the-mouseevent-interface

interface TouchEvent : UIEvent { readonly attribute TouchList touches ; readonly attribute TouchList targetTouches ; readonly attribute TouchList changedTouches ; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey;};

www.w3.org/TR/touch-events/#touchevent-interface

interface Touch { readonly attribute long identifier; readonly attribute EventTarget target; readonly attribute long screenX ; readonly attribute long screenY ; readonly attribute long clientX ; readonly attribute long clientY ; readonly attribute long pageX ; readonly attribute long pageY ;};

www.w3.org/TR/touch-events/#touch-interface

touches

all touch points on screen

targetTouches

all touch points that started on the element

changedTouches

touch points that caused the event to fire

patrickhlauke.github.io/touch/touchlist-objects

var posX, posY;...function positionHandler(e) { if ((e.clientX)&&(e.clientY)) { posX = e.clientX; posY = e.clientY; } else if (e.targetTouches) { posX = e.targetTouches[0].clientX; posY = e.targetTouches[0].clientY; e.preventDefault() ; }}...canvas.addEventListener('mousemove', positionHandler, false );canvas.addEventListener('touchmove', positionHandler, false );

patrickhlauke.github.io/touch/particle/3

tracking finger movement overtime ... swipe gestures

patrickhlauke.github.io/touch/swipe

patrickhlauke.github.io/touch/picture-slider

don't forget mouse/keyboard!

bradfrostweb.com/demo/mobile-first

touchmove fires...a lot!

do absolute minimum on eachtouchmove

(usually: store coordinates)

heavy JavaScript onsetInterval or

requestAnimationFrame

patrickhlauke.github.io/touch/touch-limit

why stop at a single point?multitouch support

interface TouchEvent : UIEvent { readonly attribute TouchList touches ; readonly attribute TouchList targetTouches ; readonly attribute TouchList changedTouches ; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey;};

www.w3.org/TR/touch-events/#touchevent-interface

/* iterate over touch array */

for (i=0; i< e.targetTouches .length; i++) { ... posX = e.targetTouches[i].clientX ; posY = e.targetTouches[i].clientY ; ...}

patrickhlauke.github.io/touch/tracker/multi-touch-tracker.html

iOS/iPad preventDefault()can't override 4-finger

gestures

iOS7/Safari preventDefault()can't override back/forward

swipe gestures

multitouch gestures

/* iOS/Safari/WebView has gesture events for size/rotation,not part of the W3C Touch Events spec. */

gesturestart / gesturechange / gestureend

function(e) { /* e.scale e.rotation */}

/* not supported in Chrome/Firefox/Opera */

/* with some trigonometry we can replicate these from basic principles. */

var distance = Math.sqrt(Math.pow(...)+Math.pow(...));var angle = Math.atan2(...);

patrickhlauke.github.io/touch/picture-organiser

not all old/cheap devices/OSssupport multitouch!

HTC Hero – Android 2.1

LG Optimus 2X – Android 2.3.7

ZTE Open – Firefox OS 1.1

what about Internet Explorer?

up to IE9 (Win7 / WinPhone7.5)only mouse events

in IE10 Microsoft introducedPointer Events

David Rousset - Unifying touch and mouse [...]

not just some“not invented here”

technology

Pointer Events - W3C Candidate Recommendation 09 May 2013

Pointer Events - W3C Editor's Draft 14 May 2014

html5labs.interoperabilitybridges.com/prototypes/...

Issue 162757: Implement pointer events in Chrome behind experimental flag

Bug 822898 - Implement pointer events

...what about Apple?

Bug 105463 - Implement pointer events RESOLVED WONTFIX

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html

events fired on tapmousemove* >

pointerover > mouseover >pointerenter > mouseenter >pointerdown > mousedown >pointermove > mousemove >

gotpointercapture >focus >

pointerup > mouseup >lostpointercapture >

pointerout > mouseout >pointerleave > mouseleave >

click

mouse events fired “inline” with pointer events(for a primary pointer, e.g. first finger on screen)

vendor-prefixed in IE10MSPointerDown etc

navigator.msMaxTouchPoints-ms-touch-action

unprefixed in IE11 (but prefixed versions still mapped for compatibility)

/* Pointer Events extend Mouse Events vs Touch Events and their completely new objects/arrays */

interface PointerEvent : MouseEvent { readonly attribute long pointerId; readonly attribute long width; readonly attribute long height; readonly attribute float pressure; readonly attribute long tiltX; readonly attribute long tiltY; readonly attribute DOMString pointerType; readonly attribute boolean isPrimary;}

/* plus all the classic MouseEvent attributes like clientX , clientY , etc */

simple feature detection forpointer events

/* detecting pointer events support */

if ( window.PointerEvent ) { /* some clever stuff here but this covers touch, stylus, mouse, etc */}

/* still listen to click for keyboard! */

/* detect maximum number of touch points */

if ( navigator.maxTouchPoints > 0 ) { /* device with a touchscreen */}

if ( navigator.maxTouchPoints > 1 ) { /* multitouch-capable device */}

are pointer events better?

no need for separate mouse ortouch event listeners

/* touch events: separate handling */

foo.addEventListener('touchmove', ... , false);foo.addEventListener('mousemove', ... , false);

/* pointer events: single listener for mouse, stylus, touch */

foo.addEventListener(' pointermove ', ... , false);

no need for separate mouse ortouch code to get x / y coords

/* Pointer Events extend Mouse Events */

foo.addEventListener(' pointermove ', function(e) { ... posX = e.clientX ; posY = e.clientY ; ...}, false);

www.w3.org/TR/pointerevents/#pointerevent-interface

but you can distinguish mouseor touch or stylus if needed

foo.addEventListener('pointermove', function(e) { ... switch( e.pointerType ) { case ' mouse ': ... break; case ' pen ': ... break; case ' touch ': ... break; default : /* future-proof */ } ...} , false);

pointer eventsvs

limitations/problems of mouseevent emulation

1.  delayed event dispatch2.  mousemove doesn't track

1.  delayed event dispatch2.  mousemove doesn't track

patrickhlauke.github.io/touch/tests/event-listener_show-delay.html(IE/Win8 has double-tap to zoom, so problem on desktop too)

patrickhlauke.github.io/touch/tests/event-listener.html

patrickhlauke.github.io/touch/tests/event-listener.html

pointer / mouse events and delaymousemove >

pointerover > mouseover >pointerenter > mouseenter >pointerdown > mousedown >pointermove > mousemove >

gotpointercapture >focus >

pointerup > mouseup >lostpointercapture >

pointerout > mouseout >pointerleave > mouseleave >

[300ms delay]click

“how can we make it feelresponsive like a native app?”

we could try a similarapproach to touch events...

•  double-up listeners - pointerup and click

•  prevent code firing twice - preventDefault

preventDefault() on pointerdown stops mouse compatibilityevents, but click is not considered mouse compatibility event

patrickhlauke.github.io/touch/tests/event-listener.html

touch-action

CSS propertytouch-action: auto | none | [ pan-x || pan-y ] | manipulation

www.w3.org/TR/pointerevents/#the-touch-action-css-propertyonly prevents default touch action (e.g. double-tap to zoom) does not

stop synthetic mouse events nor click

touch-action:none killsscrolling, long-press,

pinch/zoom

touch-action:manipulation

patrickhlauke.github.io/touch/tests/event-listener_touch[...]

Bug 979345 - Implement touch-action:manipulation [...]

Issue 349016: Add support for touch-action:manipulation

chrome://flags/#enable-experimental-web-platform-features (Chrome 35)

Bug 133114 - Implement " touch-action:manipulation " [...]

1.  delayed event dispatch2.  mousemove doesn't track

patrickhlauke.github.io/touch/particle/2

touch-action:none

patrickhlauke.github.io/touch/particle/2a

what about multitouch?

/* PointerEvents don't have the handy TouchList objects, so we have to replicate something similar... */

var points = [];switch (e.type) { case ' pointerdown ': /* add to the array */ break; case ' pointermove ': /* update the relevant array entry's x and y */ break; case ' pointerup ': /* remove the relevant array entry */ break;}

patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer.html

(note multiple isPrimary pointers)

/* like iOS/Safari, IE/Win has higher-level gestures , but these are not part of the W3C Pointer Events spec.

Replicate these from basic principles again... */

/* advanced topic: pointer capture */

gotpointercapture / lostpointercapture

element.setPointerCapture(pointerId)

patrickhlauke.github.io/touch/tests/pointercapture.html

what about backwards-compatibility?

touchstart > [touchmove]+ > touchend >[300ms delay] >

mouseover > mousemove > mousedown > mouseup > click

vs

pointerover > mouseover > pointerdown > mousedown >pointermove > mousemove > pointerup > mouseup >pointerout > mouseout > [300ms delay] > click

W3C Touch Events Community Group

/* cover all cases (hat-tip Stu Cox) */

if ('onpointerdown' in window) {

/* bind to Pointer Events: pointerdown, pointerup, etc */

} else {

/* bind to mouse events: mousedown, mouseup, etc */

if ('ontouchstart' in window) { /* bind to Touch Events: touchstart, touchend, etc */ }}

/* bind to keyboard / click */

polyfills for pointer events(code for tomorrow, today)

HandJS

www.catuhe.com/msdn/handjs/

Polymer

GitHub - Polymer/PointerEvents

/* Polymer's PointerEvents are not fired unless an element has a (custom) touch-action attribute */

<div id="foo" touch-action="none" ></div>

utility libraries(because life is too short to hand-code gesture support)

Hammer.js

/* Hammer's high-level events example */

var element = document.getElementById('test_el');

var hammertime = Hammer(element).on("swipeleft swiperight", function(event) { /* handle horizontal swipes */});

jQuery Mobile? Sencha Touch?check for support of IE10+

and “this is a touch device”

assumptions

debugging/testing

Chrome DevTools / Using the Console / Monitoring events

chrome://flags/#touch-events

beware inaccurate emulation

Issue 181204: [...] event order different from real devicesFixed in Chrome (Canary) 37

Bug 920956 - DevTools touch emulation: suppress regular mouse events ...

beware inaccurateimplementation

Bug 861876 - [...] multiple mousemoves being fired

Bug 861876 - [...] preventDefault on touchstart doesn't stop synthetic mouse events

further reading...

•  Matt Gaunt – Touch Feedback for Mobile Sites

•  Jonathan Stark – FastActive

•  Stephen Woods – HTML5 Touch Interfaces

•  Chris Wilson + Paul Kinlan – Touch And Mouse: Together Again For

The First Time

•  Ryan Fioravanti – Creating Fast Buttons for Mobile Web Applications

•  Boris Smus – Multi-touch Web Development

•  Boris Smus – Generalized input on the cross-device web

•  Boris Smus – Interactive touch laptop experiments

•  Rick Byers + Boris Smus (Google I/O) – Point, Click, Tap, Touch -

Building Multi-Device Web Interfaces

•  Grant Goodale – Touch Events

•  W3C – Touch Events Extensions

•  Mozilla Developer Network – Touch Events

•  WebPlatform.org – Pointer Events

•  Rick Byers – The best way to avoid the dreaded 300ms click delay is

to disable double-tap zoom

•  Chromium Issue 152149: All touch-event related APIs should exist if

touch support is enabled at runtime

•  Tim Kadlec – Avoiding the 300ms Click Delay, Accessibly

•  Microsoft – Pointer events updates (differences between IE10-IE11)

•  Patrick H. Lauke – Webseiten zum Anfassen

•  Patrick H. Lauke – Drei unter einem Dach: Pointer-Events für Maus,

Stylus und Touchscreen

•  Patrick H. Lauke – Make your site work on touch devices

•  Stu Cox – You can't detect a touchscreen

•  Tilo Mitra – The State of Gestures

•  Microsoft – Hover touch support (IE10/IE11)

•  W3C Media Queries Level 4 – pointer

•  Stu Cox – The Good & Bad of Level 4 Media Queries

•  Peter-Paul Koch – Touch table

•  Peter-Paul Koch – Preventing the touch default

•  Peter-Paul Koch – Mouse event bubbling in iOS

•  Edge Conference (Feb 2013 London) – Panel: Input

•  Edge Conference (Mar 2014 London) – Panel: Pointers and Interactions

•  Trent Walton – Device-Agnostic

•  Stu Cox – The Golden Pattern for Handling Touch Input

•  Matt Gaunt – ‘Focusing’ on the Web Today

•  Mobiscroll – Working with touch events

youtube.com/watch?v=AZKpByV5764

get in touch@patrick_h_lauke

github.com/patrickhlauke/touchpatrickhlauke.github.io/getting-touchy-presentation

slideshare.net/reduxpaciellogroup.com

splintered.co.uk