getting touchy - an introduction to touch and pointer events / web rebels / oslo / 22 may 2014
DESCRIPTION
TRANSCRIPT
getting touchyAN INTRODUCTION TO TOUCH AND POINTER EVENTS
Patrick H. Lauke / Web Rebels / Oslo / 22 May 2014
“how can I make my websitework on touch devices?”
you don't need touch eventsbrowsers emulate regular
mouse events
patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html
patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html
compatibility mouse events(mouseenter) > mouseover > mousemove* > mousedown >
(focus) > mouseup > click
* only a single “sacrificial” mousemove event fired
on first tap(mouseenter) > mouseover > mousemove >
mousedown > (focus) > mouseup > click
subsequent tapsmousemove > mousedown > mouseup > click
tapping awaymouseout > (blur)
focus / blur only on focusable elements in Firefoxmouseout not on iOS Safari/WebView (e.g. iOS Chrome)
Opera Mobile and
emulation works,but is limiting/problematic
1. delayed event dispatch2. mousemove doesn't track
1. delayed event dispatch2. mousemove doesn't track
patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
1. delayed event dispatch2. mousemove doesn't track
patrickhlauke.github.io/touch/particle/2
(iOS7 bug: moving finger fires mousemove on scroll)
“we need to go deeper...”
touch events
introduced by Apple, adoptedin Chrome/Firefox/Opera
www.w3.org/TR/touch-events
touchstarttouchmovetouchend
touchcancel
touchentertouchleave
patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html
patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html
Bug 128534 - 'mouseenter' mouse compat event not fired...
events fired on taptouchstart > [touchmove]+ > touchend >
(mouseenter) > mouseover > mousemove > mousedown >(focus) > mouseup > click
(mouse events only fired for single-finger tap)
on first taptouchstart > [touchmove]+ > touchend >
(mouseenter) > mouseover > mousemove > mousedown >(focus) > mouseup > click
subsequent tapstouchstart > [touchmove]+ > touchend >mousemove > mousedown > mouseup > click
tapping awaymouseout > (mouseleave) > (blur)
• too many touchmove events prevent mouse compatibility events
after touchend (not considered a "clean" tap)
• too many touchmove events on activatable elements can lead to
touchcancel (in old Chrome/Browser versions)
• not all browsers consistently send the touchmove
some browsers outright weird...
Browser/Android 4.3(AppleWebKit/534.30)
mouseover > mousemove > touchstart > touchend >mousedown > mouseup > click
Browser/Blackberry PlayBook 2.0(AppleWebKit/536.2)
touchstart > mouseover > mousemove > mousedown >touchend > mouseup > click
touch eventsvs
limitations/problems
1. delayed event dispatch2. mousemove doesn't track
1. delayed event dispatch2. mousemove doesn't track
patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
why the delay?double-tap to zoom
(mostly anyway)
what if browsers didn't wait?
Puffin/Android double-tap zooms and fires mouse events + click(also, doesn't support touch events)
when does the delay happen?
patrickhlauke.github.io/touch/tests/event-listener.html
touch / mouse events delaytouchstart > [touchmove]+ > touchend >
[300ms delay]
(mouseenter) > mouseover > mousemove > mousedown >(focus) > mouseup > click
“how can we make it feelresponsive like a native app?”
react to events fired before the300ms delay...
touchstart for an “immediate”control
(fire/jump button on a game)
touchend for a control thatfires after finger lifted
interlude: simple featuredetection for touch events
/* feature detection for touch events */
if ( 'ontouchstart' in window ) { /* some clever stuff here */}
/* older browsers have flaky support so more hacky tests needed...use Modernizr.touch or similar */
/* common performance “trick” */
var clickEvent = ( 'ontouchstart' in window ? 'touchend' : 'click' );
blah.addEventListener( clickEvent , function() { ... }, false);
don't make it touch-exclusive
/* common performance “trick” */
var clickEvent = ( 'ontouchstart' in window ? 'touchend' : 'click');
...
/* if touch events are supported, only listen to touchend, not click */
hybrid devicestouch + mouse + keyboard
Android + mouse – behaves like touchtouchstart > touchend > mouseover > mousemove > mousedown >
(focus) > mouseup > click
Blackberry PlayBook 2.0 + mouse - like desktopmouse mouseover > mousedown > mousemove > mouseup > click
Android + keyboard – like desktop keyboardfocus > click
iOS + VoiceOver (with/without keyboard) – similar to touchfocus > touchstart > touchend > (mouseenter) > mouseover >
mousemove > mousedown > blur > mouseup > click
Android + TalkBack – keyboard/mouse hybridfocus > blur > mousedown > mouseup > click > focus(?)
hacks.mozilla.org - Detecting touch [...]
/* feature detection for touch events */
if ('ontouchstart' in window) { /* browser supports touch events but user is not necessarily using touch (exclusively) */
/* it could be a mobile, tablet, desktop, fridge ... */}
touch or mouse or keyboard
touch and mouse and keyboard
/* doubled-up event listeners */
foo.addEventListener(' touchend ', someFunction, false);foo.addEventListener(' click ', someFunction, false);
/* prevent delay + mouse events */
foo.addEventListener(' touchstart ', function(e) { e.preventDefault();}, false);
/* doubled-up event listeners */
foo.addEventListener('touchend', someFunction, false);foo.addEventListener('click', someFunction, false);
preventDefault() killsscrolling, pinch/zoom, etc
apply preventDefault()carefully
(just on buttons/links, not entire page)
browsers working to removedouble-tap to zoom delay
(when page not zoomable)
<meta name="viewport" content="user-scalable=no">patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
<meta name="viewport" content="user-scalable=no">patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
... content="minimum-scale=1, maximum-scale=1"patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html
... content="minimum-scale=1, maximum-scale=1"patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html
Bug 922896 - Optimizations to remove 300ms [...] delay[RESOLVED - FIXED]
what about accessibility?
patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
Chrome 32+ / Android: ... content="width=device-width"updates.html5rocks.com/2013/12/300ms-tap-delay-gone-away
Bug 941995 - Remove 300ms [...] on "responsive" pages
patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
iOS/Safari designed themselves into a corner: “double-tap to scroll”bugs.webkit.org/show_bug.cgi?id=122212
1. delayed event dispatch2. mousemove doesn't track
patrickhlauke.github.io/touch/particle/2(iOS7 bug: moving finger fires mousemove on scroll)
events fired on taptouchstart > [touchmove]+ > touchend >
(mouseenter) > mouseover >mousemove* > mousedown > (focus) >
mouseup > click
* mouse event emulation fires only a single mousemove
too many touchmove events prevent mouse compatibility events after touchend
doubling up handling ofmousemove and touchmove
var posX, posY;...function positionHandler(e) { posX = e.clientX ; posY = e.clientY ;}...canvas.addEventListener(' mousemove ', positionHandler, false);
var posX, posY;...function positionHandler(e) { /* handle both mouse and touch */}...canvas.addEventListener(' mousemove ', positionHandler, false);canvas.addEventListener(' touchmove ', positionHandler, false);
interface MouseEvent : UIEvent { readonly attribute long screenX ; readonly attribute long screenY ; readonly attribute long clientX ; readonly attribute long clientY ; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute unsigned short button; readonly attribute EventTarget relatedTarget; void initMouseEvent(...);};
www.w3.org/TR/DOM-Level-2-Events/events.html#Events-MouseEvent
partial interface MouseEvent { readonly attribute double screenX; readonly attribute double screenY; readonly attribute double pageX ; readonly attribute double pageY ; readonly attribute double clientX; readonly attribute double clientY; readonly attribute double x ; readonly attribute double y ; readonly attribute double offsetX ; readonly attribute double offsetY ;};
www.w3.org/TR/cssom-view/#extensions-to-the-mouseevent-interface
interface TouchEvent : UIEvent { readonly attribute TouchList touches ; readonly attribute TouchList targetTouches ; readonly attribute TouchList changedTouches ; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey;};
www.w3.org/TR/touch-events/#touchevent-interface
interface Touch { readonly attribute long identifier; readonly attribute EventTarget target; readonly attribute long screenX ; readonly attribute long screenY ; readonly attribute long clientX ; readonly attribute long clientY ; readonly attribute long pageX ; readonly attribute long pageY ;};
www.w3.org/TR/touch-events/#touch-interface
touches
all touch points on screen
targetTouches
all touch points that started on the element
changedTouches
touch points that caused the event to fire
patrickhlauke.github.io/touch/touchlist-objects
var posX, posY;...function positionHandler(e) { if ((e.clientX)&&(e.clientY)) { posX = e.clientX; posY = e.clientY; } else if (e.targetTouches) { posX = e.targetTouches[0].clientX; posY = e.targetTouches[0].clientY; e.preventDefault() ; }}...canvas.addEventListener('mousemove', positionHandler, false );canvas.addEventListener('touchmove', positionHandler, false );
tracking finger movement overtime ... swipe gestures
don't forget mouse/keyboard!
touchmove fires...a lot!
do absolute minimum on eachtouchmove
(usually: store coordinates)
heavy JavaScript onsetInterval or
requestAnimationFrame
why stop at a single point?multitouch support
interface TouchEvent : UIEvent { readonly attribute TouchList touches ; readonly attribute TouchList targetTouches ; readonly attribute TouchList changedTouches ; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey;};
www.w3.org/TR/touch-events/#touchevent-interface
/* iterate over touch array */
for (i=0; i< e.targetTouches .length; i++) { ... posX = e.targetTouches[i].clientX ; posY = e.targetTouches[i].clientY ; ...}
patrickhlauke.github.io/touch/tracker/multi-touch-tracker.html
iOS/iPad preventDefault()can't override 4-finger
gestures
iOS7/Safari preventDefault()can't override back/forward
swipe gestures
multitouch gestures
/* iOS/Safari/WebView has gesture events for size/rotation,not part of the W3C Touch Events spec. */
gesturestart / gesturechange / gestureend
function(e) { /* e.scale e.rotation */}
/* not supported in Chrome/Firefox/Opera */
/* with some trigonometry we can replicate these from basic principles. */
var distance = Math.sqrt(Math.pow(...)+Math.pow(...));var angle = Math.atan2(...);
patrickhlauke.github.io/touch/picture-organiser
not all old/cheap devices/OSssupport multitouch!
HTC Hero – Android 2.1
LG Optimus 2X – Android 2.3.7
ZTE Open – Firefox OS 1.1
what about Internet Explorer?
up to IE9 (Win7 / WinPhone7.5)only mouse events
in IE10 Microsoft introducedPointer Events
David Rousset - Unifying touch and mouse [...]
not just some“not invented here”
technology
Pointer Events - W3C Editor's Draft 14 May 2014
html5labs.interoperabilitybridges.com/prototypes/...
Issue 162757: Implement pointer events in Chrome behind experimental flag
...what about Apple?
Bug 105463 - Implement pointer events RESOLVED WONTFIX
patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html
patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html
events fired on tapmousemove* >
pointerover > mouseover >pointerenter > mouseenter >pointerdown > mousedown >pointermove > mousemove >
gotpointercapture >focus >
pointerup > mouseup >lostpointercapture >
pointerout > mouseout >pointerleave > mouseleave >
click
mouse events fired “inline” with pointer events(for a primary pointer, e.g. first finger on screen)
vendor-prefixed in IE10MSPointerDown etc
navigator.msMaxTouchPoints-ms-touch-action
unprefixed in IE11 (but prefixed versions still mapped for compatibility)
/* Pointer Events extend Mouse Events vs Touch Events and their completely new objects/arrays */
interface PointerEvent : MouseEvent { readonly attribute long pointerId; readonly attribute long width; readonly attribute long height; readonly attribute float pressure; readonly attribute long tiltX; readonly attribute long tiltY; readonly attribute DOMString pointerType; readonly attribute boolean isPrimary;}
/* plus all the classic MouseEvent attributes like clientX , clientY , etc */
simple feature detection forpointer events
/* detecting pointer events support */
if ( window.PointerEvent ) { /* some clever stuff here but this covers touch, stylus, mouse, etc */}
/* still listen to click for keyboard! */
/* detect maximum number of touch points */
if ( navigator.maxTouchPoints > 0 ) { /* device with a touchscreen */}
if ( navigator.maxTouchPoints > 1 ) { /* multitouch-capable device */}
are pointer events better?
no need for separate mouse ortouch event listeners
/* touch events: separate handling */
foo.addEventListener('touchmove', ... , false);foo.addEventListener('mousemove', ... , false);
/* pointer events: single listener for mouse, stylus, touch */
foo.addEventListener(' pointermove ', ... , false);
no need for separate mouse ortouch code to get x / y coords
/* Pointer Events extend Mouse Events */
foo.addEventListener(' pointermove ', function(e) { ... posX = e.clientX ; posY = e.clientY ; ...}, false);
www.w3.org/TR/pointerevents/#pointerevent-interface
but you can distinguish mouseor touch or stylus if needed
foo.addEventListener('pointermove', function(e) { ... switch( e.pointerType ) { case ' mouse ': ... break; case ' pen ': ... break; case ' touch ': ... break; default : /* future-proof */ } ...} , false);
pointer eventsvs
limitations/problems of mouseevent emulation
1. delayed event dispatch2. mousemove doesn't track
1. delayed event dispatch2. mousemove doesn't track
patrickhlauke.github.io/touch/tests/event-listener_show-delay.html(IE/Win8 has double-tap to zoom, so problem on desktop too)
patrickhlauke.github.io/touch/tests/event-listener.html
patrickhlauke.github.io/touch/tests/event-listener.html
pointer / mouse events and delaymousemove >
pointerover > mouseover >pointerenter > mouseenter >pointerdown > mousedown >pointermove > mousemove >
gotpointercapture >focus >
pointerup > mouseup >lostpointercapture >
pointerout > mouseout >pointerleave > mouseleave >
[300ms delay]click
“how can we make it feelresponsive like a native app?”
we could try a similarapproach to touch events...
• double-up listeners - pointerup and click
• prevent code firing twice - preventDefault
preventDefault() on pointerdown stops mouse compatibilityevents, but click is not considered mouse compatibility event
patrickhlauke.github.io/touch/tests/event-listener.html
touch-action
CSS propertytouch-action: auto | none | [ pan-x || pan-y ] | manipulation
www.w3.org/TR/pointerevents/#the-touch-action-css-propertyonly prevents default touch action (e.g. double-tap to zoom) does not
stop synthetic mouse events nor click
touch-action:none killsscrolling, long-press,
pinch/zoom
touch-action:manipulation
patrickhlauke.github.io/touch/tests/event-listener_touch[...]
Bug 979345 - Implement touch-action:manipulation [...]
Issue 349016: Add support for touch-action:manipulation
Bug 133114 - Implement " touch-action:manipulation " [...]
1. delayed event dispatch2. mousemove doesn't track
touch-action:none
what about multitouch?
/* PointerEvents don't have the handy TouchList objects, so we have to replicate something similar... */
var points = [];switch (e.type) { case ' pointerdown ': /* add to the array */ break; case ' pointermove ': /* update the relevant array entry's x and y */ break; case ' pointerup ': /* remove the relevant array entry */ break;}
patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer.html
(note multiple isPrimary pointers)
/* like iOS/Safari, IE/Win has higher-level gestures , but these are not part of the W3C Pointer Events spec.
Replicate these from basic principles again... */
/* advanced topic: pointer capture */
gotpointercapture / lostpointercapture
element.setPointerCapture(pointerId)
patrickhlauke.github.io/touch/tests/pointercapture.html
what about backwards-compatibility?
touchstart > [touchmove]+ > touchend >[300ms delay] >
mouseover > mousemove > mousedown > mouseup > click
vs
pointerover > mouseover > pointerdown > mousedown >pointermove > mousemove > pointerup > mouseup >pointerout > mouseout > [300ms delay] > click
/* cover all cases (hat-tip Stu Cox) */
if ('onpointerdown' in window) {
/* bind to Pointer Events: pointerdown, pointerup, etc */
} else {
/* bind to mouse events: mousedown, mouseup, etc */
if ('ontouchstart' in window) { /* bind to Touch Events: touchstart, touchend, etc */ }}
/* bind to keyboard / click */
polyfills for pointer events(code for tomorrow, today)
/* Polymer's PointerEvents are not fired unless an element has a (custom) touch-action attribute */
<div id="foo" touch-action="none" ></div>
utility libraries(because life is too short to hand-code gesture support)
/* Hammer's high-level events example */
var element = document.getElementById('test_el');
var hammertime = Hammer(element).on("swipeleft swiperight", function(event) { /* handle horizontal swipes */});
jQuery Mobile? Sencha Touch?check for support of IE10+
and “this is a touch device”
assumptions
debugging/testing
Chrome DevTools / Using the Console / Monitoring events
beware inaccurate emulation
Issue 181204: [...] event order different from real devicesFixed in Chrome (Canary) 37
Bug 920956 - DevTools touch emulation: suppress regular mouse events ...
beware inaccurateimplementation
Bug 861876 - [...] multiple mousemoves being fired
Bug 861876 - [...] preventDefault on touchstart doesn't stop synthetic mouse events
further reading...
• Matt Gaunt – Touch Feedback for Mobile Sites
• Jonathan Stark – FastActive
• Stephen Woods – HTML5 Touch Interfaces
• Chris Wilson + Paul Kinlan – Touch And Mouse: Together Again For
The First Time
• Ryan Fioravanti – Creating Fast Buttons for Mobile Web Applications
• Boris Smus – Multi-touch Web Development
• Boris Smus – Generalized input on the cross-device web
• Boris Smus – Interactive touch laptop experiments
• Rick Byers + Boris Smus (Google I/O) – Point, Click, Tap, Touch -
Building Multi-Device Web Interfaces
• Grant Goodale – Touch Events
• W3C – Touch Events Extensions
• Mozilla Developer Network – Touch Events
• WebPlatform.org – Pointer Events
• Rick Byers – The best way to avoid the dreaded 300ms click delay is
to disable double-tap zoom
• Chromium Issue 152149: All touch-event related APIs should exist if
touch support is enabled at runtime
• Tim Kadlec – Avoiding the 300ms Click Delay, Accessibly
• Microsoft – Pointer events updates (differences between IE10-IE11)
• Patrick H. Lauke – Webseiten zum Anfassen
• Patrick H. Lauke – Drei unter einem Dach: Pointer-Events für Maus,
Stylus und Touchscreen
• Patrick H. Lauke – Make your site work on touch devices
• Stu Cox – You can't detect a touchscreen
• Tilo Mitra – The State of Gestures
• Microsoft – Hover touch support (IE10/IE11)
• W3C Media Queries Level 4 – pointer
• Stu Cox – The Good & Bad of Level 4 Media Queries
• Peter-Paul Koch – Touch table
• Peter-Paul Koch – Preventing the touch default
• Peter-Paul Koch – Mouse event bubbling in iOS
• Edge Conference (Feb 2013 London) – Panel: Input
• Edge Conference (Mar 2014 London) – Panel: Pointers and Interactions
• Trent Walton – Device-Agnostic
• Stu Cox – The Golden Pattern for Handling Touch Input
• Matt Gaunt – ‘Focusing’ on the Web Today
• Mobiscroll – Working with touch events
get in touch@patrick_h_lauke
github.com/patrickhlauke/touchslideshare.net/redux
paciellogroup.comsplintered.co.uk