getting touchy - introduction to touch (and pointer) events / jquery europe 2014 / vienna 28.02.2014
TRANSCRIPT
getting touchy AN INTRODUCTION TO TOUCH (AND POINTER) EVENTS
Patrick H. Lauke / jQuery Europe / Vienna / 28 February 2014
“how can I make my website work on touch devices?”
you don't need touch eventsbrowsers emulate regular mouse events
patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html
(mouseenter) > mouseover > mousemove* > mousedown > (focus) > mouseup > click
*only a single “sacrificial” event is fired
on first tap(mouseenter) > mouseover > mousemove > mousedown > (focus) > mouseup > click
subsequent tapsmousemove > mousedown > mouseup > click
tapping awaymouseout > (blur)
focus/blur only on focusable elements in Opera Mobile and Firefoxmouseout not on iOS Safari and embedded WebView (e.g. iOS Chrome)
emulation works,but is limiting/problematic
1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track
1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track
patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track
1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track
patrickhlauke.github.io/touch/particle/2(bug in iOS7: moving finger does seem to scroll and fire multiple mousemove events?!)
“we need to go deeper...”
touch eventsintroduced in iOS 2.0, adopted in Chrome/Firefox/Opera
www.w3.org/TR/touch-events
touchstarttouchmove
touchend
touchcanceltouchentertouchleave
patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html
patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.htmlBug 128534 - 'mouseenter' mouse compat event not fired when listeners for touch events
touchstart > [touchmove]+ > touchend > (mouseenter) > mouseover >
mousemove > mousedown > (focus) > mouseup > click
(mouse events only fired for single-finger tap)
on first taptouchstart > [touchmove]+ > touchend >
(mouseenter) > mouseover > mousemove > mousedown > (focus) > mouseup > click
subsequent tapstouchstart > [touchmove]+ > touchend > mousemove > mousedown > mouseup > click
tapping awaymouseout > (mouseleave) > (blur)
too many touchmove events abort the tap (after touchend)too many touchmove events on activatable elements lead to touchcancel
not all browsers consistently send the touchmove
some browsers outright weird...
Browser/Android 4.3(AppleWebKit/534.30)
mouseover > mousemove > touchstart > touchend > mousedown > mouseup > click
Browser/Blackberry PlayBook 2.0(AppleWebKit/536.2)
touchstart > mouseover > mousemove > mousedown > touchend > mouseup > click
touch eventsvs
limitations/problems
1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track
1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track
patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
why the delay?double-tap to zoom
(mostly anyway)
what if browsers didn't wait?
Puffin/Android (popular in Korea?) puffinbrowser.comdouble-tap zooms and fires mouse events + click
(also, doesn't support touch events at all)
when does the delay happen?
patrickhlauke.github.io/touch/tests/event-listener.html
patrickhlauke.github.io/touch/tests/event-listener.html
“how can we make it feel responsive like a native app?”
react to events fired before the 300ms delay...
touchstartfor an “immediate” control
touchendfor a control that fires after finger lifted
interlude:simple feature detection for
touch events
if ('ontouchstart' in window) {
/* some clever stuff here */
}
/* older browsers have flaky support so more hacky tests needed... use Modernizr.touch or similar */
/* common performance “trick” */
var clickEvent = ('ontouchstart' in window ? 'touchend' : 'click');
blah.addEventListener(clickEvent,function() { ... }, false);
don't make it touch-exclusive
hacks.mozilla.org/2013/04/detecting-touch...
if ('ontouchstart' in window) {
/* browser supports touch events but user is not necessarily using touch (exclusively) */
}
hybrid devicestouch + mouse + keyboard
Android + mouse – behaves like touchtouchstart > touchend > mouseover > mousemove > mousedown >
(focus) > mouseup > click
Android + keyboard – like desktop keyboardfocus > click
VoiceOver enables keyboard access on iOS
iOS + keyboard – similar to touchfocus > touchstart > touchend >
(mouseenter) > mouseover > mousemove > mousedownblur > mouseup > click
iOS + VoiceOver – similar to touchfocus > touchstart > touchend >
(mouseenter) > mouseover > mousemove > mousedownblur > mouseup > click
Android + TalkBack – keyboard/mouse hybridfocus > blur > mousedown > mouseup > click > focus(?)
(though seems a bit flakey...)
touch or mouse or keyboard
touch and mouse and keyboard
/* doubled-up event listeners */
foo.addEventListener('touchend', someFunction, false);
foo.addEventListener('click', someFunction, false);
foo.addEventListener('touchstart',function(e) {/* prevent delay + mouse events */e.preventDefault();
}, false);
/* doubled-up event listeners */
foo.addEventListener('touchend', someFunction, false);
foo.addEventListener('click', someFunction, false);
preventDefaultcan kill scrolling, pinch/zoom, etc on
touchstart/touchmove
browsers working to remove delay when possible
patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html
patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html
Bug 922896 - Optimizations to remove 300ms touch > mouse events delay[RESOLVED - FIXED]
Chrome 32 for Android removes delay also when
<meta name="viewport" content="width=device-width">
updates.html5rocks.com/2013/12/300ms-tap-delay-gone-away
patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
iOS/Safari designed themselves into a cornerwith “double-tap to scroll”
bugs.webkit.org/show_bug.cgi?id=122212
1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track
no isolated hover (or focus) on touch devices*
* iOS fakes it, Samsung Galaxy S4 + stylus, ...
patrickhlauke.github.io/touch/css-dropdown/mouse-only.html
http://developer.apple.com/library/IOS/...
www.stucox.com/blog/you-cant-detect-a-touchscreen
avoid hover-based interfaces?
complement for keyboard / touch!
patrickhlauke.github.io/touch/css-dropdown/mouse-keyboard-touch.html
1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track
touchstart > [touchmove]+ > touchend >(mouseenter) > mouseover >
mousemove* > mousedown > (focus) >mouseup > click
*mouse event emulation fires only a single mousemove
doubling up handling of mousemove and touchmove
var posX, posY;
...
function positionHandler(e) {posX = e.clientX;posY = e.clientY;
}
...
canvas.addEventListener('mousemove', positionHandler, false);
var posX, posY;
...
function positionHandler(e) {/* handle both mouse and touch? */
}
...
canvas.addEventListener('mousemove',positionHandler, false);
canvas.addEventListener('touchmove',positionHandler, false);
interface MouseEvent : UIEvent {readonly attribute long screenX;readonly attribute long screenY;readonly attribute long clientX;readonly attribute long clientY;readonly attribute boolean ctrlKey;readonly attribute boolean shiftKey;readonly attribute boolean altKey;readonly attribute boolean metaKey;readonly attribute unsigned short button;readonly attribute EventTarget relatedTarget;void initMouseEvent(...);
};
www.w3.org/TR/DOM-Level-2-Events/events.html#Events-MouseEvent
interface TouchEvent : UIEvent {readonly attribute TouchList touches;readonly attribute TouchList targetTouches;readonly attribute TouchList changedTouches;readonly attribute boolean altKey;readonly attribute boolean metaKey;readonly attribute boolean ctrlKey;readonly attribute boolean shiftKey;
};
www.w3.org/TR/touch-events/#touchevent-interface
interface Touch {readonly attribute long identifier;readonly attribute EventTarget target;readonly attribute long screenX;readonly attribute long screenY;readonly attribute long clientX;readonly attribute long clientY;readonly attribute long pageX;readonly attribute long pageY;
};
www.w3.org/TR/touch-events/#touch-interface
patrickhlauke.github.io/touch/touchlist-objects
var posX, posY;
...
function positionHandler(e) {if ((e.clientX)&&(e.clientY)) {
posX = e.clientX;posY = e.clientY;
} else if (e.targetTouches) {posX = e.targetTouches[0].clientX;posY = e.targetTouches[0].clientY;e.preventDefault();
}}
...
canvas.addEventListener('mousemove',positionHandler, false );
canvas.addEventListener('touchmove',positionHandler, false );
tracking finger movementover time ... swipe gestures
don't forget mouse/keyboard !
touchmove fires...a lot!
do absolute minimum on each touchmove
(usually: store new coordinates)
heavy JavaScript on requestAnimationFrame
setInterval
why stop at a single point?multitouch support
interface TouchEvent : UIEvent {readonly attribute TouchList touches;readonly attribute TouchList targetTouches;readonly attribute TouchList changedTouches;readonly attribute boolean altKey;readonly attribute boolean metaKey;readonly attribute boolean ctrlKey;readonly attribute boolean shiftKey;
};
www.w3.org/TR/touch-events/#touchevent-interface
for (i=0; i<e.targetTouches.length; i++) {
...
posX = e.targetTouches[i].clientX;posY = e.targetTouches[i].clientY;
...
}
patrickhlauke.github.io/touch/tracker/multi-touch-tracker.html
iOS/iPad even preventDefault()can't override 4-finger gestures
iOS7/Safari even preventDefault()can't override back/forward swipe gestures
multitouch gestures
/* iOS/Safari has gesture events for size/rotation, not supported in Chrome/Firefox/Opera, not part of the W3C Touch Events spec. */
gesturestart, gesturechange, gestureend
e.scale, e.rotation
/* with some trigonometry we can replicate these from basic principles. */
var distance = Math.sqrt(Math.pow(...)+Math.pow(...));var angle = Math.atan2(...);
patrickhlauke.github.io/touch/picture-organiser
older/cheaper devices/OS versionsand multitouch?
LG Optimus 2X – Android 2.3.7
ZTE Open – FirefoxOS 1.1
touch events andInternet Explorer...
blogs.msdn.com/...
not just some “not invented here” new approach for IE10+
html5labs.interoperabilitybridges.com/prototypes/...
code.google.com/p/chromium/issues/detail?id=162757
...what about Apple?
bugs.webkit.org/show_bug.cgi?id=105463RESOLVED WONTFIX?!
patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html
patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html
mousemove* >pointerover > mouseover >
pointerenter > mouseenter > pointerdown > mousedown > pointermove > mousemove >
gotpointercapture > focus >
pointerup > mouseup >lostpointercapture >
pointerout > mouseout >pointerleave > mouseleave >
click
mouse events fired “inline” with pointer events(for a primary pointer, e.g. first finger on screen)
vendor-prefixed in IE10MSPointerDown etc
navigator.msMaxTouchPoints-ms-touch-action
unprefixed in IE11
simple feature detection for pointer events
if (window.PointerEvent) {
/* some clever stuff here but this covers touch, stylus, mouse, etc */
}
/* still listen to click for keyboard! */
if (navigator.maxTouchPoints > 1) {
/* multitouch-capable device */
}
patrickhlauke.github.io/touch/tests/pointer-multitouch-detect.html
no need for separatemouse or touchevent listeners
/* touch events: separate handling */
foo.addEventListener('touchmove', ... , false);foo.addEventListener('mousemove', ... , false);
/* pointer events: single listener for mouse, stylus, touch */
foo.addEventListener('pointermove', ... , false);
no need for separatemouse or touch
code to get x / y coords
interface MouseEvent : UIEvent {readonly attribute long screenX;readonly attribute long screenY;readonly attribute long clientX;readonly attribute long clientY;readonly attribute boolean ctrlKey;readonly attribute boolean shiftKey;readonly attribute boolean altKey;readonly attribute boolean metaKey;readonly attribute unsigned short button;readonly attribute EventTarget relatedTarget;void initMouseEvent(...);
};
www.w3.org/TR/DOM-Level-2-Events/events.html#Events-MouseEvent
/* Pointer Events extend Mouse Events vs Touch Events and their completely new objects/arrays */
interface PointerEvent : MouseEvent {readonly attribute long pointerId;readonly attribute long width;readonly attribute long height;readonly attribute float pressure;readonly attribute long tiltX;readonly attribute long tiltY;readonly attribute DOMString pointerType;readonly attribute boolean isPrimary;
};
www.w3.org/TR/pointerevents/#pointerevent-interface
but you can distinguishmouse or touch or stylus
if needed
foo.addEventListener('pointermove', function(e) { ... switch(e.pointerType) { case 'mouse': ... break; case 'pen': ... break; case 'touch': ... break; default: /* future-proof */ ... } , false);
“how can we make it feel responsive like a native app?”
delayed event dispatch
patrickhlauke.github.io/touch/tests/event-listener.html
you can preventDefault()most mouse compatibility events on pointerdown
...but click is not considered mouse compatibility event, and the 300ms delay is not affected by this.
patrickhlauke.github.io/touch/tests/event-listener.html
patrickhlauke.github.io/touch/tests/event-listener.html
touch-action: auto|none|[pan-x][pan-y]
www.w3.org/TR/pointerevents/#the-touch-action-css-property
only prevents default touch action (e.g. double-tap to zoom)does not stop synthetic mouse events nor click
touch-action:none
patrickhlauke.github.io/touch/tests/event-listener_touch-action-none.html
touch-action:nonekills scrolling, long-press,
pinch/zoom
what about
multitouch?
/* PointerEvents don't have the handy touch arrays, so we have to replicate something similar... */
/* PointerEvents don't have the handy touch arrays, so we have to replicate something similar... */
var points = [];
switch (e.type) {
case 'pointerdown': /* add to the array */ break;
case 'pointermove': /* update the relevant array entry's x and y */ break;
case 'pointerup': /* remove the relevant array entry */ break;
}
patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer.html
/* like iOS/Safari, IE10/Win has higher-level gestures, but these are not part of the W3C Pointer Events spec.
Replicate these from basic principles again? */
pointer events as the future?
what about backwards-compatibility?
touchstart > [touchmove]+ > touchend >[300ms delay] > mouseenter > mouseover > mousemove > mousedown > mouseup > click
+mousemove > pointerover > mouseover > pointerdown > mousedown > pointermove >
mousemove > pointerup > mouseup > pointerout > mouseout > [300ms delay] > click
polyfills for pointer events(code for tomorrow, today)
utility libraries(because life is too short to hand-code gesture support etc)
jQuery Mobile? Sencha Touch? …check for support of IE10+ and “this is a touch device” assumptions
debugging/testing
Issue 181204: Emulate touch events - event order different from real devices
Bug 920956 - DevTools touch emulation: suppress regular mouse events ...
developers.google.com/chrome-developer-tools/docs/console#monitoring_events
developers.google.com/chrome-developer-tools/docs/remote-debugging
further reading...
Matt Gaunt – Touch Feedback for Mobile Siteswww.gauntface.co.uk/blog/2013/06/25/touch-feedback-for-mobile-sites
Jonathan Stark – FastActivegithub.com/jonathanstark/FastActive
Stephen Woods – HTML5 Touch Interfaceswww.slideshare.net/ysaw/html5-touch-interfaces-sxsw-extended-version
David Rousset – Unifying touch and mouse: how Pointer Events will make cross-browsers touch support easyblogs.msdn.com/b/davrous/archive/2013/02/20/handling-touch[...]
Chris Wilson + Paul Kinlan – Touch And Mouse: Together Again For The First Timewww.html5rocks.com/en/mobile/touchandmouse
Patrick H. Lauke – Webseiten zum Anfassenwebkrauts.de/artikel/2012/einfuehrung-touch-events
Ryan Fioravanti – Creating Fast Buttons for Mobile Web Applicationsdevelopers.google.com/mobile/articles/fast_buttons
Boris Smus – Multi-touch Web Developmentwww.html5rocks.com/en/mobile/touch
Boris Smus – Generalized input on the cross-device websmus.com/mouse-touch-pointer
Rick Byers + Boris Smus (Google I/O) – Point, Click, Tap, Touch - Building Multi-Device Web Interfacesdevelopers.google.com/events/io/sessions/361772634
Grant Goodale – Touch Eventswww.w3.org/conf/2011/#Touch_Events
W3C – Touch Events Extensionswww.w3.org/TR/2013/NOTE-touch-events-extensions-20131031
Mozilla Developer Network – Touch Eventsdeveloper.mozilla.org/en-US/docs/Web/Guide/API/DOM/Events/Touch_events
WebPlatform.org – Pointer Eventsdocs.webplatform.org/wiki/concepts/Pointer_Events
Rick Byers – The best way to avoid the dreaded 300ms click delay is to disable double-tap zoomplus.google.com/+RickByers/posts/ej7nsuoaaDa
Tim Kadlec – Avoiding the 300ms Click Delay, Accessiblytimkadlec.com/2013/11/Avoiding-the-300ms-click-delay-accessibly
Microsoft – Pointer events updates (differences between IE10 and IE11)msdn.microsoft.com/en-us/library/ie/dn304886(v=vs.85).aspx
Patrick H. Lauke – Webseiten zum Anfassenwebkrauts.de/artikel/2012/einfuehrung-touch-events
Patrick H. Lauke – Drei unter einem Dach: Pointer-Events für Maus, Stylus und Touchscreenwebkrauts.de/artikel/2013/drei-unter-einem-dach
Tilo Mitra – The State of Gesturesspeakerdeck.com/tilomitra/the-state-of-gestures
Microsoft – Hover touch support (IE10/IE11)msdn.microsoft.com/en-us/library/ie/dn265029(v=vs.85).aspx
Stu Cox – The Golden Pattern for Handling Touch Inputwww.stucox.com/blog/the-golden-pattern-for-handling-touch-input/
Peter-Paul Koch – Touch tablewww.quirksmode.org/mobile/tableTouch.html
Peter-Paul Koch – Preventing the touch defaultwww.quirksmode.org/mobile/default.html
Peter-Paul Koch – Mouse event bubbling in iOSwww.quirksmode.org/blog/archives/2014/02/mouse_event_bub.html
get in touch@patrick_h_lauke
github.com/patrickhlauke/touchslideshare.net/redux
paciellogroup.comsplintered.co.uk