js days mobile meow

97
Mobile Meow By YouTube's Greg Schechter Bring YouTube to the Mobile Web

Upload: greg-schechter

Post on 17-May-2015

6.877 views

Category:

Technology


0 download

DESCRIPTION

This is the story of the glory and struggle of bringing a high quality YouTube experience to the mobile web. Once upon a time there was a web developer who wanted to play videos on the web. So he filmed a cat and wrote a Flash and a HTML5 player. He spent many hours making it work on his favorite desktop browsers and even the one his grandfather still used. People could watch his cat video, he smiled, and the world was good. Then one day someone put a browser in a phone and soon there were many phones with many different browsers. This new set of environments were even harder to develop for and had a slew of new terrifying bugs. The web developer was miserable knowing people couldn’t watch his cat video. With much time and effort he figured out many of the secrets needed to combat the evils of the different mobile platforms. Once again people could watch his cat video, he smiled, and the world was good. The end.

TRANSCRIPT

Page 1: JS Days Mobile Meow

Mobile Meow

By YouTube's Greg Schechter

Bring YouTube to the Mobile Web

Page 2: JS Days Mobile Meow

Greg SchechterThe Web Warrior

[email protected]

Page 3: JS Days Mobile Meow

Video

Page 4: JS Days Mobile Meow

Video

Page 5: JS Days Mobile Meow
Page 6: JS Days Mobile Meow
Page 8: JS Days Mobile Meow

Video

Page 9: JS Days Mobile Meow

Flash

Page 10: JS Days Mobile Meow

<video>

Page 11: JS Days Mobile Meow

Video

Page 12: JS Days Mobile Meow

Playback Stats

Page 13: JS Days Mobile Meow

Flash Native <video>

Page 14: JS Days Mobile Meow

Flash Flash==Flash

Page 15: JS Days Mobile Meow

Flash Flash

Page 16: JS Days Mobile Meow

Native

Page 17: JS Days Mobile Meow

○ For fallbacks, use rtsp:// protocol (serving .3pg) if the device won't support HTML5

○ Use custom protocol / URL scheme to launch your own Android, iOS native app (no Windows Phone yet)

■ youtube://video_id

Native

Page 18: JS Days Mobile Meow

<video>

Page 19: JS Days Mobile Meow

The Different Browsers

Safari Android

Page 20: JS Days Mobile Meow

Safari Android Chrome Opera Mini Silk

IE

The Different Browsers

Firefox Opera

Page 21: JS Days Mobile Meow

Safari Android

Firefox

Chrome

Opera

Opera Mini Silk

IE

The Different Browsers

SAMPLE

Page 22: JS Days Mobile Meow

By brownpau at http://www.flickr.com/photos/brownpau/533267369/

Page 23: JS Days Mobile Meow

<video>

How do we start

Page 24: JS Days Mobile Meow

<video src="funny_cat_video">

The Source

Page 25: JS Days Mobile Meow

<video src="funny_cat_video.webm/mp4/ogg">

The Source

Page 26: JS Days Mobile Meow

Mobile Formats Support

Chrome Safari Firefox Opera IE Android

H.264

WebM Android 4+ Android 4+ Android 4+

HLS Android 3+

Platform Versions and Distribution

Page 27: JS Days Mobile Meow

The Source

<video> <source src="funny_cat_video.mp4"> <source src="funny_cat_video.webm"></video>

Page 28: JS Days Mobile Meow

HLS = HTTP Live Streaming

<video src="playlist.m3u8">

Page 29: JS Days Mobile Meow

HLS

index.m3u8

format-0.m3u8

format-0.m3u8

format-0.m3u8

video-0-1.tsvideo-0-2.ts

...

video-1-1.tsvideo-1-2.ts

...

video-2-1.tsvideo-2-2.ts

...

Page 30: JS Days Mobile Meow

HLS

Big Features○ Live and Adaptive Streaming○ Built in CDN failover○ Metadata + Cue Points

Apple App Requirements○ > 10 minutes in duration or 5 mb in 5 minutes○ at least one 64 kbps or lower stream required

Tools and more information on Apple's website:Apple's Using HTTP Live Streaming

Page 31: JS Days Mobile Meow

Power

Page 32: JS Days Mobile Meow

Power Consumption: H264 vs WebM

**fullscreen flash**windowed flash

Research done by Yossi Oren For more information visit http://iss.oy.ne.ro/

Page 33: JS Days Mobile Meow

Stick it in a page

Page 34: JS Days Mobile Meow

Basic video tag

Safari○ Renders a thumbnail poster and play button (flaky)○ Background is set to black and cannot be overwritten○ If a parent element has display:none set at any point the video fails to

playback properly

Opera○ Gives video dimentions black background if the information is

available○ Has issues with stretching WebM

IE and Android○ Play button/film icon for all video tags regardless if browser can play

the format○ Background is set to black/grey and cannot be overwritten

Page 35: JS Days Mobile Meow

Give it a poster and click it

Page 36: JS Days Mobile Meow

Poster Attribute

Use the poster attribute to get a thumbnail Safari, Chrome, Android, and Opera

○ Preserve aspect ratio of poster image

IE and Firefox○ Stretch image to size of video tag

IE○ Keeps poster as paused state

All Others○ Replace video with the paused frame

Page 37: JS Days Mobile Meow

Chrome○ Default click to toggle play/pause○ Remove with an empty onclick handler

Safari○ Will launch any supported video with or without an onclick handler

Android and IE○ Shows click interactions but does not play with empty onclick

Basic video tag

Page 38: JS Days Mobile Meow

Pro tips:○ Create your own cued state to convey you can play the video○ Set the background color to black for continuity across platforms

Basic video tag

Page 39: JS Days Mobile Meow

Controls

<video controls>

Page 40: JS Days Mobile Meow

Controls Chrome Safari Firefox Opera IE Android

Play/pause Button

Volume Controls

Only in Fullscreen

Mute toggle only

Don't work in Android 4

Seek bar

Fullscreen button

Button Gesture Button

Playback Type

Both Fullscreen only

Inline only Determined by hardware

Fullscreen only

Both

Page 41: JS Days Mobile Meow

Custom Controls

● Preserve your brand● Unified experience across platforms and

browsers

Page 42: JS Days Mobile Meow

Custom Controls

Lets get some custom controls

Page 43: JS Days Mobile Meow

Custom Controls

● Allows us to expand the set of controls and add our own○ annotations○ playlist○ captions○ more

Page 44: JS Days Mobile Meow

Custom Controls Pro Tips● User expect to be able to drag the progress

bar○ Need to remender to prevent scroll on touchstart

● Volume can't be set everywhere and users are accustomed to using device controls○ So don't build controls for it

Page 45: JS Days Mobile Meow

Custom Controls Pro Tips● Fingers are fat

○ Average finger is 11mm so make targets at least 40px with 10px padding

○ Use SVG's so icons can be scaled and shared with desktop application

○ Do what you can in css

Page 46: JS Days Mobile Meow

Custom Controls Pro Tips● Don't trigger content with hover● Fullscreen

○ The browsing context is always fullscreen so fake it

Page 47: JS Days Mobile Meow

Fullscreen

Page 48: JS Days Mobile Meow

requestFullScreen

Page 49: JS Days Mobile Meow

requestFullScreen

● Desktop Only○ Firefox, Safari, and Chrome○ Vendor Prefixed

Page 50: JS Days Mobile Meow

What about Mobile?

Page 51: JS Days Mobile Meow

Isn't it already fullscreen?

Page 52: JS Days Mobile Meow

Yes (sort of)

Page 53: JS Days Mobile Meow

Mobile Fullscreen

● Open New Tab● webkitEnterFullscreen

■ Webkit only■ Video element only■ metadata must be loaded

Page 54: JS Days Mobile Meow

Mobile Fullscreen

● Open New Tab● webkitEnterFullscreen

■ Webkit only■ Video element only■ metadata must be loaded

● Pro Tip: Remember size context changes so use viewport to scale icons and controls

Page 55: JS Days Mobile Meow

@viewport { width: device-width; }

Page 56: JS Days Mobile Meow

autoplay

<video autoplay>

Page 57: JS Days Mobile Meow
Page 58: JS Days Mobile Meow

autoplay

In Safari on iOS (for all devices, including iPad), where the user may be on a cellular network and be charged per data unit, preload and autoplay are disabled. No data is loaded until the user initiates it.

Page 59: JS Days Mobile Meow

What about everyone else?

Page 60: JS Days Mobile Meow

Autoplay

Chrome Safari Firefox Opera IE Android

Attribute

Scripted Buggy

Page 61: JS Days Mobile Meow

autoplay

Page 62: JS Days Mobile Meow

autoplay

<video onclick="this.play()">

Page 63: JS Days Mobile Meow
Page 64: JS Days Mobile Meow
Page 65: JS Days Mobile Meow

autoplayfunction someClickEvent(evt) { // In a user initiated thread. myVideoElement.load(); getVideoData(); // Triggers an ajax call.} function onGetVideoDataReturned(data) { // Not in a user initiated thread. setVideoElementSrc(data); myVideoElement.load(); myVideoElement.play();}

Page 66: JS Days Mobile Meow

autoplayfunction someClickEvent(evt) { // In a user initiated thread. myVideoElement.load(); getVideoData(); // Triggers an ajax call.} function onGetVideoDataReturned(data) { // Not in a user initiated thread. setVideoElementSrc(data); myVideoElement.load(); // For Android window.setTimeout(function() { myVideoElement.play(); }, 0);}

Page 68: JS Days Mobile Meow

Embeds

<script>

Page 69: JS Days Mobile Meow

Embeds

<script>○ We need our content to be sandboxed○ More than just a video tag

Page 70: JS Days Mobile Meow

Embeds

<script>○ We need our content to be sandboxed○ More than just a video tag

<object>

Page 71: JS Days Mobile Meow

Embeds

<script>○ We need our content to be sandboxed○ More than just a video tag

<object>○ Can load content with the data

attribute○ But no way to interact with it via

JavaScript

Page 72: JS Days Mobile Meow

Embeds

<iframe>

Page 73: JS Days Mobile Meow

Embeds

<iframe>○ Allows our content to be sandboxed○ JavaScript API communication

Page 74: JS Days Mobile Meow

<iframe type="text/html" width="640" height="385" frameborder="0" src="http://www.youtube.com/embed/VIDEO_ID" allowfullscreen></iframe>

Embeds

Page 75: JS Days Mobile Meow

Pro tip: Plan for the future (if you can) <iframe type="text/html" width="640" height="385" frameborder="0" src="http://www.youtube.com/embed/VIDEO_ID" allowfullscreen></iframe>

Embeds

Page 76: JS Days Mobile Meow

Embeds Pro Tips:

html { /** Hack to fix iPhone resizing. */ overflow: hidden;}

Page 77: JS Days Mobile Meow

Embeds Pro Tips:

body { /** Dymanic Resizing **/ background-color: #000; height: 100%; width: 100%; /** Remove highlight when use clicks **/ -webkit-tap-highlight-color: rgba(0, 0, 0, 0);}

Page 78: JS Days Mobile Meow

Testing

Page 79: JS Days Mobile Meow

● Device lab○ accurate○ costly○ space inefficient○ boring

Testing: The Old Fashioned Way

Page 80: JS Days Mobile Meow

Testing: With Software● Hardware Emulators / Simulators

○ available for major systems○ approximation varies○ still boring

● Android Emulator○ bulky, slow, no video codecs

● iOs Simulator○ both tablet and phone

Page 81: JS Days Mobile Meow

Testing: With Software● Browser Simulators

○ Poorer approximation● Opera Mobile Simulator

○ missing video tag support● Fennec (FF) Simulator

○ poor touch control mapping○ sends desktop user agent

Page 82: JS Days Mobile Meow

Testing: Automation● Selenium!

○ use with simulated or real devices○ not boring

Page 83: JS Days Mobile Meow

Testing: Automation● Android Webdriver

○ Still no video support ● iOS Webdriver

○ Need to register as ios dev○ Intermittent issues with playback

● IE○ No webdriver APIs for mobile

Page 84: JS Days Mobile Meow

Testing: Automation● Chrome, Opera

○ Driver APIs built-in○ Remote debugging

● FFMobile○ No webdriver APIs

Page 85: JS Days Mobile Meow

Testing: Automation Strategy● Test API methods in mobile context first

○ cheap○ can approximate video playback by video.

currentTime

Page 86: JS Days Mobile Meow

Testing: Automation Strategy● More sophisticated tests to follow

○ screenreader● Screen cap processing

○ check for distortion○ playback accuracy

Page 87: JS Days Mobile Meow

Testing: Automation Strategy● Screen cap processing

Page 88: JS Days Mobile Meow

Testing: Hardware Assistance

for more information check out http://bitbeam.org/

Page 90: JS Days Mobile Meow
Page 91: JS Days Mobile Meow

Or

Page 92: JS Days Mobile Meow

PointerEventshttp://msdn.microsoft.com/en-us/library/ie/hh673557(v=vs.85).aspx

Page 93: JS Days Mobile Meow

http://imgs.xkcd.com/comics/in_ur_reality.png

Page 94: JS Days Mobile Meow

Track

<video> <track src="cats_meow" kind="subtitles"></video>

Page 97: JS Days Mobile Meow

Questions?

By cloudzilla http://www.flickr.com/photos/cloudzilla/378829651/

[email protected]

can haz question?