mobile meow at mobilism

94
Mobile Meow By YouTube's Greg Schechter and Eugene Goldin Bring YouTube to the Mobile Web

Upload: greg-schechter

Post on 17-May-2015

2.862 views

Category:

Technology


0 download

DESCRIPTION

Greg Schechter & Eugene Goldin — Mobile Meow: Bringing YouTube Videos to a Mobile World This is the story of the glory and struggle of bringing a high quality YouTube experience to the mobile web. Once upon a time there was a web developer who wanted to play videos on the web. So he filmed a cat and wrote a Flash and a HTML5 player. He spent many hours making it work on his favorite desktop browsers and even the one his grandfather still used. People could watch his cat video, he smiled, and the world was good. Then one day someone put a browser in a phone and soon there were many phones with many different browsers. This new set of environments were even harder to develop for and had a slew of new terrifying bugs. The web developer was miserable knowing people couldn’t watch his cat video. With much time and effort he figured out many of the secrets needed to combat the evils of the different mobile platforms. Once again people could watch his cat vid eo, he smiled, and the world was good. The end. Greg is a fearless web warrior, fighting for browser and website progress. While training at the University of Illinois in Urbana-Champaign, he published articles with the Opera Web Standards Curriculum. Subsequently, he went on to battle alongside many different web companies, including Amazon, Yahoo, and Google. His current alliance is with YouTube, where he spearheads the movement for HTML5 video capabilities. Eugene Goldin is a javascript developer at YouTube. His interests include taking web videos where no video could go before, improving how users interact with web technologies, and long walks on the beach.

TRANSCRIPT

Page 1: Mobile Meow at Mobilism

Mobile Meow

By YouTube's Greg Schechter and Eugene Goldin

Bring YouTube to the Mobile Web

Page 2: Mobile Meow at Mobilism

Greg SchechterThe Web Warrior

[email protected]

Eugene GoldinCode Crusader

[email protected]

Page 3: Mobile Meow at Mobilism

Video

Page 4: Mobile Meow at Mobilism

Video

Page 5: Mobile Meow at Mobilism
Page 6: Mobile Meow at Mobilism
Page 8: Mobile Meow at Mobilism

Video

Page 9: Mobile Meow at Mobilism

Flash

Page 10: Mobile Meow at Mobilism

<video>

Page 11: Mobile Meow at Mobilism

Video

Page 12: Mobile Meow at Mobilism

Playback Stats

Page 13: Mobile Meow at Mobilism

Flash Native <video>

Page 14: Mobile Meow at Mobilism

Flash Flash==Flash

Page 15: Mobile Meow at Mobilism

Flash Flash

Page 16: Mobile Meow at Mobilism

Native

Page 17: Mobile Meow at Mobilism

○ For fallbacks, use rtsp:// protocol (serving .3pg) if the device won't support HTML5

○ Use custom protocol / URL scheme to launch your own Android, iOS native app (no Windows Phone yet)

■ youtube://video_id

Native

Page 18: Mobile Meow at Mobilism

<video>

Page 19: Mobile Meow at Mobilism

The Different Browsers

Safari Android

Page 20: Mobile Meow at Mobilism

Safari Android Chrome Opera Mini Silk

IE

The Different Browsers

Firefox Opera

Page 21: Mobile Meow at Mobilism

Safari Android

Firefox

Chrome

Opera

Opera Mini Silk

IE

The Different Browsers

SAMPLE

Page 22: Mobile Meow at Mobilism

By brownpau at http://www.flickr.com/photos/brownpau/533267369/

Page 23: Mobile Meow at Mobilism

<video>

How do we start

Page 24: Mobile Meow at Mobilism

<video src="funny_cat_video">

The Source

Page 25: Mobile Meow at Mobilism

<video src="funny_cat_video.webm/mp4/ogg">

The Source

Page 26: Mobile Meow at Mobilism

Mobile Formats Support

Chrome Safari Firefox Opera IE Android

H.264

WebM Android 4+ Android 4+ Android 4+

HLS Android 3+

Platform Versions and Distribution

Page 27: Mobile Meow at Mobilism

The Source

<video> <source src="funny_cat_video.mp4"> <source src="funny_cat_video.webm"></video>

Page 28: Mobile Meow at Mobilism

Power

Page 29: Mobile Meow at Mobilism

Power Consumption: H264 vs WebM

**fullscreen flash**windowed flash

Research done by Yossi Oren For more information visit http://iss.oy.ne.ro/

Page 30: Mobile Meow at Mobilism

Stick it in a page

Page 31: Mobile Meow at Mobilism

Basic video tag

Safari○ Renders a thumbnail poster and play button (flaky)○ Background is set to black and cannot be overwritten○ If a parent element has display:none set at any point the video fails to

playback properly

Opera○ Gives video dimentions black background if the information is

available○ Has issues with stretching WebM

IE and Android○ Play button/film icon for all video tags regardless if browser can play

the format○ Background is set to black/grey and cannot be overwritten

Page 32: Mobile Meow at Mobilism

Give it a poster and click it

Page 33: Mobile Meow at Mobilism

Poster Attribute

Use the poster attribute to get a thumbnail Safari, Chrome, Android, and Opera

○ Preserve aspect ratio of poster image

IE and Firefox○ Stretch image to size of video tag

IE○ Keeps poster as paused state

All Others○ Replace video with the paused frame

Page 34: Mobile Meow at Mobilism

Chrome○ Default click to toggle play/pause○ Remove with an empty onclick handler

Safari○ Will launch any supported video with or without an onclick handler

Android and IE○ Shows click interactions but does not play with empty onclick

Basic video tag

Page 35: Mobile Meow at Mobilism

Pro tips:○ Create your own cued state to convey you can play the video○ Set the background color to black for continuity across platforms

Basic video tag

Page 36: Mobile Meow at Mobilism

Controls

<video controls>

Page 37: Mobile Meow at Mobilism

Controls Chrome Safari Firefox Opera IE Android

Play/pause Button

Volume Controls

Only in Fullscreen

Mute toggle only

Don't work in Android 4

Seek bar

Fullscreen button

Button Gesture Button

Playback Type

Both Fullscreen only

Inline only Determined by hardware

Fullscreen only

Both

Page 38: Mobile Meow at Mobilism

Custom Controls

● Preserve your brand● Unified experience across platforms and

browsers

Page 39: Mobile Meow at Mobilism

Custom Controls

Lets get some custom controls

Page 40: Mobile Meow at Mobilism

Custom Controls

● Allows us to expand the set of controls and add our own○ annotations○ playlist○ captions○ more

Page 41: Mobile Meow at Mobilism

Custom Controls Pro Tips● User expect to be able to drag the progress

bar○ Need to remender to prevent scroll on touchstart

● Volume can't be set everywhere and users are accustomed to using device controls○ So don't build controls for it

Page 42: Mobile Meow at Mobilism

Custom Controls Pro Tips● Fingers are fat

○ Average finger is 11mm so make targets at least 40px with 10px padding

○ Use SVG's so icons can be scaled and shared with desktop application

○ Do what you can in css

Page 43: Mobile Meow at Mobilism

Custom Controls Pro Tips● Don't trigger content with hover● Fullscreen

○ The browsing context is always fullscreen so fake it

Page 44: Mobile Meow at Mobilism

Fullscreen

Page 45: Mobile Meow at Mobilism

requestFullScreen

Page 46: Mobile Meow at Mobilism

requestFullScreen

● Desktop Only○ Firefox, Safari, and Chrome○ Vendor Prefixed

Page 47: Mobile Meow at Mobilism

What about Mobile?

Page 48: Mobile Meow at Mobilism

Isn't it already fullscreen?

Page 49: Mobile Meow at Mobilism

Yes (sort of)

Page 50: Mobile Meow at Mobilism

Mobile Fullscreen

● Open New Tab● webkitEnterFullscreen

■ Webkit only■ Video element only■ metadata must be loaded

Page 51: Mobile Meow at Mobilism

Mobile Fullscreen

● Open New Tab● webkitEnterFullscreen

■ Webkit only■ Video element only■ metadata must be loaded

● Pro Tip: Remember size context changes so use viewport to scale icons and controls

Page 52: Mobile Meow at Mobilism

@viewport { width: device-width; }

Page 53: Mobile Meow at Mobilism

autoplay

<video autoplay>

Page 54: Mobile Meow at Mobilism
Page 55: Mobile Meow at Mobilism

autoplay

In Safari on iOS (for all devices, including iPad), where the user may be on a cellular network and be charged per data unit, preload and autoplay are disabled. No data is loaded until the user initiates it.

Page 56: Mobile Meow at Mobilism

What about everyone else?

Page 57: Mobile Meow at Mobilism

Autoplay

Chrome Safari Firefox Opera IE Android

Attribute

Scripted Buggy

Page 58: Mobile Meow at Mobilism

autoplay

Page 59: Mobile Meow at Mobilism

autoplay

<video onclick="this.play()">

Page 60: Mobile Meow at Mobilism
Page 61: Mobile Meow at Mobilism
Page 62: Mobile Meow at Mobilism

autoplayfunction someClickEvent(evt) { // In a user initiated thread. myVideoElement.load(); getVideoData(); // Triggers an ajax call.} function onGetVideoDataReturned(data) { // Not in a user initiated thread. setVideoElementSrc(data); myVideoElement.load(); myVideoElement.play();}

Page 63: Mobile Meow at Mobilism

autoplayfunction someClickEvent(evt) { // In a user initiated thread. myVideoElement.load(); getVideoData(); // Triggers an ajax call.} function onGetVideoDataReturned(data) { // Not in a user initiated thread. setVideoElementSrc(data); myVideoElement.load(); // For Android window.setTimeout(function() { myVideoElement.play(); }, 0);}

Page 65: Mobile Meow at Mobilism

Embeds

<script>

Page 66: Mobile Meow at Mobilism

Embeds

<script>○ We need our content to be sandboxed○ More than just a video tag

Page 67: Mobile Meow at Mobilism

Embeds

<script>○ We need our content to be sandboxed○ More than just a video tag

<object>

Page 68: Mobile Meow at Mobilism

Embeds

<script>○ We need our content to be sandboxed○ More than just a video tag

<object>○ Can load content with the data

attribute○ But no way to interact with it via

JavaScript

Page 69: Mobile Meow at Mobilism

Embeds

<iframe>

Page 70: Mobile Meow at Mobilism

Embeds

<iframe>○ Allows our content to be sandboxed○ JavaScript API communication

Page 71: Mobile Meow at Mobilism

<iframe type="text/html" width="640" height="385" frameborder="0" src="http://www.youtube.com/embed/VIDEO_ID" allowfullscreen></iframe>

Embeds

Page 72: Mobile Meow at Mobilism

Pro tip: Plan for the future (if you can) <iframe type="text/html" width="640" height="385" frameborder="0" src="http://www.youtube.com/embed/VIDEO_ID" allowfullscreen></iframe>

Embeds

Page 73: Mobile Meow at Mobilism

Embeds Pro Tips:

html { /** Hack to fix iPhone resizing. */ overflow: hidden;}

Page 74: Mobile Meow at Mobilism

Embeds Pro Tips:

body { /** Dymanic Resizing **/ background-color: #000; height: 100%; width: 100%; /** Remove highlight when use clicks **/ -webkit-tap-highlight-color: rgba(0, 0, 0, 0);}

Page 75: Mobile Meow at Mobilism

Testing

Page 76: Mobile Meow at Mobilism

● Device lab○ accurate○ costly○ space inefficient○ boring

Testing: The Old Fashioned Way

Page 77: Mobile Meow at Mobilism

Testing: With Software● Hardware Emulators / Simulators

○ available for major systems○ approximation varies○ still boring

● Android Emulator○ bulky, slow, no video codecs

● iOs Simulator○ both tablet and phone

Page 78: Mobile Meow at Mobilism

Testing: With Software● Browser Simulators

○ Poorer approximation● Opera Mobile Simulator

○ missing video tag support● Fennec (FF) Simulator

○ poor touch control mapping○ sends desktop user agent

Page 79: Mobile Meow at Mobilism

Testing: Automation● Selenium!

○ use with simulated or real devices○ not boring

Page 80: Mobile Meow at Mobilism

Testing: Automation● Android Webdriver

○ Still no video support ● iOS Webdriver

○ Need to register as ios dev○ Intermittent issues with playback

● IE○ No webdriver APIs for mobile

Page 81: Mobile Meow at Mobilism

Testing: Automation● Chrome, Opera

○ Driver APIs built-in○ Remote debugging

● FFMobile○ No webdriver APIs

Page 82: Mobile Meow at Mobilism

Testing: Automation Strategy● Test API methods in mobile context first

○ cheap○ can approximate video playback by video.

currentTime

Page 83: Mobile Meow at Mobilism

Testing: Automation Strategy● More sophisticated tests to follow

○ screenreader● Screen cap processing

○ check for distortion○ playback accuracy

Page 84: Mobile Meow at Mobilism

Testing: Automation Strategy● Screen cap processing

Page 85: Mobile Meow at Mobilism

Testing: Hardware Assistance

for more information check out http://bitbeam.org/

Page 87: Mobile Meow at Mobilism
Page 88: Mobile Meow at Mobilism

Or

Page 89: Mobile Meow at Mobilism

PointerEventshttp://msdn.microsoft.com/en-us/library/ie/hh673557(v=vs.85).aspx

Page 90: Mobile Meow at Mobilism

http://imgs.xkcd.com/comics/in_ur_reality.png

Page 91: Mobile Meow at Mobilism

Track

<video> <track src="cats_meow" kind="subtitles"></video>

Page 94: Mobile Meow at Mobilism

Questions?

By cloudzilla http://www.flickr.com/photos/cloudzilla/378829651/

[email protected]@google.com

can haz question?