mobile meow at mobilism

Mobile Meow

By YouTube's Greg Schechter and Eugene Goldin

Bring YouTube to the Mobile Web

Greg SchechterThe Web Warrior

schechter@google.com

Eugene GoldinCode Crusader

eugenegoldin@google.com

Video

Video

A Brief History

Video

Flash

<video>

Video

Playback Stats

Flash Native <video>

Flash Flash==Flash

Flash Flash

Native

○ For fallbacks, use rtsp:// protocol (serving .3pg) if the device won't support HTML5

○ Use custom protocol / URL scheme to launch your own Android, iOS native app (no Windows Phone yet)

■ youtube://video_id

Native

<video>

The Different Browsers

Safari Android

Safari Android Chrome Opera Mini Silk

IE

The Different Browsers

Firefox Opera

Safari Android

Firefox

Chrome

Opera

Opera Mini Silk

IE

The Different Browsers

SAMPLE

By brownpau at http://www.flickr.com/photos/brownpau/533267369/

<video>

How do we start

<video src="funny_cat_video">

The Source

<video src="funny_cat_video.webm/mp4/ogg">

The Source

Mobile Formats Support

Chrome Safari Firefox Opera IE Android

H.264

WebM Android 4+ Android 4+ Android 4+

HLS Android 3+

Platform Versions and Distribution

The Source

<video> <source src="funny_cat_video.mp4"> <source src="funny_cat_video.webm"></video>

Power

Power Consumption: H264 vs WebM

**fullscreen flash**windowed flash

Research done by Yossi Oren For more information visit http://iss.oy.ne.ro/

Stick it in a page

Basic video tag

Safari○ Renders a thumbnail poster and play button (flaky)○ Background is set to black and cannot be overwritten○ If a parent element has display:none set at any point the video fails to

playback properly

Opera○ Gives video dimentions black background if the information is

available○ Has issues with stretching WebM

IE and Android○ Play button/film icon for all video tags regardless if browser can play

the format○ Background is set to black/grey and cannot be overwritten

Give it a poster and click it

Poster Attribute

Use the poster attribute to get a thumbnail Safari, Chrome, Android, and Opera

○ Preserve aspect ratio of poster image

IE and Firefox○ Stretch image to size of video tag

IE○ Keeps poster as paused state

All Others○ Replace video with the paused frame

Chrome○ Default click to toggle play/pause○ Remove with an empty onclick handler

Safari○ Will launch any supported video with or without an onclick handler

Android and IE○ Shows click interactions but does not play with empty onclick

Basic video tag

Pro tips:○ Create your own cued state to convey you can play the video○ Set the background color to black for continuity across platforms

Basic video tag

Controls

<video controls>

Controls Chrome Safari Firefox Opera IE Android

Play/pause Button

Volume Controls

Only in Fullscreen

Mute toggle only

Don't work in Android 4

Seek bar

Fullscreen button

Button Gesture Button

Playback Type

Both Fullscreen only

Inline only Determined by hardware

Fullscreen only

Both

Custom Controls

● Preserve your brand● Unified experience across platforms and

browsers

Custom Controls

Lets get some custom controls

Custom Controls

● Allows us to expand the set of controls and add our own○ annotations○ playlist○ captions○ more

Custom Controls Pro Tips● User expect to be able to drag the progress

bar○ Need to remender to prevent scroll on touchstart

● Volume can't be set everywhere and users are accustomed to using device controls○ So don't build controls for it

Custom Controls Pro Tips● Fingers are fat

○ Average finger is 11mm so make targets at least 40px with 10px padding

○ Use SVG's so icons can be scaled and shared with desktop application

○ Do what you can in css

Custom Controls Pro Tips● Don't trigger content with hover● Fullscreen

○ The browsing context is always fullscreen so fake it

Fullscreen

requestFullScreen

requestFullScreen

● Desktop Only○ Firefox, Safari, and Chrome○ Vendor Prefixed

What about Mobile?

Isn't it already fullscreen?

Yes (sort of)

Mobile Fullscreen

● Open New Tab● webkitEnterFullscreen

■ Webkit only■ Video element only■ metadata must be loaded

Mobile Fullscreen

● Open New Tab● webkitEnterFullscreen

■ Webkit only■ Video element only■ metadata must be loaded

● Pro Tip: Remember size context changes so use viewport to scale icons and controls

@viewport { width: device-width; }

autoplay

<video autoplay>

autoplay

In Safari on iOS (for all devices, including iPad), where the user may be on a cellular network and be charged per data unit, preload and autoplay are disabled. No data is loaded until the user initiates it.

What about everyone else?

Autoplay

Chrome Safari Firefox Opera IE Android

Attribute

Scripted Buggy

autoplay

autoplay

<video onclick="this.play()">

autoplayfunction someClickEvent(evt) { // In a user initiated thread. myVideoElement.load(); getVideoData(); // Triggers an ajax call.} function onGetVideoDataReturned(data) { // Not in a user initiated thread. setVideoElementSrc(data); myVideoElement.load(); myVideoElement.play();}

autoplayfunction someClickEvent(evt) { // In a user initiated thread. myVideoElement.load(); getVideoData(); // Triggers an ajax call.} function onGetVideoDataReturned(data) { // Not in a user initiated thread. setVideoElementSrc(data); myVideoElement.load(); // For Android window.setTimeout(function() { myVideoElement.play(); }, 0);}

Embeds

Embeds

<script>

Embeds

<script>○ We need our content to be sandboxed○ More than just a video tag

Embeds

<script>○ We need our content to be sandboxed○ More than just a video tag

<object>

Embeds

<script>○ We need our content to be sandboxed○ More than just a video tag

<object>○ Can load content with the data

attribute○ But no way to interact with it via

JavaScript

Embeds

<iframe>

Embeds

<iframe>○ Allows our content to be sandboxed○ JavaScript API communication

<iframe type="text/html" width="640" height="385" frameborder="0" src="http://www.youtube.com/embed/VIDEO_ID" allowfullscreen></iframe>

Embeds

Pro tip: Plan for the future (if you can) <iframe type="text/html" width="640" height="385" frameborder="0" src="http://www.youtube.com/embed/VIDEO_ID" allowfullscreen></iframe>

Embeds

Embeds Pro Tips:

html { /** Hack to fix iPhone resizing. */ overflow: hidden;}

Embeds Pro Tips:

body { /** Dymanic Resizing **/ background-color: #000; height: 100%; width: 100%; /** Remove highlight when use clicks **/ -webkit-tap-highlight-color: rgba(0, 0, 0, 0);}

Testing

● Device lab○ accurate○ costly○ space inefficient○ boring

Testing: The Old Fashioned Way

Testing: With Software● Hardware Emulators / Simulators

○ available for major systems○ approximation varies○ still boring

● Android Emulator○ bulky, slow, no video codecs

● iOs Simulator○ both tablet and phone

Testing: With Software● Browser Simulators

○ Poorer approximation● Opera Mobile Simulator

○ missing video tag support● Fennec (FF) Simulator

○ poor touch control mapping○ sends desktop user agent

Testing: Automation● Selenium!

○ use with simulated or real devices○ not boring

Testing: Automation● Android Webdriver

○ Still no video support ● iOS Webdriver

○ Need to register as ios dev○ Intermittent issues with playback

● IE○ No webdriver APIs for mobile

Testing: Automation● Chrome, Opera

○ Driver APIs built-in○ Remote debugging

● FFMobile○ No webdriver APIs

Testing: Automation Strategy● Test API methods in mobile context first

○ cheap○ can approximate video playback by video.

currentTime

Testing: Automation Strategy● More sophisticated tests to follow

○ screenreader● Screen cap processing

○ check for distortion○ playback accuracy

Testing: Automation Strategy● Screen cap processing

Testing: Hardware Assistance

for more information check out http://bitbeam.org/

The Future

Cats in Space by WF&TD

Or

PointerEventshttp://msdn.microsoft.com/en-us/library/ie/hh673557(v=vs.85).aspx

http://imgs.xkcd.com/comics/in_ur_reality.png

Track

<video> <track src="cats_meow" kind="subtitles"></video>

Camera Access with getUserMedia

More to learn● HTML5 Video Spec

○ http://dev.w3.org/html5/spec-author-view/video.html● Browser Blogs

○ http://dev.opera.com/○ http://hacks.mozilla.org/○ http://blogs.msdn.com/b/ie/○ http://peter.sh/

● Documentation○ https://developer.mozilla.org/en-US/

● Other○ http://developer.apple.

com/library/safari/#documentation/AudioVideo/Conceptual/Using_HTML5_Audio_Video/Device-SpecificConsiderations/Device-SpecificConsiderations.html

Questions?

By cloudzilla http://www.flickr.com/photos/cloudzilla/378829651/

schechter@google.comeugenegoldin@google.com

can haz question?