Integrating Mobile Applications - Contrasting the Browser with Native OS Apps

Cary FitzGerald

caryfitz@employees.org

Workshop Leaders

• Peter Dunkley

– Acision

• Adam van den Hoven

– Frozen Mountain

Integrating Mobile Applications

Contrasting the Browser with Native OS Apps

Browser vs. Native?

Browser vs. Native

• The Web is more than Browsers

– AJAX & REST transcends browsers

– JSON is the standard data exchange format

• Real Time communication is more than Chat

• The challenge and opportunity is to find unique ways to leverage WebRTC

6

Native Apps

• Compiled, Packaged app

• True Native or Hybrid

• Deployed separate from use

• Not just Phones and Tablets– Augmented / Virtual Reality

– Embedded Systems

– Desktop Apps

– Servers

7

Browsers

PROS

• Leverage existing expertise

• Develop once, use everywhere (in theory)

• Simple API

• Simple deployment

CONS

• Design Complexity

• Limited API

• iOS not supported

• Windows Phone not supported

• IE not supported

8

Native Aps

Pros

• Background operation

• Optimized UI and I18N

• Preapproved access to resources

• Easy access to device features

Cons

• Many languages to support

• Little consistency

• More complex deployment

• Mostly DIY*

9

When to use a Browser

• Secondary Feature of Larger App

• Very Simple App

• Minimally Viable Product

10

Primary Interaction is Real Time

• Communication while in the background

• Offline notifications

• Switch Camera (front/back)

11

Headless Peers

• Recording peer

• Embedded Systems

12

Secure Communications

• Guarantee the behaviour of peers

13

Augmented Reality

• Great opportunity for unique interactions

• Expert assistance overlay

• Live Games

14

Non-camera and multiple sources

• Telemedicine

• Education

15

Managing multiple platforms

• Just bite the bullet

• Translation

• Hybrid

• Cross-compile with Xamarin

16

Demo

17

Integrating Mobile Applications – Contrasting the Browser with Native OS Apps

Peter Dunkley

Acision

Acision at-a-glance

Heritage & history

20 year history

270 customers globally

2 trillion messages/ year

Global presence & reach

35+ offices across 6

continents

850+ employees

100+ support team

1st to launch commercial

SMSC

Cloud & virtualized

services

Multi-Channel messaging

+ WebRTC

Innovation

Data and insights

Carrier grade security

Operational Excellence

24/7 Support

THE STATE OF WEBRTC ON MOBILE TODAY

Android

• Supported in Google Chrome, Mozilla Firefox, and Ericsson Bowser* for Android

• Mobile browser support trails desktop browser support

• WebRTC in WebViewssupported in Android Lollypop

iOS

• Supported in Ericsson Bowser*for iOS

WebRTC in browsers on mobile devices

* Ericsson Bowser is not yet (and might never be) a platform for consumers to use

Blackberry 10

• Natively supported within the OS

– Google Chrome rendering engine in built into BB10

Windows Mobile

• No support today

WebRTC in browsers on mobile devices

WebRTC in browsers on mobile devices

• Support for WebRTC in mobile browsers is inconsistent across the various platforms

• Even on Android it lags behind what is available in desktop browsers

• Other features needed by many apps (for example, mobile push notifications) still require native apps

For the rest of this presentation I will only consider Android and iOS

Google Media Engine (GME)

• Liberal open-source license

• Used within Google Chrome and parts used within Mozilla Firefox

• G.711 and Opus audio

• VP8 video

• DataChannel support

• Widely used today

OpenWebRTC (from Ericsson)

• Liberal open-source license (BSD)

• Used within Ericsson Bowser

• G.711 and Opus audio

• H.264* and VP8 video

• No DataChannel support (yet)

• Not widely used today

WebRTC libraries for mobile devices

* Does not confer a license to use H.264

WebRTC libraries for mobile devices

• The GME is the best known and widest used library today

– The only option if you want to use DataChannels right now

• OpenWebRTC is worth watching

– The fact that it contains an H.264 implementation doesn’t mean developers can just use it

For the rest of this presentation I will only consider the GME

WHAT CAN I DO WITH WEBRTC IN A MOBILE BROWSER TODAY?

Android

• Click-2-call style apps can be used in Google Chrome on Android– You can’t receive sessions when the browser is closed

– You can’t write WebRTC apps that require mobile push notifications

• Some hybrid apps (combination of native code and WebViews using WebRTC) are possible on Android Lollypop

iOS

• Nothing today

THE GOOGLE MEDIA ENGINE

WebRTC Architecture

RTCWeb

G.711/OPUS Codec

NetEQ for Voice

Echo Canceller /Noise Reduction

Voice Engine

H.264/VP8 Codec

Video Jitter Buffer

Image enhancements

Video Engine

SRTP

Multiplexing

P2PSTUN + TURN + ICE

Transport

Audio Capture/Render Video Capture Network I/O

Session management / Abstract signalling (Session)

WebRTC C++ API (PeerConnection)

WebRTC API

You

r we

bap

p #1

You

r we

bap

p #2

You

r we

bap

p #3. . . The web

Your browser

Based on the diagram from http://www.webrtc.org/reference/architecture

WebRTC C++ API

• The C++ API documentation at http://www.webrtc.org/is very basic

– There is a close mapping between the functions defined in the W3C WebRTC draft (for JavaScript) and the C++ API

– You can infer the C++ API behaviour by interpreting the W3C draft

• Getting the WebRTC C++ API working on a native mobile device is not easy

The media engine isn’t enough

• Even once you’ve gotten the media engine working you need more to establish your sessions– Discovering peers and exchanging media capabilities and

connection details needs to happen before the media session can be established

• You need to consider user management and provisioning too

There are many developer toolkits and SDK available to help you here (each with slightly different benefits) – take advantage of them

Consider using OAuth2 if you can

Web Service

OAuth2Provider

Authentication Code

Au

then

tica

tio

n C

od

e

Acc

ess

Toke

n

Req

ues

t Id

enti

ty Iden

tity

Identity

1

2

4

5

6

3

7

8

Consider using OAuth2 if you can

UA

SignallingServer

Web Service

5: Shared secret

Communication linknot required

You don’t have tocreate or manageaccounts on theSignalling Server

CROSS PLATFORM CODE EXAMPLE

Based on the Acision Forge SDK

Android ExampleAcisionSdkConfiguration config =

new AcisionSdkConfiguration("MYAPIKEY ", "user", "password");

config.setApplicationActivity(this);

AcisionSdk acisionSdk = new AcisionSdk(config, new AcisionSdkCallbacks() {

@Override

public void onConnected(AcisionSdk acisionSdk) {

session = acisionSdk.getWebrtc().connectTo("+441632960000",

new WebrtcSessionCallbacks() {

@Override

public void onConnected() {

session.setLocalView(localVideoView);

session.setRemoteView(remoteVideoView);

}

}, new WebrtcOptions());

}

});

iOS ExampleACSAcisionSDKConfiguration *config =

[ACSAcisionSDKConfiguration acisionSDKConfigurationWithAPIKey:@"MYAPIKEY”

username:@"user" password:@"password"];

self.acisionSDK = [ACSAcisionSDK acisionSDKWithConfig:config delegate:self];

- (void)onConnected:(ACSAcisionSDK *)acisionSDK {

ACSAcisionSDKWebRTCOptions* options =

[[ACSAcisionSDKWebRTCOptions alloc] init];

self.currentSession =

[self.acisionSDK.webrtc connectTo: @"+441632960000”,

delegate:self options:options];

}

-(void)onConnected {

self.currentSession.localView = self.localView;

self.currentSession.remoteView = self.remoteView;

}

JavaScript Example

var lib = new AcisionSDK("MYAPIKEY", {

onConnected: function() {

var session = lib.webrtc.connect("+441632960000");

session.remoteVideoElement = document.getElementById("vidRemote");

session.remoteAudioElement = document.getElementById("audRemote");

}

}, {

username: "user",

password: "password"

}

);

IOS DEEP DIVE

Based on the Acision Forge SDK

Noughts and Crosses