webrtc live q&a and screen capture session 3

WebRTC Standards & Implementation Q&A

Amir Zmora TheNewDialTone

Dan Burne3 StandardsPlay

Alex Gouaillard

WebRTC by Dr Alex / Citrix

Session sponsored by

WebRTC.ventures is a custom design and development shop dedicated to building WebRTC based applicaFons for web and mobile. We have built end-‐to-‐end broadcast soluFons for events and entertainment clients, telehealth soluFons for mulFple clients, live support tools, as well as communicaFon tools for a variety of other applicaFons. WebRTC.ventures is a recognized development partner of TokBox and has also built naFve WebRTC soluFons

We use CrowdCast….It’s WebRTC

WebRTCStandards.info

About Us •  Amir Zmora •  Dan Burnett •  Alex Gouaillard

Screen Capture & Screen Sharing with WebRTC

Screen Sharing in WebRTC •  Is WebRTC plus Screen Capture

•  Screen capture gives you MediaStreamTrack •  WebRTC lets you send it

•  We will talk about the Screen Capture piece

Security in native apps •  If you install it, the app has complete access to your device •  So, choosing not to install is the first level of security

Security in the Web model •  Visiting a site is the "install" •  But visiting a site needs to be safe •  So, the Web uses site origin as security

•  By default, limited access to the device browser runs on •  Also, page has access to JS it loads but no access to JS from other tabs/windows

Problem - API keys in stupid sites

Screen capture breaks web model •  Browser controls allow Site A to do a user View-Source on Site B •  Normally, user can see B's popped up source but A can't read •  But with screen capture, A can read

Nasty scenario •  Site A uses WebRTC with user permission to access camera, screen •  Site A scrapes screen image to see what other tabs/windows user has open in browser •  Site A tracks user's eyes with camera •  When user looks away, Site A does view-source on a tab, scrapes the screen, closes

view-source window

WebRTC Screen Capture standard

•  http://w3c.github.io/mediacapture-screen-share/

•  Still very new

navigator.mediaDevices.getDisplayMedia({ video: true }) .then(stream => { // we have a stream, attach it to a feedback video element videoElement.srcObject = stream; }, error => { console.log("Unable to acquire screen capture", error); });

Protections in the standard •  By default no viewing of other tabs or other browser windows, even in other browser apps

(e.g., Chrome app can't see FF browser) •  Requirement for explicit, elevated permissions in order to view these since one app could

control what is presented on the others •  In practice,

•  Permissions will probably be a form of whitelist similar to what FF uses today •  Likely no way for WebRTC apps to get exemptions in advance

Screen Sharing with Chrome

Chrome Specific WebRTC Bits

Media Stream Manager (singleton@browser) GUM JS API

(tab/sandbox)

1. Send request



(tab/sandbox)

2. Check if MST is already available



(tab/sandbox)

Security Manager (source, origin)

3. Check rights

2. Check if MST is already available -‐ NO


Media Stream Manager (singleton@browser)

Audio Capturer

Video Capturer


GUM JS API (tab/sandbox)

4. Ask Corresponding capturer type to start capturing

3. Check rights -‐ OK



Audio Capturer

Video Capturer


A


4. Ask Corresponding capturer type to create one -‐ OK

V

5. Store the MST



Audio Capturer

Video Capturer


A


V

6. Trigger callback

Keep feeding frames



Audio Capturer

Video Capturer


A


V

NOTE 1: second call for same device with same constraints will directly return the MST, that allows to share streams across tabs without blocking



Audio Capturer

Video Capturer


A


V

NOTE 2: Recently, a second call for the same device but with different constraints (think simulcast) will indeed return a different resoluFon. Before it would return the first resoluFon asked.



Audio Capturer

Video Capturer


A


V

NOTE 3: Not only this allow to share cams across processes, it allows for global echo cancellaFon (yes, including the key strokes). Before tabs could cross feed.

Chrome Screensharing 2 steps (1)


Screen/Windows/Tab Capturer


Screensharing (extension)

1 2

3 4

S 5

Chrome Screensharing 2 steps (2)


Screen/Windows/Tab Capturer

S


With ID

Screen Sharing with Firefox

Firefox •  Whitelisting (wiki.mozilla.org/Screensharing)

•  Manual •  Hardcoded •  Extension

Firefox •  Whitelisting - Manual

•  Manual •  Hardcoded •  Extension

Firefox •  Whitelisting (wiki.mozilla.org/Screensharing)

•  Hardcoded ⇒ open a bug! ⇒ Attack surface?

Firefox

•  webex.com,*.webex.com,ciscospark.com,*.ciscospark.com,projectsquared.com,*.projectsquared.com,

•  *.room.co,room.co,

•  beta.talky.io,talky.io,

•  *.clearslide.com,

•  appear.in,*.appear.in,

•  tokbox.com,*.tokbox.com, *.opentok.com,

•  *.sso.francetelecom.fr,*.si.francetelecom.fr,*.sso.infra.ftgroup,*.multimedia-conference.orange-business.com,*.espacecollaboration.orange-business.com,

•  example.com,

•  *.mypurecloud.com,*.mypurecloud.com.au,

•  spreed.me,*.spreed.me,*.spreed.com,

•  air.mozilla.org,

•  *.circuit.com,*.yourcircuit.com,circuit.siemens.com,yourcircuit.siemens.com,circuitsandbox.net,*.unify.com,tandi.circuitsandbox.net,

•  *.ericsson.net,*.cct.ericsson.net,

•  *.conf.meetecho.com,

•  meet.jit.si,*.meet.jit.si,

•  web.stage.speakeasyapp.net,web.speakeasyapp.net,

•  *.hipchat.me,

•  *.beta-wspbx.com,*.wspbx.com,

•  *.unifiedcloudit.com,

•  *.smartboxuc.com,

•  *.smartbox-uc.com,

•  *.panterranetworks.com,

•  pexipdemo.com,

•  *.pexipdemo.com,pex.me,*.pex.me,*.rd.pexip.com,

•  1click.io,*.1click.io,

•  *.fuze.com,*.fuzemeeting.com,

•  *.thinkingphones.com,

•  free.gotomeeting.com,g2m.me,*.g2m.me,gotomeeting.com,*.gotomeeting.com,gotowebinar.com,*.gotowebinar.com,gototraining.com,*.gototraining.com,citrix.com,*.citrix.com,expertcity.com,*.expertcity.com,citrixonline.com,*.citrixonline.com,g2m.me,*.g2m.me,gotomeet.me,*.gotomeet.me,gotomeet.at,*.gotomeet.at

Both Firefox and Chrome •  FF-Whitelisting – Extension / addOn •  Cr – Extension See e.g. here: Blog - https://tokbox.com/developer/guides/screen-sharing/js/

Code - https://github.com/opentok/screensharing-extensions

Thank You

Amir Zmora TheNewDialTone

Dan Burne3 StandardsPlay

Alex Gouaillard WebRTC by Dr Alex

webrtc live q&a and screen capture session 3

Technology