beyond the mcu

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

D3-4 Media Services -Beyond the MCU

Chad Hart

WebRTC Expo V

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 2Image source: http://thewrestlinglifeupdate.tumblr.com/

Multi-PARTY! Video Conferencing

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 3

Easy & cheap approach to multi-party: Mesh

3

Full Mesh

Works for a few parties

No server cost

Lowest latency


Mesh does not scale for many video conferees

4

Full Mesh

Clients get overloadedEncode costs more than decode

Limited uplink bandwidth

Inconsistent performance across participants


Traditional Telephony Approach: MCU

5

Multipoint Control Unit (MCU)Good at manipulating all media

Transcoding

Transizing

Transrating

Interworking

Server-side CPU intensive

Client can request different conference mixes

Usually client friendly

Downsizing & mixing can reduce bandwidth

Fewer client streams to process

MCU


Traditional MCU resource usage

6

Worst-case process:1. Each stream decoded

at the MCU

2. MCU mixes each conferee

3. MCU encodes individual mix for each conferee

MCU

1

1 22

3

3

44

5

5

6

6

Ingress streams 6

Egress streams 6

MCU decodes 6

MCU encodes 6


Efficient MCU resource usage

7

Efficient process:1. Each stream decoded

at the MCU

2. MCU mixes each conferee

3. MCU shares same encoder for all conferees

MCU

12

3

45

6

Ingress streams 6

Egress streams 6

MCU decodes 6

MCU encodes 1

1


Resource efficient MCU with encoder sharing

30%-50% reduction in compute resources

0%

20%

40%

60%

80%

100%

1 2 3 4 5 6 7 8 9 10 11 15 17 20

Syst

em

CP

U U

tiliz

atio

n

Conferees

Encoder sharing vs. multi-encode

Traditional MCU Dialogic Evolved MCU


Newer approach: SFU

9

SFU

Selective Forwarding Unit (SFU) routing

Clients send one & receive many

Client can instruct SFU which streams to send

High throughput

Can be lots of downlink bandwidth

Low latency


Newer approach: SFU

10

SFU

Selective Forwarding Unit (SFU) routing

Clients send one & receive many

Client can instruct SFU which streams to send

High throughput

Can be lots of downlink bandwidth

Low latency


New WebRTC approach: Simulcast

11

SFU

High bitrate

Low bitrate

Selective Forwarding Unit (SFU) with Simulcast

Clients send multiple streams to SFU

one high-bit rate

one or more lower-bit

Client directs SFU which streams to receive

Reduces bandwidth vs. SFU

Simulcast in WebRTC coming


A look at Google Hangouts with Simulcast

http://webrtchacks.com/hangout-analysis-philipp-hancke/

http://webrtchacks.com/hangout-analysis-philipp-hancke/


Future Approach with VP9 – SVC?

13

SFU

MediumHigh LowLayered bitrates:

Selective Forwarding Unit (SFU) with SVC

Clients send layered stream to SFU

Layers have varying bitrates –time, size, quality

SFU directs who gets what

Coming to WebRTC eventually


Multi-party video summary comparison

http://networkfuel.dialogic.com/webrtc-whitepaper

http://networkfuel.dialogic.com/webrtc-whitepaper


Building a compelling UX with a MCU


Video UX: Traditional method

MCU1 2

3 4

Mixed stream 1 2

3 4

3G Device


Video UX: Traditional method

MCU1 2

3 4

Mixed stream 1 23

3G DeviceActive Talker

Media Server Chooses Layout


Client appApp Server2

Modern UX with MCU mixing, HTML5 & JavaScript

Browser

MCU1 2

3 4

HTML5 CanvasJavascript

Mixed stream

Application-defined UX

Layout

"Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum."

Benefits:

Application defined UX

User specific & customizable layout

Lower client BW & CPU vs. mesh


What do you care more about?


CPU & processing power is expensive

Client view Provider View


Bandwidth is not always ubiquitous or free

Client-side Server-side


Why do you need a media server?

Multi-party conferencing InterworkingTranscoding

Stream processing Person-to-machineRecording


Case Study: Video Recording Service

Application highlightsRecord live events – E.g. Kids soccer gameShare event recording with family/friends – Multi-party stream

Supported Recording Formats

.3GP

.mp4

.wav

CodecsG.711, Opus, AMR-NB and AMR-WB (voice)VP8, H.264 (video)

Message StorageLocal storageRemote message deposit via MSRP

HTTP/MSRP

REST (HTTP)

Application Server

Media Server

Message Storage

RTP (media)

Solution HighlightsSoftware-based Media Server

Virtualized (VM) and COTS deployment

High Availability and Geographical Redundancy

Scalability via Media Resource Broker

Architected to support 3.5M Subscriber Network

Recording

Remote Message Deposit

Key Functions

Transcoding

WebRTC Signaling

WAN


The future: gesture controls

https://github.com/mtschirs/js-objectdetect/


The future: machine vision


The future: augmented reality

http://jooink.blogspot.com/2012/10/gwt-augmented-reality-howto-step-1.html

beyond the mcu

Technology

mcu encodesindividual

resource efficient mcu

mcu mixes eachconferee3

mcu shares sameencoder

newer approach

client streams

egress streams

future approach