openwebrtc – not just for video chat
TRANSCRIPT
OpenWebRTC – Not just for Video Chat
Kristofer Dovstam
Master Researcher
Multimedia Applications and Protocols, Ericsson Research
ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 2
› Mobile first WebRTC client framework
› Permissive Open Source BSD 2-
clause license
› Independent, interoperable
implementation
› Being integrated into WebKit
› Native applications
› GitHub
What isOpenWebRTC?
ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 3
› What is it?
– Not just audio and video, but data too
– Can control reliability and ordering properties depending on needs
– Encrypted real-time data
› OpenWebRTC now with Data Channel support!
– Using the reference SCTP implementation - libusrsctp
– Contributed by Collabora Ltd.
– Has been tested in real use cases…
WebRTC Data Channel
ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 4
› Remote Control over Mobile Networks
– Remote control an excavator from a long
way away
– Use WebRTC technology
– Optimize solution for high-performance and
low-latency
› Demo at MWC 2014
– ‘Toy’ excavator
– Earlier internal version of OpenWebRTC
– iPhone
– Oculus Rift DK1
IDEA and History
ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 5
…2015
ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 6
Excavator
› Research collaboration
with Volvo CE
› Not a product!
ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 7
Excavator
› Excavator
– Excavators from Volvo CE
– Modified to handle electrical control
signals
– Powerful Linux PC
– Python application with custom
signaling built on top of
OpenWebRTC
ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 8
Audio / VideoCapture
› 2048x2048 24fps
camera
› 360 degree lens
› 52.5 degrees up, 62.5
degrees down
› Avoids stitching artifacts
› Microphone array of four
microphones
ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 9
Control Rig› Simulator from Oryx and
Volvo CE
› Mac OS X computer
› OS X Cocoa application
with custom signaling
built on top of
OpenWebRTC
ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 10
› Oculus Rift DK2 (1920x1080 75Hz –
960x1080 for each eye)
– Still much too low resolution!
› Doughnut to panorama texture
mapped on a sphere
› Local rendering minimizes motion-to-
light latency of head movements
› Four channel audio rendered as four
independent sound sources in a 3D
space
Audio / VideoRendering
ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 11
Controls
› Simulator rig with analogue controls
› Computer measures analogue signals
and converts to digital measurements
› Control signals sent via WebRTC Data
Channel to excavator
› Excavator computer converts digital
signals into analogue electrical signals
› It’s alive!
ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 12
Media Processing Architecture
Camera
Microphone Array
Excavator
OpenWebRTC
Audio / Video
Data Channel
OpenWebRTC
Audio / Video
Data Channel
Oculus Rift DK2
Headphones
Control Rig
Network
Mobile, fixed, …
Network
Mobile, fixed, …
Audio
Video
Data
Audio
Video
Data
Audio
Video
Data
Audio
Video
Data
ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 13
› Data Channel
– 1 stream
– Unordered
– Unreliable (full control signal
state in every message so
only the newest message is
relevant)
– libusrsctp
› Video
– 2048 x 2048
– ~24fps
– H.264
– OpenH264
– 10Mbps
› Audio
– Single 4-channel stream
– Signed 16-bit
– 48kHz
– Opus
– libopus
– 256 kbps total bitrate
Media Stream Specifications
ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 14
Mobile world congress - 2015
ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 15