spatial conferencing

© Fraunhofer IIS 1 Spatial Conferencing

Redmond, 16 Oct 2014

Spatial Conferencing

HP Baumeister


Telepresence Rooms: High End Conferencing


General Reaction: “As if speaking to someone in the same room”


General Reaction: “As if speaking to someone in the same room”

Why?

In part, because the audio is “transparent”, and comes from the appropriate

direction!

“Spatial Audio”


“Spatial Audio”

• audio: 3 channels of Fraunhofer AAC-LD @ 48kHz

• video: 3 x H.264 @ 1080p


Telepresence for Everyone: Spatial Conferencing

Bring the “telepresence experience” to smartphones

Using regular headsets/earphones


Telepresence for Everyone: Spatial Conferencing

“Billions of smartphone and tablets”


Spatial Conferencing: Key Elements

Highest Quality Audio: “Full-HD Voice”

“Spatialization”


Spatial Conferencing: “Full-HD Voice”

14 kHz+ audio bandwidth


Full-HD Voice Audio Quality

source: http://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/TSGS4_52/Docs/S4-090080.zip (Feasibility study on EVS audio bandwidth, Ericsson 3GPP/SA4 01/2009)

0 1 2 3 4 5

Mean Opinion Score

Fair Good Excellent Bad Poor

NB

WB

SWB


Spatial Conferencing “Full-HD Voice”

14 kHz+ audio bandwidth

Very low delay

Multi-channel capability


Spatial Conferencing “Full-HD Voice”

ELD = Enhanced Low Delay

AAC-ELD is a low latency mode of the widely adopted music codec AAC.

CD-like audio quality for telephone calls

full audio bandwidth up to 20 kHz

low coding delay down to 15 ms, crucial for natural conversations

optimized for a bit-rate range of 24 kbit/s to 64 kbit/s per channel

AAC-ELDv2 enhances bit-rate range down to 24 kbit/s stereo

Most widely adopted codec offering Full-HD Voice today

FaceTime

Native in iOS

Native in Android


Spatial Conferencing for Everyone “Spatialization”

Geometric Localization of Participants

With or without Video……


Spatial Conferencing “Head Related Transfer Function”


Spatial Conferencing “Head Related Transfer Function”

Credit: University of Maryland


Spatial Conferencing “Out of Head Experience”

Stereo = “Inside Head”

Spatial Rendering = “outside Head”


Audio Conferencing

MCU


Audio Conferencing “single channel” – “everyone is in the same location”

MCU


Spatial Conferencing “Spatialization” – “participants can be localized as if in the room”

MCU

positioning &

out of head

Full-HD Voice (spatialized)


Spatial Conferencing MCU


3-Point conference via IP

Full Band Audio, 44.1 kHz

Immersive, spatial experience

Spatial Conferencing Demo Setup

3x Nexus 5 - Android 4.4.3.2.1 - VoIP App using native ELD

standard in-ear headset

Mac-Mini running MCU with binaural processing

64 kbps mono

128 kbps stereo


Developed and evaluated in collaboration with

Spatial Conferencing Trials

Full-HD VoicePSTN

“I found it really amazing, because one really

thought you are sitting in the same room”


Member Companies should consider spatial conferencing in future systems

Define uses cases

Interface Specification / Standardisation

Interoperability

Spatial Conferencing IMTC Opportunities


See me: HP Baumeister

Call me: 408 573 9903

Mail Me [email protected]

Thank you! Questions? Live Demo?

mailto:[email protected]

mailto:[email protected]


Spatial Conferencing “Spatialization”

Full-HD Voice (mono)

MCU

positioning &

out of head

Full-HD Voice (spatialized)

spatial conferencing

Technology

spatial conferencing

spatial audio fraunhofer

room fraunhofer iis

android fraunhofer iis

video fraunhofer iis

tablets fraunhofer iis

p fraunhofer iis

outside head fraunhofer