prins - euroitv 2013 - a hybrid architecture for delivery of panoramic video

A hybrid architecture for delivery of panoramic video

Martin Prins, Research Scientist

EuroITV 2013, June 26th


Martin Prins, Research Scientist, TNO

O.A. Niamut, R. Van Brandenburg (TNO)

J.F. Macq, N.Verzijp, P. R. Alface (Alcatel-Lucent)

EuroITV – June 2013




Outline

Some trends and backgrounds

The Fascinate Delivery Network

A hybrid approach

Putting it into practice

Conclusions




8K

4K

HD




Trends8K screening

@ Olympics London ’12

HDTV

Smart TV1st iPhone 1st iPad

4K TV3DTV




Produce format-agnostic audiovisual media and

decide on format at delivery or rendering time

Allow end-users to interactively view and navigate

around high resolution video panorama with the

accompanying audio automatically changing to match

the selected view

“Record everything, focus on one perspective”

How do we support both trends?




7K/2K

Watch on

mobileZoom in

Watch in

public




Football

Premier League Game

Chelsea FC – Wolverhampton Wanderers (London, October 2010)

Dance Performance

Carmen Suite

Berlin Philharmonic Orchestra conducted by Sir Simon Rattle &

Dance Compagnie Sasha Waltz (Berlin, May 2012)

Concert

BBC Proms (London, August 2012)

Three Demo Productions (so far)




Network

The FascinatE Consortium




Cope with huge bandwidth requirements

Support interactivity and provide low latency for high QoE

Content segmentation tiled video streaming

In-network rendering adapt to end-user device

Support pan/tilt/zoom navigation virtual cameraman

How do we support both trends in the network?




The FascinatE Delivery Network

Broadcast & IPTV

Media Internet




Functional components in the delivery network

A/V Proxy:

• interfaces with end-devices

• in-network rendering




A/V Ingest: the Tiling Approach

ROI streams




Layered

Scene

Layer

Omnicam

Layer

BCam

Scale

HD

Grid

2x2

Grid

4x4

Layer

…

Generic Segment

Time_Segment

Representation

Bitrate 1

Scale

…

Scale

7kx2k

Grid

…

Time_Segment

Time_Segment

Representation

Bitrate 2

Tile

T1

Tile

T2

Tile

T3

Tile

T4

A/V Ingest: Common Tiling Format




Unicast

Multicast

A/V Relay: Publisher/Subscriber

Rationale: seamlessly combine efficiency of

Multicast: push content useful for many devices

Unicast: pull content on-demand for single device

Fit with next-gen managed video delivery

Se

gm

en

ted

Con

ten

t

Lowres full panorama

Hires Tile 1

Hires Tile 2

A/V Relay




Segment

Client

Frame

Combiner

A/V Proxy

2nd Screen for control/display

HTTP server

A/V RelayHAS delivery of spatial segments

Event Signalling

CDN servers

A/V Relay: Tiled HTTP Adaptive Streaming

Extension of HAS/DASH with spatial segmentation for tiled streaming

Control plane signaling for manifest and events

AV relay implementation in live CDN

Spatial Manifest




HTTP Adaptive Streaming




TILED-HAS

MPEG-DASH

Zoom LayerZoom Layer structure

Media Presentation

Period

Period

1920x1080

Period Representation A Segment Info

Adaptation Set

Adaptation Set

Grid 2x2

Segment Info

Representation B

Segment Info

Media Segment

Media Segment

Grid

Layout=2x2

Tile X0Y0

Tile X0Y1

6984x1920

Media Presentation

LayerOMNICAM

LayerBROADCAST CAM

... ... ...

Signalling ...

Grid 8x8

...

...

Representation A Segment Info

Segment Info Media Segment

Media Segment

...

Tile

Representation

Representation

...

Media Presentation Description




A/V Relay: The Hybrid approach

Hybrid A/V

Relay

Reduce bandwidth usage

Increase flexibility & scalability

Integrated solution for delivery of panoramic video




Pub/Sub client subscribes to all streams

Video streams are packaged and time segmented

Segments are stored and a tiled HAS manifest is created

The A/V Proxy is signalled of the session

A common tiling format simplifies life




iPad…

Hybrid delivery to a big screen


And 2nd Screen




Camera ingest (6K x 2K) ~4.5 Gbit/s

Pub/Sub panorama stream ~ 150 Mbit/s

85 parallel encodings on a single machine

Tiled-HAS aggregate 10 – 15 Mbit/s

Depends on tiling needed to recreate a specific view

End-to-end delay 15s

Mostly implementation related

Interaction delay: instantaneous

HQ access delay: ~4 seconds

Some figures




Conclusions

A hybrid approach leverages a common tiling format extending the

flexibility and scalability of delivery of ultra high resolution video.

By means of content tiling, panoramic video can be delivered to various

types of devices, regardless of screen resolution and bandwidth, whilst

offering full control of navigation by the user.

HAS based mechanisms like MPEG-DASH can easily be extended to

support ultra high resolution video content such that a user can interact

with it.




Future work




Questions? More info?

Martin Prins

Research Scientist Media Networking

[email protected]

+31 (0)88 866 78 16

Web: http://www.fascinate-project.eu/

Twitter: Fascinate_Prjct

Facebook and Vimeo

Come see our demo’s during the demo session!

mailto:[email protected]

prins - euroitv 2013 - a hybrid architecture for delivery of panoramic video

Documents