Verdione

Remote presence in a

World Opera

Carsten Griwodz Jason Geistweidt

griff@simula.no jason@geistweidt.com

November 2010

Verdione project relations

Nofima Matforsk

Lividi

Netview

NTNU

Univ. Tromsø

Norwegian Academy of Music

Telenor

The MET New York

Bang & Olufsen

The World Opera Verdione

Perceval

National University of Singapore

Ilmenau Univ. of Tech.

Tampere Univ. of Tech.

Stanford

McGill

NYU

Univ. Oslo

Simula

Verdione project relations

Univ. Tromsø

Verdione

Netview

Univ. Oslo

Simula

Telenor

The MET New York

Bang & Olufsen

The World Opera

Lividi

Norwegian Academy of Music

National University of Singapore

Stanford

McGill

NYU

Nofima Matforsk

NTNU Perceval

Ilmenau Univ. of Tech.

Tampere Univ. of Tech.

Verdione project relations

Univ. Tromsø

Verdione

Netview

Univ. Oslo

Simula

NTNU

BI Nor. School of Mgmt.

Microsoft/FAST

Accenture

iAd

Cornell

DCU

7

Tromsø - Stockholm - Copenhagen

? ?

The World Opera

primary stage

live performance, fully

synchronized with artistic

output from other stages

audience see both live

and virtual elements

secondary stages

present a fully virtual

performance to the audience

Interaction spaces

surround artists

detect movement,

sounds, etc

Display wall

remote stages as part of

a pixel space

se

co

nd

ary

sta

ge

s

audience

audience

audience

notion of primary and

secondary stages changes

Interleaved live performances

secondary stages –send to

primary stage

present on display walls

large-scale

camera

arrays

Cameras

multiple cameras

Depth Estimation for Free-View Rendering

Stereo System • Capture two views

(left and right)

• Depth estimated by

disparity

• Novel view points

by warping left right

depth

output

Depth Estimation for Free-View Rendering

Structured Light • Project structured

light on object

• Depth estimated by

ray-plane

intersection

• Novel view points

through warping or

modelling

setup

output

Stage challenges • must use infrared

light

• handle movement

during refinement illuminated by

structured light

origina

l

Service overlay

Main goal Architecting a solution for transporting

media between opera sites

Requirements • Very low delay (to

support interactivity)

• Efficient bandwidth

usage (for increased

media quality)

• High reliability (given

by a live audience)

• Handle link and

media router failures

(with minimum impact

for the opera

audience)

Current approach • An overlay network between operas

• Quickly identify failed links and re-route using "standby" backup routes

• Media forwarding and filtering for very low delay (no media transcoding)

• Routing algorithms for short delay and bandwidth efficiency

14

Client (receiver)

App

Server (sender)

App

Proxy

Default Flow

Redirected Part of Flow

Network Interfaces

NAT

NAT

Multilink proxy

Transparent load leveling Two or more access networks

Useful for capacity bundling

and transparent backup

Results Compensation for latency variations

Exploitation of heterogeneous

bandwidth

400x improved tunneling (redirect)

speed

Questions