mmsys14 amphi - slideshare

3DTI AMPHITHEATER

A Manageable 3DTI Environment with Hierarchical Stream Prioritization

Shannon Chen Klara Nahrstedt

Indranil Gupta

University of Illinois at Urbana-Champaign

3D TELE-IMMERSION

• 3D virtual reality

• Interactive

• Free view point

• Multi-user

IP-BASED LIVE BROADCASTING

• Existing IP-based broadcasting frameworks• 2D• Single producer at any given time• Single view• View decided by producer

3DTI + LIVE BROADCAST

• Challenges• High bandwidth demand• Low latency demand• Subscription dynamics• Synchronization• Content dissemination

3DTI + LIVE BROADCAST

• Challenges• High bandwidth demand• Low latency demand• Subscription dynamics• Synchronization• Content dissemination

3DTI + LIVE BROADCAST

• Challenges• High bandwidth demand• Low latency demand• Subscription dynamics• Synchronization• Content dissemination

3DTI + LIVE BROADCAST

• Challenges• High bandwidth demand• Low latency demand• Subscription dynamics• Synchronization• Content dissemination

3DTI + LIVE BROADCAST

• Challenges• High bandwidth demand• Low latency demand• Subscription dynamics• Synchronization• Content dissemination

3DTI AMPHITHEATER

Virtual Stage

3D models of performers

Audience Physical Space

Non-immersive

Performer Physical Space

Immersive

SYSTEM MODEL

• Performer Sites

• Audience Sites

• Session management Site

SYSTEM MODEL

• Performer Sites• Immersive users• Interact on the virtual stage• Depth camera array, head-

mounted display, acc meter• Low latency Demand

• Audience Sites

• Session management Site

SYSTEM MODEL

• Performer Sites

• Audience Sites• Non-immersive users• Passive observer in virtual

seats• Regular display• Relaxed latency requirement

• Session management Site

*beep*

SYSTEM MODEL

• Performer Sites

• Audience Sites

• Session management Site• One manager per session• Dissemination network

construction• Subscription admission• Subscription updating• Production registration

USER MODEL

• Surrounding virtual seats• Manageable view changes• Effective content sharing• Omni-directional coverage

• Hierarchical Stream Prioritization• Stream differentiation:

view-based priority• Site differentiation:

role-based priority

Source: www.stub.com/jamestown-mall-universoul-circus-tickets/

VIRTUAL SEATS

• Manageable view changes• Fixed position• Changeable direction• Virtual opera glasses

• Effective content sharing

• Omni-directional coverage

VIRTUAL SEATS

• Manageable view changes• Fixed position• Changeable direction• Virtual opera glasses

• Effective content sharing

• Omni-directional coverage

VIRTUAL SEATS

• Manageable view changes• Fixed position• Changeable direction• Virtual opera glasses

• Effective content sharing

• Omni-directional coverage

VIRTUAL SEATS

• Manageable view changes

• Effective content sharing• Pre-assigned seats• Evenly spread• Overlapping views

• Omni-directional coverage

Source: homepages.inf.ed.ac.uk/rbf/CVDICT/cvd.htm

VIRTUAL SEATS

• Manageable view changes

• Effective content sharing

• Omnidirectional coverage• Every camera of a

performer is subscribed by some audience

• ‘Hubs’ that alleviates the dissemination burden

Source: skull-the-kid.deviantart.com/

HIERARCHICAL STREAM PRIORITY

• Stream differentiation

• Site differentiation

HIERARCHICAL STREAM PRIORITY

• Stream differentiation – view based priority• Not all cameras are equally important to a viewer• Depends on the view

• View-based priority ≡ max(CF,0)

• Site differentiation

𝐶𝑜𝑛𝑡𝑟𝑖𝑏𝑡𝑖𝑜𝑛𝐹𝑎𝑐𝑡𝑜𝑟

𝐶𝐹=1 𝐶𝐹=0

𝐶𝐹=−1𝐶𝐹=0

• Stream differentiation

• Site differentiation – role based priority• Not all performers are equally important to a viewer• Depends on the roles of the performer and the viewer

• Examples for determine role-based priority • Uniform priority• User-defined priority• Objective priority

HIERARCHICAL STREAM PRIORITY

A performer is a viewer too

since they also have to see

other performers!

• Stream differentiation

• Site differentiation• Not all performers are equally important to a viewer• Depends on the roles of the performer and the viewer

• Examples for determine role-based priority • Uniform priority

• Scenario: sport game• Viewer: player1• Performers:

{player2, oppo1, oppo2}• Role-based priority:

{3.33, 3.33, 3.33}

HIERARCHICAL STREAM PRIORITY

Source: www.allabouttabletennis.com

• Stream differentiation

• Site differentiation• Not all performers are equally important to a viewer• Depends on the roles of the performer and the viewer

• Determination of role-based priority • User-defined priority

• Scenario: school play• Viewer: Ann’s dad• Performers:

{kid1, kid2, Ann}• Role-based priority:

{1.00, 1.00, 8.00}

HIERARCHICAL STREAM PRIORITY

Source: www.abbyofftherecord.com/2012/07/25/board-meeting-or-school-play/

• Stream differentiation

• Site differentiation• Not all performers are equally important to a viewer• Depends on the roles of the performer and the viewer

• Examples for determine role-based priority • Objective priority

• Scenario: cocktail party• Viewer: guest• Performers:

{other guests}• Role-based priority:

{1/distance}

HIERARCHICAL STREAM PRIORITY

Source: www.crowneventsandconferences.com.au

• Construction of dissemination network• For each stream

there is a publisherand itssubscribers

• Together theyform a disseminationtree

• How does the session manager construct the forest?

• Pub/Sub model

STREAM DELIVERY MODEL

P2P overlay

• Construction of dissemination network• All trees will be sharing

the resource ofthe same P2P overlay

• So the structureof the forestis important

• How does the session manager construct the forest?

• Pub/Sub model

STREAM DELIVERY MODEL

P2P overlay

STREAM DELIVERY MODEL

• Construction of dissemination network

• How does the session manager construct the forest?• Pub/Sub model

Publisher SubscriberBroker

Registration Subscription request

Info of the receiver Info of the sender

Establish connection

Stream

Publisher: performers

Subscriber: viewers

(audience + performers)

Broker: session manager

Forest Planning

STREAM DELIVERY MODEL

• Construction of dissemination network

• How does the session manager construct the forest?• Pub/Sub model• Registration

• Cameras• Shooting angles

Publisher SubscriberBroker

Registration Subscription request

Info of the receiver Info of the sender

Establish connection

Stream

Forest Planning

STREAM DELIVERY MODEL

• Construction of dissemination network

• How does the session manager construct the forest?• Pub/Sub model• Registration• Subscription

• Site type• Position/direction• Priorities• Capabilities

Publisher SubscriberBroker

Registration Subscription request

Info of the receiver Info of the sender

Establish connection

Stream

Forest Planning

STREAM DELIVERY MODEL

• Construction of dissemination network

• How does the session manager construct the forest?• Pub/Sub model• Registration• Subscription• Forest planning

• Role-based priority• View-based priority• Bandwidth• Latency(…more details in the paper)

Publisher SubscriberBroker

Registration Subscription request

Info of the receiver Info of the sender

Establish connection

Stream

Forest Planning

EVALUATION

• Experiment 1: overall performance

• Experiment 2: service quality of performers

• Experiment 3: effect of virtual seats

• Settings• Network: Netmap database• 3DTI sites: TEEVE prototype

• Metrics• Request rejection ratio• AQoS: admission rate weighted by stream importance

EVALUATION

• Experiment 1: overall performance

• Compare to 4D TeleCast [2012]

• Amphitheater sustains 1,010 more requests and provides higher AQoS

100

300

500

700

900

0%

2%

4%

6%

8%

10%Reject Ratio

Amphitheater 4D TeleCast

Number of audiences

100

200

300

400

500

600

700

800

90010

000.9

0.92

0.94

0.96

0.98

1AQoS

Amphitheater 4D TeleCast

Number of audiences

EVALUATION

• Experiment 2: service quality of performers

• Compare to Nahrstedt et al. [2011]

• x2.8 AQoS for virtual play and x1.4 AQoS for sport arena

3 6 9 12 15 180

0.2

0.4

0.6

0.8

1Virtual Play (User-de-

fined)

Amphitheater

Nahrstedt et al. [8]

Number of participating sites

AQ

oS

3 4 5 6 7 8 9 100

0.2

0.4

0.6

0.8

1Sport Arena (Uni-

form)

Amphitheater

Nahrstedt et al. [8]

Number of participating sites

AQ

oS

EVALUATION

• Experiment 3: effect of virtual seats

• Amphitheater w/ and w/o audience

• Substantial gain in both admission ratio and AQoS

3 6 9 12 15 180

0.10.20.30.40.50.60.70.80.9

1

w/ audience

w/o audience

Number of performers

Request

reje

ctio

n

rati

o

3 6 9 12 15 180

0.2

0.4

0.6

0.8

1

w/ audience

w/o audience

Number of performers

AQ

oS

FUTURE WORK AND CONCLUSION

• Proposition of 3DTI Amphitheater: a new 3DTI live broadcast framework, which introduce a more manageable 3DTI environment

• Identification of role-based priority, which improves the utilization by granting resource to streams that are semantically important

• Future directions• Stress testing: intensive view change• Seat changing: tradeoff efficiency and user freedom• Audience churn: seat reassignment• Role-based priority determination: ML-based

THANK YOU