mmsys14 amphi - slideshare
TRANSCRIPT
3DTI AMPHITHEATER
A Manageable 3DTI Environment with Hierarchical Stream Prioritization
Shannon Chen Klara Nahrstedt
Indranil Gupta
University of Illinois at Urbana-Champaign
3D TELE-IMMERSION
• 3D virtual reality
• Interactive
• Free view point
• Multi-user
IP-BASED LIVE BROADCASTING
• Existing IP-based broadcasting frameworks• 2D• Single producer at any given time• Single view• View decided by producer
3DTI + LIVE BROADCAST
• Challenges• High bandwidth demand• Low latency demand• Subscription dynamics• Synchronization• Content dissemination
3DTI + LIVE BROADCAST
• Challenges• High bandwidth demand• Low latency demand• Subscription dynamics• Synchronization• Content dissemination
3DTI + LIVE BROADCAST
• Challenges• High bandwidth demand• Low latency demand• Subscription dynamics• Synchronization• Content dissemination
3DTI + LIVE BROADCAST
• Challenges• High bandwidth demand• Low latency demand• Subscription dynamics• Synchronization• Content dissemination
3DTI + LIVE BROADCAST
• Challenges• High bandwidth demand• Low latency demand• Subscription dynamics• Synchronization• Content dissemination
3DTI AMPHITHEATER
Virtual Stage
3D models of performers
Audience Physical Space
Non-immersive
Performer Physical Space
Immersive
SYSTEM MODEL
• Performer Sites
• Audience Sites
• Session management Site
SYSTEM MODEL
• Performer Sites• Immersive users• Interact on the virtual stage• Depth camera array, head-
mounted display, acc meter• Low latency Demand
• Audience Sites
• Session management Site
SYSTEM MODEL
• Performer Sites
• Audience Sites• Non-immersive users• Passive observer in virtual
seats• Regular display• Relaxed latency requirement
• Session management Site
*beep*
SYSTEM MODEL
• Performer Sites
• Audience Sites
• Session management Site• One manager per session• Dissemination network
construction• Subscription admission• Subscription updating• Production registration
USER MODEL
• Surrounding virtual seats• Manageable view changes• Effective content sharing• Omni-directional coverage
• Hierarchical Stream Prioritization• Stream differentiation:
view-based priority• Site differentiation:
role-based priority
Source: www.stub.com/jamestown-mall-universoul-circus-tickets/
VIRTUAL SEATS
• Manageable view changes• Fixed position• Changeable direction• Virtual opera glasses
• Effective content sharing
• Omni-directional coverage
VIRTUAL SEATS
• Manageable view changes• Fixed position• Changeable direction• Virtual opera glasses
• Effective content sharing
• Omni-directional coverage
VIRTUAL SEATS
• Manageable view changes• Fixed position• Changeable direction• Virtual opera glasses
• Effective content sharing
• Omni-directional coverage
VIRTUAL SEATS
• Manageable view changes
• Effective content sharing• Pre-assigned seats• Evenly spread• Overlapping views
• Omni-directional coverage
Source: homepages.inf.ed.ac.uk/rbf/CVDICT/cvd.htm
VIRTUAL SEATS
• Manageable view changes
• Effective content sharing
• Omnidirectional coverage• Every camera of a
performer is subscribed by some audience
• ‘Hubs’ that alleviates the dissemination burden
Source: skull-the-kid.deviantart.com/
HIERARCHICAL STREAM PRIORITY
• Stream differentiation
• Site differentiation
HIERARCHICAL STREAM PRIORITY
• Stream differentiation – view based priority• Not all cameras are equally important to a viewer• Depends on the view
• View-based priority ≡ max(CF,0)
• Site differentiation
𝐶𝑜𝑛𝑡𝑟𝑖𝑏𝑡𝑖𝑜𝑛𝐹𝑎𝑐𝑡𝑜𝑟
𝐶𝐹=1 𝐶𝐹=0
𝐶𝐹=−1𝐶𝐹=0
• Stream differentiation
• Site differentiation – role based priority• Not all performers are equally important to a viewer• Depends on the roles of the performer and the viewer
• Examples for determine role-based priority • Uniform priority• User-defined priority• Objective priority
HIERARCHICAL STREAM PRIORITY
A performer is a viewer too
since they also have to see
other performers!
• Stream differentiation
• Site differentiation• Not all performers are equally important to a viewer• Depends on the roles of the performer and the viewer
• Examples for determine role-based priority • Uniform priority
• Scenario: sport game• Viewer: player1• Performers:
{player2, oppo1, oppo2}• Role-based priority:
{3.33, 3.33, 3.33}
HIERARCHICAL STREAM PRIORITY
Source: www.allabouttabletennis.com
• Stream differentiation
• Site differentiation• Not all performers are equally important to a viewer• Depends on the roles of the performer and the viewer
• Determination of role-based priority • User-defined priority
• Scenario: school play• Viewer: Ann’s dad• Performers:
{kid1, kid2, Ann}• Role-based priority:
{1.00, 1.00, 8.00}
HIERARCHICAL STREAM PRIORITY
Source: www.abbyofftherecord.com/2012/07/25/board-meeting-or-school-play/
• Stream differentiation
• Site differentiation• Not all performers are equally important to a viewer• Depends on the roles of the performer and the viewer
• Examples for determine role-based priority • Objective priority
• Scenario: cocktail party• Viewer: guest• Performers:
{other guests}• Role-based priority:
{1/distance}
HIERARCHICAL STREAM PRIORITY
Source: www.crowneventsandconferences.com.au
• Construction of dissemination network• For each stream
there is a publisherand itssubscribers
• Together theyform a disseminationtree
• How does the session manager construct the forest?
• Pub/Sub model
STREAM DELIVERY MODEL
P2P overlay
• Construction of dissemination network• All trees will be sharing
the resource ofthe same P2P overlay
• So the structureof the forestis important
• How does the session manager construct the forest?
• Pub/Sub model
STREAM DELIVERY MODEL
P2P overlay
STREAM DELIVERY MODEL
• Construction of dissemination network
• How does the session manager construct the forest?• Pub/Sub model
Publisher SubscriberBroker
Registration Subscription request
Info of the receiver Info of the sender
Establish connection
Stream
Publisher: performers
Subscriber: viewers
(audience + performers)
Broker: session manager
Forest Planning
STREAM DELIVERY MODEL
• Construction of dissemination network
• How does the session manager construct the forest?• Pub/Sub model• Registration
• Cameras• Shooting angles
Publisher SubscriberBroker
Registration Subscription request
Info of the receiver Info of the sender
Establish connection
Stream
Forest Planning
STREAM DELIVERY MODEL
• Construction of dissemination network
• How does the session manager construct the forest?• Pub/Sub model• Registration• Subscription
• Site type• Position/direction• Priorities• Capabilities
Publisher SubscriberBroker
Registration Subscription request
Info of the receiver Info of the sender
Establish connection
Stream
Forest Planning
STREAM DELIVERY MODEL
• Construction of dissemination network
• How does the session manager construct the forest?• Pub/Sub model• Registration• Subscription• Forest planning
• Role-based priority• View-based priority• Bandwidth• Latency(…more details in the paper)
Publisher SubscriberBroker
Registration Subscription request
Info of the receiver Info of the sender
Establish connection
Stream
Forest Planning
EVALUATION
• Experiment 1: overall performance
• Experiment 2: service quality of performers
• Experiment 3: effect of virtual seats
• Settings• Network: Netmap database• 3DTI sites: TEEVE prototype
• Metrics• Request rejection ratio• AQoS: admission rate weighted by stream importance
EVALUATION
• Experiment 1: overall performance
• Compare to 4D TeleCast [2012]
• Amphitheater sustains 1,010 more requests and provides higher AQoS
100
300
500
700
900
0%
2%
4%
6%
8%
10%Reject Ratio
Amphitheater 4D TeleCast
Number of audiences
100
200
300
400
500
600
700
800
90010
000.9
0.92
0.94
0.96
0.98
1AQoS
Amphitheater 4D TeleCast
Number of audiences
EVALUATION
• Experiment 2: service quality of performers
• Compare to Nahrstedt et al. [2011]
• x2.8 AQoS for virtual play and x1.4 AQoS for sport arena
3 6 9 12 15 180
0.2
0.4
0.6
0.8
1Virtual Play (User-de-
fined)
Amphitheater
Nahrstedt et al. [8]
Number of participating sites
AQ
oS
3 4 5 6 7 8 9 100
0.2
0.4
0.6
0.8
1Sport Arena (Uni-
form)
Amphitheater
Nahrstedt et al. [8]
Number of participating sites
AQ
oS
EVALUATION
• Experiment 3: effect of virtual seats
• Amphitheater w/ and w/o audience
• Substantial gain in both admission ratio and AQoS
3 6 9 12 15 180
0.10.20.30.40.50.60.70.80.9
1
w/ audience
w/o audience
Number of performers
Request
reje
ctio
n
rati
o
3 6 9 12 15 180
0.2
0.4
0.6
0.8
1
w/ audience
w/o audience
Number of performers
AQ
oS
FUTURE WORK AND CONCLUSION
• Proposition of 3DTI Amphitheater: a new 3DTI live broadcast framework, which introduce a more manageable 3DTI environment
• Identification of role-based priority, which improves the utilization by granting resource to streams that are semantically important
• Future directions• Stress testing: intensive view change• Seat changing: tradeoff efficiency and user freedom• Audience churn: seat reassignment• Role-based priority determination: ML-based
THANK YOU