medianet: user-defined adaptive scheduling for streaming data michael hicks adithya nagarajan...
Post on 21-Dec-2015
229 views
TRANSCRIPT
MediaNet: User-defined Adaptive Scheduling for
Streaming Data
Michael Hicks
Adithya NagarajanUniversity of Maryland
Robbert van RenesseCornell University
Motivation
• Multi-user streaming data applications– Sources
• Sensor reports• Movies• Live video or audio• Weather reports• Stock quotes
• Support Quality of Service (QoS)– Application-specific metrics– Fair and efficient sharing of resources
– Situations• Military• Disaster• Home network
MediaNet
• Adaptive– Schedules flows using available resources– Adapts to loads not under its control
• User-directed– Adaptations are directed by users, based on
relative preferences and user priority
• Comprehensive– Global view of the network, accounting for
overall network utilization, and per-user utility
MediaNet Architecture
Video description,location, &
resource info
User’s desiredstream &
adaptation prefs
Globalscheduling
service
publish
subscribe
Video source
Video player
schedules
feedback
• Continuous Media Network (CMN)– Directed acyclic graph of operations
• frame droppers, transcoders, compressors and decompressors, filters, aggregators, etc.
• User specification– One or more CMN’s, each with associated
utility value
• Goal: Maximize users’ utility while utilizing the network efficiently
Streaming Computations
Operations
Op
frmn
IntervalFrame sizeOther attributes• Fixed location?• Transitions only?
…
frm1
…
Example User Specification
1.0
0.3
0.1
Utility CMN
Vid
Prio*Vid
Vid Prio*DropPB
Prio* User
DropB
Prio* User
Prio* UserpcS
pcS
pcS
pcD
pcD
pcD
Global Scheduling Service
• Schedules each user specification on the actual network– Locates each operation on a node– Inserts send and receive operations between
nodes; can have varying transport attributes
• Scheduling choices based on current network resources (i.e. operates on-line)
Global Scheduling Algorithm
• Simulated annealing technique– Cost function for network configurations– Maximize minimum utility for all users, plus
• Optimize individual user utilities above minimum• Use best-cost configuration at these utilities
• Cost function– Relates resource cost of a configuration to the
total resources available (CPU, bandwidth)
Creating Configurations
• Gather all user specs at utility u, combine them, and partition into distinct trees
• For each tree– Calculate network “shortest” paths
• Actually, most bandwidth-plentiful paths
– Find “best” placement of operations to nodes
Example: Creating Config
pc1 pc5
Vid1 Prio*
pc1 pc7
Vid1 Prio*
pc1 pc8
pc2 pc4
Vid2 Prio*
pc2 pc8
5 user specifications, utility 1.0
User2
User4
Vid1 Prio* User1
User3
User5
Vid2 Prio*
Example: Creating Config
Vid1 Prio* User1
pc1 pc5
User3
pc7
User5
pc8
Vid2 Prio* User2
pc2 pc4
User4
pc8
Combining the specs
Example: Scheduling
pc1
pc2
pc3
pc5
pc6
pc4
pc7
pc8
300
300 150
300
300
300
300
150
1. Initial network conditions
Example: Scheduling
pc1
pc2
pc3
pc5
pc6
pc4
pc7
pc8
300
300 150
300
300
300
300
150
2. Scheduling the 1st tree
v1 u1 u3
u5
p* p*
Example: Scheduling
pc1
pc2
pc3
pc5
pc6
pc4
pc7
pc8
150
300 150
150
300
150
150
0
3. Adjusting edge weights
v1 u1 u3
u5
p* p*
Example: Scheduling
pc1
pc2
pc3
pc5
pc6
pc4
pc7
pc8
150
300 150
150
300
150
150
0
4. Scheduling the 2nd tree
v1 u1 u3
p*
p*
p*
u2v2
u4
u5
Prototype Implementation
• Global scheduling service– implemented on a single node– eventually hierarchical
• Local, per-node schedulers– monitor and report available bandwidth;
eventually CPU + memory usage– implement local CMNs
• Global scheduler reconfigures schedules on-line
Local Scheduling
• Implement the CMN given by the GS– Must correctly reconfigure on-line
• Report monitoring info back to GS• Implemented in Cyclone
– Type-safe, C-like language– One component per operation, dynamically
reconfigurable
• Current uses TCP for send/receive– Other transports possible/useful
Monitoring
• Monitor available bandwidth– Keep track of TCP throughput, attempted vs.
actual bandwidth
• Report to global scheduler when – attempted ≠ actual bandwidth (i.e. at peak)– after a regular timeout
• Too pessimistic– “creep” bandwidth estimate additively to
optimistically attempt higher utility configs
On-line Reconfiguration
• New configuration is applied in parallel with the current configuration– Old operations are “flushed” along the
dataflow path– New operations are enabled when all old ones
are flushed on a particular node
• Challenges– Rapid reconfiguration– Low disruption to stream
Experiments• Conducted on 8 850 MHz PIII’s, 512 MB RAM, 100 Mb/s Ethernet,
RedHat Linux 7.1 using a “bowtie” topology:
MPEG clip Bandwidth Requirements
Frame rate I+P+B I+P I+B
30 fps 144 KB/s 88 KB/s 28 KB/s
Single-flow Performance Comparison
No adaptation Priority-based frame dropping
Local, proactive frame dropping MediaNet
Multi-User Performance
Multi-User Performance
Multi-User Performance
B frame dropping op
Multi-User Performance
B frame dropping op
Related Work
• Layered Multicast
• In-network stream processors– MEGA, Active Nets
• Flow planning systems– Ninja, Darwin, CANS, End-to-End Paths,
Conductor, PATHS, Choi et al.
• Reservation-based QoS– OMEGA
Future Work
• Hierarchy of global schedulers– Better scalability
• Scaling user utilities– Based on user priority and resource usage– Enforces fairness
• Better on-line monitoring• More experiments
– Real wireless– Simulation for larger scenarios
Conclusions
• Application-specific QoS via user specs• High network utilization and per-user utility
via global scheduling:– Share resources between flows in a multicast-
like manner, but generalized to CMNs– Utilize multiple, redundant paths– Intelligently place operations to reduce
network utilization
• Adapts to resource availabilities on-line
Papers, Software at
http://www.cs.umd.edu/projects/medianet
For More Information