sparrow distributed low-latency scheduling kay ousterhout, patrick wendell, matei zaharia, ion...
TRANSCRIPT
![Page 1: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/1.jpg)
SparrowDistributed Low-Latency Scheduling
Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica
![Page 2: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/2.jpg)
Sparrow schedules tasks in clusters
using a decentralized, randomized approach
![Page 3: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/3.jpg)
Sparrow schedules tasks in clusters
using a decentralized, randomized approach
support constraints and fair sharing
and provides response times
within 12% of ideal
![Page 4: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/4.jpg)
Scheduling Setting
…
…
…
Map Reduce/Spark/
Dryad Job
…
Task
Task
Task
Map Reduce/Spark/
Dryad JobTask
Task
![Page 5: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/5.jpg)
Job Latencies Rapidly Decreasing
10 min.
10 sec.
100 ms
1 ms
2004: MapReducebatch job
2009: Hive
query
2010: Dremel Query
2012: Impala query
2010:In-
memory Spark query
2013:Spark
streaming
![Page 6: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/6.jpg)
Scheduling challenges:
Millisecond Latency
Quality Placement
Fault Tolerant
High Throughput
![Page 7: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/7.jpg)
10 min.
10 sec.
100 ms
1 ms
2004: MapReducebatch job
2009: Hive
query
2010: Dremel Query
2012: Impala query
2010:In-
memory Spark query
2013:Spark
streaming
1000 16-core machines
26decisio
ns/second
Scheduler
throughput
1.6K
decisions/
second
160Kdecisio
ns/second
16M
decisions/
second
![Page 8: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/8.jpg)
Millisecond Latency
Quality Placement
Fault Tolerant
High Throughput
Today: Completely Centralized Less centralization
Sparrow:Completely
Decentralized
✗
✗
✓
✗ ✓
✓
✓
?✓
![Page 9: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/9.jpg)
SparrowDecentralized approach
Existing randomized approachesBatch SamplingLate BindingAnalytical performance evaluation
Handling constraints
Fairness and policy enforcement
Within 12% of ideal on 100 machines
![Page 10: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/10.jpg)
Scheduling with Sparrow
WorkerWorkerWorkerWorkerWorker
Scheduler
Scheduler
Scheduler
SchedulerJob
Worker
![Page 11: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/11.jpg)
WorkerWorkerWorkerWorkerWorker
Scheduler
Scheduler
Scheduler
SchedulerJob
Worker
Random
![Page 12: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/12.jpg)
Simulated Results
100-task jobs in 10,000-node cluster, exp. task durations
Omniscient: infinitely fast centralized
scheduler
![Page 13: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/13.jpg)
Per-task sampling
WorkerWorkerWorkerWorkerWorker
Scheduler
Scheduler
Scheduler
SchedulerJob
Worker
Power of Two Choices
![Page 14: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/14.jpg)
Per-task sampling
WorkerWorkerWorkerWorkerWorker
Scheduler
Scheduler
Scheduler
SchedulerJob
Worker
Power of Two Choices
![Page 15: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/15.jpg)
Simulated Results
100-task jobs in 10,000-node cluster, exp. task durations
![Page 16: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/16.jpg)
70% cluster load
Response Time Grows with Tasks/Job!
![Page 17: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/17.jpg)
Per-Task Sampling
WorkerWorkerWorkerWorkerWorker
Scheduler
Scheduler
Scheduler
SchedulerJob
Worker
✓
✓
Task 1
Task 2
Per-ta
sk
![Page 18: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/18.jpg)
Per-task Sampling
WorkerWorkerWorkerWorkerWorker
Scheduler
Scheduler
Scheduler
SchedulerJob
Worker
Place m tasks on the least loaded of dm slaves
Per-ta
sk✓
✓
4 probes (d =
2)
Batch
![Page 19: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/19.jpg)
Per-task versus Batch Sampling
70% cluster load
![Page 20: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/20.jpg)
Simulated Results
100-task jobs in 10,000-node cluster, exp. task durations
![Page 21: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/21.jpg)
Queue length poor predictor of wait time
Worker
Worker
80 ms155
ms
530 ms
Poor performance on heterogeneous workloads
![Page 22: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/22.jpg)
Late Binding
Worker
Worker
Worker
Worker
Worker
Scheduler
Scheduler
SchedulerSchedulerJob
Worker
Place m tasks on the least loaded of dm slaves
4 probes (d =
2)
![Page 23: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/23.jpg)
Late Binding
Scheduler
Scheduler
SchedulerSchedulerJob
Place m tasks on the least loaded of dm slaves
4 probes (d =
2)
Worker
Worker
Worker
Worker
Worker
Worker
![Page 24: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/24.jpg)
Late Binding
Scheduler
Scheduler
SchedulerSchedulerJob
Place m tasks on the least loaded of dm slaves
Worker
requests
task
Worker
Worker
Worker
Worker
Worker
Worker
![Page 25: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/25.jpg)
Simulated Results
100-task jobs in 10,000-node cluster, exp. task durations
![Page 26: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/26.jpg)
What about constraints?
![Page 27: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/27.jpg)
Job Constraints
Scheduler
Scheduler
Scheduler
SchedulerJob
Worker
Worker
Worker
Worker
Worker
Worker
Restrict probed machines to those that satisfy the constraint
![Page 28: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/28.jpg)
Per-Task Constraints
Scheduler
Scheduler
Scheduler
SchedulerJob
Worker
Worker
Worker
Worker
Worker
Worker
Probe separately for each task
![Page 29: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/29.jpg)
Technique Recap
Scheduler
Scheduler
Scheduler
SchedulerBatch
sampling+
Late binding+
Constraint handling
WorkerWorkerWorkerWorkerWorker
Worker
![Page 30: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/30.jpg)
How does Sparrow perform on a real cluster?
![Page 31: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/31.jpg)
Spark on Sparrow
WorkerWorkerWorkerWorkerWorker
Worker
Query: DAG of Stages
Sparrow
Scheduler
Job
![Page 32: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/32.jpg)
Spark on Sparrow
WorkerWorkerWorkerWorkerWorker
Worker
Query: DAG of Stages
Sparrow
Scheduler
Job
![Page 33: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/33.jpg)
Spark on Sparrow
WorkerWorkerWorkerWorkerWorker
Worker
Query: DAG of Stages
Sparrow
Scheduler
Job
![Page 34: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/34.jpg)
How does Sparrow compare to Spark’s native scheduler?
100 16-core EC2 nodes, 10 tasks/job, 10 schedulers, 80% load
![Page 35: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/35.jpg)
TPC-H Queries: Background
TPC-H: Common benchmark for analytics workloads
Sparrow
Spark: Distributed in-memory analytics framework
Shark: SQL execution engine
![Page 36: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/36.jpg)
TPC-H Queries
100 16-core EC2 nodes, 10 schedulers, 80% load
95
75
25
50
Percentiles
5
![Page 37: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/37.jpg)
TPC-H Queries
100 16-core EC2 nodes, 10 schedulers, 80% load
Within 12% of ideal
Median queuing delay of 9ms
![Page 38: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/38.jpg)
Fault Tolerance
Scheduler 1
Scheduler 2
Spark Client 1 ✗Spark
Client 2
Timeout: 100msFailover: 5ms
Re-launch queries: 15ms
![Page 39: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/39.jpg)
When does Sparrow not work as well?
High cluster load
![Page 40: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/40.jpg)
Related Work
Centralized task schedulers: e.g., Quincy
Two level schedulers: e.g., YARN, Mesos
Coarse-grained cluster schedulers: e.g., Omega
Load balancing: single task
![Page 41: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/41.jpg)
Batch sampling
+Late binding
+Constraint handling
www.github.com/radlab/sparrow
Sparrows provides near-ideal job response
times without global visibility
Scheduler
Scheduler
Scheduler
Scheduler
WorkerWorkerWorkerWorkerWorker
Worker
![Page 42: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/42.jpg)
Backup Slides
![Page 43: Sparrow Distributed Low-Latency Scheduling Kay Ousterhout, Patrick Wendell, Matei Zaharia, Ion Stoica](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649db05503460f94a9e8de/html5/thumbnails/43.jpg)
Can we do better without losing simplicity?
Policy Enforcement
SlaveHigh Priority
Low Priority SlaveUser A (75%)
User B (25%)
Fair SharesServe queues using
weighted fair queuing
PrioritiesServe queues based on strict priorities