Scalable, Fast Cloud Computing with Execution Templates

Omid Mashayekhi, Hang Qu, Chinmayee Shah, Philip LevisStanford University

{omidm, quhang, chshah}@stanford.edu pal@cs.stanford.edu

AbstractLarge scale cloud data analytics applications are oftenCPU bound. Most of these cycles are wasted: bench-marks written in C++ run 10-51 times faster than frame-works such as Naiad and Spark. However, calling fasterimplementations from those frameworks only sees mod-erate (3-5x) speedups because their control planes cannotschedule work fast enough.

This paper presents execution templates, a controlplane abstraction for CPU-bound cloud applications,such as machine learning. Execution templates leveragehighly repetitive control flow to cache scheduling deci-sions as templates. Rather than reschedule hundreds ofthousands of tasks on every loop execution, nodes in-stantiate these templates. A controller’s template spec-ifies the execution across all worker nodes, which it par-titions into per-worker templates. To ensure that tem-plates execute correctly, controllers dynamically patchtemplates to match program control flow. We have im-plemented execution templates in Nimbus, a C++ cloudcomputing framework. Running in Nimbus, analyticsbenchmarks can run 16-43 times faster than in Naiad andSpark. Nimbus’s control plane can scale out to run thesefaster benchmarks on up to 100 nodes (800 cores).

1 IntroductionThe CPU has become the new bottleneck for analyticsbenchmarks and applications. One recent study foundthat the big data benchmark (BDBench), TCP decisionsupport benchmark (TCP-DS), and production work-loads from Databricks were all CPU-bound. Improvingnetwork I/O would reduce their median completion timeby at most 2% and improving disk I/O would reduce theirmedian completion time by at most 19% [30].

At the same time, systems such as DMLL [13] andDimmWitted [26] have shown it is possible to achieveorders-of-magnitude improvements in CPU performanceover frameworks such as Spark [38]. Comparing theperformance of C++ and Spark implementations of twostandard machine learning benchmarks, we find that theC++ implementations run up to 51 times faster. Modern

analytics frameworks are CPU-bound, but most of thesecycles are wasted.

One straw man solution to improve performance is tohave a framework call into C++ implementations of com-putational kernels, e.g., through the Java Native Interface(JNI). In Section 2.2, we show that this only sees mod-est speedups (5x rather than 50x): worker nodes spend90% of their cycles idle. The central Spark controller,which is responsible for telling to workers to executetasks, cannot schedule tasks quickly enough. The frame-work’s control plane becomes a bottleneck and workersfall idle. In Section 5 we show that Naiad [28], anotherframework, has similar control plane bottlenecks.

Current frameworks do not scale to run optimizedtasks on many nodes. They can either run on many nodesor run optimized tasks, but not both, because the controlplane cannot schedule tasks fast enough. Prior scalablescheduling systems such as Sparrow [31], Omega [33],Apollo [12], Mercury [25], Hawk [16] and Tarcil [17] allpropose ways to distribute the scheduling of many jobswhich together overwhelm a single controller. Schedul-ing a job requires centralized state, and so for all thesesystems, tasks from a single job still go through a singlescheduler. Optimized tasks, however, mean that a singlejob can saturate a controller.

Section 3 presents execution templates a control planeabstraction which scales to schedule optimized tasks onmany nodes. The key insight behind execution templatesis that long-running CPU-bound computations are repet-itive: they run the same computation (e.g., a loop body)many times. Rather than reschedule each repetition fromscratch, a runtime caches scheduling decisions as an ex-ecution template of tasks. A program invokes a tem-plate, potentially creating thousands of tasks, with a sin-gle message. We call this abstraction a template becauseit can cache some decisions (e.g., dependencies) but fullyinstantiating it requires parameters (e.g., task identifiers).

Section 4 describes an implementation of executiontemplates in Nimbus, a C++ analytics framework that in-corporates execution templates. Compared to Spark andNaiad, benchmarks in Nimbus run 16-43 times faster.Rewriting benchmarks in Spark and Naiad to use opti-mized tasks reduces their completion time by a factor of

3.7-5. However, Section 2.2 shows results that neithercan scale out past 20 worker nodes because the controlplane becomes a bottleneck: running on more than 20nodes increases completion time. Using execution tem-plates, implementations of these benchmarks in Nimbusscale out to 100 nodes (800 cores), seeing nearly linearspeedups.

Execution templates allow a centralized controller tohandle tasks shorter than 1ms, or 100 times shorter thanwhat prior systems support [31]. This makes wholenew applications possible. We have ported PhysBAM,a graphical simulation library [18] used in many featurefilms1 to Nimbus. PhysBAM has tasks as short as 100µs,yet execution templates can execute extremely large sim-ulations within 15% of the speed of PhysBAM’s hand-tuned MPI libraries.

This paper makes five contributions:

1. A detailed analysis of how Spark spends CPU cycles,finding that C++ implementations run 51 times fasterand most of Spark’s cycles are wasted due to runtimeand programming language overheads (Section 2.1).

2. Results showing Spark and Naiad’s control planesare a bottleneck when running optimized (C++) tasksand so they can only provide modest speedups (Sec-tion 2.2).

3. Execution templates, a novel control plane abstractionthat allows optimized tasks to run at scale (Section 3).

4. The design of Nimbus, an analytics framework thatincorporates execution templates and a data modelbased on mutable data objects which permit in-placemodifications (Section 4).

5. An evaluation of execution templates, finding theyallow Nimbus to run optimized tasks with almostno overhead, scaling out to 100 nodes (800 cores)while running 30-43 times faster than Spark and 16-23 times faster than Naiad. Execution templates alsoallow Nimbus to support large, complex applicationswith tasks as short as 100µs (Section 5).

Section 4 provides details on the Nimbus implementa-tion of execution templates, including the dynamic pro-gram analysis that ensures they execute properly despitevariations in control and data flow. Section 6 presentsrelated work and Section 7 concludes.

2 MotivationA recent study found that Spark analytics applicationsare CPU-bound [30]. Increasing server RAM and easy

1PhysBAM is a cornerstone of special effects at Industrial Light andMagic and is also used Pixar.

C++(4.1ms) Scala(206ms)

85%8%5%

boxing/un-boxingdataduplicaAonJVM

Figure 1: Logistic regression execution time imple-mented in Scala and C++. C++ is 51 times faster thanScala. These results are averaged over 30 iterations anddiscard the first iteration to allow Java Virtual Machine(JVM) to warm up and just-in-time compile.

Code Nodes Task Length Completion Time

Scala 100 nodes 206ms 2.86sC++ 100 nodes 4ms 1.00sC++ 20 nodes 20ms 0.53s

Table 1: Effect of running optimized logistic regressiontasks in Spark. Although C++ tasks can run 51 timesfaster, a job using C++ tasks on 100 nodes only runs 2.8xfaster. It run 5x faster when run on 20 nodes. Both aremuch slower than the expected speedups and 20 nodes isfaster than 100 due to the control plane being unable toschedule tasks fast enough.

parallelization means that many applications can keeptheir entire working set in RAM and completion time islimited by CPU performance.

This section motivates the need for a new control planein cloud data analytics frameworks. It starts by exam-ining where Spark’s CPU cycles go: 98% of them arewasted. Re-implementations in C++ run up to 51 timesfaster. However, if a Spark job uses these faster re-implementations, it only sees modest (5x) speedups be-cause the control plane (messages to schedule and dis-patch tasks) become the bottleneck. The section con-cludes by observing an important property of CPU-bound applications, that their control flow and executionexhibits very regular patterns, which can be calculated,cached and reused.

2.1 Where the Cycles Go

Frameworks such as Spark [38] and Naiad [28] focuson applications whose data sets can fit in memory whenspread across many nodes. At the same time, a push forgreater programmer productivity has led them to supporthigher-level languages: 70% of Spark applications arewritten in Scala [37].

These two trends (in-memory datasets and higher-

2

level languages) conflict: for applications that operateon in-memory data, higher-level language overheads be-come significant. Figure 1 shows the execution timeof logistic regression, a common analytics benchmark,implemented in Spark using Scala and implemented inC++. The C++ implementation runs 51 times faster thanthe Spark one.

This poor performance has three major causes.2 First,since Scala’s generic methods cannot use primitive types(e.g., they must use the Double class rather than adouble), every generic method call allocates a new ob-ject for the value, boxes the value in it, un-boxes forthe operation, and deallocates the object. In addition tocost of a malloc and free, this results in millions oftiny objects for the garbage collector to process. 85%of logistic regression’s CPU cycles are spent boxing/un-boxing.

Second, Spark’s resilient distributed datasets (RDDs)forces methods to allocate new arrays, write into them,and discard the source array. For example, a mapmethodthat increments a field in a dataset cannot perform theincrement in-place and must instead create a whole newdataset. This data duplication adds an additional factorof ≈ 2x slowdown.

Third, using the Java Virtual Machine has an addi-tional factor of ≈ 3x slowdown over C++. This resultis in line with prior studies, which have reported 1.9x-3.7x for computationally dense codes [22, 21]. In total,this results in Spark code running 51 times slower thanC++.

2.2 Implications of Optimized Tasks

To determine how much tasks running at C++ speedscould improve performance, we replaced the logistic re-gression benchmark’s Spark Scala code with loops thattake as long as the C++ implementations. This repre-sents the best-case performance of Spark calling into anative method (there is no overhead).

Table 1 shows the results. While the computationaltasks run 51 times faster, on 100 nodes the overall com-putation only runs 2.8 times faster. Worker nodes spendmost of the time idle because the central Spark controllercannot schedule tasks fast enough. Each core can execute250 tasks per second (each task is 4ms), and 100 nodes(800 cores) can execute 200,000 tasks per second. We

2To determine the cause of this slowdown, we configured the JVMto output the JIT assembly and inspected it. We inserted performancecounters in the Scala code re-inspected the assembly to verify they cap-tured the correct operations. To separate the cost of Scala from JVMbytecode interpretation, we decompiled the JVM bytecodes Scala gen-erated into Java, rewrote this code to remove its overheads, recompiledit, and verified that the bytecodes for the computational operations re-mained unchanged.

measured Spark’s controller to be able to issue ≈8,000tasks per second.

This control plane bottleneck is not unique to Spark.Naiad [28] is the best available distributed cloud frame-work. In Naiad, worker nodes directly coordinate withone another rather than acting through a central con-troller. While Naiad code is in C# rather than Scala andso sees overall better performance than Spark, its all-to-all synchronization also becomes a bottleneck above 20nodes. We defer detailed experimental results on Naiadto Section 5.1.

Scheduling techniques such as Sparrow [31],Omega [33], Apollo [12], Mercury [25], Hawk [16] andTarcil [17], address the scheduling bottleneck that occurswhen there are many concurrent jobs. In aggregate,many jobs can execute more tasks per second than asingle controller can schedule. But since these jobsshare underlying computing resources, they need to bescheduled cooperatively to prevent overloading workersor contention. Each of these systems propose ways formany separate, per-job controllers to coordinate theirresource allocation and scheduling decisions. Thesesystems all solve the problem of when the aggregatetask rate of many jobs is greater than what one controllercan handle. Optimized tasks, however, mean that singlejob can saturate a controller. None of these systems candistribute a single job’s scheduling.

2.3 Observation: Repetition

Cloud computing applications are increasingly advanceddata analytics including machine learning, graph pro-cessing, natural language processing, speech/imagerecognition, and deep learning. These applications areusually implemented on top of frameworks such asSpark [38] or Naiad [28], for seamless parallelization andelastic scalability. A recent survey [6] of Spark users,for example, shows 59% of them use the Spark machinelearning library [5]. Efforts such as Apache Mahout [4]and Oryx [9] provide machine learning libraries on topof Spark. Cloud providers, in response to this need, nowoffer special services for machine learning models [8, 1].

One important property of analytics jobs is their com-putations have repetitive patterns: they execute a loop(or set of nested loops) until a convergence condition.The Ernest system [35], for example, leveraged this ob-servation for predicting the performance and managingresources. Logistic regression, for example, often exe-cutes until parameters have converged and are no longerchanging or a large fixed number of iterations (whicheverhappens first).

For example, Figure 2 shows the execution graph ofthe hold-out cross validation method, a common ma-chine learning method used for training regression algo-

3

TrainingData

EsBmaBonData

Parameters

ErrorEsBmaBonIteraBveOpBmizer

IteraBveModelSelecBonCo

efficien

ts

Figure 2: Execution graph of training a regression al-gorithm. It is iterative with an outer loop for updatingmodel parameters based on the estimation error, and aninner loop for optimizing the feature coefficients.

TasksController

Worker

Worker

Worker

Driver

TasksStats

Figure 3: Architecture of a canonical cloud framework:a driver program specifies the application logic for a cen-tralized controller, which drives the worker nodes to ex-ecute tasks.

rithms [20]. It has two stages, training and estimation,which form a nested loop. The training stage uses aniterative algorithm, such as gradient descent, to tune co-efficients. The estimation stage calculates the error of thecoefficients and feeds this back into the next iteration ofthe training phase.

Each iteration generates the same tasks and schedulesthem to the same nodes (those that have the data residentin memory). Re-scheduling each iteration repeats thiswork. This suggests that a control plane cached thesedecisions and reused would schedule tasks much fasterand scale to support fast tasks running on more nodes.The next section describes execution templates, a controlplane abstraction that achieves this goal.

3 Execution TemplatesWe describe execution templates, a control plane abstrac-tion for cloud computing. Execution templates make itpossible for workers to inexpensively generate and exe-cute large batches of tasks. If a program has a loop init, rather than resend all of the tasks for that loop to thecontroller on every iteration, it should instead send themonce. For subsequent iterations, it can tell the controller“execute those tasks again.” The same is true for the con-

while (error > threshold_e) {while (gradient > threshold_g) {// Start 1gradient = Gradient(tdata, coeff, param)coeff += gradient// End 1

}// Start 2error = Estimate(edata, coeff, param)param = update_model(param, error)// End 2

}

Figure 4: Driver program pseudocode for the iterativeapplication in Figure 2. There are two basic blocks.Gradient and Estimate are both parallel operationsthat execute many tasks on partitions of data.

Execution template JIT compiler

Template FunctionTask (Driver→Controller) Bytecode instructionTask (Controller→Worker) Native instructionData object Register

Table 2: Execution templates are analogous to a just-in-time compiler for a data analytics control plane.

troller; rather than resend all of the tasks to the workers,it can tell each worker to “execute those tasks again.”

The execution and control structure of cloud frame-works places requirements on how templates operate.Figure 3 shows the architecture of a cloud computingframework. A driver program generates tasks, which itsends to a centralized controller. The driver and con-troller may or may not reside on the same node. Thecontroller processes these tasks and dispatches them to acluster of workers. The controller balances load acrossworkers and recovers execution when one fails.

Templates optimize repeated control decisions. In thisway, they are similar to a just-in-time (JIT) compiler forthe control plane. A JIT compiler transforms blocks ofbytecodes into native instructions; execution templatestransform blocks of tasks into dependency graphs andother runtime scheduling structures. Table 2 shows thecorrespondences in this an analogy: an execution tem-plate is a function (the granularity JIT compilers typi-cally operate on), a task from the driver to the controlleris a bytecode instruction, and a task executing on theworker is a native instruction.

The rest of this section describes six requirements forhow templates operate. While the analogy to JIT compi-lation fits well and many of these requirements followfrom it, the driver-controller-worker execution modeladds an additional requirement, the need to validate and

4

patch templates before executing them.

1. Templates must be dynamically generated. Con-trollers and workers do not have the driver program.They receive a stream of tasks, which they dynamicallyschedule and execute. They therefore need to generatetemplates in response to this dynamic stream of informa-tion. Furthermore, because a controller can dynamicallyshift how it schedules tasks to workers (e.g., in responseto load imbalance or changing resources), it needs tobe able to correspondingly dynamically create new tem-plates. Put another way, templates cannot be staticallycompiled: they must instead be created just-in-time.

2. Templates must be parameterizable. Similarly tohow a program must be able to pass parameters to just-in-time compiled functions, a driver must be able to pass pa-rameters to execution templates. Analytics jobs involvemany repetitions of the same loop or computation, butthe repetitions are not identical. The cross-validation jobin Figure 2, for example, updates parameters, whichare then passed to the optimizer block. Each instantiationof the optimizer block must fill in parameters to thefind_gradient tasks. In addition to data parameters,templates also require control parameters, such as whichtask identifiers to use, to ensure that two workers do notuse the same globally unique identifier.

3. Workers must locally resolve dependencies. Largeblocks of tasks often have data dependencies betweenthem. For example, the line coeff += gradientin Figure 4 cannot run until the previous line comput-ing gradient completes. For a worker to be able toexecute the tasks for both lines of code locally, withoutcoordinating with the controller, it must know this de-pendency and correctly determine when that line of codecan run. This is similar to how a CPU uses data flow toknow when it can execute an instruction that depends onthe output of other instructions.

4. Workers must directly exchange data. Optimizedtasks read and write in-memory data objects on workers.Often, within a single template, the output of a task onone worker is needed as the input for a task on another.As part of executing the template, the two workers needto directly exchange this data. This is similar to howtwo cores accessing the same memory need to be ableto to update each other’s caches rather than always writethrough to main memory.

5. Controllers must be able to quickly validate andpatch templates. The driver-controller-worker execu-tion model adds additional complexities that JIT com-pilers do not need to handle. Just as function calls as-sume that arguments are in certain registers or stack po-sitions, when a controller generates execution templatesfor workers, it must assume certain preconditions on

where data is located. However, a driver can insert newtasks at any time, which might violate these precondi-tions. For example, it might insert instructions that in-crement a variable and store it in a different register.When a controller instantiates a template, it must vali-date whether a template’s preconditions hold, and if not,insert tasks to patch it. In the above example, the con-troller needs to detect the variable is now in a new reg-ister and issue a move instruction to put it back in theregister the function call expects.

6. Templates must be fast. Finally, as the overall goalof templates is to allow the control plane to support opti-mized tasks at scale, the performance gains of instantiat-ing them must be greater than their cost to generate andinstantiate.

Execution templates are tightly entwined with aframework’s data model and execution. The next sec-tion describes a concrete implementation of them in thecontext of an analytics framework designed to executeoptimized tasks at scale.

4 ImplementationThis section describes the design and implementation ofexecution templates in a C++ analytics framework wehave implemented, called Nimbus. We chose to imple-ment execution templates in a new framework in order toexplore their tradeoffs when not limited by prior designdecisions that might conflict with their goals. There isalso discussion on how execution templates can be intro-duced into existing frameworks.

4.1 NimbusBecause execution templates are tightly entwined with aframework’s data and execution model, we first explainthe relevant details of Nimbus. The core Nimbus imple-mentation is 15,000 semicolons of C++ code.

4.1.1 Data Model

Nimbus has a data flow model similar toDryadLINQ [36], Naiad [28], and Spark [38]. Ajob is decomposed into stages. Each stage is a computa-tion over a set of input data and produces a set of outputdata. Each data set is partitioned into many data objectsso that stages can be parallelized. Each stage typicallyexecutes as many tasks, one per object, that operate inparallel. In addition to the identifiers specifying the dataobjects it accesses, each task can be passed parameters,such as a time-step value or constants.

Unlike Spark’s RDDs, and to avoid the cost of datacopying noted in Section 2.1, Nimbus allows tasks to mu-

5

Parameter: A

C

PA BParameter: PB

Parameter: PCread

no-access

DataAccess:

1 2 3 DataAccess:

1 2 3

DataAccess:

1 2 3read&write

(a) Simple task graph example with three tasks and threedata objects. The data flow among tasks forms a DAG. Forexample, task C reads the updated data objects 2, and 3 afterexecution of task A, and B.

A B

C

Worker1

X

Worker2

PA PB

PC

DataCopy

(b) Mapping of the task graph in Figure 5(a) over two work-ers. Each task graph embeds per worker task dependenciesand data copy among workers. Task graph dependencies al-low workers to proceed without controller’s middling.

Figure 5: Simple task graph example (a) and how it maps into per worker task graph in Nimbus (b).

tate data in place. Mutable data has the additional benefitthat multiple iterations of a loop can access the same ob-jects and reuse their identifiers. This makes templatesmore efficient to parameterize, as the object identifierscan be cached rather than recomputed on each iteration.There can be multiple copies of a data object. However,since objects are mutable they are not always consistent.If one worker writes to its copy of an object, other work-ers who later read it will need to receive the latest update.

Data flow between tasks forms a directed acyclicgraph (DAG), called task graph, whose vertices are tasksand edges are data dependencies. Figure 5(a) shows asimple task graph with three tasks that operate over threedata objects. The rest of this section uses this exampletask graph to explain how templates are generated andinstantiated.

4.1.2 Dependencies and Data Exchange

The goal of execution templates is to allow workers togenerate and correctly schedule large batches of tasks.Not all of these tasks are immediately runnable. For ex-ample, when a worker instantiates the template in Fig-ure 5(a), it cannot run task C until both A and B havecompleted. The ability to locally determine when it issafe to run a task is critical for reducing load on a con-troller; otherwise, the controller would need to publishwhen every task has completed. Workers need to be ableto know this both when the dependent tasks are local aswell as when they run on another node.

To enforce the correct execution order, each task in-cludes a set of dependencies. These dependencies areidentifiers for tasks that must complete before the workerschedules the task. As shown in Figure 5(b), these depen-dencies can also be across workers: task B on worker 2must complete before task C can run on worker 1. Nim-bus represents this dependency by introducing a pair ofcontrol tasks, a send task on worker 2 and a receive taskon worker 1, and inserting the receive task as a depen-

dency in C. These explicit dependencies allow workersto know when a task is ready to run without involvingthe controller.

4.2 Dynamic Template GenerationNimbus generates execution templates on the granularityof basic blocks. A basic block is a code sequence in thedriver program with only one entry point and no branchesexcept at the exit. For example, in Figure 4, there aretwo basic blocks, the optimizer and the estimator. Eachiteration of the algorithm executes the optimizer block(inner loop) multiple times and the estimator block (outerloop) once.

There are two types of execution templates. Con-troller templates are installed on a controller by the driverprogram; they encode the task graph of a basic blockacross all of the workers. Controller templates reducethe control overhead between the driver and the con-troller. Worker templates are installed on workers by thecontroller; they encode the task graph of a basic blockfor that worker. Once a template is installed, it can beinvoked with a single message. When the driver pro-gram invokes a controller template, the controller typi-cally invokes the corresponding worker template on eachworker.

The two types of templates are decoupled to enable dy-namic scheduling decisions. If a worker fails or the con-troller re-balances load across worker, two invocations ofthe same controller template can result in two differentpartitioning of tasks among workers. For every partition-ing strategy a separate worker template is installed on theworkers.

4.2.1 Controller Templates

Controller template stores control dependencies betweentasks as well as which data tasks access. To create a con-troller template, the driver program sends a start message

6

Parameter: t1

t3

P1 t2Parameter: P2

Parameter: P3

Parameters:

TaskIDs: t1p1

t2p2

t3p3

DataAccess:

1 2 3 DataAccess:

1 2 3

DataAccess:

1 2 3read&write

read

no-access

(a) A controller template represents the common structure ofa task graph metadata. It stores task dependencies and dataaccess patterns. It is invoked by filling in task identifiers andparameters to each task.

t1 t2

t3

WorkerI

t4

WorkerII

P1 P2

P3

TaskIDs: t1p1

t3p3

t4 t2p2

t4Parameters:

DataCopy

(b) Each worker template stores the common structure of atask graph for execution including the data copies amongworkers. It is invoked by passing the task identifiers, andparameters to each task.

Figure 6: Controller template (a) and worker templates (b) for the task graph in Figure 5(a).

to the controller, which instructs to start building a tem-plate. As the controller receives each subsequent task, itboth schedules it normally and adds it to the template.At the end of the basic block, the driver sends a finishmessage to the controller. At this point the controllerprocesses the template it has built and generates workertemplates from it. For the successive instance of the samebasic block, driver only invokes the installed template bypassing the task identifiers and parameters to each task.Figure 6(a) shows how the controller templates for thegraph in Figure 5(a) are installed and invoked.

4.2.2 Worker Templates

A worker template has two halves. The first half existsat the controller and represents the entire computationacross all of the workers. This centralized half allowsthe controller to cache how the template’s tasks are dis-tributed across workers and which data objects the tasksaccess. The second half is distributed among the workersand caches the per-worker local task graph with depen-dencies and data exchange directives.

As with controller templates, to generate a workertemplate the controller sends all tasks to workers ex-plicitly. Workers execute these tasks and simultaneouslybuild a template. The next time the driver program in-vokes the controller template, the controller invokes thetemplates at each worker by passing new task ids, andparameters. Figure 6(b) shows how the controller tem-plates for the scheduling strategy in Figure 5(b) are in-stalled and invoked.

4.3 Patching Worker TemplatesTemplates, when generated, assume that the latest up-dates to data objects are distributed in a certain way. For

example, in Figure 5(b), the template on worker 2 as-sumes that data objects 1 and 3 contain the latest up-date. Since templates are installed dynamically, the run-time does not know the complete control structure of theprogram. It can be that there are code paths which donot leave the latest update in every object. Put in otherwords, the driver may have issued tasks which invalidatethe template’s assumptions. At the same time, the driverprogram does not know where data objects are placed ortasks execute, so cannot correct for these cases.

Because this problem is subtle, we provide an analogybased on JIT compilers. JIT generated blocks of nativeinstructions assume that variables are in particular reg-isters. If the registers do not hold the correct variableswhen the block of native instructions is invoked, thenmove, store, and load instructions must be added so theregisters do hold the correct variables.3

Whenever a template is invoked, the controller needsto first validate whether the corresponding worker tem-plates will operate on the latest updates. If validationfails, the controller patches the template by inserting con-trol tasks that copy the latest updates to the objects. Forexample, if immediately after the template in Figure 6(b)the same template is invoked, then controller needs totransfer the latest update of first object (updated by taskt3) to worker 2 to satisfy the preconditions. Only afterpatching it is safe to invoke the template again.

Validating and patching must be fast, specially whenthere are many workers, data objects, and nodes. Forexample, the complex graphics application in Section 5.5has almost one million data objects. Nimbus uses twooptimizations to make validation and patching fast.

First, for the common case of a template executing3This is one reason why JITs often operate on function boundaries,

since function calling conventions specify how variables must be laidout in registers.

7

twice back to back, the controller ensures that the in-put objects to a template hold the latest updates whenthe template completes. This is especially important forwhen there are small, tight loops: the controller can by-pass both validation and patching. Second, for basicblocks that can be entered from multiple places in theprogram (e.g., the block after an if/else clause), the con-troller generates a separate template for each possiblecontrol flow.

4.4 Load Balancing and Fault Tolerance

Nimbus balances load across workers by periodicallycollecting performance statistics at the controller. Whenthe controller detects that certain workers are busier thanothers, it redistributes tasks across the workers, regener-ating templates for any workers whose load has changed.

To recover from worker failures, the Nimbus con-troller periodically checkpoints system state. To create acheckpoint, the controller inserts tasks that commit dataobjects to durable storage as well as metadata on wherein program execution this checkpoint is. If a worker failsand the system loses the latest update to an object, thecontroller halts all tasks on the workers. It restores thelost objects to their last checkpoint as well as any otherobjects which have been modified since that checkpoint.It then restarts execution, regenerating any worker tem-plates as needed. If the controller fails, it can restart andrestore the entire system from the checkpoint.

4.5 Templates in Other Frameworks

Templates are a general abstraction that can be appliedto many frameworks. However, the requirements in Sec-tion 3 can be simpler to incorporate in some systemsthan others. For example, incorporating execution tem-plates into Spark would require three significant changesto its data model and execution model, particularly itslazy evaluation and scheduling. First, it would need tosupport mutable data objects. When data is immutable,each execution of a template is on new data object iden-tifiers. Second, the Spark controller needs to be able toproactively push updates to each worker’s block man-ager. Otherwise, every access of a new data object re-quires a lookup at the controller. Third, in Spark the con-troller is completely responsible for ensuring tasks runin the correct order, and so tasks sent to workers do notcontain any dependency information. Adding executiontemplates would require adding this metadata to tasks aswell as worker control logic. While these changes are allquite tractable, together they involve a significant changeto Spark’s core execution model and so we are beginningto discuss this with its developers.

We are have not yet considered adding templates toNaiad since it is no longer actively supported (the lastcode commit was Nov 9, 2014).

5 EvaluationThis section evaluates how execution templates can sup-port fast, optimized data analytics jobs at scale. It com-pares the performance of k-means and logistic regres-sion benchmarks implemented in Nimbus with imple-mentations in Spark and Naiad. It measures the costs ofcomputing and installing templates as well as the perfor-mance effect of needing to recompute worker templatesdue to load re-balancing. Finally, it evaluates how farexecution templates can scale by measuring their effecton a distributed graphics workload whose median tasklength is 13ms and 10th percentile task length is 3ms.

In summary, our findings show:

• Execution templates support orders of magnitude moretasks per second than existing centralized (Spark) anddecentralized (Naiad) designs. Task throughput scalesalmost linearly with the number of workers.

• Using execution templates, Nimbus is able to run lo-gistic regression and k-means benchmarks 16-43 timesfaster than Spark and Naiad implementations.

• Half of this performance benefit is from optimizedtasks, the other half is from execution templatesscheduling optimized tasks at scale. If Spark and Na-iad use optimized tasks, they cannot scale out past 20nodes; execution templates allow Nimbus to scale outto at least 100 nodes and cut completion times by afactor of 4-8.

• Using execution templates, Nimbus is able to run acomplex graphical simulation with tasks as short as100µs within 15% of the performance of a hand-tunedMPI implementation. Without templates, completiontime increases by 520% as the controller cannot sched-ule tasks quickly enough.

All experiments use Amazon EC2 compute-optimizedinstances since they are the most cost effectivefor compute-bound workloads. Worker nodes arec3.2xlarge instances, which have 8 virtual cores and15GB of RAM. Because we wish to evaluate how thecontroller can become a bottleneck, we run it on a morepowerful instance than the workers, a c3.4xlarge in-stances, with 16 cores and 30GB of RAM. This showsthe performance of the controller even when it has moreresources than the workers. We measure completion timeof different jobs on 20–100 worker nodes. Nodes are al-located within a placement group and so have full bisec-tion bandwidth.

8

20 50 100 20 50 100 20 50 100 20 50 100 20 50 100

Nimbus Spark Naiad Spark-opt Naiad-opt

Number of Workers

0

1

2

3

Itera

tion t

ime (

s)

0.2

14

0.0

96

0.0

66

2.8

6

1.9

81.0

41.2

8

0.5

30.5

81.0

0

0.2

83

0.3

52

0.9

5

12.5

65.3

0

Runtime Overhead

Task Execution Time

(a) Logistic regression

20 50 100 20 50 100 20 50 100 20 50 100 20 50 100

Nimbus Spark Naiad Spark-opt Naiad-opt

Number of Workers

0

1

2

3

Itera

tion t

ime (

s)

0.3

39

0.1

49

0.0

90

3.9

52.6

6

3.9

42.0

72.5

4

0.5

10.5

80.9

6

0.4

80.6

91.8

6

9.6

6

Runtime Overhead

Task Execution Time

(b) K-means clustering

Figure 7: Iteration time of logistic regression and k-means for a data set of size 100GB. Spark, Naiad and Nimbus, runScala, C# and C++ respectively. Spark-opt and Naiad-opt show the performance when the computations are replacedwith spin-wait as fast as tasks in C++. Execution templates helps Nimbus scale out almost linearly.

Iteration time is averaged over 30 iterations and ex-cludes the first iteration due to its overhead of data load-ing and JIT compilation. We observed negligible vari-ance in iteration times. For Nimbus, the first iterationincludes the cost of template installation. We thereforequantify this cost separately from overall performance.

5.1 Data Analytics BenchmarksFigure 7 shows the completion time for logistic regres-sion and k-means when run in Spark, Naiad and Nimbus.In addition to a Scala implementation in Spark and a C#implementation in Naiad, we also measure performanceif these frameworks could execute tasks as quickly asNimbus. We consider the best case performance of nooverhead for invoking native code by having them run abusy loop.

For logistic regression, Naiad’s C# runs 6 times fasterthan Spark’s Scala. The fastest Spark configuration is100 nodes, while for Naiad it is 50 nodes. This is be-cause Naiad’s faster task execution means its controlplane overhead overwhelms the benefits of running onmore workers. Naiad’s control overhead grows quicklybecause it requires O(n2) communication among Naiadnodes, where n is the number of nodes.

C++ tasks run 51 times faster than Scala and 9 timesfaster than C#. When Spark and Naiad’s tasks are re-placed by tasks running as quickly as C++ code, neitherscale out past 20 nodes. We ran them on fewer than 20nodes: 20 is the fastest configuration. For example, run-ning on 100 nodes, Naiad-opt runs almost 3 times slowerthan on 50 nodes, as its n2 coordination overhead grows.

Nimbus runs 43 times faster than Spark and almost 16times faster than Naiad. Its control overhead is almostnegligible, even when scaled out to 100 nodes. This al-lows it to come very close to the expected performancebenefits of C++. Even if Spark and Naiad were to runoptimized tasks, execution templates lead Nimbus to run4-8 times faster.

10 20 30 40 50 60 70 80 90 100Number of Workers

0

50

100

150

200

Task

Thro

ughput

(Thousa

nds

per

seco

nd)

Nimbus

Naiad

Spark

Figure 8: Task throughput of cloud frameworks as thenumber of workers increases. Spark and Naiad saturateat about 8,000 tasks per second, while Nimbus grows al-most linearly as the number of workers increases.

K-means shows similar results to logistic regression:Nimbus runs almost 30 times faster than Spark withScala and 23 times faster than Naiad with C#. It runs5 times faster than Spark or Naiad even when they useoptimized tasks.

5.2 Task ThroughputThe results in Figure 7 show that neither Naiad nor Sparkcan scale out to handle optimized tasks at scale. Sinceprogress bottlenecks at the controller, workers spend alarger fraction of time idle. Figure 8 shows the taskthroughput (the number of tasks per second that work-ers execute) each system sustains for logistic regres-sion. Both Spark and Naiad saturate at about 8,000tasks per second. Using execution templates, Nimbus isable to scale almost linearly, supporting almost 200,000tasks/second for 100 nodes.

Execution templates scale slightly sub-linearly be-cause the scheduling cost at the controller increases lin-early with the number of workers. If these benchmarkswere run on 800 workers with 1 core each (rather than100 workers with 8 cores each), each worker template

9

10 15 20 25 30 35Iteration Index

0

1

2

Itera

tion

Tim

e(s

)manually disabled templates

installing controller template on controllerinstalling preconditions on controller for running on 100 workers

installing worker templates on workers

resource manager evicts half of the workers from the cluster

installing preconditions on controller for running on 50 workers

installing worker templates on workers

resource manager brings all the workers back

Figure 9: Adaptive behavior of execution templates as resources change. If the number of available workers changes,a controller can recompute new templates or fall back to templates it has already computed.

Per-task cost Iter. overhead

Controller Template 25µs 20%Worker Template @cntrl 15µs 12%Worker Template @work 9µs 7%

Table 3: Costs of installing templates on the first iter-ation of logistic regression running on 100 nodes. Thecost is predominantly at the controller. Nonetheless, theone-time cost of installing templates on the first iterationcauses the iteration to run 39% slower.

Completion time

No templates 1.07sController template only 0.49sWorker & controller template 0.07s

Table 4: Execution time of logistic regression iterations(100 nodes) with and without templates.

would be 1/8th the size and the controller would have toprocess 8 times as many template instantiation messages.If T is the number of tasks to execute in a block andW isthe number of workers, Spark’s controller cost is O(T ),Naiad’s is O(W 2) and execution templates are O(W ).

5.3 Template Overhead and Gains

To filter out the startup cost of the JVM and CLR loadingobject files and just-in-time compilation, the results inFigure 7 do not include the first iteration of either compu-tation. This also excludes out the cost of generating andinstalling templates. Table 3 shows the costs of installingtemplates in logistic regression with 100 workers.

Installing templates increases the execution time of thefirst iteration by 39%. This cost is predominantly at thecontroller, as it must generate both the controller tem-plate as well as the controller half of the worker template.

Processing each task at the controller takes 40µs. A con-troller is therefore limited to processing at most 25,000tasks/second on the first iteration: this is approximately3 times what available controllers can handle.

Table 4 shows how controller and worker templatesreduce control plane costs. Both controller and workertemplates cut the overhead significantly as they trans-form thousands of tasks into a single message. Their ben-efits are roughly equal. A controller template transformstens of thousands of messages from the driver to the con-troller to a single message. Worker templates transformtens of thousands of messages from the controller to theworkers to one message per worker. Together, they re-duce control plane overhead from 93% to negligible.

5.4 Template Adaptation

If a controller decides to re-balance a job across workers,remove workers, or add workers, it must recompute newworker task graphs and install the corresponding tem-plates on workers whose responsibilities have changed.Figure 9 shows the time it takes for each iteration of lo-gistic regression as a cluster manager adjusts the avail-able workers. The run begins with templates disabled: it-erations take 1.07s. On iteration 10, templates are turnedon. This iteration takes 1.2s due to controller templateinstallation (20% overhead). On iteration 11, the con-troller’s half of worker templates is installed. On iter-ation 12, the worker’s half of the worker templates isinstalled. We intentionally separated each phase of thetemplate installation on progressive iterations to showthe cost and gain from each. However, all phases couldoverlap on a single iteration (39% overhead). Once alltemplates are installed, the iteration time drops to 0.07s.

On the 20th iteration, the controller receives a com-mand from a cluster manager to stop using half of itsworkers. This does not change the controller template,but forces the controller to recompute worker templates.It then executes at half speed (0.14s/iteration), until iter-

10

10-4 10-3 10-2 10-1 100

Task length (seconds)

0.0

0.2

0.4

0.6

0.8

1.0C

DF

Figure 10: CDF of task durations in a PhysBAM simu-lation. The median task is 13ms, the 10th percentile is3ms, and some tasks are as short as 100µs.

Figure 11: Still of a PhysBAM simulation of water beingpoured into a glass.

ation 30. At iteration 30, the 50 workers are restored tothe controller. It is then able to go back to using its firstset of templates, which are still cached.

5.5 Complex ApplicationsThis final set of experiments examines how templatesscale to support complex applications. PhysBAM isan open-source library for simulating many phenom-ena in computer graphics [18]. It is the result of over50 developer-years of work and has won two Academy

196.8

Nimbus/wotemplates

Nimbus

MPI

Itera;onTime(s)

36.5

31.7

Figure 12: Iteration time of a PhysBAM water simula-tion in Nimbus with and without templates as well as itsstandard MPI implementation.

Awards. We ported PhysBAM to Nimbus, wrappingPhysBAM functions inside tasks and interfacing Phys-BAM data objects into Nimbus so they can be copiedand transferred.

We wrote a driver program for a canonical fluid simu-lation benchmark, water being poured into a vessel (e.g.,Figure 11). This simulation uses the particle-levelsetmethod [19], maintaining the simulation as a volume offixed grid cells but using particles along the surface ofthe water to simulate it in much higher detail. The sim-ulation is the same core simulation used in films suchas The Perfect Storm and Brave and has a triply-nestedloop with 26 different computational stages that accessover 40 different variables.

We ran a 10243 cell simulation (512GB-1TB of RAM)on 64 workers, comparing the performance of Nimbuswith PhysBAM’s hand-tuned MPI implementation. TheMPI implementation cannot re-balance load, and in prac-tice developers rarely use it due to its brittle behavior andlack of fault tolerance.

Figure 10 shows the CDF of task duration in Phys-BAM. While the main computational tasks are 60-70mssome tasks run for only 100µs. These tasks computingminimum and maximum values over small sets. Fig-ure 12 shows PhysBAM’s performance using Nimbusand MPI. Without templates, the simulation generatestasks 8 times faster than a controller can handle: Nim-bus takes 520% longer than MPI, because controller be-comes a bottleneck. With templates, it runs within 15%of the MPI implementation.

6 Related WorkThis paper builds on a long history of related work fromseveral disparate fields: cloud computing, high perfor-mance computing, and programming languages.

Fast data analytics: Within the database and paral-lel computing communities, prior work has exploredthe computational inefficiency of Spark code, propos-ing new programming models and frameworks to replaceit [26, 13]. Facebook’s AI research group has open-sourced GPU modules for the Torch machine learningframework. [7]. There is also ongoing research on acommon intermediate language for Spark that providesa glossary of data-parallel optimizations (including vec-torization, branch flattening and, prediction), suggestingperformance in some cases even faster than, hand-writtenC [32, 37]. The trend shows that the next generation ofcloud computing frameworks will execute tasks whichrun orders of magnitude faster than today.

Cloud programming frameworks: MapReduce [15] isa widely used programming model for processing large

11

data sets. Open source MapReduce frameworks such asHadoop and Hive [2, 3] are I/O bound: they fetch sta-ble input data from disks, and save intermediate and fi-nal results to disks. Spark [38] uses resilient distributeddatasets (RDDs) to perform computations on in-memorydata, while providing the reliability that data on disk pro-vides. For optimized data analytics with short task, how-ever, Spark’s centralized runtime system becomes a bot-tleneck. While Nimbus also uses a centralized controller,execution templates enable Nimbus to handle orders ofmagnitude higher task rate.

Naiad [28] is another framework for in-memory com-putations. While the distributed event-based runtimehelps scalability without creating a centralized bottle-neck, the cost of synchronization dominates as the num-ber of workers grows. Logical to physical graph transla-tion on Naiad nodes resembles the worker templates onNimbus, however the lack of centralized controller to re-solves the inter-worker dependencies leaves the burdenof synchronization on the runtime system.

Dataflow frameworks such as Dryad [23],DryadLINQ [36], CIEL [29] and FlumeJava [14]focus on abstractions for parallel computations thatenable optimizations and high performance. This paperexamines a different but complementary question: howthe runtime scales out to support very fast computations.In fact, our framework implementation that incorporatesexecution templates, Nimbus, resembles the data flowmodel in DryadLINQ [36].

Distributed scheduling systems: There is a huge bodyof work on distributed scheduling. They deploy var-ious mechanisms to provide efficient scheduling deci-sions with high throughput. For example, Sparrow [31]uses a stateless scheduling model based on batch sam-pling and late binding. Omega [33], on the other hand,leverages a shared global state through atomic transac-tions to improve the allocation decisions. Apollo [12]benefits from a similar model, and adds task completiontime estimations to optimize the scheduling decisions.Tarcil [17] is a hybrid model based on both sampling andperformance estimation. Hawk [16], and Mercury [25]suggest a hybrid distribute/centralized solution to realizebetter efficiency in the cluster.

At a very high level, all these systems solve the sameproblem as execution templates do: providing highertask throughput at the runtime system. However thereis a very important and subtle difference: these systemsdistribute the scheduling across the job boundaries. Fora single job with high task rate the scheduling still goesthrough a single node. The distributed solution onlysolves the problem of multiple jobs producing high ag-gregate task rate in the cluster by directing the schedulingof each job to a different node. In a way, execution tem-plates are orthogonal to theses systems. Every node in

the distributed implementation could benefit from execu-tion templates to support jobs with orders of magnitudehigher task rate.

High performance computing (HPC): MPI [34] pro-vides an interface to exchange messages between par-allel computations, and is the most commonly usedframework to write distributed computations in theHPC domain. MPI does not include any support forload-balancing or fault recovery. Frameworks such asCharm++ [24] and Legion [11] provide abstractions todecouple control flow, computation and communication,similar to cloud data flow frameworks. Their fundamen-tal difference, however, is that they provide mechanismsand very little policy; applications are expected to de-cide on how their data is placed as well as were tasksrun. The scale and cost of the machines they are designedfor (supercomputers) is such that they demand more pro-grammer effort in order to achieve more fine-tuned andoptimized use of hardware resources.

Just-in-time (JIT) compilation: Finally, the idea ofmemoizing control flow and dynamic decisions in anexecution path closely resembles the approach taken injust-in-time (JIT) compilers [10] as well as the Synthesiskernel [27]. Both of these approaches note that particulardecisions, while dynamic in the general case, might leadto deterministic results in any particular case. Therefore,optimizing that deterministic result can remove all of thesurrounding overhead. While a JIT compiler and theSynthesis kernel generate optimized native code for par-ticular functions, execution templates generate optimizedstructures for scheduling distributed computations.

7 Conclusion And Future WorkThis paper presents execution templates, a novel abstrac-tion for cloud computing runtime systems that allowsthem to support extremely high task rates. The need forhigh task rates is driven by the observation that manymodern workloads are CPU-bound, and rewriting themin high performance code can easily lead to task ratesthat overwhelm modern schedulers. Long-running appli-cations with high task rates, however, usually consist ofmany executions of a loop. Rather than reschedule eachiteration of the loop from scratch, execution templatesallow a controller to cache its scheduling decisions andinvoke large, complex sets of tasks on worker nodes witha single message. Using execution templates, the papershows that some benchmark applications reimplementedin C++ can run up to 40 times faster; without templates,their speedup is limited only a factor of 5. Finally, execu-tion templates enable whole new classes of applicationsto run in the cloud, such as high performance simulationsused in computer graphics.

12

References[1] Amazon Machine Learning. https://aws.

amazon.com/machine-learning/.

[2] Apache Hadoop. http://wiki.apache.org/hadoop.

[3] Apache Hive. http://hadoop.apache.org/hive.

[4] Apache Mahout. http://mahout.apache.org/.

[5] Apache Spark MLLib. https://spark.apache.org/mllib/.

[6] Apache Spark, Preparing for the Next Wave of Re-active Big Data. http://goo.gl/FqEh94.

[7] Facebook AI Research open sources deep-learningmodules for Torch. https://research.facebook.com/blog/fair-open-sources-deep-learning-modules-for-torch/.

[8] Microsoft Azure Machine Learning.https://azure.microsoft.com/en-us/services/machine-learning/.

[9] Oryx 2. http://oryx.io/.

[10] V. Bala, E. Duesterwald, and S. Banerjia. Dy-namo: a transparent dynamic optimization system.In ACM SIGPLAN Notices, volume 35, pages 1–12.ACM, 2000.

[11] M. Bauer, S. Treichler, E. Slaughter, and A. Aiken.Legion: Expressing locality and independence withlogical regions. In High Performance Computing,Networking, Storage and Analysis (SC), 2012 Inter-national Conference for, pages 1–11. IEEE, 2012.

[12] E. Boutin, J. Ekanayake, W. Lin, B. Shi, J. Zhou,Z. Qian, M. Wu, and L. Zhou. Apollo: scalable andcoordinated scheduling for cloud-scale computing.In 11th USENIX Symposium on Operating SystemsDesign and Implementation (OSDI 14), pages 285–300, 2014.

[13] K. J. Brown, H. Lee, T. Rompf, A. K. Sujeeth,C. De Sa, C. Aberger, and K. Olukotun. Have ab-straction and eat performance, too: Optimized het-erogeneous computing with parallel patterns. InProceedings of the 2016 International Symposiumon Code Generation and Optimization, pages 194–205. ACM, 2016.

[14] C. Chambers, A. Raniwala, F. Perry, S. Adams,R. R. Henry, R. Bradshaw, and N. Weizenbaum.Flumejava: easy, efficient data-parallel pipelines.In ACM Sigplan Notices, volume 45, pages 363–375. ACM, 2010.

[15] J. Dean and S. Ghemawat. Mapreduce: simplifieddata processing on large clusters. Communicationsof the ACM, 51(1):107–113, 2008.

[16] P. Delgado, F. Dinu, A.-M. Kermarrec, andW. Zwaenepoel. Hawk: hybrid datacenter schedul-ing. In 2015 USENIX Annual Technical Conference(USENIX ATC 15), pages 499–510, 2015.

[17] C. Delimitrou, D. Sanchez, and C. Kozyrakis. Tar-cil: reconciling scheduling speed and quality inlarge shared clusters. In Proceedings of the SixthACM Symposium on Cloud Computing, pages 97–110. ACM, 2015.

[18] P. Dubey, P. Hanrahan, R. Fedkiw, M. Lentine, andC. Schroeder. Physbam: Physically based simu-lation. In ACM SIGGRAPH 2011 Courses, SIG-GRAPH ’11, pages 10:1–10:22, New York, NY,USA, 2011. ACM.

[19] D. Enright, R. Fedkiw, J. Ferziger, and I. Mitchell.A hybrid particle level set method for improvedinterface capturing. Journal of ComputationalPhysics, 183(1):83–116, 2002.

[20] J. Friedman, T. Hastie, and R. Tibshirani. The Ele-ments of Statistical Learning: Data Mining, Infer-ence, and Prediction, volume 1. Springer series instatistics Springer, 2009.

[21] L. Gherardi, D. Brugali, and D. Comotti. A java vs.c++ performance evaluation: a 3d modeling bench-mark. In Simulation, Modeling, and Programmingfor Autonomous Robots, pages 161–172. Springer,2012.

[22] R. Hundt. Loop recognition in c++/java/go/scala.Proceedings of Scala Days, 2011:38, 2011.

[23] M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fet-terly. Dryad: distributed data-parallel programsfrom sequential building blocks. In ACM SIGOPSOperating Systems Review, volume 41, pages 59–72. ACM, 2007.

[24] L. V. Kale and S. Krishnan. CHARM++: aportable concurrent object oriented system basedon C++, volume 28. ACM, 1993.

[25] K. Karanasos, S. Rao, C. Curino, C. Douglas,K. Chaliparambil, G. M. Fumarola, S. Heddaya,

13

R. Ramakrishnan, and S. Sakalanaga. Mercury:Hybrid centralized and distributed scheduling inlarge shared clusters. In 2015 USENIX AnnualTechnical Conference (USENIX ATC 15), pages485–497, 2015.

[26] J. Liu and S. J. Wright. Asynchronous stochas-tic coordinate descent: Parallelism and conver-gence properties. SIAM Journal on Optimization,25(1):351–376, 2015.

[27] H. Massalin and C. Pu. Threads and input/output inthe synthesis kernal. In Proceedings of the TwelfthACM Symposium on Operating Systems Principles,SOSP ’89, pages 191–201, New York, NY, USA,1989. ACM.

[28] D. G. Murray, F. McSherry, R. Isaacs, M. Isard,P. Barham, and M. Abadi. Naiad: a timely dataflowsystem. In Proceedings of the Twenty-Fourth ACMSymposium on Operating Systems Principles, pages439–455. ACM, 2013.

[29] D. G. Murray, M. Schwarzkopf, C. Smowton,S. Smith, A. Madhavapeddy, and S. Hand. Ciel:A universal execution engine for distributed data-flow computing. In NSDI, volume 11, pages 9–9,2011.

[30] K. Ousterhout, R. Rasti, S. Ratnasamy, S. Shenker,B.-G. Chun, and V. ICSI. Making sense of perfor-mance in data analytics frameworks. In Proceed-ings of the 12th USENIX Symposium on NetworkedSystems Design and Implementation (NSDI), pages293–307, 2015.

[31] K. Ousterhout, P. Wendell, M. Zaharia, and I. Sto-ica. Sparrow: distributed, low latency scheduling.In Proceedings of the Twenty-Fourth ACM Sympo-sium on Operating Systems Principles, pages 69–84. ACM, 2013.

[32] S. Palkar, J. Thomas, and M. Zaharia. Nested vectorlanguage: Roofline performance for data parallelcode. http://www.mit.edu/~jjthomas/nvl-poster.pdf.

[33] M. Schwarzkopf, A. Konwinski, M. Abd-El-Malek,and J. Wilkes. Omega: flexible, scalable schedulersfor large compute clusters. In Proceedings of the8th ACM European Conference on Computer Sys-tems, pages 351–364. ACM, 2013.

[34] M. Snir. MPI–the Complete Reference: The MPIcore, volume 1. MIT press, 1998.

[35] S. Venkataraman, Z. Yang, M. Franklin, B. Recht,and I. Stoica. Ernest: Efficient performance predic-tion for large-scale advanced analytics. In Proceed-ings of the 13th USENIX conference on NetworkedSystems Design and Implementation. USENIX As-sociation, 2016.

[36] Y. Yu, M. Isard, D. Fetterly, M. Budiu, Ú. Erlings-son, P. K. Gunda, and J. Currey. Dryadlinq: Asystem for general-purpose distributed data-parallelcomputing using a high-level language. In OSDI,volume 8, pages 1–14, 2008.

[37] M. Zaharia. New developments in sparkand rethinking apis for big data. http://platformlab.stanford.edu/Seminar%20Talks/stanford-seminar.pdf.

[38] M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma,M. McCauley, M. J. Franklin, S. Shenker, andI. Stoica. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster comput-ing. In Proceedings of the 9th USENIX conferenceon Networked Systems Design and Implementation,pages 2–2. USENIX Association, 2012.

14