SEUSS: Skip Redundant Paths to Make

Serverless Fast

The Proceedings of EuroSys, 2020 April 29th, 2020

James Cadden, Thomas Unger, Yara Awad, Han Dong, Orran Krieger, Jonathan Appavoo

Department of Computer Science Boston University

Isolated Execution Environments

Application database.pyf2

.rbf3

.jsfn…

Platform API

.jsf1

Compute Resources

1. Function-as-a-Service (FaaS): on-demand execution of a client code snippet (functions)

2. Applications are deployed and scaled automatically

3. Function start time is dominated by deterministic setup & install paths⇝

E1

1. event

2. pull app

3. setup & install

4. Run!

Serverless Computing

Isolated Execution Environments

Application database.pyf2

.rbf3

.jsfn…

Platform API

.jsf1

Compute Resources

4. Functions deploy quickly using a pre-initialized environment!

⇝

1. event

2. Run!

E1

Serverless Computing

Isolated Execution Environments

Function database.pyf2

.rbf3

.jsfn…

FaaS API

.jsf1

Compute Resources

5. FaaS platform utility becomes a matter of cache efficiency!

6. Mechanism of the system-level defines the security, cache density, and responsiveness

⇝

P1 EXECUTION CACHE

Serverless Computing

Machine Density Start Time

Linux Process 4200 0.3 s

Docker Container 3000 0.5 to 4 s

Container in a MicroVM 450 1 to 7 sLinux v4.15; Docker v18.09; [Xeon 2.20 GHz; 88GB]

less overhead

more overhead

lesssecure

more secure

P

VM

C

P

C

VM

Node.js “launcher” provides a REST API to import and run an arbitrary JavaScript function

Cache Primitive:

P .jsf1

FaaS Environment Caching

Cadden, et al. “Skip Redundant Paths to Make Serverless Fast”, In The Proceedings of EuroSys ‘20

Serverless Executionvia Unikernel SnapShots

Is there… a method to better enable reuse across the entire memory footprint of the function?

We want to… 1. Shorten the setup time of new function invocations

2. Improve cache density for fast repeat invocations

1. Unikernels support strong isolation semantics

2. Enable “black box” capture of environment’s memory footprint into an snapshot (object)

3. Page-level sharing can be applied ubiquitously across the application and kernel layers

Cadden, et al. “Skip Redundant Paths to Make Serverless Fast”, In The Proceedings of EuroSys ‘20

Serverless Executionvia Unikernel SnapShots

lightweight kernel

F1 F2 F3 F4

Unikernels

shared libsfilesystem

Function #1+

Language Runtime

single address space

TCP/IP VMM scheduler

hardware-lvl interface

What is a unikernel?

In SEUSS, functions are deployed inside of dedicated unikernels

SEUSS operating system

snapshot cache

cold warm

hot

time

construct environment

initialize runtime

import run arguments

startrun

import code

generatebytecode

initialization on boot

7.67 ms2.95 ms0.82 ms

12 MB

IO core core 1 core 2 core 3 core 4

requestsreplies

runtime snapshots

9 MB

4 MB

211 MB

function snapshots

unikernelbinaries

7.67 MB115 MB

21 430

time

coldstart

1P

.jsf1

P

Functions code & libraries

⇝

1

{`foo`}

Function invocation times are dominated by deterministic import & initialization procedures

Environment Snapshots

Snapshots captured at strategic points in time can be used as templates for deploying execution

A B

SEUSS operating system

snapshot cache

cold warm

hot

time

construct environment

initialize runtime

import run arguments

startrun

import code

generatebytecode

initialization on boot

7.67 ms2.95 ms0.82 ms

12 MB

IO core core 1 core 2 core 3 core 4

requestsreplies

runtime snapshots

9 MB

4 MB

211 MB

function snapshots

unikernelbinaries

7.67 MB115 MB

21 430

time

1P

.jsf1

⇝

1

{`foo`}.jsf1

P

Functions code & libraries

Immutable snapshot images acts as a reusable launch point for new function invocations

warmstart

Environment Snapshots

runtime snapshot used for new invocations

SEUSS operating system

snapshot cache

cold warm

hot

time

construct environment

initialize runtime

import run arguments

startrun

import code

generatebytecode

initialization on boot

7.67 ms2.95 ms0.82 ms

12 MB

IO core core 1 core 2 core 3 core 4

requestsreplies

runtime snapshots

9 MB

4 MB

211 MB

function snapshots

unikernelbinaries

7.67 MB115 MB

21 430

time

1P

.jsf1

⇝

1

{`foo`}

Environment Snapshots

.jsf1

P

Functions code & libraries

hot start

function snapshot used for repeat invocations

Function-specific snapshots provide the near-immediate execution of function bytecode

SEUSS operating system

snapshot cache

cold warm

hot

time

construct environment

initialize runtime

import run arguments

startrun

import code

generatebytecode

initialization on boot

7.67 ms2.95 ms0.82 ms

12 MB

IO core core 1 core 2 core 3 core 4

requestsreplies

runtime snapshots

9 MB

4 MB

211 MB

function snapshots

unikernelbinaries

7.67 MB115 MB

21 430

time

Snapshot Lineages

Registers

Pages

Page-level sharing & copy-on-write (COW) can be applied to drastically reduce replicated state

Page refs

runtime snapshot function snapshot

Written page

SEUSS operating system

snapshot cache

cold warm

hot

time

construct environment

initialize runtime

import run arguments

startrun

import code

generatebytecode

initialization on boot

7.67 ms2.95 ms0.82 ms

12 MB

IO core core 1 core 2 core 3 core 4

requestsreplies

runtime snapshots

9 MB

4 MB

211 MB

function snapshots

unikernelbinaries

7.67 MB115 MB

21 430

time

Snapshot Lineages

Registers

Pages

Page refs

Anticipatory optimization enabled by accumulating state within the origin snapshot

Written page

Child snapshots contain only a memory ‘diff’ of written pages

SEUSS OS OS specialized for FaaS compute plane

Snapshot capture region

KVM-QEMU

SEUSS operating system

Ring 3Ring 0

VCPU TX/RX queues

EbbRT LibraryOSvirtio

Rumprun unikernel

Node.js V8 JavaScript engine

invocationdriver.js <*.js>

Solo5

• Foundation event-driven multi-core kernel (x86_64 native)

• Per-core job scheduler & network (NAT) layer

• In-memory snapshot cache

• Unprivileged unikernel guest:

• POSIX0ish unikernel (Rumprun)

• Minimal domain interface (Solo5)

Compute Plane

API Server

Control Plane

Controller

Benchmark

B

• 3-node OpenWhisk cluster • custom benchmark tool

• Functions run in Docker containers

Internal Databases

FaaS Performance(single node)

• 12-core, 88GB nodes

FaaS Platform Cache

Star

t Tim

e (m

s)350

700

1050

1400

1 4 16 64 256 1024 4096No. of functions

cache limit

cache thrashing

using Docker containers

time

f():

f(),g(), h(),i():

Sequential invocation requests to an Apache OpenWhisk compute node

timex = 1

x = 4

Report the average start time

(Linux v4.15; Xeon 2.20 GHz; 88GB)

FaaS Platform Cache

Star

t Tim

e (m

s)350

700

1050

1400

No. of functions

1 4 16 64 256 1024 4096

vs. unikernel snapshotsusing Docker containers

time

f():

f(),g(), h(),i():

Sequential invocation requests to an Apache OpenWhisk compute node

timex = 1

x = 4

Report the average start time

Invo

catio

n G

oodp

ut (R

eq/s

)

• 64 concurrent requests

• NOP (‘hello world) invocations

No. of function (log scale)

FaaS Platform Throughput

Resiliency to Traffic Bursts

blue/purple: Blocking IO requests to an external HTTP host (~250ms)

red: 128 concurrent CPU-bound functions (~150 ms)

32-second Burst Intervals 16-second Burst Intervals

Final Thoughts• Unikernel snapshots promote reuse

in a safe, simple, and effective way

• Prototype demonstrates a major advantage for serverless applications models

• In the end, high-performance cloud computing will continue to challenged our infrastructure software in new

• It will be the operating system (design, mechanisms & techniques) that will address challenge and enable new workloads

lightweight kernel

F1 F2 A F2 B …

SEUSS OS

Unikernels contexts

snap0

snap1 snap2 read-onlysnapshots