noria - usenix...frontend!4 stories votes 90% reads 10% writes join count filter query. backend...

Dynamic, partially-stateful data-flow forhigh-performance Web applications

Jon Gjengset

Jonathan Behrens Lara Timbó Araújo Martin Ek

Eddie Kohler M. Frans Kaashoek Robert Morris

Malte Schwarzkopf

Noria

!2

Frontend!2

🌎 Frontend!2

Backend

🌎 Frontend!2

Backend

Frontend

!3

Backend

Frontend

!3

Stories Votes

Backend

Frontend

!4

Stories Votes

JOIN

COUNT

FILTERQuery

Backend

Frontend

!4

Stories Votes

JOIN

COUNT

FILTERQuery

Backend

Frontend

!4

Stories Votes

90% reads10% writes

JOIN

COUNT

FILTERQuery

Backend

Frontend

!4

Stories Votes

Slow reads, repeated work!

☹90% reads

10% writesJOIN

COUNT

FILTERQuery

Frontend

!5

Stories Votes

Precomputed results

22

JOIN

COUNT

FILTERQuery

Frontend

!5

Stories Votes

Precomputed results

22

READ

JOIN

COUNT

FILTERQuery

Frontend

!5

Stories Votes

Precomputed results

22

READ

JOIN

COUNT

FILTERQuery

Store in base table?— manual, slow.

Frontend

!5

Stories Votes

Precomputed results

22

READ

JOIN

COUNT

FILTERQuery

Store in base table?— manual, slow.

memcached?— complex [Facebook NSDI’13].

Frontend

!5

Stories Votes

JOIN

COUNT

FILTER

22

Streamingdata-flow?

Store in base table?— manual, slow.

memcached?— complex [Facebook NSDI’13].

Frontend

!6

Stories Votes

JOIN

COUNT

FILTER

22

Streamingdata-flow?

INSERT

Frontend

!6

Stories Votes

JOIN

COUNT

FILTER

22

Streamingdata-flow?

3

Frontend

!6

Stories Votes

JOIN

COUNT

FILTER

Materializedview

22

Streamingdata-flow?

3

Frontend

!6

Stories Votes

JOIN

COUNT

FILTER

Materializedview

22

Fast reads. Efficient writes. Parallelizes well.

Streamingdata-flow?

3

Frontend

!7

Stories VotesChallenges

JOIN

COUNT

FILTER

23

13

2

Frontend

!7

Stories Votes

‣Change queries? Restart!

ChallengesState-of-the-artdata-flow systems:

JOIN

COUNT

FILTER

23

13

2

Frontend

!7

Stories Votes

‣Change queries? Restart!

ChallengesState-of-the-artdata-flow systems:

JOIN

COUNT

FILTERSUM 4

2#

$

42

#

$

23

2

31

Frontend

!7

Stories Votes

‣Change queries? Restart!‣Memory footprint? Grows!

ChallengesState-of-the-artdata-flow systems:

JOIN

COUNT

FILTERSUM 4

2#

$

42

#

$

23

2

31

Frontend

!8

Stories VotesNoria

Frontend

!8

Stories Votes

JOIN

COUNT

FILTER

32

Noria

3

21

Frontend

!8

Stories Votes

JOIN

COUNT

FILTER

32

‣Change queries? Live.

Noria

3

21

Frontend

!8

Stories Votes

JOIN

COUNT

FILTER

32

‣Change queries? Live.

Noria

42

#

$

42

#

$

SUM

3

21

Frontend

!8

Stories Votes

JOIN

COUNT

FILTER

32

‣Change queries? Live.‣Memory footprint? Bounded.

Noria

42

#

$

42

#

$

SUM

3

21

Frontend

!8

Stories Votes

JOIN

COUNT

FILTER

3

‣Change queries? Live.‣Memory footprint? Bounded.

Noria

42

#

$

42

#

$

SUM

3

Frontend

!8

Stories Votes

JOIN

COUNT

FILTER

3

‣Change queries? Live.‣Memory footprint? Bounded.‣No global coordination.

Noria

42

#

$

42

#

$

SUM

3

New model: Partially-stateful data-flow

!9

!10

Stories Votes

JOIN

COUNT

FILTER

3

13

2

2

Partially-stateful data-flow

Data-flow state is partial: entries for some keys are absent ( ).

Frontend

!10

Stories Votes

JOIN

COUNT

FILTER

3

13

2

Partially-stateful data-flow

Data-flow state is partial: entries for some keys are absent ( ).

Frontend

!10

Stories Votes

JOIN

COUNT

FILTER

3

3

2

Partially-stateful data-flow

Data-flow state is partial: entries for some keys are absent ( ).

Frontend

!10

Stories Votes

JOIN

COUNT

FILTER

3

3

2

Partially-stateful data-flow

Data-flow state is partial: entries for some keys are absent ( ).

Lower memory footprint.

Frontend

!10

Stories Votes

JOIN

COUNT

FILTER

3

3

2

Partially-stateful data-flow

Data-flow state is partial: entries for some keys are absent ( ).

Lower memory footprint.No need to update absent entries.

Frontend

!10

Stories Votes

JOIN

COUNT

FILTER

3

3

2

Partially-stateful data-flow

Data-flow state is partial: entries for some keys are absent ( ).

Lower memory footprint.No need to update absent entries.Enables live data-flow changes.

Frontend

!11

Stories Votes

JOIN

COUNT

FILTER

3

3

2

Partially-stateful data-flow: upqueries

READ

Frontend

!11

Stories Votes

JOIN

COUNT

FILTER

3

3

2

Partially-stateful data-flow: upqueries

??? Need to fill absent entry!READ

Frontend

!11

Stories Votes

JOIN

COUNT

FILTER

3

3

2

Partially-stateful data-flow: upqueries

??? Need to fill absent entry!READ

Solution: upquery through data-flow.• Compute missing entry from

upstream state

Frontend

Frontend

!12

Stories Votes

JOIN

COUNT

FILTER

3

3

2

Partially-stateful data-flow: upqueries

Solution: upquery through data-flow.• Compute missing entry from

upstream state • Response fills missing entry

READ

Frontend

!12

Stories Votes

JOIN

COUNT

FILTER

3

3

2

Partially-stateful data-flow: upqueries

Solution: upquery through data-flow.• Compute missing entry from

upstream state • Response fills missing entry

2

READ

!13

Start new views and operator state empty, fill via upqueries.

Partial state enables live data-flow changes

Stories Votes

JOIN

COUNT

FILTER

32

3

21

Frontend

!13

#

$

SUM

#

$

Start new views and operator state empty, fill via upqueries.

Partial state enables live data-flow changes

Stories Votes

JOIN

COUNT

FILTER

32

3

21

Frontend

!13

#

$

SUM

#

$

Start new views and operator state empty, fill via upqueries.

Partial state enables live data-flow changes

Stories Votes

JOIN

COUNT

FILTER

32

3

21

Frontend

READ #

!13

#

$

SUM

#

$

Start new views and operator state empty, fill via upqueries.

Partial state enables live data-flow changes

Stories Votes

JOIN

COUNT

FILTER

32

3

21

Frontend

READ #

!13

#

$

SUM

#

$

Start new views and operator state empty, fill via upqueries.

4

4

Partial state enables live data-flow changes

Stories Votes

JOIN

COUNT

FILTER

32

3

21

Frontend

READ #

!14

#

$

SUM

#

$

4

4

Stories Votes

JOIN

COUNT

FILTER

32

3

21

Frontend

High performance requires concurrency

!14

#

$

SUM

#

$

4

4

Stories Votes

JOIN

COUNT

FILTER

32

3

21

Frontend

High performance requires concurrency

Process operators concurrently.Read from views concurrently.Process shards concurrently.

Without global coordination!

!14

#

$

SUM

#

$

4

4

Stories Votes

JOIN

COUNT

FILTER

32

3

21

Frontend

High performance requires concurrency

Process operators concurrently.Read from views concurrently.Process shards concurrently.

Without global coordination!

!14

#

$

SUM

#

$

4

4

Stories Votes

JOIN

COUNT

FILTER

32

3

21

Frontend

High performance requires concurrency

Process operators concurrently.Read from views concurrently.Process shards concurrently.

Without global coordination!

Challenges implementing partially-stateful data-flow

!15

Challenges implementing partially-stateful data-flow

!15

1. Concurrent upqueries and forward processing — races!

Must maintain correctness under concurrency!

Challenges implementing partially-stateful data-flow

!15

1. Concurrent upqueries and forward processing — races!

Must maintain correctness under concurrency!

Correctness under concurrency

!16

Goal: upquery restores state as if present all along.

Correctness under concurrency

!16

Goal: upquery restores state as if present all along.

COUNT2

Correctness under concurrency

!16

Goal: upquery restores state as if present all along.

12

COUNT2

Correctness under concurrency

!16

Goal: upquery restores state as if present all along.

12

COUNT2

Correctness under concurrency

!16

Goal: upquery restores state as if present all along.

12

COUNT2

Correctness under concurrency

!16

Goal: upquery restores state as if present all along.

12

Upquery response is a snapshot of state

COUNT2

2

Correctness under concurrency

!16

Goal: upquery restores state as if present all along.

12

Upquery response is a snapshot of state

COUNT2

2

includes 12

does not include33

Correctness under concurrency

!16

Goal: upquery restores state as if present all along.

12

Upquery response is a snapshot of state

COUNT2

2

includes 12

does not include

Solution: Maintain order of upquery response and surroundingupdates, despite lack of global coordination.

33

Upquery responses in total order with updates

!17

Goal: upquery restores state as if present all along.

Upquery responses in total order with updates

!17

Goal: upquery restores state as if present all along.

223

3

1

resulting staterespects total order

Upquery responses in total order with updates

!17

Goal: upquery restores state as if present all along.

223

3

1

resulting staterespects total order

232

2

1

resulting stateviolates total order

Upquery responses in total order with updates

!17

Goal: upquery restores state as if present all along.

223

3

1

resulting staterespects total order

232

2

1

resulting stateviolates total order

More complex cases: merged upquery responses, evictions (Paper).

Challenges implementing partially-stateful data-flow

!18

1. Concurrent upqueries and forward processing — races!

2. Update processing may require absent state

Must maintain correctness under concurrency!

Challenges implementing partially-stateful data-flow

!18

1. Concurrent upqueries and forward processing — races!

2. Update processing may require absent state

COUNT

3

2

…

absent

Must maintain correctness under concurrency!

Challenges implementing partially-stateful data-flow

!18

1. Concurrent upqueries and forward processing — races!

2. Update processing may require absent state

COUNT

3

2

…

absent

Must maintain correctness under concurrency!

Drop updates that touch absent state, future upquery repeats them.

Challenges implementing partially-stateful data-flow

!18

1. Concurrent upqueries and forward processing — races!

2. Update processing may require absent state

COUNT

3

2

…

absent

Must maintain correctness under concurrency!

Drop updates that touch absent state, future upquery repeats them.

(see Paper)

!19

Noria implementation

!19

Noria implementation

!19

Noria implementation

MySQL adapter

!19

Noria implementation

Data-flow graph

MySQL adapter

Transform

!19

Noria implementation

Data-flow graph

MySQL adapter

Transform

!19

Noria implementation

Data-flow graph

MySQL adapter

• 45k lines of Rust + 15k libraries• RocksDB for base table storage• ZooKeeper for leader election

Transform

1. Can Noria improve a real web application’s performance?

2. How does Noria compare to alternative approaches?

3. Can Noria change queries without downtime?

!20

Evaluation

1. Can Noria improve a real web application’s performance?

2. How does Noria compare to alternative approaches?

3. Can Noria change queries without downtime?

!20

Evaluation

Amazon EC2 c5.4xlarge instance (16 vCPUs)Open-loop clients, measuring latency & throughput

Setup

1. Can Noria improve a real web application’s performance?

2. How does Noria compare to alternative approaches?

3. Can Noria change queries without downtime?

!20

Evaluation

Amazon EC2 c5.4xlarge instance (16 vCPUs)Open-loop clients, measuring latency & throughput

Setup

multi-machine experimentscomparison with differential dataflow } see Paper

!21

Case study: Lobsters (http://lobste.rs)

!21

Case study: Lobsters (http://lobste.rs)

‣Ruby-on-Rails applicationwith MySQL backend

!21

Case study: Lobsters (http://lobste.rs)

‣Ruby-on-Rails applicationwith MySQL backend‣Hand-optimized by

developers to pre-compute aggregations

!21

Case study: Lobsters (http://lobste.rs)

‣Ruby-on-Rails applicationwith MySQL backend‣Hand-optimized by

developers to pre-compute aggregations‣Noria data-flow with

235 operators, 35 views

!21

Case study: Lobsters (http://lobste.rs)

‣Ruby-on-Rails applicationwith MySQL backend‣Hand-optimized by

developers to pre-compute aggregations‣Noria data-flow with

235 operators, 35 views‣Emulate production load

!22

Can Noria improve Lobsters’ performance?

!22

Can Noria improve Lobsters’ performance?Be

tter

Better

!22

Can Noria improve Lobsters’ performance?Be

tter

Better

!22

Can Noria improve Lobsters’ performance?Be

tter

Better

!22

Noria with natural queries supports 5x MySQL’s throughput.

Can Noria improve Lobsters’ performance?Be

tter

Better

!23

How does Noria compare to alternatives?Be

tter

Better

!23

How does Noria compare to alternatives?

‣Zipf-distributed story ID,95% reads, 5% writes ‣No TX, all in-memory

Bette

r

Better

!23

How does Noria compare to alternatives?

‣Zipf-distributed story ID,95% reads, 5% writes ‣No TX, all in-memory

Bette

r

Better

!24

How does Noria compare to alternatives?Be

tter

Better

‣Zipf-distributed story ID,95% reads, 5% writes ‣No TX, all in-memory

!24

How does Noria compare to alternatives?Be

tter

Better

‣Zipf-distributed story ID,95% reads, 5% writes ‣No TX, all in-memory

!24

How does Noria compare to alternatives?

Noria outperforms an in-memory key-value store and simplifies its interface.

Bette

r

Better

‣Zipf-distributed story ID,95% reads, 5% writes ‣No TX, all in-memory

32

!25

Can Noria change queries without downtime?

JOIN

COUNT

FILTER

StoriesVotes

32

!25

Can Noria change queries without downtime?

JOIN

COUNT

FILTER

StoriesVotes

3 ⭐

1.5 ⭐

AVG

JOIN

COUNT

Ratings⭐⭐⭐

⭐

⭐⭐

!26

Can Noria change queries without downtime?Be

tter

!26

Can Noria change queries without downtime?

new table & query added

Bette

r

!26

Can Noria change queries without downtime?

new table & query added

Bette

r

!26

Can Noria change queries without downtime?

new table & query added

Bette

r

‣Zipf-distributed story ID, 95% reads; 2M existing votes at transition

!27

Can Noria change queries without downtime?Be

tter

‣Zipf-distributed story ID, 95% reads; 2M existing votes at transition‣Old view reads are live throughout

!27

Can Noria change queries without downtime?

instantaneous transition,no downtime for writes

Bette

r

‣Zipf-distributed story ID, 95% reads; 2M existing votes at transition‣Old view reads are live throughout

!27

Can Noria change queries without downtime?

instantaneous transition,no downtime for writes

80% of reads from new view proceedwithout upquery after 1 second

Bette

r

‣Zipf-distributed story ID, 95% reads; 2M existing votes at transition‣Old view reads are live throughout

!27

Can Noria change queries without downtime?

instantaneous transition,no downtime for writes

80% of reads from new view proceedwithout upquery after 1 second

Noria achieves downtime-free query change with partial state.

Bette

r

‣Zipf-distributed story ID, 95% reads; 2M existing votes at transition‣Old view reads are live throughout

• New partially-stateful data-flow model.

• Noria: new web application backend based on data-flow.

• Partial state saves space and allows live change.

• Supports high throughput on one or more machines.

• Open source, try it out!

!28

Noria — Summary

• New partially-stateful data-flow model.

• Noria: new web application backend based on data-flow.

• Partial state saves space and allows live change.

• Supports high throughput on one or more machines.

• Open source, try it out!

!28

https://pdos.csail.mit.edu/noria

Noria — Summary

(see our demo at poster #37 today!)