Just-RightConsistency!

Valter Balegas,NunoPreguiça,MarcShapiro,AnnetteBieniusa,ChristopherS.Meiklejohn,CarlaFerreira&manyothers

1

Agenda

• Geo-replication;• Consistency/availabilitytrade-off;• Just-RightConsistency!• … andhowtouseit.

2

Geo-replication

3

A

CB

Consistency

High availability

Low latency

Geo-Replication

4

A

CB

H

D

EF

G

IConsistency →High latency

Consistency/availabilitytrade-off

• Weakconsistencyisdifficulttogetright:• Notwell-definedsemantics;• Difficulttoprogram;• Unpredictableerrorsinproductionduetouncoordinatedexecutions.

• Strongconsistencyperformsworse,butissafe:• Well-definedsemantics;• Application’slogicisprotectedfromconcurrencyerrors;• Scalableandlow-latencyinsomedeployments;• Cross-replicacoordinationaffectsperformanceinthewide-area.

5

Consistency/availabilitytrade-off

• Whatwereallycareaboutisapplicationcorrectness.

• Serializingoperationsisa(overlyconservative)wayofmaintainingcorrectness.

• Example:• Operationsonabankaccountcanbeappliedconcurrentlyaslongasmoneydoesnotdisappear,getduplicated,orusedwithoutpermission.

6

Just-RightConsistency!

• Ensureapplication correctness withminimalcoordination.

• Howtodoit?

7

BankingApplication:

• Allreplicasmustconvergetothesamestate.• Accountbalanceisequaltoinitbalance+∑ deposit()�

� – ∑withdrawals()�� ;

• Themoneycannottemporarilydisappearinatransfer.• Accountbalancemustbenon-negative.

8

Problem1:Statedivergence

9

• Concurrentupdatesleadtostatedivergence.

Clt1

DC1

DC2

Clt2

a:5

a:5

a:2

a:3

a:?

a:?

withdraw(3,a)

withdraw(2,a)

write(a)

write(a)

Replicateddatatypes

10

updt(-3,a)

updt(-2,a)

Clt1

DC1

DC2

Clt2

withdraw(3,a)

withdraw(2,a)

a:5

a:5

a:2

a:3

a:0

a:0

• Ensurestateconvergencewithoutlosingupdates.

Problem2:Atomicoperations

11

Clt1

DC1

DC2

Clt2

transfer(3,ac,as)

balance(ac) balance(as)2 1 2+1!=6balance(ac+s) 6

ok

updt(-3,ac)updt(+3,as)

ac:5,as:1

ac:5,as:1 ac:2,as:1 ac:2,as:4

ac:2,as:4

• ACIDtransactionsarenothighlyavailable.

Transactionalconsistency

12

Clt1

DC1

DC2

Clt2

transfer(3,ac,as)

balance(ac) balance(as)5 1 5+1==6balance(ac+s) 6

ok

updt(-3,ac)updt(+3,as)

ac:5,as:1

ac:5,as:1 ac:2,as:4

ac:2,as:4

• Ensureatomicitywithweakerisolation.• Snapshotreads;atomicupdates.

Problem3:Asynchronousreplication

13

• Doesnotpreservetheexecutionorderofoperationsattheorigin.

Clt1

DC1

DC2

Clt2

transfer(3,ac,as) ok notify(clt2,’updated_as’)

balance(as) 1

ac:5,as:1

ac:5,as:1

balance(as) 1

notify()

Causalconsistency

14

• Executionorderofoperationsatdifferentreplicasrespectthehappens-beforerelationship.

Clt1

DC1

DC2

Clt2

transfer(3,ac,as) ok notify(clt2,’updated_as’)

balance(as) 1

ac:5,as:1

ac:5,as:1

balance(as) 4

notify()

Problem4:Invariantviolation

15

• Concurrentexecutionsmightbreakthecorrectnessofapplications.à Usecoordination?

Clt1

DC1

DC2

Clt2

withdraw(3,a) ok

ok

a:5

a:5

withdraw(3,a)

a:2

a:2

updt(-3,a)

updt(-3,a) a:-1

a:-1

Minimizecoordination

• Usecoordinationonly whennotsafetoexecuteoperationsasynchronously.

16

Clt1

DC1

DC2

Clt2

withdraw(3,a) ok

fail

a:{5,4}

a:{5,1}

withdraw(3,a)

a:{2,1}

updt(-3,a)

req(a,3)grant(1,a)

a:{2,3}

a:{2,0}

Performance

17

0 20 40 60 80

100 120 140 160 180 200

0 20 40 60 80 100 120

Late

ncy

[ms]

Time [s]

Latencyofoperationsforasinglesite,ina3DCdeployment.

OtherapplicationInvariants

• Bidirectionalrelationshipà usetransactions;• Referentialintegrityà automaticrepair;• Overdraftà compensations;• Sequentialidentifiersà fallbacktocoordination.

18

Just-RightConsistency!

• Ensureapplication correctness withminimalcoordination.

• Howtodoit?• Convergentdata-types;• Transactions;• Causality;• Maintainapplication-levelinvariants.

• Toolstosupportapplicationdevelopment.

19

JRC!Tools

• Helpprogrammersverifyapplicationcorrectnessusingasoundapproach:

• Staticprogramanalysis.

• Detectconcurrencyconflicts.

• Modifyapplicationsandtestagain.

• Publicly-availabletoolsontheway.

20

JRCToolsvideo:http://tiny.cc/JRC-TOOLS-VIDEO

AntidoteDB

21

http://syncfree.github.io/antidote/

• Just-RightConsistencydatabase.• Geo-/Partial-replication.• SQL-likeinterfaceontheway:

• Well-knownandwidelyadopted;• Out-of-the-boxsupportformaintainingmanycommonclassesofinvariants.

AntidoteDB performanceinFMKe Benchmark

22

0

5

10

15

20

25

30

35

0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000

Late

ncy

(ms)

Throughput (tx/s)

1DC-Normal2DC-Nornal3DC-Normal

Latencyofoperationsformultiplesiteswith4nodespersite.

Questions?

23

• JRCToolsvideo:http://tiny.cc/JRC-TOOLS-VIDEO

• AntidoteDB:http://syncfree.github.io/antidote/

• FMKe Benchmark• http://tiny.cc/fmke

Backupslides

24

25

Conflictdetectionalgorithm

ValterBalegasetal.– NOVALINCS,DI,FCT,UniversidadeNOVAdeLisboa@SyncFreeMonth36 25

enroll(“Mario”,”World Cup”)enrolled(“Mario”, World Cup”) := true

remove_tournament(“WorldCup”)tournament(“World Cup”) := false

R1 R2InitialState

tournament(“World Cup”)

player(“Mario”)

R1State

tournament(“World Cup”)

player(“Mario”)

enrolled(“Mario”, WC”)

R2State

tournament(“World Cup”)

player(“Mario”)

MergedState

player(“Mario”)

enrolled(“Mario”, WC”)

INV=enrolled(p,t)⇒ player(p)∧ tournament(t)

T⇒ F

a1,b1

Referentialintegrityexample

a1a2a3

b1

b3

b2

26

a1,b1

Referentialintegrityexample

a1a2a3

b1

b3

DC1

DC2

a1,b2add

b2

27

a1,b1

Referentialintegrityexample

a1a2a3

b1

b2

b3

DC1

DC2

a1,b2add

rem

b2

28

Referentialintegrityexample

a1a2a3

b1

b3

DC1

DC2

a1,b2add

Danglingpointer

a1,b1a1,b2

29

rem b2

Preventiverepairexample

a1a2a3

b1

b3

DC1

DC2

a1,b2add

rem

a1,b1a1,b2

30

b2

b2

Preventiverepairexample

a1a2a3

b1

b3

DC1

DC2

a1,b2add

rem

a1,b1a1,b2

31

b2

b2

*,b2

Preventiverepairexample

a1a2a3

b1

b3

DC1

DC2

a1,b2add

a1,b1

32

*,b2rem b2

Scalability

33