node fails

Phase 2recovery

Adaptive Hybrid Quorums in Practical

Settings

What are Quorums?

[1] Peter Bailis, Shivaram Venkataraman, Michael J. Franklin, Joseph M. Hellerstein, and Ion Stoica. Probabilistically bounded staleness for practical partial quorums. PVLDB, 5(8):776787, 2012.

t-visibility

Failure Modeling

Replication → consistency + durability + availability.

quorum – subset of the replicasWrite and read from a quorum → efficiency

Traditionally: full quorums – every 2 quorums intersect →

consistency

In practice: partial quorums →sacrificing consistency for

efficiency Dynamo: send to N replicas (full quorum),

wait for R/W (partial quorum).

Eventually all replicas receive request → eventual consistency

Bailis et al. [1]: bound the time for consistency

Adaptive Hybrid Quorums Use PBSPredictor() and WARS Model [1] to track latencies and predict consistency

Track workload properties, e.g. is it read- or write- oriented?

Optimization algorithm → adaptively find the best quorum for current workload and system performance

References

Aaron Davidson, Aviad Rubinstein, Anirudh Todi, Peter Bailis and Shivaram

Venkataraman

Experimental Latency

Experimental Consistency

Node failure model:

Incorporate model into adaptive hybrid consistency

Ongoing work: model message drops…

Introduction Implementation in Cassandra Evaluation

Key Observation

Quorums are temporary → change them adaptively!

Phase 1 – node crashesbut we don’t know it…

detection

Phase 2 – Writes go to hinted node

N-1 replicas remain

Phase 3 – Hinted-handoffRecovering node is partially

available

node restarts

node fails

Phase 2recovery

(1,2)

node fails

(1,1)recovery