cloud era transactional processing -- problems, strategies and solutions

Cloud Era Transactional ProcessingProblems, Strategies and Solutions

Yu Liu

HUE&ATE Div

March 8, 2016

Yu Liu (HUE&ATE Div) Cloud Era Transactional Processing March 8, 2016 1 / 33

[email protected]

[email protected]

Contents

1 Modeling & FormalizingDefinitions and Concepts

2 Choose A Sufficient Consistency Model

3 The State-of-the-art in AcademiaCoordination AvoidanceConsistency ModelsEfforts on Hardware

4 Appendix


[email protected]

Modeling & Formalizing

Contents




4 Appendix


[email protected]


Consistency Dominates the Complexity of Applications

Biggest challenge for developing applications on Eventual-consistencyNoSQL databases:

All the intermediate status of applications need to be maintained bythemselves

Controlling referential integrity between processes and data is difficult

No generalized application-specific logic for guarantee/verifycorrectness


[email protected]


Constrains on Consistency

object-level: consistency between replications

session-level: read your write, writes follow reads, monotonic reads ...

process-level: handling concurrent accesses to data


[email protected]


An Example For Isolating Two Concurrent Processes

If databases cannot handle such coordinations for us then it will be verytroublesome.


[email protected]


Overhead and Drawback of Performance

Application-level concurrency control is usually inefficient, here are somereasons:

using coarse-grained barriers/datasets

lack of global information

optimization by hands

experience, skill-level and constraints on time/schedule


[email protected]

Modeling & Formalizing Definitions and Concepts

Contents




4 Appendix


[email protected]


ANSI SQL Isolation Levels

ANSI’92 SQL defined four levels of isolation in terms of the four phenomena.

Isolation Level Dirty write Dirty read Fuzzy Read Phantom

READ UNCOMMITTED Ö X X XREAD COMMITTED Ö Ö X XREPEATABLE READ Ö Ö Ö X

SERIALIZABLE Ö Ö Ö Ö

Many other isolation levels and phenomena are defined besides the onesdefined in ANSI’92.


[email protected]


About Consistency Models and Isolation Levels

Read Uncommitted (RU), Read Committed (RC), Monotonic Atomic View (MAV), Item Cut Isolation (I-CI), Predicate Cut

Isolation (P-CI), Writes Follow Reads (WFR), Monotonic Reads (MR), Monotonic Writes (MW)


[email protected]


One-copy Serializability (1SR)

1SR is the desired level of data consistency for replicated systems.

One-copy serializability (1SR) guarantees a replicated DB actsequivalent to a non-replicated DB

1SR is important for ACID transactions on replicated storages

To achieve1SR coordinations between replicas is necessary, like 2PL,or consensus algorithms like Paxos


[email protected]


Snapshot Isolation

Many high performance txn systems adopt SI

all reads within a transaction see a consistent view

read never block write

non-serializable stronger than ANSI RR isolation level

SI can be extended to serializable1

1Patent: Optimistic serializable snapshot isolation, Microsoft CorporationYu Liu (HUE&ATE Div) Cloud Era Transactional Processing March 8, 2016 12 / 33

[email protected]


Causal Consistency

Causal consistency provides apartial order over operationsaccording to:

Thread-of-Execution

Reads-From.

Transitive-Closure


[email protected]


Comparison of Isolation Levels

Anomaly 1SR SI C-SI Eventual

Dirty read Ö Ö Ö X

Non-repeatable read Ö Ö Ö X

Lost update Ö Ö Ö X

Short fork Ö X X X

1SR is impossible forGeo-replicated HA DBs

Eventual is a bit too weak

SI is sufficient for most usages

Causal-SI can be implementedon top of Geo-distributeddatabases


[email protected]

Choose A Sufficient Consistency Model

Contents




4 Appendix


[email protected]


Trade-off: CAP, ACID, and Availability

All the theories about them give clear messages:

Pay for High Availability

CAP means neither Linearizability nor Serializability can beachieved with presence of network partition

For Geo-replicated system, no guarantee on arbitrary global integrityconstraints (no 1SR semantics) on data

But, it does not mean ACID transaction is not possible with highavailability


[email protected]


Low-latency and Deterministic Response

Causal consistency is the best choice for Geo-distributed HA transaction.

high volume transactionswith strict time limits

high availability inmulti-DC environments

strong reliability for txnon large clusters


[email protected]


A Bolt-On Causal Architecture

A Causability transaction framework can be implemented on top of anyeventual consistent store 2

application define itsown causal relationships

a shim layer holds thecausal consistency

the under-lying store isresponsible for liveness /durability / convergence

2Bailis, et. al., Bolt-on causal consistency. Acm Sigmod’13.Yu Liu (HUE&ATE Div) Cloud Era Transactional Processing March 8, 2016 18 / 33

[email protected]


Example: Using Causal Txn For Copy Convergence

Using state vector (clock vector) to capture caulsalablity and implementconcurrent editing


[email protected]

The State-of-the-art in Academia

Contents




4 Appendix


[email protected]

The State-of-the-art in Academia Coordination Avoidance

Contents




4 Appendix


[email protected]


Maximizing Scalability, Availability, and High Performance

Minimizing coordination blocking communication between concurrentlyexecuting operations.

lock-based, optimistic, and pre-scheduling mechanisms (2PL/2PC,Paxos, Spanner ...)

commutativity and associativity of operations (FOL, Datalog)

monotonicity and convergence (CALM theorem and CRDT objects)

Term rewriting (transaction chopping)

Coordination-free algorithms and semantics (RAMP transaction)


[email protected]


Concurrency Control

Traditional Two-Phase Locking (2PL) guarantees Serializability how everyintroduced overhead is heavy.

Most of multiversion concurrency control (MVCC) databases

avoid read-write conflictsnot always necessary to achieve 1SRcontention and convergence restrictions are relaxed

some novel ways to implement MVCC fast and efficient, both for SIand for full serializability (Hekaton, Hyder)

Efficient and scalable commit protocol (Silo)


[email protected]

The State-of-the-art in Academia Consistency Models

Contents




4 Appendix


[email protected]


Geo-replicated Causal Consistency Transactions

Many prototype transactional systems with causal consistency areimplemented.

Orbe is a distributed KVS that incurs relatively small overhead thaneventually consistent KVS

provide a scalable and efficient implementation of causal consistencyusing physical clocksprovide read-only transactions

Eiger provides the following features:

low latency read-only and write-only transactionwrite-only transactions can atomically update multiple columns ofmultiple keys spread across multiple servers in a DCread-only transactions are non-blocking and partition tolerantmuch faster than Cassandra’s strongly consistent operations


[email protected]


Parallel Snapshot Isolation (PSI)

PSI provides a balance between consistency and latency

Asynchronous replication acrosssites

Efficient update-anywhere forcertain objects

Freedom from conflict-resolutionlogic

Strong isolation within each site

Anomaly 1SR SI PSI Eventual

Dirty read Ö Ö Ö X

Non-repeatable read Ö Ö Ö X

Lost update Ö Ö Ö X

Short fork Ö X X X

Long fork Ö Ö X X

Conflict fork Ö Ö Ö X


[email protected]


Non-Monotonic Snapshot Isolation (NMSI)

satisfies strong safety properties,and addresses the scalabilityproblems of PSI

using dependence vectors toenable the efficient computationof consistent snapshots

coordination overhead is close toRead-Committed and up to twotimes faster than PSI

Anomaly 1SR SI PSI NMSI

Dirty read Ö Ö Ö Ö

Non-repeatable read Ö Ö Ö Ö

Lost update Ö Ö Ö Ö

Short fork Ö X X X

Long fork Ö Ö X X

Conflict fork Ö Ö Ö Ö


[email protected]

The State-of-the-art in Academia Efforts on Hardware

Contents




4 Appendix


[email protected]


Hardware Transactional Memory

HTM like Intel restricted transactional memory (RTM)

HTM allows multi-core scalability without statically partitioning thedatabase

recent mainstream CPUs reduce the overhead of locking and latching

HTM can efficiently scale out many MVCC implementations


[email protected]


Remote Direct Memory Accesses (RDMA) and SSD

RDMA can help distributed transactions with strict serializability, highperformance, durability, and high availability Recent report of:

surpass 1.3 million ops (90% gets) using a single CPU core comparedwith 55K for Memcached and 59K for Redis (Pilaf)

26 million key-value operations per second with 5 µs average latencyin single machine (HERD)

140 million TATP transactions per second on 90 machines with a 4.9TB database, and and it recovers from a failure in less than 50ms(FaRM)


[email protected]

Appendix

Contents




4 Appendix


[email protected]

Appendix

Not Be afraid of Weak Consistency

Most of modern databases provides weaker isolation level.

Database Isolation Level Default Maximum

MySQL 5.6 RR S

SQL Server 2012 RC S

Oracle 11g RC SI

Postgres 9.2.2 RC S

SAP HANA RC SI

VoltDB S SRead Committed (RC), Serializability (S), Snapshot Isolation (SI)


[email protected]

Appendix

Discussion on Consensus Algorithms

2PC

Paxos and its extensions

Multi-PaxosGeneralized PaxosRampMDCC


[email protected]

cloud era transactional processing -- problems, strategies and solutions

Technology