Download - Return of the transaction king
Ryan Knight - @knight_cloud
Return of the Transaction King
Brief History of Data
RDBMS + ACID Transactions
Atomic
Consistent
Isolated
Durable
Traditional ACID Transactions
Did not Scale with the Internet
Leader w/ Read Replicas
Distributed Locks
CAP Theorem Forced Choices
Global Scale or Strong Consistency?
Rise of Eventual Consistency
Eventual Consistency
Really a Necessary Evil
Last Write Wins
Event Sourcing
Write now and figure it out later
● DNS, IoT, Time Series, Monitoring, Web Metrics, Followers, etc.
● Modify Business to Fit - Allow for Inconsistency○ Inventory - Oversell and cancel order if
necessary○ Bank - Overdraft and charge
Where Event Sourcing Works
We live in a stateful and finite world! - James Ward
But ….
Where Event Sourcing BreaksWhen Order of Events Matters
Can not write from 2 hosts and guarantee ordering
Reconcile Diverging History of Events
Possible to Lose History
Strong Consistency
Differing Views of Strong Consistency
Write / Read Local Quorum
No Read before Write
No Compare and Swap
No Snapshot Isolation
Write SkewTwo concurrent transactions each determine what they are writing based on reading a data set which overlaps what the other is writing
begriffs.com
Can we Ignore Consistency?
Pushes consistency problem into the application where they are far harder to solve
Eventual Consistency really is a necessary Evil
Consistency in the Application Tier
Events Sourcing w/ Command
Serialization
Serialization Points - Kafka
Consumers
What happens when they fail?
Cloud Native / Microservices
Partitioning of the Application Domain and Related Data
Increased Coordination between Data
Number of Microservices Increases Number of Interactions
Exponential Growth in Complexity
Cloud Native + NoSQL - ACID==========Rube Goldberg machine
ACID Transactions
Atomic
Consistent / Correct
Isolated
Durable
ACID TRANSACTIONS
+ EVENT SOURCING
============RETURN OF
TRANSACTIONS
Modern Transactional Databases
Fauna
Google Spanner
CockroachDB
GLOBAL SCALE + STRONG CONSISTENCY
Value of Transactions
Decrease Application Tier Complexity
Reduce Cognitive Overhead
Increased Developer Productivity
Increased Focus on Business Value
● Deal with People's Money ● Financial Transactions● Inventory ● Stock Exchange● Auctions● Points
When do you need ACID TXN?
Do we really have to sacrifice AP to get C?But there are two important caveats: first, you only need forfeit something during an actual partition, and even then there are many mitigations. Second, the actual theorem is about 100% availability, while the interesting discussion here is about the tradeoffs involved for realistic high availability.
Eric Brewer Spanner Paper
Problems with CAP, and Yahoo’s little known NoSQL system - Daniel Abadi
Tradeoff between latency (L) and consistency under normal conditions
Google Spanner
● Private global network along with GPS receivers and atomic clocks.
● Atomic Clocks for Serializability Guarantees● AdWords & Google Play for 5+ years● Google Cloud Platform Control Plane ● No Scheduled Downtime● Can do maintenance by moving data between nodes.
CockroachDBHybrid Logical Clock similar to a vector clock for ordering of transactions
Challenges with clock skew
Waits up to 250 MS on reads
What is Fauna?
Globally distributed, transactional database
Born out of Twitter and the Calvin Pape
Functional / Type-Safe Query Language
Indexes, Constraints and Views
Fauna Transaction Resolution
Resolves transactions before committing to DB
Execute phase and Commit phase
All Transaction w/ in a epoch our ordered and applied as a batch
Uses a RAFT replicated log that derives a total order of all transactions.
Distributed ACID Transaction Engine
Consensus protocols from Raft and Calvin Paper
Can span thousands of records and indexes
Designed for high throughput and WAN replication
Ensures multi-node and multi- datacenter correctness
Security, Identity, IsolationNative Security and Access Control
Row-level security, identity, and isolation
Complete Data Isolation across datasets
Quality of Service Managed Resource Scheduler
Operating System Like Design
Single Cluster for Differing and Competing Workloads
Temporality
FaunaDB keeps all instances of a data, not overwriting them but creating new ones when a write is performed. This is useful especially when auditing data, verifying its evolution over time.
Fauna Data RepresentationDatabase
Class(es)
Instanc(es)
Arrays and Scalar Types
Nested Data
Database
Database(s)
Class(es)
Class(es)
InstancesEach record stored as an object.
All objects are instances of a specific class.
Every instance has an identifier called a ref.
An instance’s ref encodes its class along with a unique id
FunctionalTransaction Scope
Not traditional begin / end transaction
Entire Function / Query Block is scope of transaction
Paradigm will feel natural to Functional Programmers
Similar to Traditional RDBMS Indexes Terms and Values
Transform, cover, and order instance data
Enforce unique constraints.
Referenced explicitly in query expressions;
Optimizer does not make index applications
Indexes
Example “Smart Contract”
● User Nodes w/ Simulated Account Balance
● Server Node acts as Intermediary and
Notary on Contract
● User Nodes Propose Borrowing from Peers
● Peer Accepts or Rejects Proposal
● Contract is Created between Peers Lending
Fixed Amount