the potential dangers of causal consistency and an explicit solution - socc 2012
DESCRIPTION
Paper: http://www.bailis.org/papers/explicit-socc2012.pdf Narration: http://www.vimeo.com/pbailis/explicit-socc12TRANSCRIPT
the potential dangersof causal consistencyand an explicit solutionPeter Bailis, Alan Fekete, Ali Ghodsi,Joseph M. Hellerstein, Ion Stoica
UC Berkeley SOCC 2012
A Story
A Story
“Sally’s in a coma!”
A Story
“Sally’s in a coma!”“Sally’s in a coma!”
A Story“Sally’s okay!”
“Sally’s in a coma!”“Sally’s in a coma!”
A Story“Sally’s okay!”
“Sally’s in a coma!”“Sally’s in a coma!”
A Story
“Great news!”
“Sally’s okay!”
“Sally’s in a coma!”“Sally’s in a coma!”
A Story
“Great news!”
“Sally’s okay!”
“Sally’s in a coma!”“Sally’s in a coma!”
“Great news!”
“Sally’s okay!”
The Wrong Story
“Sally’s in a coma!”
“Great news!”
The Wrong Story
“Sally’s in a coma!”
“Great news!”
The Wrong Story
“Sally’s in a coma!”
“Great news!”
???The Wrong Story
The Right Order
“Sally’s in a coma!”
“Great news!”
“Sally’s okay!”
The Right Order
“Sally’s in a coma!”
“Great news!”
“Sally’s okay!”
The Right Order
“Sally’s in a coma!”
“Great news!”
“Sally’s okay!”
CAUSALITY: TL;DRMETADATA AND MODERN SERVICESTHE SITE SCALABILITY WALLSCALING WITH EXPLICIT CAUSALITY
CAUSALITY: TL;DRMETADATA AND MODERN SERVICESTHE SITE SCALABILITY WALLSCALING WITH EXPLICIT CAUSALITY
CAUSALITY: TL;DRMETADATA AND MODERN SERVICESTHE SITE SCALABILITY WALLSCALING WITH EXPLICIT CAUSALITY
CAUSALITY: TL;DRMETADATA AND MODERN SERVICESTHE SITE SCALABILITY WALLSCALING WITH EXPLICIT CAUSALITY
CAUSALITY: TL;DRMETADATA AND MODERN SERVICESTHE SITE SCALABILITY WALLSCALING WITH EXPLICIT CAUSALITY
CAUSALITY: TL;DRMETADATA AND MODERN SERVICESTHE SITE SCALABILITY WALLSCALING WITH EXPLICIT CAUSALITY
1978“Time, Clocks, and the Ordering of Events
in a Distributed System” by Leslie Lamport
1978“Time, Clocks, and the Ordering of Events
in a Distributed System” by Leslie Lamport
1994“Causal Memory” by Ahamad et al.
1978“Time, Clocks, and the Ordering of Events
in a Distributed System” by Leslie Lamport
1994“Causal Memory” by Ahamad et al.
2011-12
2011-12
2011-12
2011-12
IEEE CAP
There aremany hardtrade-offs,
2011-12
IEEE CAP SOSP: COPS
There aremany hardtrade-offs,
butcausalconsistencycan workwell
2011-12
IEEE CAP SOSP: COPS Texas: CAC
There aremany hardtrade-offs,
butcausalconsistencycan workwell
and it’sthe bestyou cando*!
“Sally’s okay!”
“Sally’s okay!”
“Sally’s okay!”
“Sally’s okay!”
“Sally’s okay!”
“Sally’s okay!”
“Great news!”
“Sally’s okay!”
“Great news!”
“Sally’s okay!”
“Great news!”
“Sally’s okay!”
“Great news!”
“Sally’s okay!”
Deliver writesin the “right”order
Deliver writesin the “right”order
“Great news!”
“Sally’s okay!”
“Sally’s in a coma!”
Causal Consistencyreads obey partial ordercalled “happens-before”
Write value
Write valueHappens-beforemetadata
Write valueHappens-beforemetadata
Happens-beforemetadata
Happens-before metadata
per-DC vector clocksO(datacenters)
Happens-before metadata
per-DC vector clocksO(datacenters)
Happens-before metadata
LESS PARALLELISM
per-DC vector clocksO(datacenters)
per-key vector clock(dependency set)
O(keys)
Happens-before metadata
LESS PARALLELISM
per-DC vector clocksO(datacenters)
per-key vector clock(dependency set)
O(keys)
Happens-before metadata
MORE PARALLELISM
LESS PARALLELISM
per-DC vector clocksO(datacenters)
per-server vector clocksO(servers)
per-key vector clock(dependency set)
O(keys)
Happens-before metadata
MORE PARALLELISM
LESS PARALLELISM
per-DC vector clocksO(datacenters)
per-server vector clocksO(servers)
per-key vector clock(dependency set)
O(keys)
Happens-before metadata
MORE PARALLELISM
LESS PARALLELISM
Have we seen alldependencies?
Have we seen alldependencies?
Have we seen alldependencies?
Y
Have we seen alldependencies?
Have we seen alldependencies?
Y
Have we seen alldependencies?
Yes: apply write
Have we seen alldependencies?
Yes: apply write
Have we seen alldependencies?
No: waitYes: apply write
Sounds great, right?
CAUSALITY: TL;DRMETADATA AND MODERN SERVICESTHE SITE SCALABILITY WALLSCALING WITH EXPLICIT CAUSALITY
Causal Consistencyreads obey partial ordercalled “happens-before”
Causal Consistency
traditional happens-before:potential causality
reads obey partial ordercalled “happens-before”
If you wish to make an apple pie from scratch,you must first inventthe universe.
-Carl Sagan
“Great news!”
“Great news!”
“Sally’s okay!”
“Great news!”
“Sally’s okay!”
“Great picture!”
“Rad party!” “Lol!”
“Can’t waitfor SOCC!”
“Want to go skiing?”
“Great news!”
“Sally’s okay!”
“Great picture!”
“Rad party!” “Lol!”
“Can’t waitfor SOCC!”
“Want to go skiing?”
“I hope my paper gets in.”
“Youwho?”
“I love Tahoe!”
“Coming tonight?
“Great news!”
“Sally’s okay!”
“Great picture!”
“Rad party!” “Lol!”
“Can’t waitfor SOCC!”
“Want to go skiing?”
“Snow rocks!”
“I hear the PC is great!”
“You.”“Hello,
“I hope my paper gets in.”
“Youwho?”
“I love Tahoe!”
“Coming tonight?
“Great news!”
“Sally’s okay!”
“Great picture!”
“Rad party!” “Lol!”
“Can’t waitfor SOCC!”
“Want to go skiing?”
“Snow rocks!”
“I hear the PC is great!”
“You.”“Hello,
“Who’s there?”
“Are you submitting?”
“Have you met Larry?”
“Great food here”
“I hope my paper gets in.”
“Youwho?”
“I love Tahoe!”
“Coming tonight?
“Great news!”
“Sally’s okay!”
“Great picture!”
“Rad party!” “Lol!”
“Can’t waitfor SOCC!”
“Want to go skiing?”
“Snow rocks!”
“I hear the PC is great!”
“You.”“Hello,
“Who’s there?”
“Are you submitting?”
“Have you met Larry?”
“Great food here”
“Knock, knock.”
“I hope my paper gets in.”
“Youwho?”
“I love Tahoe!”
“Coming tonight?
“Great news!”
“Sally’s okay!”
“Great picture!”
“Rad party!” “Lol!”
“Can’t waitfor SOCC!”
“Want to go skiing?”
“Snow rocks!”
“I hear the PC is great!”
“You.”“Hello,
“Who’s there?”
“Are you submitting?”
“Have you met Larry?”
“Great food here”
“Knock, knock.”
“I hope my paper gets in.”
“Youwho?”
“I love Tahoe!”
“Coming tonight?
Twitter.com20 Tweetsat login
20 moreauto-scroll
600+ per min
Twitter.comquicklyapproachesupper bound
rec
ap
potential dangerPotential causality graphs are huge,limiting local apply rates and throughput
rec
ap
CAUSALITY: TL;DRMETADATA AND MODERN SERVICESTHE SITE SCALABILITY WALLSCALING WITH EXPLICIT CAUSALITY
DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!
DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!Unstable queue!
DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!Unstable queue!
Sustained throughputlimited to slowest DC
DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!Unstable queue!
Sustained throughputlimited to slowest DC
DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!Sustained throughputlimited to slowest DC
DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!
Adding DCs doesn’t helpslowest site
Sustained throughputlimited to slowest DC
DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!
During partitions/failures,sustainable throughput iszero writes/szero
Sustained throughputlimited to slowest DC
DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!During partitions/failures,sustainable throughput iszero writes/szero
DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!During partitions/failures,sustainable throughput iszero writes/szero
DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!During partitions/failures,sustainable throughput iszero writes/szero
Metadata garbage collectionstallsstalls
DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!During partitions/failures,sustainable throughput iszero writes/szero
Metadata garbage collectionstallsstalls
DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!DANGER!During partitions/failures,sustainable throughput iszero writes/szero
Metadata garbage collectionstallsstalls
rec
ap
rec
ap
potential dangerWrite throughput limited to slowest DCViolation ⇒ arbitrarily high visibility latencyAdding DCs does not increase throughput
Potential dangers spell trouble for causality
bad news
Potential dangers spell trouble for causality
to it?Are live without wedestined
bad news
Potential dangers spell trouble for causality
to it?Are live withoutwe destined
bad news
CAUSALITY: TL;DRMETADATA AND MODERN SERVICESTHE SITE SCALABILITY WALLSCALING WITH EXPLICIT CAUSALITY
ObservationWhy track dependencies that don’t matter?
ObservationWhy track dependencies that don’t matter?
ObservationWhy track dependencies that don’t matter?
Need to know application semantics
ObservationWhy track dependencies that don’t matter?
Need to know application semantics
Explicit Causalityapp-defined “happens-before”
transitivity enforced
subset of potential causality
Explicit Causalityapp-defined “happens-before”
transitivity enforced
subset of potential causality
not a new idea(e.g., Cheriton and Skeen SOSP 1993,
Ladin et al. PODC 1990)
but...
Explicit Matters
Explicit MattersTwitter28% of Tweets in conversations69% of convos are depth twoaverage depth is 10.7
[Ye and Wu SocInfo 2010, Ritter et al. HLT 2010]
Explicit MattersTwitter28% of Tweets in conversations69% of convos are depth twoaverage depth is 10.7
[Ye and Wu SocInfo 2010, Ritter et al. HLT 2010]
reply-to degree and depth are limited
Explicit MattersTwitter28% of Tweets in conversations69% of convos are depth twoaverage depth is 10.7
[Ye and Wu SocInfo 2010, Ritter et al. HLT 2010]
reply-to degree and depth are limited
109 smaller graph for a year of Tweets
Explicit APIput(key, value)
put_after(key, value, deps)
Explicit API
put_after(key, value, deps)
Explicit API
put_after(key, value, deps)
Explicit API
(possibly empty) set ofreferences to other writes
put_after(key, value, deps)
Explicit API
track what matters
(possibly empty) set ofreferences to other writes
put_after(key, value, deps)
Explicit API
track what mattersfrequently in data model already
(possibly empty) set ofreferences to other writes
put_after(key, value, deps)
Explicit API
track what mattersfrequently in data model alreadycan simulate fencing
(possibly empty) set ofreferences to other writes
put_after(key, value, deps)
Explicit API
track what mattersfrequently in data model alreadycan simulate fencing
(possibly empty) set ofreferences to other writes
won’t track non-explicit references
Extra Note:
paper for more details
can treat foreign key references in relational data model as explicit dependencies automatically!
can use schema to describe explicit deps...
Extra Note:
paper for more details
NATURAL EXTENSIONNOT ROCKET SCIENCE
NATURAL EXTENSIONNOT ROCKET SCIENCE
http://twitter.com/rbranson/status/256795094387142657
potentialcausality
A WORLD OFPHYSICAL
OPERATIONSMESSAGESREGISTERS
READSWRITES
MODEST SCALE
A WORLD OF HUGE SCALE
MASSIVEINTERACTIVITYPLANET-WIDE
NETWORKSREAL-WORLD
INTERFACESUBICOMPBIG DATA
potential
dangers
potential
dangers
huge causality graphs
potential
dangers
huge causality graphs
throughput scalability limited
potential
dangers
huge causality graphs
throughput scalability limited
explicitcausalitysemantic context to the rescueconsider modern appshelps with #1, indirectly with #2
