consistency rationing in the cloud: pay only when it matters tim kraska, martin hentschel, gustavo...
Post on 15-Jan-2016
214 views
TRANSCRIPT
![Page 1: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/1.jpg)
1
Consistency Rationing in the Cloud: Pay only when it matters
Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann27.09.2009 Systems
Modified by Raja Amadala - 08305078
Sandeep N - 08305063
![Page 2: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/2.jpg)
2
![Page 3: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/3.jpg)
3
You own a Kiosk o Items are not as
valuableo Strong protection is not requiredo It is easier to handleo Less expensiveo It scales better
Of course,more can go wrong!
![Page 4: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/4.jpg)
4
Key Observation
Not all data needs to be treated at the same level consistencyEg : Web shop : Credit card information
User Preferences – (who bought this item also …)
Why :
Why we are looking at data divisions ?ConsistencyIn cloud storage consistency not only determines correctness but also the actual cost per transaction.
![Page 5: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/5.jpg)
5
Relation between cost, consistency and transaction
o We are using cloud database serviceso For that we have to payo In terms of transaction basiso How we are going to measure the
transaction cost.o Number of service callso For higher consistency – more number of
service calls needed.o For lower consistency – less number of
service calls enough.
![Page 6: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/6.jpg)
6
Consistency Rationing - Ideao Strong consistency is expensiveo ACID prevents scaling & availabilityoBut not everything is worth gold!
Transaction Cost vs. Inconsistency Cost• Use ABC-analysis to categorize the data• Apply different consistency strategies per category
![Page 7: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/7.jpg)
7
Outline
Consistency Rationing Adaptive Guarantees Implementation & Experiments Conclusion and Future Work
![Page 8: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/8.jpg)
8
Outline
Consistency Rationing Adaptive Guarantees Implementation & Experiments Conclusion and Future Work
![Page 9: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/9.jpg)
9
Consistency Rationing (Classification)
![Page 10: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/10.jpg)
10
Consistency Rationing (Category A)
o Serializabilityo Always stays consistento In cloud – expensive
• transaction cost• Performance issues
o Complex protocols neededo Require more interaction with additional
services (e.g., lock services, queuing services)o Higher costo Lower performance(response time)o 2PL protocol, serializability.
![Page 11: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/11.jpg)
11
Consistency Rationing (Category C)
o Session consistencyo It is the minimum consistency levelo Below this – application can’t see it’s own updateso Connects system in terms of Sessionso System guarantees read-your-own-writes monotonicity.o Session of different clients will not always see each other’s
updates.o Only uses Eventual consistencyo Conflict resolution in terms of type of updateso Non –commutative updates (overrides)o Commutative updates (numerical operations e.g., add)o It is very cheapo Permits extensive cachingo Should always place data in C- category
![Page 12: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/12.jpg)
12
Consistency Rationing (Category B)
Adaptive consistency Switches between Serializability and Session
consistency
![Page 13: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/13.jpg)
13
Consistency Rationing – Transactions on mixes
Consistency guarantees per category instead of transaction level Different categories can mix in a single operation/transaction For joins, unions, etc, the lowest category wins Issues: transactions may see different consistency levels
![Page 14: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/14.jpg)
14
Outline
Consistency Rationing Adaptive Guarantees Implementation & Experiments Conclusion and Future Work
![Page 15: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/15.jpg)
15
Adaptive Guarantees for B-Data
B-data: Inconsistency has a cost, but it might be tolerable Often the bottleneck in the system Here, we can make big improvements Let B-data automatically switch between A and C guarantees
Use policy to optimize:
Transaction Cost vs. Inconsistency Cost
![Page 16: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/16.jpg)
16
B-Data Consistency Classes
![Page 17: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/17.jpg)
17
General Policy - Idea
Apply strong consistency protocols only if the likelihood of a conflict is high
1. Gather temporal statistics at runtime2. Derive the likelihood of an conflict by means of a simple stochastic model3. Use strong consistency if the likelihood of a conflict is higher than a certain threshold
Consistency becomes a probabilistic guarantee
![Page 18: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/18.jpg)
18
General Policy - Model
n servers Servers cache data withcache interval - CI Load equally distributed Two updates considered asa conflict Conflicts for A and B datacan be detected and resolved after every CI Every server makes localdecisions(no synchronization)
![Page 19: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/19.jpg)
19
General Policy - Model
Likelihood of a conflict inside one CI
X is a stochastic variable corresponding to the number of updates to the same record within the cache interval. P(X > 1) is the probability of more than one update over the same record in one cache interval. the probability that the concurrent updates happen on the same server
![Page 20: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/20.jpg)
20
General Policy - Model
Simplification of this equation using Poisson processλ - arrival rate -
![Page 21: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/21.jpg)
21
General Policy – Temporal Statistics
On every server: Collect update rate for a window ω with sliding factor per item
Window size ω is a smoothingfactor Sliding factor and window sizeinfluence adaptability and spaceRequirement Sliding factor specifies the granularity which the window moves
![Page 22: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/22.jpg)
22
General Policy – Temporal Statistics
All complete intervals of a window build a histogram of the updates.Larger window size – better the statisticsIssues also present – like hot spot dataWe can calculate arrival rate λ (useful at Model)
X – is arithmetic mean of the number of the updates to a recordn – number of serversδ - Sliding factor
Issues: local statistics can easily mislead the statisticsSo local statistics can combined into a centralized view. Broadcast statistics from time to time to all other servers.
![Page 23: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/23.jpg)
23
General Policy – Setting the Threshold
Use strong consistency if the savings are bigger than the penalty cost
Cost of A transaction Cost of C transaction Cost of inconsistency (O)
![Page 24: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/24.jpg)
24
Numeric Data – Value Constraint
• Most of the conflicting updates around numerical values.
• Eg., : Stock of items in a store.• Available tickets in a reservation system• Often characterized by integrity constraint
defined limit• Optimize the general policy by considering the
actual update values to decide on which consistency level enforce
![Page 25: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/25.jpg)
25
Numeric Data – Value Constraint
Fixed threshold policyvalue is below a certain threshold, the record
is handled under strong consistency
Current value – amount <= Threshold
Issues : Inconsistency can occur if the sum of all updates on different servers let the value under watch drop below the limit.Finding the optimal Threshold - experimentally determine over time.Static threshold.
![Page 26: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/26.jpg)
26
Numeric Data – Value Constraint
Demarcation policyThe idea of the protocol is to assign a certain amount of the
value (e.g., the stock) to every server with overall amount being distributed across the servers.Servers may request additional shares from other servers
n – number of the serversv – valueT - Threshold
Issues : Large number of servers n, so this policy will treat B data as A data. All most all transactions require locking
![Page 27: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/27.jpg)
27
Value Constraint – Dynamic Policy
Apply strong consistency protocols only if the likelihood of violating a value constraint becomes high
Commutative updates Without loss of generalitythe limit is assumed to be “0” Assumptions as for thegeneral policy Statistics: Value changeover time
![Page 28: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/28.jpg)
28
Numeric Data – Value Constraint
Dynamic policy:It implements probablistic consistency guarantees by adjusting threshold.
Y is a stochastic variable corresponding to the sum of update values within the cache interval CI.
![Page 29: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/29.jpg)
29
Dynamic policy:
Temporal statistics:probability density function (PDF) f for the cache interval.
![Page 30: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/30.jpg)
30
Dynamic policy:
Using central limit theorem- approximating the CDF
![Page 31: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/31.jpg)
31
Outline
Consistency Rationing Adaptive Guarantees Implementation & Experiments Conclusion and Future Work
![Page 32: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/32.jpg)
32
Implementation
Architecture of “Building aDB on S3” [Sigmod08] Extended protocols Additional services
Locking Service Reliable Queues Counter Service
Every call is priced Approach is not restrictedto this architecture
PNUTS Distributed DB Traditional DB
![Page 33: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/33.jpg)
33
Implementation
![Page 34: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/34.jpg)
34
• Session consistency• Serializability• Meta data• Implementation Alternative using
PNUT
Implementation
![Page 35: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/35.jpg)
35
Experiments
Modified TPC-W
No refilling of the stock Different demand distributions Stock is uniformly distributed between 10-100 Order mix (up to 6 items per basket (80-20 rule) 10 app servers, 1000 products but up to 12,000 products are sold in 300sec Rationed consistency
Results represent just one possible scenario!!!
![Page 36: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/36.jpg)
36
Overall Cost (including the penalty cost)per TRX [$/1000]
![Page 37: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/37.jpg)
37
Overall Cost per TRX [$/1000], vary penalty costs
![Page 38: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/38.jpg)
38
Response Time [ms]
![Page 39: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/39.jpg)
39
Cost per TRX [$/1000]
![Page 40: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/40.jpg)
40
Response Time [ms]
![Page 41: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/41.jpg)
41
![Page 42: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/42.jpg)
42
![Page 43: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/43.jpg)
43
![Page 44: Consistency Rationing in the Cloud: Pay only when it matters Tim Kraska, Martin Hentschel, Gustavo Alonso, Donald Kossmann 27.09.2009 Systems Modified](https://reader035.vdocuments.site/reader035/viewer/2022062804/56649d7d5503460f94a602aa/html5/thumbnails/44.jpg)
44
Conclusion and Future Work
Rationing the consistency can provide big performance and cost benefits
With consistency rationing we introduced the notion of probabilistic consistency guarantees
Self-optimizing system – just the penalty cost is required
Future work Applying it to other architectures Other optimization parameters (e.g. energy) Better and faster statistics