megastore - id2220 presentation
Post on 10-May-2015
511 Views
Preview:
DESCRIPTION
TRANSCRIPT
Megastore
Providing Scalable, Highly Available Storage for Interactive Services
Paper by Jason Barker et al. Presented by Arinto Murdopo
arinto@kth.se
7/11/2012 1
Outline
7/11/2012 2
Motivation Megastore:
Features Scalability Availabilty Putting them all together
Observation Conclusions
Motivation
Conflicting requirements • RDBMS – easy to use, but not scale • NoSQL – scale, but not easy to use
Interactive online services • Highly available and fast response time
7/11/2012 3
Here comes Megastore
easy to use • ACID semantics
scalable • data partitioning
highly available • synchronous replication through modified
Paxos
7/11/2012 4
Easy to use - Features
cost-transparent APIs • No API for joins • Joins are implemented in application code
data model • schema, table (entity), property • entity clustering • indexes: local, global • Bigtable column name == Megastore table
name and property name, i.e User.name
7/11/2012 5
Easy to use - Features
transactions and concurrency control • Bigtable for concurrency control • transaction lifecycle: read, application logic,
commit, apply, clean up
others • backup system of transaction logs • encryption
7/11/2012 6
Scalable
Scale the replication scheme Data partitioning
• Entity group concept
Data locality
• Entity group locality • Bigtable instances locality
7/11/2012 7
Entity Groups
7/11/2012 8
Entity is like instance of table. Entity group is group of entities. i.e
Email Application • Email account
Blog Application • User Profile • Blog post + metadata • Blog unique name
Entity Groups
7/11/2012 9
Highly Available
7/11/2012 10
Replicate mutations of write-ahead log inside entity groups using modified Paxos, but let’s revisit original Paxos…
Modified Paxos – Fast Reads
7/11/2012 11
Read in original Paxos
Modified Paxos – Fast Reads
7/11/2012 12
Contact Coordinator and read locally if possible
Modified Paxos – Fast Writes
7/11/2012 13
Skip “prepare” stage in subsequent write of same leader, provided no write from other writers
Modified Paxos – New Replica Types
7/11/2012 14
Full Replicas all replicas that we have seen until now
Witness Replicas
are able to vote store but do not apply write-ahead logs do not store entity data
Read-only Replicas
are not able to vote snapshots of entity data
Putting them all together
7/11/2012 15
Megastore Architecture
Reads
7/11/2012 16
Query Local
Find Position
Catchup
Validate
Query Data
Writes
7/11/2012 17
Accept Leader
Prepare
Accept
Invalidate
Apply
Observation - Availability
7/11/2012 18
Observation – Latency
7/11/2012 19
Conclusion
7/11/2012 20
Megastore and its motivation
Features of megastore • It has ACID semantics • But need to define entity groups • Need to handle inter-group updates
Scalability and Availability
More experiments are needed
top related