Locking Range Resources

Locking Key Ranges with Unbundled Transaction Services1David LometMicrosoft ResearchMohamed MokbelUniversity of Minnesota1Talk OutlineUnbundled Transaction ServicesChallenge: Key Range LockingUsing Conventional Locking Managers for Unbundled Transaction ServicesThe Partition Lock ProtocolPerformance AnalysisConclusion and Summary22Present Day engine ArchitectureSplit between Query Processing and DB Kernel

DB Kernel Provides Tightly Integrated CodeAccess methodsData caching & persistenceConcurrency controlRecovery

Query ProcessingConcurrency ControlRecoveryCachingAccess MethodsDB Kernel33Big Picture of New Kernel Architecture*Split the DB Kernel to Transaction component (TC) and Data component (DC)

Transaction Component (TC) Concurrency control and recoveryHas a logical level knowledge about keys and recordsNo knowledge of pages, buffers, physical structureData Component (TC) Access methods & cacheProvides atomic logical operationsNo knowledge of how they are grouped in user transactions

Query ProcessingConcurrency ControlRecoveryDB Kernel4CachingAccess MethodsTransaction Component(TC)Data Component (DC)* David Lomet, Alan Fekete, Gerhard Weikum, and Mike Zwiling, Unbundling Transaction Services in the Cloud. In CIDR 2009

4Why might this be interestingExtensible DBMSProvider of new access method does not need to deal with transactional concurrency control & recoveryArchitectural advantage whether this is user or system builder extensionCloud Data Store with TransactionsTC coordinates transactions across distributed collection of DCs Can add TC to data store that already supports atomic operations on data; without changing, perhaps, the data storeQuery ProcessingConcurrency ControlRecoveryDB KernelCachingAccess MethodsTransaction Component(TC)Data Component (DC)5Multi-core ArchitectureRun TC and DC on separate cores to better exploit parallelism and reduce instruction cache misses5Talk OutlineUnbundled Transaction ServicesChallenge: Key Range LockingUsing Conventional Locking Managers for Unbundled Transaction ServicesThe Partition Lock ProtocolPerformance AnalysisConclusion and Summary

66Challenge: Key Range LockingPresent Day Engine ArchitectureThe storage engine does know all the keys within R and can request a lock for each single key in R

UPDATE Student SET GPA= GPA+0.25 WHERE ID 10 and ID 40Required Locks: Need to lock all the records in the range R = [10,40]

New Kernel ArchitectureThe TC has no idea about the keys in R and thus has no way of requesting any lock on any specific key

There is a need for a new locking protocol that allows the TC to lock the required range without knowing the internal keys

77Key Range Locking in SQL ServerSQL Server lock manager has special range modes that are used to lock ranges between keys to prevent phantomsA key range lock mode is interpreted by the storage engine as a lock on the key and a lock on the interval between the key and the next keyWe are not planning to replace key range locking. Our proposal is to supplement it to: (a) avoid locking each single key and (b) work without the knowledge of keys in the range8Key range locking requires locking each single key in the requested range, and thus, cannot be used in the new DB Kernel architecture as it requires the knowledge of the keys inside the range

8Talk OutlineUnbundled Transaction ServicesChallenge: Key Range LockingUsing Conventional Locking Managers for Unbundled Transaction ServicesThe Speculative Visit ApproachThe Table Lock ApproachThe Partition Lock ProtocolPerformance AnalysisConclusion and Summary

99The Speculative Visit ApproachMain idea:The TC takes a speculative visit to the DC to find the key that requires locking. The TC acquires a lock on the returned key from the DCThe TC takes a certification visit to the DC to confirm that there are no changes between the two visitsIf there is no difference between the two visits, the record access is completed, otherwise, the DC returns failure and the protocol is restarted10Speculative visit

Certification visitTCDC10The Speculative Visit ApproachEnhancement:Combine the certification visit with the speculative visit for the next key

Advantage:No change in the lock managerHigh concurrency

DisadvantageData records are accessed at least twiceConcurrent transactions may result in having multiple visits to the same record before being able to acquire a lock1111The Table Lock ApproachMain idea: Acquire a lock on a larger granule than a recordWe cannot go for pages as we would have the same problem as with the records (the TC does not know what are the pages to be accessed)So, we need to lock the whole table in order to safely access the records

12Advantage:No change in the lock manager. Very simple to implement

DisadvantageVery low concurrency as we turn the system off by locking the whole table

12Talk OutlineUnbundled Transaction ServicesChallenge: Key Range LockingUsing Conventional Locking Managers for Unbundled Transaction ServicesThe Partition Lock ProtocolPerformance AnalysisConclusion and Summary

1313The Partition Covering LockIdea: Have the range as a resource (similar to keys, pages, and tables) rather than a mode (similar to Key Range locking in MS SQL Server)

The TC can request a lock on a range resource from the DC

Each table is partitioned to a set of range resourcesThere are several alternatives for this partitioning

The boundary of the range resource does not have to be keys in the table. This means that the TC can know these boundaries without the need to lock at the data

1414The Partition Locking ProtocolThe TC maps the range R=[kl,ku], requested by a query, into the set of partition resources on which the TC can request locks.(The TC does know the available range resources)

The TC requests a lock on each partition before accessing its recordsThis lock acts as a cover lock in which the TC can talk safely with the DC to discover the records inside the locked range resource

When multiple partitions are required to cover the range R (the most common case), we lock the partitions one at a time, as we access records sequentially within R.

1515Partition Locking Protocol: ExampleA shared lock (S) is requested on a range of values R=[kl,ku] under the SERIALIZABLE isolation level.

16Boundary PartitionBoundary PartitionInternal PartitionsKlKuS LockS LockIS LockS LockS LockS LockIS LockPreventing Phantoms at the end points of the rangeLocking a key and its preceding range

Indicating the end of the partitionWhen there are no more records within the partition, the DC indicates this by providing an end record sign.

Preventing phantoms between partitionsNeed to place a key range lock on the first key of an internal partition before demoting the partition lock on the boundary partition

16Talk OutlineUnbundled Transaction ServicesChallenge: Key Range LockingUsing Conventional Locking Managers for Unbundled Transaction ServicesThe Partition Lock ProtocolPerformance AnalysisConclusion and Summary

1717Performance AnalysisLocking ProtocolsKey Lock: Conventional lock managers Though it is not applicable to unbundled transaction servicesThe Speculative Visit protocolThe Table Lock protocolThe Range Lock approach

Performance measuresConcurrency control overhead due to locking and reading the data in the requested rangeLoss of concurrency incurred by holding locks on resources for some period of time1818Performance AnalysisConcurrency Control Overhead19

19Performance AnalysisLoss of Concurrency20

20Talk OutlineUnbundled Transaction ServicesChallenge: Key Range LockingUsing Conventional Locking Managers for Unbundled Transaction ServicesThe Partition Lock ProtocolPerformance AnalysisConclusion and Summary2121Conclusion and SummaryUnbundled transaction services call for a new locking protocol to handle locking a range of data

Conventional lock managers can be used to support unbundled transaction services, yet, with very inefficient performanceSpeculative visit protocolTable lock protocol

The partition locking protocol is proposed to support efficient key range locking for unbundled transaction servicesThe main idea is to add a new range resource in the locking hierarchy; in which the TC has a fair knowledge about it

Analytical models show that the new proposed partition locking protocol has a much lower locking overhead and provides better concurrency than other locking protocols2222Questions..?2323Partition Locking Protocol: Dealing with Empty PartitionsInternal partitionsThe locking protocol is unchangedFinal boundary partitionTreat the empty end partition as an internal partitionInitial boundary partitionDemote the partition lock to an intention lock, but need to ensure that the first record in the range has a key range lock to protect this early part of the range

24KlKuS LockIS LockS LockS LockS Lock24