federations in sql azure: building large scale, elastic data tiers in the cloud
DESCRIPTION
Federations simply bring in the sharding pattern into SQL Azure as a first class citizen. Sharding pattern is used for building many of your favorite sites on the web such as social networking sites, auction sites or scalable email applications such as Facebook, eBay and Hotmail. By bringing in the sharding pattern into SQL Azure, federations enable building scalable and elastic database tiers and greatly simplify developing and managing modern multi-tenant cloud applications. Federations scalability model is something you are already greatly familiar with: Imagine a canonical multi-tier application: these applications scale-out their front and middle tiers for scalability. As the demand on the application varies, administrators add and remove new instances of the front end and middle tier nodes to handle the variance in the workload. With Windows Azure Platform this is easily achieved through easy provisioning of new capacity and the pay as you go model of the cloud. However database tiers have typically not provided support for the same elastic scale-out model. However with federations, SQL Azure enable database tiers to scale-out in a similar model. With federations, database tiers can be elastically scaled-out much like the middle and front tiers of the application based on application workload. Using federations, applications can expand and contract the number of nodes that service the database workload without requiring any downtime!-Cihan BiyikogluTRANSCRIPT
– INTERNAL ONLY
Building Large Scale Database Applications in the CloudFederations in SQL Azure
Cihan Biyikoglu Program Manager – SQL Azure
– INTERNAL ONLY
Scalability Model for the Cloud• Cloud Apps Require Scale Beyond Scale-Up
– Massive aggregate capacity: 100s of nodes available for use• Cloud Apps Demand the Best Economics
– Best Price/Performance• Many commodity nodes for the economics
– Elasticity + Pay-as-you-go• Provision just in time and without downtime!• Reduce overcapacity
– INTERNAL ONLY
Introducing Federations in SQL Azure• Canonical 3 tier app scales by adding and removing nodes in front and
middle tiers.• Federations extend the model to the DB Tier
SQL AzureLarger User
Traffic
FrontTierFrontTierFrontTier
FrontTier
FrontTierFrontTierFrontTierMiddle
Tier
– INTERNAL ONLY
Why use Federations?• Gain practically unlimited scale by harnessing 100s of SQL Azure
nodes– Go beyond the limits of a single SQL Azure Database
• Create an elastic database tier that can expand and contract with your applications workload… And without downtime!
– INTERNAL ONLY
Why use Federations?• Build Multi-tenant Solutions– Single tenants per db works… But what about very small tenants and
very large tenants?• Tenant Management with Federations– Federations makes tenant placement and replacement easy.– Change your tenant placement any time without downtime.
Single tenant per databaseMultiple-tenants per database Multiple databases per tenant
Tenancy Models
– INTERNAL ONLY
Who are Federations for? • A few examples– Web Scale DB Solutions– Multi Tenant SaaS ISVs – Workloads with Spikes, Bursts, Peaks etc…– …And also NoSQL Applications• Apps that need bigdata, big scale, massive parallelism, eventual consistency,
lightweights local storage, semi structured data etc.
FEDERATIONS ARCHITECTURE
– INTERNAL ONLY
Overview - Architecture• Federations: Federation define the scheme for the federation
– Federations are objects contained within a user database just like other objects. • Federation Root: DB that houses federation object
– This is the central repository for information about distribution of scaled-out data• Federation Members: System managed SQL Azure DBs that make up the federation
– Each federation member contain parts of federations data
CREATE FEDERATION Orders_Federation(tenant_id BIGINT RANGE)
SalesDBOrders_federationOrders_federationOrders_Fed
Federation MembersFederation Root
Federations
– INTERNAL ONLY
Overview - Concepts• Federation Distribution Key– The key used for data distribution in the federations.
• Atomic Unit– Represent a single instance of a federation key. An AU contains all rows in all
federated tables with the same federation key value.
SalesDB
Orders_federationOrders_federationOrders_Fed
Federation MembersFederation Root
Federations member: Range [1000, 2000)
AUPK=5
AUPK=25
AUPK=35
AUPK=5
AUPK=25
AUPK=35
AUPK=1005
AUPK=1025
AUPK=1035
Atomic Units
– INTERNAL ONLY
Overview – Architecture cont.• Repartitioning Operations without Downtime!– SPLIT members to spread workloads over to more nodes– DROP members to shrink back to fewer nodes
SalesDB
Orders_federationOrders_federationOrders_Fed
[5000, 10000)
ALTER FEDERATION Orders_Federation SPLIT AT (tenant_id=7500)
[5000, 7500) & [7500, 10000)
– INTERNAL ONLY
Reliable Routing • Built-in Data-Dependent Routing (DDR)– DDR ensure app can discover where the data is just-in-time – Apps no longer has to cache ‘shard map’ – Federations guarantee routing to the right member even when
repartitioning operations are going on in the background.
SalesDB
Orders_federationOrders_federationOrders_Fed
USE FEDERATION Orders_Federation(tenant_id=7509) WITH RESET
[5000, 7500) & [7500, 10000)
– INTERNAL ONLY
Related Content
• Federations on the Web– Online Product Documentation• http://msdn.microsoft.com/en-us/library/windowsazure/hh597452.aspx
– A fresh overview of federations• http://social.technet.microsoft.com/wiki/contents/articles/2281.aspx
– My blog• http://blogs.msdn.com/b/cbiyikoglu
– INTERNAL ONLY
© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions,
it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.