SQL Server Data Services

Eric NelsonDeveloper & Platform GroupMicrosoft Ltderic.nelson@microsoft.com http://geekswithblogs.net/IUpdateable http://twitter.com/ericnel

SQL Data Services

Eric NelsonDeveloper & Platform GroupMicrosoft Ltderic.nelson@microsoft.com http://geekswithblogs.net/IUpdateable http://twitter.com/ericnel

The plan...

Cloud101Microsoft and CloudAzure Services Platform

Storage in the CloudSQL Data Services + a little Windows Azure Storage

SQL Data ServicesDrill down

Apologies – this is a demo free session!

If you were watching carefully – we did a u-turn on how we surfaced “database in the cloud” last week. All hail the power of early adopters...

As your time is precious

SQL Data Services programming model is...

TSQL over TDS - SqlClient ODBC, OLEDB, ADO.NET, LINQ to Entities etc.

Exposes most of SQL ServerBeta July, Release November

This session looks at how we are supporting the above, the limitations, the “cunning plan” etc.

No code. No demos

Cloud?

Cloud Computing 101

Run/Store stuff in the cloud -Somebody else has the data center

Amazon.com key role in Cloud Computing

Amazon S3 (Simple Storage Service) – objectsAmazon EC2 (Elastic Compute Cloud) – virtual machines

2008 and 2009 – exciting for Cloud Computing with MicrosoftSQL Server Data Services (SSDS) – announced

at MIX 08 (March 2008)“SQL Server in the cloud”“Code far”

ADO.NET Data Services - part of .NET Framework 3.5 SP1(July 2008)

Not just about Cloud but all about RESTUsed by Windows AzureBeing explored by SQL Data Services

Azure Services Platform – announced at PDC 08 (October 2008)

Windows Azure “O.S. for the Cloud”SQL Services+ more

Azure Services Platform

ServiceBus

AccessControl

Workflow

…

Database

SQL Data Services

Reporting

Analytics

…

Compute Storage Manage

Identity

Devices

Contacts

…

…

…

Your Applications

An internet-scale cloud services platform hosted in Microsoft data centers, which provides an operating system and a set

of developer services that can be used individually or together.

Storage in the Cloud

Azure Services Platform

ServiceBus

AccessControl

Workflow

…

Database

SQL Data Services

Reporting

Analytics

…

Compute Storage Manage

Identity

Devices

Contacts

…

…

…

Your Applications

23

Windows Azure and SDS

Azure Storage SQL Data Services

Vision

Access

Relational? (today)

Relational? (tomorrow)

Analogy

Windows Azure and SDS

Azure Storage SQL Data Services

Vision Highly scalable, highly available store in the Cloud

Access Uses ADO.NET Data Services - REST

Relational? (today)

No

Relational? (tomorrow)

No

Analogy

Windows Azure and SDS

Azure Storage SQL Data Services

Vision Highly scalable, highly available store in the Cloud

Highly scalable, highly available relational store in the Cloud

Access Uses ADO.NET Data Services - REST

SqlClient + TSQL(Use Azure Web Role + ADO.NET Data Services if REST is required)

Relational? (today)

No Yes – but with some limitations

Relational? (tomorrow)

No Yes – with less and less limitations

Analogy

Windows Azure and SDS

Azure Storage SQL Data Services

Vision Highly scalable, highly available store in the Cloud

Highly scalable, highly available relational store in the Cloud

Access Uses ADO.NET Data Services - REST

SqlClient + TSQL(Use Azure Web Role + ADO.NET Data Services if REST is required)

Relational? (today)

No Yes – but with some limitations

Relational? (tomorrow)

No Yes – with less and less limitations

Analogy File System RDBMS – as it is

Windows Azure Datacenter

Your Service

Code far vs Code near

LB

Internet

Web Site(ASPX, ASMX,

WCF)

Web Site(ASPX, ASMX,

WCF)Web Role

(ASPX, WCF)

Worker Service

Worker Role

LB

StorageTables Blobs

Queue

SQ

L D

ata

Serv

ices

Easy

Trickier

SQL Data Services

Extending SQL Data Platform to Cloud

SQL Data Services (SDS)

Data SyncReference Data

Reporting

Business Intelligenc

e

Symmetric Programming Model

• Initial services – core RDBMS capabilities as a service (SDS), Data Sync and Data Hub

• Future Offerings• Additional data platform capabilities as a service: BI/DSS, DW• New services: Reference Data, Secure Data Hub

• Enable new uses of data to deliver differentiated business value

Data Hub Aggregation

The New SQL Data Services

Familiar SQL Server relational modelUses existing APIs & toolsBuilt for the Cloud with availability and scaleAccessible to all from PHP, Ruby, and Java

Clear Feedback: “I want a database in the Cloud”

Focus on combining the best features of SQL Server running at scale with low friction

The Evolution of SDS

Evolv

es

SDS Current

REST/SOAP + ACE Model

Web App

REST Client

Windows Azure

BrowserApplicatio

n

Application

REST Client

Cloud

HT

TP

HT

TP

+R

ES

T

Data

Cente

r

SDS Next

TDS + TSQL Model

Web App

SQL Client*

Windows Azure

BrowserApplicatio

n

Application

REST Client

REST (Astoria)

ADO.Net + EF

Application

SQL Client*

Cloud

HT

TP

HT

TP

+R

ES

T

TD

S

* Client access enabled using TDS for ODBC, ADO.Net, OLEDB, PHP-SQL, Ruby, …

Data

Cente

r

ODBC, OLEDB, ADO.Net PHP, Ruby, …

SDS v1 Scenarios

Web and departmental 5 to 10GBStrip away blob, many applications < 3GB of data

ISVs delivering SaaSAlthough more to do

New scenarios – data sync Aggregation of enterprise, partner, desktop & device data

Logical User DatabasesUses shared infrastructure at SQL database and below

Each user database is replicated to one or more servers (configurable based on SLA)Client requests are routed to current “primary server” for read and write operations (based on SQL session)Security, lockdown and isolation enforced in SQL tier

Highly scalable and state-of-the-art HA technologyAutomatic failure detection; client request re-routed to new primary on failure High SLA guarantee using replication (hot standby replicas)Automatic management, self-healing and load balancing across shared resource pool

SDS provides provisioning, metering and billing infrastructure

Data Node (Machine)

SQL Server

UserDB1

UserDB2

UserDB3

UserDB4

Scalability and Availability: Fabric, Failover, Replication, and Load balancing

SDS Provisioning - databases, accounts, roles, …, metering and billing

Data Node (Machine)

SQL Server

UserDB1

UserDB2

UserDB3

UserDB4

Data Node (Machine)

SQL Server

UserDB3

UserDB4

Data Node (Machine)

SQL Server

UserDB1

UserDB2

UserDB3

Scalability and Availability: Fabric, Failover, Replication, and Load balancing

Multi-Tenancy

Multi-tenancy considerationsThrottling

Get less

Limits DB sizeDuration of transaction…

Load Balancing

Automatic Load Balancing3 replicas

Primary and two hot secondary

Can switch primary to a secondary with no interruption

TDS Gateway

Front-end NodeProtocol Parser

Gateway Logic

Front-end NodeProtocol Parser

Gateway LogicTDS

Session 2

High Level Architecture

Master Cluster

Master Node

Data Node Components

Partition Manager

Master Node

Data Node Components

Partition Manager

Service Platform

Monitoring

Load Balancing

Data Cluster

Data Node

Mgmt. Service

s

SQL Server

Fabric

Data Node

Mgmt. Service

s

SQL Server

Fabric

Data Node

Mgmt. Service

s

SQL Server

Fabric

Data Node

Mgmt. Service

s

SQLServer

Fabric

Fabric Replication

TDSProtocol

Data Node

Mgmt. Service

s

SQL Server

Fabric

Utility Layer

Account Database

Billing/Usage

Database

Database Provisionin

g

RolesTDS

Session 1

Data Cluster

Data Node

Mgmt. Service

s

SQL Server

Fabric

Data Node

Mgmt. Service

s

SQL Server

Fabric

Data Node

Mgmt. Service

s

SQL Server

Fabric

Data Node

Mgmt. Service

s

SQLServer

Fabric

Fabric Replication

TDSProtocol

Data Node

Mgmt. Service

s

SQL Server

Fabric

TDS Gateway

Front-end NodeProtocol Parser

Gateway Logic

TDSSession 1

Front-end NodeProtocol Parser

Gateway LogicTDS

Session 2

High Level Architecture

Master Cluster

Master Node

Data Node Components

Partition Manager

Master Node

Data Node Components

Partition Manager

Service Platform

Monitoring

Load Balancing

1

2

4

5

6

7

8

Utility Layer

Account Database

Billing/Usage

Database

Database Provisionin

g

Roles

3

SDS Features & CompatibilityR

eso

urc

es

Dedicated

Shared

Low High“Friction”/Control

• Deliver unique value propositions to core v1 scenarios

• High compatibility between on/off-premises offering – SQL Data Platform extension

SDS (RDBMS)

Value Props:• Auto HA, Fault-

Tolerance• Friction-free scale• Self-provisioning• High compatibility

•Virtual DB server•Logical user database (LUDB)•Resource governance @ LUDB•Security @LUDB

Value Props:• XSPs, Server Ops• SQL CLR• 100% compatibility

Roll-your-own HA/DR/scale

Hosted RDBMS•Hosted SQL Server

•Resource governance @ VM•Security @ SQL Server/OS

SQL Server•SQL Server on-premises•Resource governance @ machine•Security @ SQL Server/OS

Value Props:• Full h/w control –

size/scale• 100% compatibility• Roll-your-own

HA/DR/scale

Compatibility Goals

Symmetrical ModelPatterns work for Cloud and On Premise

Common application patternsSecurity, schema, code

Logical/policy based administration

V1: Address the needs of 95% or more web and departmental application

SQL CompatibilityIn Scope for v1

Tables, indexes and viewsStored ProceduresTriggersConstraintsTable variables, session temp tables (#t)…

Out of Scope for v1

Distributed TransactionsDistributed QueryCLRService BrokerSpatialPhysical server or catalog DDL and views

Security Model

Standard SQL Server security model

Authenticate logins, map to users and rolesAuthorize users and roles to SQL objectsLimited to standard SQL Auth logins

Username + password

Future AD Federation, WLID, etc as alternate authentication protocols

Security model is 100% compatible with on-premise SQL

Logical vs Physical Administration

SDS focus on logical administrationMove to policy based management

Data management, config and availability

HA “out of box”Transparent failover

DBA role places more focus on policy/logical management

Service Provisioning ModelEach account has

a billing relationship with Microsoftowns one or more virtual servers

Each virtual server has one or more databases including virtual mastereach database limited in size one or more logins

Each logical database has one or more SQL users

Account

Server

DatabaseServer=server1.data.database.windows

.netDatabase=testDBLogin=ericnel[@server1] (maps to

testuser)

Proven Platform powers SDS

Exchange Hosted Archive (EHA) is high scale archival serviceRuns on SDS infrastructure

Goes into production in Spring ’09Rebuilt to address cost and scale issueshttp://go.microsoft.com/fwlink/?LinkId=145374

Reduced COGS with increased customer and business benefit:• Larger scale (e.g. larger customers ~24TB

each)• More self-managing (e.g. fault-tolerance)

• Better query performance for fan-out• Faster provisioning of new customers

Release Plan

Invitation CTP – On-board invitation-only customers + some current

Public CTP – On-board all early adopters

Commercial Availability with Windows Azure in 2009

MIX ‘09

TechEd WWPC

InvitationCTP

Public CTP

Commercial Availability

Migration

Mar 09

May 09 Jul 09 Nov 09

Sep 09

Apr 09

Jun 09 Aug 09

Oct 09

PDC

InternalAdoption

And then what …

Partitioned databasesMove management from application into service

Distributed queryFan-out over partitions or collection of database instances

SecuritySQL Login support federated with AD, WLID, …

SaaSSchema management and deploymentBilling-on-behalf, tenant provisioning and management

Consolidation “dial”Part of server (current), whole server, set of servers, …Enable dedicated resource assignment and differentiated SLA

Key Takeaways

Underlying platform is real and in production

Delivery of rich relational database service – PDC ’09

Relational database is key capability of the Azure Services Platform

SDS design and architecture supports a wide range of scenarios

What next?

•Play with Windows Azure – http://www.azure.com•Do not play with SDS just yet•Invest time in ADO.NET Data Services (3.5 SP1)•Invest time in LINQ, Entity Framework•Watch the MIX 09 recordings http://live.visitmix.com

2009“Learn”

•Live applications running on Azure Services Platform•(There already are e.g. http://www.aws.net/services/cloud-services/case-study )

2010“Do”

Resources

Team Bloghttp://blogs.msdn.com/ssds

MSDN Development Centerhttp://msdn.microsoft.com/en-us/sqlserver/dataservices

My bloghttp://geekswithblogs.net/IUpdateable

© 2008 Microsoft Ltd. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the

date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.