high availability & scalability in sql server 2005 rick brewis, microsoft technology specialist...

High Availability & Scalability High Availability & Scalability in SQL Server 2005 in SQL Server 2005

Rick Brewis, MicrosoftRick Brewis, MicrosoftTechnology SpecialistTechnology [email protected]@microsoft.com

Brian Madden, NuSoft SolutionsBrian Madden, NuSoft SolutionsSenior ConsultantSenior [email protected]@nusoftsolutions.com

mailto:[email protected]

mailto:[email protected]

High Availability and High Availability and Scalability: AgendaScalability: Agenda

High AvailabilityHigh AvailabilityAlso known as business continuanceAlso known as business continuance

Refers to technologies to ensure Refers to technologies to ensure continued availability – a matter of kindcontinued availability – a matter of kind

Primary new technology: database Primary new technology: database mirroringmirroring

ScalabilityScalabilityKeeping a database available as you scale Keeping a database available as you scale up or outup or out

Primary new technology: partitioningPrimary new technology: partitioning

High Availability OverviewHigh Availability Overview

HA

Technology

ProcessPeople

High Availability – Overview High Availability – Overview

Blend of people, process, and Blend of people, process, and technologytechnology

Process enforces “best practices”Process enforces “best practices”

Technology enables people and Technology enables people and processes to come together in processes to come together in providing maximum uptimeproviding maximum uptime

SQL Server 2005 HA SQL Server 2005 HA TechnologiesTechnologies

Cold StandbyCold StandbyNo failover with potential data lossNo failover with potential data lossBackup / restoreBackup / restoreDetach / copy / attachDetach / copy / attach

Warm StandbyWarm StandbyManual failover with potential data lossManual failover with potential data lossLog ShippingLog ShippingTransactional ReplicationTransactional ReplicationDatabase Mirroring - high performance modeDatabase Mirroring - high performance mode

Hot StandbyHot StandbyAutomatic failover and zero data lossAutomatic failover and zero data lossFailover ClusteringFailover ClusteringDatabase Mirroring - high availability modeDatabase Mirroring - high availability mode

Cold Standby SolutionsCold Standby SolutionsBackup / RestoreBackup / Restore

Data files may be smaller – only used Data files may be smaller – only used pages are backed uppages are backed upLog backups allow restore to point in timeLog backups allow restore to point in timeGenerally a longer restore timeGenerally a longer restore time

Detach / Copy / AttachDetach / Copy / AttachCopies entire filesCopies entire filesNo possibility of rolling forward No possibility of rolling forward subsequent logssubsequent logs

For a cold standby:For a cold standby:Manually detect failure and fail overManually detect failure and fail overSome data lossSome data lossPer database only, not per serverPer database only, not per serverWorks on standard serversWorks on standard serversClient must know how to re-connectClient must know how to re-connectSlowest failover with most downtimeSlowest failover with most downtime

Backup / RestoreBackup / Restore

Detach / Copy / AttachDetach / Copy / Attach

Warm Standby 1: Log Warm Standby 1: Log ShippingShipping

Provide Multiple copies and Provide Multiple copies and Manual failoverManual failover

Log ShippingLog ShippingBasic idea: Automated Backup, Basic idea: Automated Backup, Copy, Restore Log Copy, Restore Log

Database scopeDatabase scope

Secondary database is accessible Secondary database is accessible but read-onlybut read-only

Users must exit for next log to be Users must exit for next log to be appliedapplied

Database backups no longer block Database backups no longer block log backupslog backups

Log ShippingLog Shipping

Log ShippingLog ShippingMonitoring Server

Transfer Logins

Primary Server

Secondary Server(s)

3. Transaction log restored

1. Transaction log backed up

LogBackup

2. Transaction log copied

LogBackup

Log Shipping NotesLog Shipping Notes

Automatically copies and restores Automatically copies and restores transaction log files from the primary to transaction log files from the primary to one or more secondary databasesone or more secondary databases

Transfers all database objects, all changesTransfers all database objects, all changes

Managing large log files can be a problemManaging large log files can be a problem

SQL Server 2005 log shippingSQL Server 2005 log shippingRevised from SQL Server 2000Revised from SQL Server 2000

Available on EE, Std, and WGAvailable on EE, Std, and WG

Warm Standby 2: Peer-to-peer Warm Standby 2: Peer-to-peer Transactional ReplicationTransactional Replication

Primarily designed for Primarily designed for availabilityavailability

Often used for scale out of read Often used for scale out of read activityactivityOnly data is copied, not system Only data is copied, not system tablestables

Failover must be Failover must be customized, but can be customized, but can be automaticautomaticOnly copies dataOnly copies data

Can work with a subset of a Can work with a subset of a databasedatabaseReplicated data is accessible for Replicated data is accessible for read activity (i.e. reporting)read activity (i.e. reporting)Very low latency: secondsVery low latency: seconds

ReplicationReplication

Distribution Distribution AgentAgent

DistDistDBDB

Logreader Logreader AgentAgent


DistDistDBDB



DistDistDBDB


““West”West” ““East”East”

““South”South”

Peer-to-peer Transactional Peer-to-peer Transactional ReplicationReplication

Peer-to-Peer Transactional Peer-to-Peer Transactional Replication NotesReplication Notes

New in SQL Server 2005New in SQL Server 2005All nodes are peers of each otherAll nodes are peers of each other

Table schema must be identical at all sitesTable schema must be identical at all sitesEach node publishes updates made on its dataEach node publishes updates made on its dataEach node subscribes to other nodes and receives their Each node subscribes to other nodes and receives their changeschanges

A given set of data (row) can be updated at only A given set of data (row) can be updated at only one site at a timeone site at a time

Data range ownership is application-definedData range ownership is application-definedNo conflict detectionNo conflict detectionSQL Server prevents an update from cascadingSQL Server prevents an update from cascading

Enables load-balancing and high availabilityEnables load-balancing and high availabilityCan be used for a warm or hot standbyCan be used for a warm or hot standby

Small possibility of data loss on failure, Small possibility of data loss on failure, depending on latencydepending on latency

Hot Standby 1: ClusteringHot Standby 1: Clustering

Automatic failoverAutomatic failoverAt the SQL Server instance level, At the SQL Server instance level, not database levelnot database levelPreserves server and database Preserves server and database dependenciesdependenciesNo committed work loss No committed work loss

Based on Microsoft Cluster Based on Microsoft Cluster Server (MSCS)Server (MSCS)

Multiple nodes provide Multiple nodes provide availabilityavailability

Maximum of 2, 4, or 8 nodes Maximum of 2, 4, or 8 nodes depending on OS editiondepending on OS edition

Automatic detection and failover, Automatic detection and failover, transparent to clienttransparent to clientRequires Cluster-certified Requires Cluster-certified hardware:hardware:

See the Windows Catalog: See the Windows Catalog: ClusteredClustered

Failover Cluster

ClusteringClustering

Client PCs

Node A Node B

Shared Disk Array

HeartbeatSQL ServerSQL Server SQL ServerSQL Server

Clustering Notes (SQL Clustering Notes (SQL Server 2005)Server 2005)

More nodes are now supportedMore nodes are now supportedMatches the Windows OS limitsMatches the Windows OS limitsTwo-node Failover Clustering is available in StandardTwo-node Failover Clustering is available in Standard

Unattended setup now availableUnattended setup now availableSupport for mounted volumes (Mount Points)Support for mounted volumes (Mount Points)Full support for Windows Majority Node Set quorumFull support for Windows Majority Node Set quorumAll SQL Server data services participateAll SQL Server data services participate

Database Engine, SQL Server Agent, Database Engine, SQL Server Agent, Full-Text SearchFull-Text SearchAnalysis Services – Now has multiple instancesAnalysis Services – Now has multiple instances

Managing and troubleshooting a cluster can be Managing and troubleshooting a cluster can be complexcomplex

Hot Standby 2: Database Hot Standby 2: Database MirroringMirroring

Databases are mirrored, not Databases are mirrored, not instancesinstances

Does not account for database and Does not account for database and server dependenciesserver dependencies

Automatic FailoverAutomatic FailoverVery fast failoverVery fast failover

Less than five seconds in many casesLess than five seconds in many cases

No loss of committed workNo loss of committed work

Manual failover also possibleManual failover also possibleNew mirror can automatically re-sync New mirror can automatically re-sync after failoverafter failover

Potential for automatic, Potential for automatic, transparent client redirecttransparent client redirect

Client must use ADO.NET or SQL Client must use ADO.NET or SQL Native ClientNative Client

Database Mirroring

Database Mirroring: HA Database Mirroring: HA modemode

MirrorPrincipal

Witness

ApplicationApplication

SQLSQL ServerServer SQL ServerSQL Server

1

2 3

2

4

5

Database Mirroring ModesDatabase Mirroring Modes

High-Availability ModeHigh-Availability ModeSafety set to FULL: synchronous operationSafety set to FULL: synchronous operationDatabase is available whenever a quorum existsDatabase is available whenever a quorum existsWitness is required for automatic failoverWitness is required for automatic failover

High-Protection ModeHigh-Protection ModeSafety FULL: synchronous operationSafety FULL: synchronous operationNo witness – quorum provided by partnersNo witness – quorum provided by partnersIf the principal loses quorum, it stops servicing the If the principal loses quorum, it stops servicing the databasedatabaseEnsures high protection; database is never in ‘exposed’ Ensures high protection; database is never in ‘exposed’ statestateManual failover only, no automatic failoverManual failover only, no automatic failoverDesigned as a transition modeDesigned as a transition mode

High-Performance ModeHigh-Performance ModeSafety set to OFF: asynchronous operationSafety set to OFF: asynchronous operationManual failover only: forced service with potential data lossManual failover only: forced service with potential data loss

Database Mirroring Notes 1Database Mirroring Notes 1

HardwareHardwareWorks with standard computers, storage, Works with standard computers, storage, and networksand networks

No shared storage components, virtually No shared storage components, virtually no distance limitationsno distance limitations

Impact to transaction throughputImpact to transaction throughputZero to minimal, depending on Zero to minimal, depending on environment / workloadenvironment / workload

Database Mirroring Notes 2 Database Mirroring Notes 2

Role of the Witness is to verify Role of the Witness is to verify principal and mirror for producing a principal and mirror for producing a quorumquorum

To automatically survive the loss of one To automatically survive the loss of one server you must have at least threeserver you must have at least three

Prevents a “split brain”Prevents a “split brain”

Two out of three vote required for failoverTwo out of three vote required for failover

To become a new principal, the mirror To become a new principal, the mirror must talk to at least one other servermust talk to at least one other server

The witness settles the issue of whether The witness settles the issue of whether the principal is downthe principal is down

WitnessWitness

The witness must be an instance of The witness must be an instance of SQL Server 2005SQL Server 2005

Can be a SQL Server Express instanceCan be a SQL Server Express instance

A server can be a witness for multiple A server can be a witness for multiple mirroring sessionsmirroring sessions

Witness consumes very little resourcesWitness consumes very little resources

The witness is not a single point of The witness is not a single point of failurefailure

The mirroring partners can form a quorum The mirroring partners can form a quorum on their ownon their own

Safety versus PerformanceSafety versus PerformanceDatabase Mirroring has two safety Database Mirroring has two safety levelslevels

FULL – commit when logged on MirrorFULL – commit when logged on MirrorAllows automatic failoverAllows automatic failover

No data lossNo data loss

OFF – commit when logged on PrincipalOFF – commit when logged on PrincipalSystem does its best to keep upSystem does its best to keep up

Prevents failover; to make mirror availablePrevents failover; to make mirror availableMust ‘force’ serviceMust ‘force’ service

Or terminate Database Mirroring sessionOr terminate Database Mirroring session

Transparent Client RedirectTransparent Client Redirect

Client automatically redirected if the Client automatically redirected if the principal is downprincipal is down

Upon initial connection to a principal, Upon initial connection to a principal, the client library caches the mirror the client library caches the mirror name from the principalname from the principal

If the client attempts to reconnect If the client attempts to reconnect because the connection is lostbecause the connection is lost

If the old principal is not available, the If the old principal is not available, the client library automatically redirects client library automatically redirects connection to mirror (new principal)connection to mirror (new principal)

Comparing SQL Server Comparing SQL Server 2005 HA Technologies2005 HA Technologies

HA Technology Enterprise Standard Workgroup Express

Log Shipping Yes Yes Yes N/A

Peer-to-Peer Replication

Yes ? ? N/A

Failover Clustering Yes 2-node only

N/A N/A

Database Mirroring Yes Safety Full

Witness only

Witness only

Scalability/Availability Scalability/Availability EnhancementsEnhancements

Availability EnhancementsAvailability Enhancements

Database SnapshotsDatabase Snapshots

Table PartitioningTable Partitioning

Availability EnhancementsAvailability EnhancementsOnline OperationsOnline Operations

Checksum verificationChecksum verificationFast recoveryFast recoveryOnline restoreOnline restoreBackup to mirrored mediaBackup to mirrored mediaInstant file initializationInstant file initializationFull-text backup and restoreFull-text backup and restoreOnline Index OperationsOnline Index Operations

Snapshot IsolationSnapshot IsolationReaders don’t block Writers, Writers don’t block Readers don’t block Writers, Writers don’t block readersreaders

Database SnapshotsDatabase SnapshotsTable partitioningTable partitioning

Database SnapshotsDatabase Snapshots

User ErrorUser Error

Users, applications, and DBAs do Users, applications, and DBAs do make errorsmake errors

Database Snapshots solve the problem Database Snapshots solve the problem by allowing the database to go back in by allowing the database to go back in timetime

Data lost as database goes back in timeData lost as database goes back in time

Must be created before the errorMust be created before the error

SnapshotsSnapshotsGeneralGeneral

Snapshot of a database at a point in Snapshot of a database at a point in timetime

Created instantlyCreated instantlyRead onlyRead only

Base database continues to changeBase database continues to changeSnapshot does not restrict the base Snapshot does not restrict the base databasedatabase

Snapshot requires a different database Snapshot requires a different database name from base databasename from base databaseRevert to previously created Snapshot Revert to previously created Snapshot to recover from errorsto recover from errors

SnapshotSnapshotTechnologyTechnology

Extremely space efficientExtremely space efficient

Does Does notnot require a complete copy of require a complete copy of the datathe data

Shares unchanged pages of the databaseShares unchanged pages of the database

Only requires extra storage for changed Only requires extra storage for changed pagespages

Uses a “copy on write” mechanismUses a “copy on write” mechanism

Snapshot does affect performance on Snapshot does affect performance on the base databasethe base database

ValueValue

RR

DD

BB

HH

JJ

LL

YY

MM

Space Used

Command

Create Northwind_SS

Northwind Northwind_SS

Update Northwind

0%

ValueValue

DDXX

Read Northwind_SS

12.5%

Result:

Snapshot (Copy on Write)Snapshot (Copy on Write)

DD

SnapshotsSnapshotsDatabase MirroringDatabase Mirroring

Multiple Snapshots are allowed on the Multiple Snapshots are allowed on the MirrorMirror

Each Snapshot is taken at a different timeEach Snapshot is taken at a different time

Each Snapshot has a different nameEach Snapshot has a different name

Snapshot on Mirror may affect Snapshot on Mirror may affect performance on Principalperformance on Principal

Snapshots can exist foreverSnapshots can exist foreverConstrained by resourcesConstrained by resources

Reporting on a MirrorReporting on a MirrorUse database Snapshots on MirrorUse database Snapshots on Mirror

Mirror

Principal

Reporting Client

Database MirroringDatabase Mirroring

OLTP Clients

SnapshotSnapshot@ Noon@ Noon

Witness

Snapshot2Snapshot2@ 2PM@ 2PM

Reporting on the MirrorReporting on the MirrorLimitationsLimitations

Schema of Snapshot cannot changeSchema of Snapshot cannot change

Snapshot is staticSnapshot is staticNew data is not availableNew data is not available

Snapshot may affect performanceSnapshot may affect performance

Consider Replication for a dedicated, Consider Replication for a dedicated, scale out reporting solutionscale out reporting solution

Table PartitioningTable Partitioning

Table PartitioningTable PartitioningScale-up:Scale-up:

Allow easy management of very large tables and Allow easy management of very large tables and indexes (data scale-up)indexes (data scale-up)

For example Fast Insert or Delete of large For example Fast Insert or Delete of large quantities of dataquantities of data

SQL Server Enterprise Edition featureSQL Server Enterprise Edition feature

Take advantage of large machinesTake advantage of large machines Better scaling of some operatorsBetter scaling of some operators

Take advantage of collocation Take advantage of collocation Joins of large tablesJoins of large tables

Improve concurrencyImprove concurrencyLock escalation only within a partitionLock escalation only within a partition

How Partitioning WorksHow Partitioning Works

Data in a Table is separated into Data in a Table is separated into different pieces instead of a single heap different pieces instead of a single heap or B-Treeor B-Tree

Data still logically belongs to a single Data still logically belongs to a single tabletable

A, B, C D, E, F G, H, I J, K, L

A, B, C, D, E, F, G, H, I, J, K, LNon-Partitioned

Partitioned

What Can Be Partitioned?What Can Be Partitioned?

Objects that may be partitionedObjects that may be partitionedBase Tables Base Tables

Indexes (Secondary or Clustered)Indexes (Secondary or Clustered)

Indexed ViewsIndexed Views

Row is the smallest unit of partitioning Row is the smallest unit of partitioning (only “horizontal” partitioning)(only “horizontal” partitioning)

Partitioned Table Partitioned Table

TableTableIDID c1c1 c2c2 c3c3 c4c411 3030 AA

22 5050 BB

33 2020 BB

44 1010 LL

55 5050 YY

66 5050 AA

77 2020 FF

Filegroup3Filegroup5Filegroup2

Filegroup5

Filegroup1

Filegroup5Filegroup2

30

50

20

50

10

50

20

Part. Function

Part. Scheme

Partition Function ObjectPartition Function Object

CREATE PARTITION FUNCTIONCREATE PARTITION FUNCTIONPFR (smallint) PFR (smallint) AS AS RANGE LEFT RANGE LEFT FOR VALUES (10,20,30,40,50,60)FOR VALUES (10,20,30,40,50,60)RANGE partition function specifies the boundaries RANGE partition function specifies the boundaries of the rangesof the ranges

LEFT or RIGHT keyword specifies to which of the intervals LEFT or RIGHT keyword specifies to which of the intervals the boundary value belongsthe boundary value belongs

There are no holes in the partitioned domain even if There are no holes in the partitioned domain even if not all values are attainablenot all values are attainable

Discourages use of the PF as a constraintDiscourages use of the PF as a constraint

Partition Scheme ObjectPartition Scheme Object

Assigns partitions to physical storage Assigns partitions to physical storage (Filegroups)(Filegroups)

Partitions can be mapped to the same or Partitions can be mapped to the same or different Filegroupsdifferent Filegroups

In SQL Server 2000, Tables and Indexes In SQL Server 2000, Tables and Indexes could be mapped to Filegroups – now they could be mapped to Filegroups – now they can also be mapped to partition schemescan also be mapped to partition schemes

A Partition Function can participate in A Partition Function can participate in multiple Partition Schemesmultiple Partition Schemes

Partitioning Key RestrictionsPartitioning Key Restrictions

Must be based on columns in the tableMust be based on columns in the table

Restrictions similar to index key Restrictions similar to index key limitationslimitations

Only “Native Types” – no User-Defined Only “Native Types” – no User-Defined TypesTypes

Column values must be deterministicColumn values must be deterministic

Deterministic imprecise columns must be Deterministic imprecise columns must be persistedpersisted

New Varchar(Max) is supported as New Varchar(Max) is supported as partitioning keypartitioning key

Other Partitioning Other Partitioning RestrictionsRestrictions

Maximum of 1000 Partitions per table in Maximum of 1000 Partitions per table in SQL Server 2005SQL Server 2005

Partitions must all be on a single Partitions must all be on a single databasedatabase

Distributed Partitioned Views can be used Distributed Partitioned Views can be used in conjunction with table partitioningin conjunction with table partitioning

Other Query EnhancementsOther Query Enhancements

Many other operations can be Many other operations can be performed “per-partition”performed “per-partition”

Grouping, Filtering, ProjectionGrouping, Filtering, Projection

Inserts, Updates, DeletesInserts, Updates, Deletes

Create Index, Bulk InsertCreate Index, Bulk Insert

SQL Server tries to find large groups of SQL Server tries to find large groups of operations that can be performed per-operations that can be performed per-partition to improve query performancepartition to improve query performance

Adding/Dropping PartitionsAdding/Dropping Partitions

ALTER PARTITION FUNCTION pfr() {SPLIT|ALTER PARTITION FUNCTION pfr() {SPLIT|MERGE} [RANGE (boundary_value)]MERGE} [RANGE (boundary_value)]

Ranges are added by specifying a new boundary Ranges are added by specifying a new boundary point – this SPLITs the existing partitionpoint – this SPLITs the existing partition

Note that ALL tables and indexes using the Note that ALL tables and indexes using the partition function are affected by this partition function are affected by this operationoperation

Example: Add a new month to a partitioned Example: Add a new month to a partitioned sales table (partitioned by month)sales table (partitioned by month)

Partition OperationsPartition Operations

SPLIT - new partition is automatically SPLIT - new partition is automatically populated populated

For RANGE - by moving some rows from the For RANGE - by moving some rows from the partition that is splitpartition that is split

MERGE – data is automatically moved MERGE – data is automatically moved RANGE – to the neighboring “remaining” RANGE – to the neighboring “remaining” partitionpartition

SWITCHSWITCH

SWITCH PartitionSWITCH Partition

Method to move a whole partition into Method to move a whole partition into or out of a table efficientlyor out of a table efficiently

Enables important manageability Enables important manageability scenariosscenarios

All indexes on table must be “aligned” All indexes on table must be “aligned” (plus other restrictions)(plus other restrictions)

SWITCHing a Partition in SWITCHing a Partition in

Q1 Q2 Q3Q4 =

Empty

Table1

Q4

Table2

ALTER TABLE Table2 SWITCH PARTITION TO Table1 PARTITION 4

SWITCH BenefitsSWITCH Benefits

Performance - allow building new and Performance - allow building new and removing old partition fast removing old partition fast

Availability - allow adding new and Availability - allow adding new and removing old partition with minimal removing old partition with minimal downtimedowntime

Per-partition manageability – support Per-partition manageability – support as many utilities as possible to work on as many utilities as possible to work on a per-partition basisa per-partition basis

Large Table ChallengeLarge Table Challenge

Load and Delete from a large tableLoad and Delete from a large tableTable has indexes => load is slow; about Table has indexes => load is slow; about 10+ times slower than loading into a heap 10+ times slower than loading into a heap and creating indexes afterwardsand creating indexes afterwards

Delete is slow – deleting rows is orders of Delete is slow – deleting rows is orders of magnitude slower than truncating a multi-magnitude slower than truncating a multi-GB tableGB table

Cost of running a utility usually grows Cost of running a utility usually grows linearly with the table sizelinearly with the table size

Large Table SolutionLarge Table Solution

Q: How to populate empty partition? Q: How to populate empty partition?

A: Build it outside the partitioned table A: Build it outside the partitioned table (BCP into a heap, create indexes), and then (BCP into a heap, create indexes), and then make it a new partition (SWITCH in)!make it a new partition (SWITCH in)!

Q: How to delete all data in one partition? Q: How to delete all data in one partition?

A: SWITCH partition out of the partitioned A: SWITCH partition out of the partitioned table into another table then back it up, table into another table then back it up, drop it or truncate itdrop it or truncate it

SWITCH SolutionSWITCH Solution

Allows you to add and remove whole Allows you to add and remove whole partitionpartition

Advantage Advantage PerformancePerformance

DisadvantagesDisadvantagesNew commandNew command

Restrictions – tables must satisfy certain Restrictions – tables must satisfy certain conditions to allow SWITCHconditions to allow SWITCH

SWITCH SolutionSWITCH Solution

Allows a separate table to become a Allows a separate table to become a partition of existing partitioned table partition of existing partitioned table (move partition “IN”)(move partition “IN”)

Allows a partition of a table to become Allows a partition of a table to become a new separate table (move partition a new separate table (move partition “OUT”)“OUT”)

In most general solution it allows to In most general solution it allows to “SWITCH” a partition between two “SWITCH” a partition between two partitioned tablespartitioned tables

SWITCH RequirementsSWITCH Requirements

Both the source and target tables must Both the source and target tables must existexist

Target table or partition must be emptyTarget table or partition must be empty

Metadata only operation – no data Metadata only operation – no data scans, copies. Thereforescans, copies. Therefore

+ SWITCH is perfectly scalable and fast (1 + SWITCH is perfectly scalable and fast (1 second or so)second or so)

- SWITCH has some restrictions- SWITCH has some restrictions

SWITCH RestrictionsSWITCH Restrictions

Restrictions are derived from the Restrictions are derived from the principles of no data movement, scansprinciples of no data movement, scans

Columns must have same namesColumns must have same names

All target indexes must exist in the All target indexes must exist in the sourcesource

All indexes must be alignedAll indexes must be aligned

Corresponding target index is in the Corresponding target index is in the same filegroup as the sourcesame filegroup as the source

Sliding Window ScenarioSliding Window Scenario

Large Database Large Database

Each hour/day/week/month… add new Each hour/day/week/month… add new partition and remove the oldest onepartition and remove the oldest one

New partition – may need load, scrub, New partition – may need load, scrub, transformation before becoming a part transformation before becoming a part of the whole table; or it starts as empty of the whole table; or it starts as empty and is populated by transactionsand is populated by transactions

Old partition – may need backup, Old partition – may need backup, archive, restore archive, restore

Sliding Window ScenarioSliding Window Scenario

Think through your scenarioThink through your scenario

Ensure minimal data movement forEnsure minimal data movement forSPLITSPLIT

MERGEMERGE

Empty partition should be the one that Empty partition should be the one that is “SPLIT”is “SPLIT”

Watch how filegroups are assigned to Watch how filegroups are assigned to partitionspartitions

MERGE may cause data movement even if MERGE may cause data movement even if one partition is emptyone partition is empty

Partitioning SummaryPartitioning Summary

Partitioning in SQL Server 2005 is Partitioning in SQL Server 2005 is primarily forprimarily for

Data ManagementData Management

SCALE UP on large (>8 way) machinesSCALE UP on large (>8 way) machines

Plan for itPlan for itUnderstand the benefits and costsUnderstand the benefits and costs

Identify the problem that you want to solve Identify the problem that you want to solve with partitioningwith partitioning

ResourcesResources

SQL Server 2005 Mission Critical High AvailabilitySQL Server 2005 Mission Critical High Availability http://www.microsoft.com/technet/prodtechnol/sql/themes/high-avhttp://www.microsoft.com/technet/prodtechnol/sql/themes/high-availability.mspxailability.mspx

Database Mirroring Best Practices and Performance Database Mirroring Best Practices and Performance ConsiderationsConsiderations

http://www.microsoft.com/technet/prodtechnol/sql/2005/http://www.microsoft.com/technet/prodtechnol/sql/2005/technologies/dbm_best_pract.mspx technologies/dbm_best_pract.mspx

SQL Server 2005 Failover Clustering White Paper SQL Server 2005 Failover Clustering White Paper http://www.microsoft.com/downloads/details.aspx?http://www.microsoft.com/downloads/details.aspx?familyid=818234dc-a17b-4f09-b282-c6830fead499&displaylang=en familyid=818234dc-a17b-4f09-b282-c6830fead499&displaylang=en

SQL Server 2005: Upgrade Your SkillsSQL Server 2005: Upgrade Your Skillshttp://www.microsoft.com/events/series/http://www.microsoft.com/events/series/technetsqlserver2005.mspx technetsqlserver2005.mspx

SQL Server 2005 Upgrade Technical Reference Guide SQL Server 2005 Upgrade Technical Reference Guide http://www.microsoft.com/downloads/details.aspx?http://www.microsoft.com/downloads/details.aspx?FamilyID=3d5e96d9-0074-46c4-bd4f-FamilyID=3d5e96d9-0074-46c4-bd4f-c3eb2abf4b66&DisplayLang=en c3eb2abf4b66&DisplayLang=en

SQL Server 2005: Virtual LabsSQL Server 2005: Virtual Labshttp://msdn.microsoft.com/virtuallabs/sql/default.aspx http://msdn.microsoft.com/virtuallabs/sql/default.aspx

SQL Server 2005 Understanding Database Pricing SQL Server 2005 Understanding Database Pricing http://www.microsoft.com/sql/howtobuy/understdbpricing.mspx http://www.microsoft.com/sql/howtobuy/understdbpricing.mspx

http://www.microsoft.com/technet/prodtechnol/sql/themes/high-availability.mspx

http://www.microsoft.com/technet/prodtechnol/sql/themes/high-availability.mspx

Customer ExamplesCustomer Examples

Mediterranean Shipping Company – Mediterranean Shipping Company – MSC runs it MSC runs it 5 TB SQL database and 15 billions transacions on Database 5 TB SQL database and 15 billions transacions on Database MirroringMirroring

http://www.microsoft.com/casestudies/casestudy.aspx?http://www.microsoft.com/casestudies/casestudy.aspx?casestudyid=49075casestudyid=49075

Austrian Broadcast Corporation (ORF) – Austrian Broadcast Corporation (ORF) – with with SQL Server 2005,SQL Server 2005, ORFORF generates results 70% faster and generates results 70% faster and also runs Database Mirroringalso runs Database Mirroring


ServiceU Corporation – ServiceU Corporation – ServiceU runs Database ServiceU runs Database Mirroring between two data centers 300 miles apartMirroring between two data centers 300 miles apart


Tuev Nord - Tuev Nord - one of Germany's largest technical service one of Germany's largest technical service providers) runs its SAP Business Warehouse (BW) on SQL providers) runs its SAP Business Warehouse (BW) on SQL Server 2005Server 2005


high availability & scalability in sql server 2005 rick brewis, microsoft technology specialist...

Documents

log backup slide

log backups log shipping

log shipping notes

transaction log files

large log files

partitioning slide

wg slide

database backups