gopal ashok program manager microsoft corp. agenda sql server consolidation virtualization &...

Virtualization and Consolidation Considerations for Microsoft SQL Server 2008DAT305

Gopal AshokProgram ManagerMicrosoft Corp

Agenda

SQL Server Consolidation

Virtualization & Microsoft Hyper-VArchitecture

Consolidation Performance

High Availability & Manageability offerings

Achieving scalability with Virtualization

Best Practices and RecommendationsSQL Relational Workload

A Case Study

Key Takeaways

Forces Driving ConsolidationReduce Cap-Ex

Upgrade & standardize to fewer hardware

Reduce space, power and thermal needs

Reduce Op-ExImprove Hardware Utilization Efficiency

Improve Management Efficiency

Lowered cost and complexity for High Availability

Infrastructure AgilityLoad Balancing & Dynamic Provisioning

Standardization of services

Number of database apps

Number of DBA’s

1990 2000 2010

Hardware computing capacity Underutilized hardware

Overburdened Administrators

SQL Server Consolidation H

ighe

r Iso

latio

n, H

ighe

r Cos

ts Higher D

ensity, Lower Costs

Databases InstancesVirtualMachines

Sales_1

Marketing_1

Online_SalesERP_10

ERP_10

DB_1

DB_2DB_3

Consolidate_1

Currently a variety of consolidation approaches exist and are utilized. Typically, as isolation goes up, density goes down and operation cost

goes up.

IT ManagedEnvironment

Schemas

SQL Server Consolidation Options H

ighe

r Iso

latio

n, H

ighe

r Cos

tsH

igher Density, Low

er Costs

Database level consolidation Multiple databases are collapsed into a single instance Common security, manageability and compatibility models required Might require changes to existing applications and scripts Lower manageability costs Better resource isolation and allocation capabilities through Resource

Governor

Databases

DB_1

DB_2DB_3

Consolidate_1

InstancesVirtual

Machines

Sales_1

Marketing_1

Online_SalesERP_10

ERP_10

Instance level consolidation Multiple Instances running on one system

Full schema and security isolation Partial system resource & management isolation Potential conflicts in namespace, resources and system roles IO, System Memory and CPU are typical density limiters

SQL Server Consolidation OptionsH

ighe

r Iso

latio

n, H

ighe

r Cos

tsH

igher Density, Low

er Costs

Databases

DB_1

DB_2DB_3

Consolidate_1

InstancesVirtual

Machines

Sales_1

Marketing_1

Online_SalesERP_10

ERP_10

SQL Server Consolidation Options H

ighe

r Iso

latio

n, H

ighe

r Cos

ts Higher D

ensity, Lower Costs

Virtual Machine consolidation Strong isolation between applications Ease of capturing and moving execution loads Out of the box High Availability configuration Flexible storage management Fewer systems, but just as many OS images to manage Increased resource usage

Databases

DB_1

DB_2DB_3

Consolidate_1

InstancesVirtual

Machines

Sales_1

Marketing_1

Online_SalesERP_10

ERP_10

Agile

ITRe

duce

Op-

ExRe

duce

Cap

-Ex

Choosing the Right Consolidation Approach Evaluate key parameters for your environment

Isolation between apps Security isolation Predictable Performance : Resource isolation HA : Failure isolation

Density of apps Performance : Resource utilization efficiency Manageability Impact HA : mitigating single point of failure

Time to Market How long does it take to consolidate? Can my solution Scale?

Agenda







A Case Study

Key Takeaways

Hyper-V Architectural OverviewRoot Partition

I/O Stacks

I/O Stacks

DriversDrivers

I/O StacksTCP/IP , File systems


ServerServer

VMBusVMBus

Virtual Service Clients

Virtual Service Clients

Devices Processors Memory

VMBusVMBus

Virtual Service Providers

Virtual Service Providers

Child Partition

DriversDrivers

Child PartitionEnlightened OS

UserMode

KernelMode

HypervisorMode

Fast Path DeviceEmulation



DriversDrivers

DeviceEmulation

ServerServer

Child PartitionNon-Enlightened OS

Applications

Hypervisor

Hardware

Windows Server 2008Windows Server 2008 R2

Agenda







Analysis Services Workload

A Case Study

Key Takeaways

Monitoring Performance: CPU Terminology

Logical Processor: One logical computing engine in the OS, application and driver view

Virtual Processor: Virtual logical processor (upto 1:8 over commit of LP)

% Processor Time% Processor Time counters in either guest or root may not be accurate

Hyper-V Processor Counters The best way to get true measure of CPU utilization

Hyper-V Logical Processor: Total CPU time across entire server

Hyper-V Virtual Processor: CPU time for each guest virtual processor

Monitoring Performance: Storage

Configuration Considerations

Guest VM w/ Passthrough Disks

Use Physical disk counters within root partition to monitor I/O of passthrough disks

Guest VM w/ VHD Use Logical or physical disk counters within guest VM to monitor IO rates of a VHD Disk counters at the root partition provide aggregate IO of all VHDs hosted on the underlying partition/volume

Either configuration

Very little difference in the values reported by the counters from the root partition with those within guest VMSlightly higher latency values (Avg. Disk/sec Read and Write) observed within the guest VM

TerminologyPassthrough Disk: Disk Offline at Root Partition

VHD (Virtual Hard Disk)Fixed Size VHD : Space allocated staticallyDynamic VHD : Expand on demand

SQL Server Consolidation Performance: Native vs. VirtualConfiguration

Workload : Stock trading applicationHardware:

PowerEdge R900 Intel Xeon 2.4GHz, 4 cores x 4 CPU = Total 16 cores Hitachi Data Systems AMS1000

Virtual Machines: 4 Virtual Processors, 14GB RAM, 12 GB for SQLPassthrough and Fixed-Size VHDs (2 Data, 1 Log)

1Instances 2Instances 3Instances 4Instances0

5

10

15

20

25

30

35

40

45

50

55

60

0

500

1000

1500

2000

2500

Batches/sec_Native

Batches/sec_VM

Batches/sec/Inst_NativeBatches/sec/Inst_VM

% Processor_Native

% Processor_VM

Relative Through-put_Native

Relative Throughput_VM

Performance and Scalability Comparison between Native and Virtual Instances: 16core 64GB RAM 12GB Per SQL Instance

Results Relative Throughput = Batches/sec/%Processor Same throughput attainable

There is more CPU overhead with hyper-v enabled or when running within a VM

Passthrough Performance ~5% Better than fixed size VHDs

Native Instances and Virtual Instances achieves the same level of scalability

SQL Server Consolidation Performance: Native vs. Virtual (overcommit)

1 Inst 2 Inst 3 Inst 4 Inst0

10

20

30

40

50

60

70

80

90

100

110

0

200

400

600

800

1000

1200

1400

1600

1800

2000

Batches/sec_Native

Batches/sec_VM

Batches/sec/Inst_NativeBatches/sec/Inst_VM

% Processor_Native

% Processor_VM

Relative Through-put_Native

Relative Throughput_VM

Performance and Scalability Comparison between Native and Virtual Instances: 8core 64GB RAM 12GB Per SQL Instance

Configuration Workload : Stock trading application Hardware:

PowerEdge R900 Intel Xeon 2.4GHz, 4 cores x 2 CPU = Total 8 cores Hitachi Data Systems AMS1000

Virtual Machines: 4 Virtual Processors, 14GB RAM, 12 GB for SQLPassthrough and Fixed-Size VHDs (2 Data, 1 Log)

Results Relative Throughput = Batches/sec/%Processor Same throughput attainable with CPU overcommit

Additional overhead with over subscribe logical processors

Taking into consideration of the additional overhead for capacity planning

Native Instances and Virtual Instances achieves the same level of scalability

Passthrough vs. Fixed Size VHD VHD’s on Shared Storage vs. Dedicated Spindles using Passthrough Disks

Measuring average reads per second vs. latency

VHDs on shared disks has slight latency overhead and less throughput Graph bars = Reads/sec Lines = Avg. Disk/sec Read (.001 = 1 ms)

Low OLTP Workload Med OLTP Workload High OLTP Workload0

1,0002,0003,0004,0005,0006,0007,000

0

0.002

0.004

0.006

0.008

0.01

Dedicated Pass-through vs. VHDs on Shared DisksTotal IO/s per Second and Disk Latency

Total Reads/sec (Dedicated LUNs)Total Reads/sec (Common Vol-ume with VHDs)Average Read Latency (Ded-icated LUNs)

Tota

l Rea

d IO

’s p

er S

econ

d

Transaction Response Time

Inst2 Inst4 Inst6 Inst8 Inst10 Inst12 Inst 14 Inst 160

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0

10

20

30

40

50

60

70

80

90

Response Time_VM Response Time_Native %Processor_VM %Processor_Native

% P

roce

ssor

tim

e

Resp

onse

Tim

e (s

ec)

Results:• Transaction response time comparable

• sub seconds application response time• When system under stress, VM appears to have better

scalability• Test your own workload, may scale differently

Configuration:• OS: Microsoft® Windows Server® 2008 R2 Hyper-V™• Hardware:

HP DL785 (32 core)Hitachi Data Systems AMS2500 Storage

• Virtual Machines: 4 virtual processors and 7 GB RAM per virtual machine; Fixed size VHD

Agenda







A Case Study

Key Takeaways

High Availability with Guest Clustering Using iSCSIScenario Description:• Improve high availability with a combination of

guest clustering and host clustering• Only iSCSI is supported for guest clustering• iSCSI Initiator runs within the VM allowing storage

to be fully visible to the VM• Storage is fully visible to the guest cluster and

enables high-availability of services and applications in the virtual layer

• Support the use of multiple redundant paths using Microsoft Multipath IO (MPIO) or MCS (multiple connections per session) from within the VM

Virtualization Benefits: • Provide fault tolerance both at application and

host level• All applications can run in the context of the VM• Management efficiency based on SQL Server® and

System Center management tools• Backup applications have full visibility to data

within the application context• Storage providers such as MPIO, VDS, VSS run

within the VM

Shared StorageiSCSI

Guest Cluster

Guest Cluster

1 2

VM

Redundant Paths to storage

High Availability with Live MigrationScenario Description:• Manage high availability with multipathing and live

migration for planned downtime situations, such as hardware and software maintenance

• Migrating individual virtual machines (VMs) to other hosts within a cluster by using Cluster Shared Volume (in Windows Server® 2008 R2)

• Use Microsoft ® System Center Virtual Machine Manager for migrations. System Center VMM can perform host compatibility checks before migrations and manage multiple Live Migrations with queues.

• Nodes in cluster can be active-active• Ensure there is enough CPU capacity for the failover

nodes in cluster

Virtualization Benefits: • No loss of service during migration with live

migration. • Improve availability with less complexity• Load balancing VMs across physical machines as

needed• Better server utilization due to consolidation

• Easier management through System Center VMM

Shared StorageiSCSI, SAS, Fibre

LiveMigration

1 2

Host cluster

1 2

VM

Consolidation increases the importance of HA, since there is a high cost to single system failure

Creating and managing Virtual Machines• Maximize Resources

– Centralized virtual machine deployment and management for Hyper-V, Virtual Server, and VMware ESX servers

– Intelligent placement of Virtual Machines– Fast and reliable P2V and V2V conversion– Comprehensive application and service-level

monitoring with Operations Manager– Integrated Performance and Resource Optmization

(PRO) of VMs• Increase Agility

– Rapid provisioning of new virtual machines with templates

– Centralized library of infrastructure components– Leverage and extend existing storage infrastructure

and clusters– Allow for delegated management and access of VMs

• Leverage Skills– Familiar interface, common foundation , powershell

scripting

Agenda







A Case Study

Key Takeaways

NUMA NODE 0 NUMA NODE 1

Memory

CPU 0

CPU 1

CPU 2

CPU 3

Local Memory Access

Foreign Memory Access 4x local

Memory

CPU 4

CPU 5

CPU 6

CPU 7

Memory

CPU 0 CPU 1 CPU 2 CPU 3 CPU 4 CPU 5 CPU 6 CPU 7

SMP

NUMA

Consolidation Hardware – NUMA & 64bit

NUMA Node isolation presents inherent & significant advantages to software that is designed to take advantage of locality

Microsoft Hyper-V localizes VM resource utilization to a NUMA boundary

64bit allows significantly more addressable space than 32bit

Front side bus contention increases w/ higher #CPUs

Consolidation Hardware – SLATWith Virtualization an additional level of mapping is required in address translation Second Level Address Translation (SLAT) - 2nd generation virtualization technology in Intel VT-x with EPT and AMD –V with NPT chips accelerate VM performance to be almost on par with bare metal for common workloadsContinuing innovations in hardware assists – Device and IO virtualization

Guest Physical Memory Pages

Guest Physical view

The Virtual Machine view Guest Physical Memory PagesGuest Physical view

Virtual Machine 1

Hyper Visor

Virtual Machine 1

Virtual Machine 3

Operating System

Guest Physical Memory PagesGuest Physical view

Physical Memory PagesHost Physical / real view

SQL Server Performance: SLAT ImpactResults:• Increased throughput with consolidation• Near linear scale in throughput with no CPU over-

commit• Improved performance with Windows Server 2008

R2 and SLAT processor architecture


HP DL585 (16 core) with SLAT HP EVA 8000 storage


)

1VM 2VM 3VM 4VM 5VM 6VM 7VM 8VM0

10

20

30

40

50

60

70

80

0

500

1000

1500

2000

2500

3000

3500

Batch req/sec %CPU Relative Throughput

Virtual Instances ScalabilityVirtual Instances Scalability% CPU

Throughput(Batch requests/sec)

Relative Throughput for Windows Server 2008

HeavyLoad

ModerateLoad

LowLoad

CPU over-commitAlmost Linear ScaleNo CPU over-commit

Consolidation Hardware – Snoop Filtering


Memory

CPU 0

CPU 1

CPU 2

CPU 3

Memory

CPU 4

CPU 5

CPU 6

CPU 7

Mem

ory

C0 C1 C2 C3 MCH0 C4 C5 C6 C7 MCH1Fetch addr (first time)1. Issue Fetch to

Memory Controller Hub (MCH)

2. Snoop local caches

3. Caches respond4. Snoop foreign

caches5. Caches respond6. Get data from

memory controller

Consolidation Hardware – Snoop Filtering


Memory

CPU 0

CPU 1

CPU 2

CPU 3

Memory

CPU 4

CPU 5

CPU 6

CPU 7

Mem

ory

C0 C1 C2 C3 MCH0 C4 C5 C6 C7 MCH1

Fetch addr (first time)1. Issue Fetch to

Memory Controller Hub (MCH)

2. Check local snoop filter

3. Check foreign snoop filter

4. Get data from memory controller

Snoop Filter reduces internode traffic

SQL Server Performance : Snoop Filter ImpactResults:• Drop-in compatibility of Istanbul processors with existing

infrastructure • AMD HyperTransport Assist (Intel snoop filter) feature

huge benefit to SQL workload• keep cache coherency traffic between the two sockets

from appearing on the external bus• ~50% performance improvement


HP DL785 with and without snoop filter support (Shanghai vs. Istanbul)Hitachi Data Systems AMS2500 Storage


VM1 VM2 VM4 VM6 VM80

10

20

30

40

50

60

70

80

90

100

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

82.12 90.2893.12 91.70

94.05

57.8460.15

57.8459.79 60.67

Batches/sec_IstanbulBatches/sec_Shanghai%Processor Time_Istanbul%Processor Time_ShanghaiRelative Through-put_IstanbulRelative Through-put_Shanghai

Comparing Istanbul vs. Shanghai

42%

55%

Agenda







A Case Study

Key Takeaways

Best Practices and Recommendations

Running SQL Server workloads within Hyper-V guest VM’s is a good option for production environment

When compared against native the same throughput can be achieved within a guest VM at a cost of slightly increased CPU utilization

Hyper-V benefits from newer generation of processor architecture (SLAT, Snoop filter)Assuming limitations of Guest VM meet requirements of the workload.

Proper hardware sizing is critical to SQL Server performance

Test/Monitor your workloads

CPU Resources Over Commit on case by case basis for Higher Density

May introduce noticeable performance overhead when all workloads are busy

Best Practices and Recommendations

Integrated Component : “Enlightenments”Better IO performance

Reduce memory access overhead

Passthrough and Fixed Size VHD for Better I/O performanceIO Performance Impact is minimalSQL IO performance and sizing recommendations applyDynamic VHD not recommended for SQL Server deployments

Proper sizing of memory capacity. Memory is allocated for VMs in a static fashion and can only be modified when a guest is offline

CPU Affinity Not SupportedNetwork intensive workload might experience more overhead

Case Study – MSIT SQL Consolidation

Microsoft IT Infrastructure Overview• Pre-Consolidation State

● ~2,700 Applications in MSIT Portfolio● ~4797 SQL Server Instances● ~100,000 databases● ~20% end-of-life hosts/year● ~10% CPU utilization across hosts

• Consolidation Approach● Microsoft IT evaluated database, instance and host based consolidation

• Resource Management Approach● Resource Management effectiveness considered critical issue● Instance based would utilize WSRM ● Hyper-V allows explicit allocation of CPU and IO channels

Case Study – MSIT SQL Consolidation

Microsoft IT Consolidation Conclusions• Consolidation Strategy

● Host Consolidation utilizing Hyper-V● Target of 6 to 1 consolidation ratio● Fixed Virtual Drives (VHDs) over Dynamic and Pass Through

• Consolidation Approach● Decision starting point was instance based consolidation● Evaluation led to decision change: Hyper-V based host consolidation

• WSRM vs. Hyper-V ● Microsoft IT evaluated WSRM vs. Hyper-V & ultimately chose Hyper-V

Case Study - Microsoft IT Consolidation

0500,000

1,000,0001,500,0002,000,0002,500,0003,000,0003,500,0004,000,000

Power and Cooling at 6:1 Consolidation

Legacy Consolidated$0

$10,000$20,000$30,000$40,000$50,000

Annual Recycle Cost at 6:1 Consolida-tion

Legacy Consolidated

$0$2,000,000$4,000,000$6,000,000$8,000,000

$10,000,000$12,000,000$14,000,000$16,000,000$18,000,000

Annual Operating Cost at 6:1 Consolidation

Legacy Consolidated

Key Takeaways

No One-size Solution for ConsolidationHyper-V is a solid platform for SQL Server, both for scalability and performanceUnderstand performance and functional needs of workloads being consolidated

If possible, avoid consolidation of heterogeneous workloads on the same Windows Server

I/O design is very significantEspecially tempdb and log filesMonitor I/O latency and ensure healthy I/O performanceAvoid spindle sharing with other servers/applications when possibleFollow Best Practices for storage

Ensure No/minimal Contention for Memory Resources Determine and set upper memory limits for SQL Server and SSAS (both single and multiple instances)

Take into account memory needed for OS, other applicationsUnderstand and test the impact of SSIS packages being run locallyEnsure enough memory for CLR procedures

Key Takeaways

Resource ManagementHyper-V CPU management tools to manage CPU resources at VM LevelWSRM to manage multiple instances CPU resourceResource Governor to manage within an instance

SQL Server Analysis ServicesDetermine and set upper memory limits for SQL Server and SSAS (both single and multiple instances)

SSAS consolidation enables better utilization of hardware (new or existing)

SSAS consolidation requires planning to resolve resource contention and availability / single point of failure issues

Taking advantage of New Processor ArchitectureSLAT

Snoop Filter

Share your Lessons Learned!

SQL Solution Accelerators

Infrastructure Planning and Design Guide

Outline SQL Server 2008 infrastructure design decisions and ensure that business and technical requirements are metOffer easy-to-follow steps to design architecture including decision flowsDownload at www.microsoft.com/IPD

Microsoft Assessment and Planning Toolkit

Identify SQL instances automaticallyAssess Hyper-V virtualization candidates for server consolidationDownload at www.microsoft.com/MAP

http://www.microsoft.com/IPD

http://www.microsoft.com/MAP

question & answer

www.microsoft.com/teched

Sessions On-Demand & Community

http://microsoft.com/technet

Resources for IT Professionals

http://microsoft.com/msdn

Resources for Developers

www.microsoft.com/learning

Microsoft Certification & Training Resources

Resources

http://www.microsoft.com/teched

http://microsoft.com/technet

http://microsoft.com/msdn

http://www.microsoft.com/learning

ReferencesRunning SQL 2008 in Hyper-V Environment

http://sqlcat.com/whitepapers/archive/2008/10/03/running-sql-server-2008-in-a-hyper-v-environment-best-practices-and-performance-recommendations.aspx

Green IT in Practices: SQL Server Consolidation in Microsoft IThttp://www.msarchitecturejournal.com/pdf/Journal18.pdfhttp://msdn.microsoft.com/en-us/architecture/dd393309.aspx

Support Policies of SQL Server in virtualized environments. http://support.microsoft.com/?id=956893

http://blogs.msdn.com/psssql/archive/2008/10/08/sql-server-support-in-a-hardware-virtualization-environment.aspx

Windows Virtualization Validation Programhttp://windowsservercatalog.com/svvp.aspx?svvppage=svvp.htm

Windows Server Hyper-V site http://www.microsoft.com/windowsserver2008/en/us/virtualization-consolidation.aspx

Hyper-V Technet center http://technet2.microsoft.com/windowsserver2008/en/servermanager/virtualization.mspx

SQL Server 2008 Business Value Calculator: www.moresqlserver.com

http://www.microsoft.com/windowsserver2008/en/us/virtualization-consolidation.aspx

Complete an evaluation on CommNet and enter to win an Xbox 360 Elite!

© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS,

IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

gopal ashok program manager microsoft corp. agenda sql server consolidation virtualization &...

Documents

sales erp

lower costs databases

consolidation considerations

resource governor databases

higher costshigher density

resource isolation ha

system memory

system rolesio