architecting private clouds using windows server 2012

Architecting Private Clouds Using Windows Server 2012Michael PascoeManaging Director – Olikka@OlikkaTech

WSV313

Architecting Private Clouds Using Windows Server 2012Michael PascoeManaging Director – Olikka@OlikkaTech

Architecting Win Server 2012 for Private CloudsMichael PascoeManaging Director – Olikka@OlikkaTech

partner

Michael PascoeManaging Director @OlikkaTech

Olikka - Gold Virtualisation & Management- Gold Desktop- Silver Server

Olikka

We are experts inInfrastructure Management and Virtualization

We specialise in theDesign and Deployment of

Microsoft and Citrix software

We focus on two areasPrivate Cloud and Windows Client

is a specialist systems integrator

AgendaWindows Server 2012 - The cloud infrastructureConsiderations in building clouds

WorkloadsNetworkingStorageResiliency

Example scenarios and configurations

Windows Server 2012 is Cloud Optimized

Manageable& Extensible

High Scale & Low Cost Datacenters

Multi-Tenant Clouds

Windows Server 2012 : Summary of Cloud Enabling Features


High Scale & Low CostDatacenters

• PowerShell• Hyper-V Extensible Switch

Compute• Network Virtualization• Policy-Based Isolation• QoS• Performance Metrics• Live & Storage

Migrations• Cross-Premise

Connectivity

Multi-Tenant Clouds

Network

Storage

StorageNetwork


Multi-Tenant Clouds

ComputeHigh Scale & Low Cost Datacenters

• Larger hosts• Large VMs• Higher VM density• Large clusters

• Datacenter Bridging (DCB) on 10GbE Networks

• Hardware offloads• SR-IOV• RSC / RSS• RDMA

• NIC Teaming

• Hyper-V over SMB• Copy offload (ODX)• Storage Spaces• Storage Thin Provisioning • Synthetic Fibre Channel

Windows Server 2012 : Summary of Cloud Enabling Features

http://aka.ms/bzm1rl

Scope of this talk…DDCReferenceArchitecture



OK. I’m In! How do I build my Cloud Infrastructure?

Requirements Influence ChoicesCost

Scalability

Reliability

Security

Performance

High Availabilit

y Workloads

Networking

StorageResiliency

Primary ConsiderationsWorkloads

Networking

Storage

Resiliency

Understanding Workloads

Trusted?

Performance Requirements

Cloud Aware?Do you Manage Servers or Apps?

Will apps be cloud ready or aware?IaaS clouds fully supports legacy appsRequirements of Each

Workload?2 sockets servers usually offer best ROIApp networking patterns - need SR-IOV?Mix of servers to serve different workloads?

Will the Workloads Play Nice?Isolation between workloadsQoS policies

Workloads


Networking

Storage

Resiliency

Primary Network ConsiderationsNetworking

Type of Infrastructure

Isolation of Traffic Flows

Hardware Offload

Typical Hyper-V Server Traffic Flows

Virtual Machines trafficCluster trafficStorage traffic / CSVLive MigrationsManagement

Networking

Traffic Isolation / Separation Approaches

Use Port ACLs, QoS, DCB & VM QoS to enforce isolation and performance guarantees

How many NICs should be in the server?1. Separate NICs for different traffic flows

- WS2008 R2 best practice2. Separate “Datacenter” traffic from “Tenant” Traffic

- Server 2012 common practice3. Run everything through the virtual switch, one physical

network - Now possible with server 2012

Workloads

InfiniBand vs. 10GbE vs. 1GbE10 Gb Ethernet

Great performance on familiar fabricRDMA optional (for SMB 3.0 file access)Offers Quality of Service (DCB) & Flexible bandwidth allocationNew offloads…But physical switch ports more expensive

InfiniBand (32 Gb and 56 Gb)Very high performance, low latencyRDMA included (for SMB 3.0 file access)…But Network management different than Ethernet

1 Gb EthernetAdequate performance for many workloads

InfiniBand

1 GbE

10 GbE

Networking

Hardware Offloads for Scalability & Performance

Hyper-V Extensible

Switch

VM 1

Live

M

igra

tio

n

Stor

age

Man

age

Hyper-V Server

VM n

Windows Server 2012 supports many NIC features including:• HW QoS via Datacenter Bridging

(DCB)• Remote Direct Memory Access

(RDMA)• Receive Segment Coalescing (RSC)• Receive Side Scaling (RSS)• Virtual Machine Queue (VMQ)• Guest IPsec Task Offload (IPsecTO)• Single Root I/O Virtualization (SR-

IOV)

DCB

RDMA RSS RSC

SRIOV

VMQ

NOTE: Offloads require capable HW, Windows Server 2012 drivers and potentially BIOS support (e.g. SR-IOV)

DCB

RSSIPsecT

O

Networking

Selecting the right NIC Offloads *

Flexibilityand

ScalabilityRaw

PerformanceRDMASR-IOV

Hyper-V Extensible SwitchHyper-V Network Virtualization

NIC TeamingRSSVMQ

IPsecTO

Some Performance offloads bypass Windows functionality

* Per NIC constraints

Networking


Networking

Storage

Resiliency

Storage Selection Considerations

Cost / PerformanceBlock vs. FileManageabilityVendor preferenceExisting InvestmentsApproach to scaling

Storage

Storage Scaling ApproachesWhen?• Very low cost• Disk-bound

VMs

Compute & Storage Scale TogetherCompute & Storage Scale Independently

Hyper-V Server

SAS

Hyper-V Servers

Compute/Storage

Compute/Storage

SAS / iSCSI / FC

Hyper-V Servers

Compute

Storage

File Servers

10 GbE / RDMA

Optional Controlle

r

Hyper-V Servers

SAN Controller

When?• Grow capacity on two

axis• Compute-bound VMs• Storage offloading

iSCSI / FC

Storage

demo

Storage Spaces are Awesome!


Networking

Storage

Resiliency

What happens when hardware fails?

VMs designed to handle failures (e.g. Guest Clustering) or downtime acceptable.Lower End Industry Standard Server, single infrastructure

App-Level Resiliency

Application-controlled failover / Guest clustering

Switch

Switch

Parent VMsVMsVMs

Parent VMsVMsVMs

Resiliency Approaches

Cluster

VMs not designed to handle failures, H/A at server level, Failover clustering as another layer of protection, high-end servers, redundant power and network gear

Hyper-V Failover Clustering

Infrastructure Resiliency

Parent VMsVMsVMs

Parent VMsVMsVMs

Switch Switc

h

Resiliency

Building Your Cloud Infrastructure: Scenario Examples

Optimized for Windows Server 2012Validated at Microsoft’s Enterprise Engineering Center (EEC)Step by Step guidance & PowerShell Script Samples Available

http://technet.microsoft.com/en-us/library/hh831441http://blogs.technet.com/b/privatecloud/archive/2012/05/02/let-s-build-a-cloud-with-powershell.aspx

1: Non-converged Enterprise Configuration2: Converged Datacenter with File Server Storage3: Converged Datacenter without Dedicated Storage Nodes

http://technet.microsoft.com/en-us/library/hh831441

http://technet.microsoft.com/en-us/library/hh831441

http://blogs.technet.com/b/privatecloud/archive/2012/05/02/let-s-build-a-cloud-with-powershell.aspx



Easiest transition Path• Physically isolated networks• Regular SANs• Similar to Windows Server

2008 R2 guidance• Backward compatible

design, taking advantage of 10GbE, SR-IOV, LBFO, RSS, RSC, VMQ, ODX

SAN

FC / iSCSI

Hyper-V Extensible

Switch

VM 1

VM 2

Live

M

igra

tio

n

Clus

ter

Man

age

Hyper-V Server

Stor

age

HBA / 10GbE 1 GbE 1 GbE 1 GbE 1GbE /

10 GbE + SR-IOV

1: Non-converged Enterprise Configuration



design, taking advantage of 10GbE, SR-IOV, LBFO, RSS, RSC, DCB, VMQ, ODX

SAN

FCoE / iSCSI

Hyper-V Ext. Switch

VM 1

VM 2

Live

M

igra

tio

n

Clus

ter

Man

age

Hyper-V Server

Stor

age

10GbE 1 GbE 1 GbE 1 GbE 1GbE /10 GbE + SR-IOV

RSS

ODX

NIC Teaming

Scale & Resiliency for storage access

DCB


Hyper-V Ext. Switch

VM 1

VM 2

Live

M

igra

tio

n

Clus

ter

Man

age

Hyper-V Server

Stor

age

1 GbE 1 GbE 1 GbE

NIC Teaming




SAN

FC / iSCSI

HBA / 10GbE

1GbE /10 GbE + SR-IOV

Scale & Resiliency for VMs

VMQ

1 GbE 1 GbE 1 GbE





SAN

FC / iSCSI

Hyper-V Ext. Switch

VM 1

VM 2

Live

M

igra

tio

n

Clus

ter

Man

age

Hyper-V Server

Stor

age


10 GbE + SR-IOV

Switch Extensions

Network Virt.

QoS

Flexible & Extensible





SAN

FC / iSCSI

Hyper-V Ext. Switch

VM 1

VM 2

Live

M

igra

tio

n

Clus

ter

Man

age

Hyper-V Server

Stor

age


10 GbE + SR-IOV

SRIOV

Switch Extensions

Network Virt.

QoS

Very High Scale VMs w/ SRIOVBut, there are tradeoffs!



Hyper-V Server

SAS

Hyper-V Servers

SAS / iSCSI / FC

Hyper-V Servers

File Servers

10 GbE / RDMA

Optional Controlle

r

Hyper-V Servers

SAN Controller

iSCSI / FC


Storage Scaling Approaches

SAN / JBODs

File Server

Clus

ter

Stor

age

Man

age

HBA(Optional)

Hyper-V Extensible

Switch

VM 1

Live

M

igra

tio

nCl

uste

r /

Stor

age

Man

age

Hyper-V Server

VM n

iSCSIFC/SAS Datacenter Network Tenants Network

“Green Field”

• 10GbE Network(s)• File Server for VM storage

• Actual storage may be an existing FC/iSCSI SANs or JBODs+Spaces

• Highlighted features: 10GbE w/DCB, QoS, LBFO, Hyper-V over SMB, Spaces

10GbE(RDMA)

10GbE(RDMA) 1/10GbE

2: Converged Datacenter with File Server Storage

JBODs

File Server

Clus

ter

Stor

age

Man

age

SAS

Hyper-V Extensible

Switch

VM 1

Live

M

igra

tio

nCl

uste

r /

Stor

age

Man

age

Hyper-V Server

VM n

SAS Datacenter Network Tenants Network

“Green Field”




10GbE(RDMA)

10GbE(RDMA) 1/10GbE

Pool

CSV

SMB File Share

Space

Scale out FS

Storage Spaces lowers cost of storage


JBODs

File Server

Clus

ter

Stor

age

Man

age

ClusteredRAID

Hyper-V Extensible

Switch

VM 1

Live

M

igra

tio

nCl

uste

r /

Stor

age

Man

age

Hyper-V Server

VM n

SAS Datacenter Network Tenants Network

“Green Field”




10GbE(RDMA)

10GbE(RDMA) 1/10GbE

CSV

SMB File Share

Scale out FS

Using hardware RAID controllers(Cluster In a Box, And more…)


SAN

File Server

Clus

ter

Stor

age

Man

age

HBA

Hyper-V Extensible

Switch

VM 1

Live

M

igra

tio

nCl

uste

r /

Stor

age

Man

age

Hyper-V Server

VM n

FC Datacenter Network Tenants Network

“Green Field”




10GbE(RDMA)

10GbE(RDMA) 1/10GbE

CSV

SMB File Share

Scale out FS

Also possible : Front-end the SAN with an SMB Share

LUN


File Server

Clus

ter

Stor

age

Man

age

SAS

Hyper-V Extensible

Switch

VM 1

Live

M

igra

tio

nCl

uste

r /

Stor

age

Man

age

Hyper-V Server

VM n

Datacenter Network Tenants Network

“Green Field”




10GbE(RDMA) 1/10GbE

OS QoS

DCB

Using Qos & DCB JBODs

SAS

10GbE(RDMA)


OS QoS

DCB

File Server

Clus

ter

Stor

age

Man

age

SAS

Hyper-V Extensible

Switch

VM 1

Live

M

igra

tio

nCl

uste

r /

Stor

age

Man

age

Hyper-V Server

VM n

Datacenter Network Tenants Network

“Green Field”




10GbE(RDMA) 1/10GbE

OS QoS

DCB

Switch QoS

Using Qos & DCB JBODs

SAS

10GbE(RDMA)


OS QoS

DCB

QoSNew-NetQosPolicy

Switch QoSSet-VMNetworkAdapter

DCBAdd-WindowsFeature Data-Center-BridgingImport-Module DcbQosEnable-NetAdapterQosEnable-NetQosFlowControlNew-NetQoSTrafficClass

File Server

Clus

ter

Stor

age

Man

age

SAS

Hyper-V Extensible

Switch

VM 1

Live

M

igra

tio

nCl

uste

r /

Stor

age

Man

age

Hyper-V Server

VM n

Datacenter NetworkTenants Network

“Green Field”




• Note: LBFO & RDMA can’t coexist on same NICs.

10GbE 1/10GbE

NIC Teaming

NIC Teaming

Use of NIC Teaming & Qos/DCB SAN / JBODs

SAS


OS QoS

DCB

OS QoS

DCB

10GbE(RDMA)

10GbE(RDMA)

10GbE

10GbE(RDMA)

10GbE(RDMA)

File ServerCl

uste

r

Stor

age

Man

age

SAS

Hyper-V Extensible

Switch

VM 1

Live

M

igra

tio

nCl

uste

r /

Stor

age

Man

age

Hyper-V Server

VM n

Tenants Network

“Green Field”




1/10GbE

RDMA with two Datacenter NICsSAN / JBODs

SAS

• SMB Multichannel• 2 Cluster Networks• 2 Networks for LM• Multiple Mgmt IPs• QoS + DCB


Datacenter Network

10GbE

10GbE(RDMA)

10GbE(RDMA)

10GbE

10GbE(RDMA)

10GbE(RDMA)

Hyper-V Server

SAS

Hyper-V Servers

SAS / iSCSI / FC

Hyper-V Servers

File Servers

10 GbE / RDMA

Optional Controlle

r

Hyper-V Servers

SAN Controller

iSCSI / FC


Storage Scaling Approaches2: Converged Datacenter with File Server Storage

demo

Migrating VM workload to non-clustered host

JBODs

Hyper-V Extensible

Switch

VM 1

Hyper-V Server

VM n

Converged Network

Converged Compute + Storage:

• 10GbE Network(s), fully converged, routing all traffic through the switch

• Shared JBODs, scale compute and storage together

• Highlighted features: Storage Spaces on the Hyper-V nodes, Hyper-V Switch ports for VM and Host traffic, QoS policies.

SAS

SAS

Clus

ter

/ St

orag

eLi

ve

Mig

rati

on

Man

age

10GbE

3: Converged Datacenter without Dedicated Storage Nodes

JBODs

Hyper-V Extensible

Switch

VM 1

Hyper-V Server

VM n

Converged Network

SAS

SASM

anag

e

ACLs,

QoS

VM NICs, with ACLs + QoS on the Switch





VM NICs

Clus

ter

/ St

orag

eLi

ve

Mig

rati

on

10GbE


JBODs

Hyper-V Extensible

Switch

VM 1

Hyper-V Server

VM n

Converged Network





SAS

SAS

Clus

ter

/ St

orag

eLi

ve

Mig

rati

on

Man

age

Cluster/Storage

Tenants

Live MigrationManagement Setup VLANs to separate traffic flows

10GbE

VLANs


JBODs

Hyper-V Extensible

Switch

VM 1

Hyper-V Server

VM n

Converged Network

SAS

SASM

anag

e

Pool

CSVSpace

Spaces and CSV directly on the Hyper-V Servers





Clus

ter

/ St

orag

eLi

ve

Mig

rati

on

10GbE


JBODs

Hyper-V Extensible

Switch

VM 1

Hyper-V Server

VM n

Converged Network

SAS

SASM

anag

e

1/10GbE

Pool

CSVSpace

Add RDMA NICs for fast CSV Redirection





10GbERDMA

CSV Traffic

Clus

ter

/ St

orag

eLi

ve

Mig

rati

on


Hyper-V Server

SAS

Hyper-V Servers

SAS / iSCSI / FC

Hyper-V Servers

File Servers

10 GbE / RDMA

Optional Controlle

r

Hyper-V Servers

SAN Controller

iSCSI / FC


Storage Scaling Approaches3: Converged Datacenter without Dedicated Storage Nodes

Hyper-V Extensible

Switch

VM 1

Live

M

igra

tio

n

Man

age

Hyper-V Server

VM n

Converged Network

Very Low Cost (Huge Scale Clouds)

• 10GbE Network(s), fully converged

• No clusters, no H/A, local DAS

• For cloud-ready applications

• Highlighted feature: Live migration w/o clusters

DAS

4: DAS, Non-Clustered Configuration

Hyper-V Server

SAS

Hyper-V Servers

SAS / iSCSI / FC

Hyper-V Servers

File Servers

10 GbE / RDMA

Optional Controlle

r

Hyper-V Servers

SAN Controller

iSCSI / FC


Storage Scaling Approaches4: DAS, Non-Clustered Configuration

demo Migrate from SMB storage to local


Networking

Storage

Resiliency

Windows Server 2012 is Cloud Optimized


High Scale & Low Cost Datacenters

Multi-Tenant Clouds

Related ContentBreakout Sessions

WSV313 Diving into SMB 3.0VIR316 Comparing Private Cloud CapabilitiesVIR314 Hyper-V Live Migration and Live Storage MigrationWSV316 File System Enhancements: ReFS and Storage SpacesWSV321 NIC Teaming and Multichannel SolutionsWSV325 File and Storage Services Management

Find Me Later At…Twitter @OlikkaTechBlog http://bolikka.com

TechEd 2012; WSV313

© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the

part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

architecting private clouds using windows server 2012

Documents