elastic vsphere?

1© Copyright 2010 EMC Corporation. All rights reserved.

Elastic vSphere?Design Considerations for Building Stretched ClustersScott Lowe, vSpecialist, EMC CorporationVCDX #39, VMware vExperthttp://blog.scottlowe.orghttp://twitter.com/scott_lowe

http://blog.scottlowe.org/

http://twitter.com/scott_lowe


Agenda• Reasons for building stretched clusters

• Storage configurations for stretched clusters

• Design considerations for stretched clusters

• EMC VPLEX in stretched clusters

• Q&A


Reasons for Building Stretched Clusters• Valid reasons:– Provide high availability across sites– Workload balancing across sites

• Invalid reasons:– Because you can/because it’s cool (is that a valid

business justification?)– Enable vMotion over distance (stretched clusters

are not a prerequisite)– Use vMotion as a DR mechanism (vMotion not

applicable when both ends aren’t up and available)


Storage Configurations for Stretched ClustersA review of storage configurations to support stretched cluster designs


Stretched SAN Configuration• Literally just stretching the SAN fabric

between locations

• Requires synchronous replication

• Limited in distance to ~100km in most cases

• Typically read/write in one location, read-only in second location

• Implementations with only a single storage controller at each location create a SPoF (single point of failure)


Stretched SAN Configuration

XX

Read/Write Read-Only


Distributed Virtual Storage Configuration• Leverages new storage technologies to

distribute storage across multiple sites

• Requires synchronous mirroring

• Limited in distance to ~100km in most cases

• Read/write storage in both locations, employs data locality algorithms

• Typically uses multiple controllers in a scale-out fashion

• Must address “split brain” scenarios


Distributed Virtual Storage Configuration

X XRead/Write Read/Write


EMC VPLEX Overview• EMC VPLEX falls into the distributed virtual

storage category

• Keeps data synchronized between two locations but provides read/write storage simultaneously at both locations

• Uses scale-out architecture with multiple engines in a cluster and two clusters in a Metro-Plex

• Supports both EMC and non-EMC arrays behind the VPLEX


Preferred Site in VPLEX Metro• VPLEX Metro provides read/write

storage in two locations at thesame time (AccessAnywhere)

• In a failure scenario, VPLEX uses “detach rules” to prevent split brain– A preferred site is defined on a per-

distributed virtual volume basis– Preferred site remains read/write;

I/O suspended at non-preferred site

• Invoked only by entire cluster failure, entire site failure, or cluster partition

Distributed Virtual Volume

IP/FC linksfor Metro-PlexX

Read/write

Read/write

I/O Suspended

Preferred Site

Non-Preferred

Site


Design Considerations for Stretched ClustersA review of design considerations and design impacts when using stretched clusters


Stretched Cluster Considerations #1

Consideration: Without read/write storage at both sites, roughly half the VMs incur a storage performance penalty.

• With stretched SAN configurations:– VMs running in one site are accessing storage in

another site– Creates additional latency for every I/O operation

• With distributed virtual storage configurations:– Read/write storage provided, so this doesn’t

apply



Consideration: Prior to vSphere 4.1, you can’t control HA/DRS behavior.

• With stretched SAN configurations:– Additional latency introduced when VM storage

resides in other location– Storage vMotion required to remove this latency

• With distributed virtual storage configurations:– Need to keep cluster behaviors in mind– Data is access locally due to data locality

algorithms


Stretched Cluster Considerations #3Consideration: With vSphere 4.1, you can use DRS host affinity rules to control HA/DRS behavior.

• With all storage configurations:– Doesn’t address HA primary/secondary node selection

• With stretched SAN configurations:– Beware of single-controller implementations– Storage latency still present in the event of a controller

failure

• With distributed virtual storage configurations:– Plan for cluster failure/cluster partition behaviors



Consideration: Host affinity rules don’t affect VMware HA admission control.

• With all storage configurations:– Must configure admission control for 50% failure

in order to guarantee resource availability– Can’t configure “per site” admission control rules– Impacts the reasons people would build stretched

clusters, especially workload balancing



Consideration: There is no supported way to control VMware HA primary /secondary node selection.

• With all storage configurations:– Limits cluster size to 8 hosts (4 in each site)– das.preferredprimaries is not a supported

mechanism for controlling primary/secondary node selection

– Methods for increasing the number of primary nodes also not supported by VMware



Consideration: Stretched HA/DRS clusters require Layer 2 adjacency (or equivalent) at the network layer.

• With all storage configurations:– Complicates the network infrastructure– Involves technologies like OTV, VPLS/Layer 2

VPNs

• With stretched SAN configurations:– Can’t leverage vMotion at distance without

storage latency

• With distributed virtual storage configurations:– Data locality enables vMotion at distance without

latency



Consideration: The network lacks site awareness, so stretched clusters introduce new networking challenges.

• With all storage configurations:– The movement of VMs from one site to another

doesn’t update the network– VM movement causes “horseshoe routing” (LISP,

a future networking standard, helps address this)– You’ll need to use multiple isolation addresses in

your VMware HA configuration


Horseshoe Routing


Stretched Cluster Recommendations• Use separate HA/DRS clusters in each

datacenter

• Use separate distributed VMFS datastores for each clusters

• Use vMotion to move VMs as needed between clusters– Keep preferred/non-preferred site behavior in

mind!– Try to keep related VMs together in a site– Change detach rules to switch preferred site if

necessary

• A VMware KB article is available discussing HA/DRS clusters with VPLEX


For More Information…• VMware support with NetApp MetroCluster:

http://kb.vmware.com/kb/1001783

• Using VPLEX Metro with VMware HA:http://kb.vmware.com/kb/1026692

• vMotion over Distance Support with VPLEX Metro:http://kb.vmware.com/kb/1021215

• The Case For and Against Stretched ESX Clusters:http://virtualgeek.typepad.com/virtual_geek/2008/06/the-case-for-an.html




http://virtualgeek.typepad.com/virtual_geek/2008/06/the-case-for-an.html

http://virtualgeek.typepad.com/virtual_geek/2008/06/the-case-for-an.html


Q&A


THANK YOU

elastic vsphere?

Technology

virtual storage configurations

stretched san configurations

storage latencywith

stretched cluster designs5

stretched clusters12

stretched clustersqa3

vm storage

casesreadwrite storage