Virtual SAN

Introduction

VSAN is many things • A Storage Solution that is fully integrated with vSphere• A Platform for Policy Based Storage to simplify Virtual Machine deployments decisions• A Highly Available Clustered Storage Solution• A Scale-Out Storage System• A Quality Of Service implementation (for its storage objects)

Layout

VMware’s plan for software defined storage is to focus on a set of VMware initiatives around local storage, shared storage and storage/data services. In essence, to make vSphere a platform for storage services. Software defined storage aims to provide storage services and SLA automation through a software layer on the hosts that integrates with, and abstracts, the underlying hardware.

Key Components• Hypervisor-based software-defined storage• Aggregates local HDDs to provide a clustered data store for VM consumption• Leverages local SSDs as a cache• Distributed object-based RAIN (Redundant Array of Independent Nodes) architecture provides no

single point of failure• Policy-based VM-storage management for end-to-end SLA enforcement• Integrated with vCenter• Integrated with vSphere HA, DRS and vMotion• Scale-Out Storage: 3-8 nodes

Customer benefitsUsing a hybrid approach of SSD for performance and HDD for capacity, VSAN is aimed at re-enabling projects that require a less expensive storage solution.

Each node can host up to 32 disks and up to 100 virtual machines (VMs). That means that a 32 node Virtual SAN can host 3,200 VMs . It can also provide a maximum of 4.4 petabytes of storage space, and offer 2M IOPS (100% read)

The SSDs themselves don't affect the storage capacity of each node - 30% of their capacity is used as a non-persistent write buffer, and 70% is used a read cache.

Virtual SAN Use Cases

Hardware & Software RequirementsA vCenter Server running 5.5. VSAN can be managed by both the Windows version of vCenter Server and the vCenter Server Appliance (VCSA). VSAN is configured and monitored via the vSphere Web Client and this also needs to be version 5.5.

At least 3 vSphere hosts (where each host has local storage) in order to form a supported VSAN cluster. This is to allow the cluster to meet the minimum availability requirements of tolerating at least one host failure. The vSphere hosts must be running vSphere version 5.5 at a minimum. With fewer hosts there is a risk to the availability of virtual machines if a single host goes down. The maximum number of hosts supported is 8 in the initial release of VSAN.

Each vSphere host in the cluster that contributes local storage to VSAN must have at least one hard disk drive (HDD) and at least one solid state disk drive (SSD).

Contd..• A HBA or a Pass-thru RAID Controller is required (a RAID Controller which can present

disks directly to the host without a RAID configuration)• A combination of HDD & SSD devices are required (a minimum of 1 HDD & 1 SSD [SAS or

SATA]), although we expect SSDs to make up 10% of total storage• The SSD will provide both a write buffer and a read cache. The more SSD capacity in the

host, the greater the performance since more I/O can be cached.• Not every node in a VSAN cluster needs to have local storage. Hosts with no local

storage can still leverage distributed datastore.• Each vSphere host must have at least one network interface card (NIC). The NIC must be

1Gb capable. However, as a best practice, VMware is recommending 10Gb network interface cards.

• A Distributed Switch can be optionally configured between all hosts in the VSAN cluster, although VMware Standard Switches (VSS) will also work.

• A Virtual SAN VMkernel port must be configured for each host. With a Distributed Switch, NIOC can also be enabled to dedicate bandwidth to the VSAN network.

• The VMkernel port is labelled Virtual SAN. This port is used for inter- cluster node communication and also for read and writes when one of the vSphere hosts in the cluster owns a particular virtual machine, but the actual data blocks making up the virtual machine files are located on a different vSphere host in the cluster. In this case, I/O will need to traverse the network configured between the hosts in the cluster.

Cost

Virtual SAN license may cost around $750 per CPU, and if you estimate a server cost of about $5000 - $6000, with an extra $5000 or so for SSD and HDD drives, you get an all in per-node price of about $11,000 to $12,000.

That would make a 3 node cluster come in at around $35,000, and a large 32-node cluster would cost around $370,000.

Add VSAN Network

Select Target Device

Note: You MUST have already attached each host to a distributed switch and created a “VSAN” port group to use it

Specify Port Group for VSAN traffic

Enable VSAN

Cluster Nodes participating in VSAN

Claim Disks for VSAN Use (Manual Method)

Disk group created per Cluster Node

VSAN Datastore

The capacity of “vsanDataStore” displayed is an aggregate of the HDDs taken from each of the ESXi hosts in the cluster (less some VSANdatastore overhead).

The SSD volumes are not considered when the capacity calculation is made.

Verify Storage Provider StatusFor each ESXi host to be aware of the capabilities of VSAN and to communicate between vCenter and the storage layer a Storage Provider is created. Each ESXi host has a storage provider once the VSAN cluster is formed. The storage providers will be registered automatically with SMS (Storage Management Service) by vCenter. However, it is best to verify that the storage providers on one of the ESXi hosts has successfully registered and is active, and that the other storage providers from the remaining ESXi hosts in the cluster are registered and are in standby mode.

Navigate to the vCenter server > Manage > Storage Providers to check the status.

In this four-node cluster, one of the VSAN providers is online and active, while the other three are in Standby. Each ESXi host participating in the VSAN cluster will have a provider, but only one needs to be active to provide VSAN datastore capability information.

Should the active provider fail for some reason one of the standby storage providers will take over.

ESX CLI VSAN Commands

Enable Virtual FlashYou can configure Virtual Flash Read Cache on an ESXi host as a swap cache. The cache reservation is allocated from the Virtual Flash resource.

The Virtual Flash can be reserved for any individual VMDK in a Virtual Flash Read Cache pool. A resource is created only when a virtual machine is powered on, and it is discarded when a virtual machine is suspended or powered off. You have the option to migrate the cache. By default the cache can be migrated if the Virtual Flash module on the source and destination hosts are compatible. You can change the properties of the cache while a virtual machine is powered on. In this instance, the existing cache is discarded and a new cache is created, which results in a cache warm up period. When the Virtual Flash Read Cache is reconfigured, you must stun and unstun the virtual machine.

You can set up a Virtual Flash Read Cache resource or add capacity to an existing Virtual Flash resource.

To set up a Virtual Flash resource - you use an SSD device connected to your host. If you need to increase the capacity of your Virtual Flash resource, you can add more SSD devices, up to eight total. This functionality is only available when using the vSphere Web Client.

Add Virtual Flash Resource CapacityPrerequisites:• Hosts appear in the Add Virtual Flash

Capacity list only if they are configured for Virtual Flash and have eligible SSD devices.

• You can configure Virtual Flash for a virtual machine with hardware version 10 compatible with ESXi 5.5 or later.

Assigning Flash Cache to a VM

Note that it is possible to change the Virtual Flash block size, ranging from 4 KB to 1024 KB.

This is an important feature, in order to get the best performance from Virtual Flash, you should adjust the block size to match your application IO pattern.

Your app IO pattern can be identified by various tools including vscsistats which is part of ESXi.

VM Storage Policy & VSAN

When you use Virtual SAN, you can define virtual machine storage requirements, such as performance and availability, in the form of a policy. The policy requirements are then pushed down to the Virtual SAN layer when a virtual machine is being created. The virtual disk is distributed across the Virtual SAN datastore to meet the requirements.When you enable Virtual SAN on a host cluster, a single Virtual SAN datastore is created. In addition, enabling Virtual SAN configures and registers the Virtual SAN storage provider that uses VASA to communicate a set of the datastore capabilities to vCenter Server.

When you know storage requirements of your virtual machines, you can create a storage policy referencing capabilities that the datastore advertises.

You can create several policies to capture different types or classes of requirements.

Number of failures to tolerate: Defines the number of host, disk or network failures a storage object can tolerate. For n failures tolerated "n+1" copies of the object are created and "2n+1" hosts contributing storage are required. Default value:1, Maximum value:3.

Number of disk stripes per object: The number of HDDs across which each replica of a storage object is striped. A value higher than 1 may result in better performance ( e.g. when flash read cache misses need to get serviced from HDD), but also results in higher use of system resources. Default value:1, Maximum value:12.

Flash read cache reservation (%): Flash capacity reserved as read cache for the storage object. Specified as a percentage of the logical size of the object. To be used only for addressing read performance issues. Reserved flash capacity cannot be used for other objects. Unreserved flash is shared fairly among all objects. Default value: 0%, Maximum value: 100%

Force provisioning: If this option is "Yes", the object will be provisioned even if the policy specified in the storage policy is not satisfiable with the resources currently available in the cluster. VSAN will try to bring the object into compliance if and when resources become available. Default value: No.

Object space reservation (%): Percentage of the logical size of the storage object that will be reserved ( thick provisioned) upon VM provisioning. The rest of the storage object is thin provisioned. Default value: 0%, Maximum value: 100%

Create/Edit VM Storage Policy