virtual volumes (vvol) · vendor specific vasa providermatched to storage array extends vsphere...
TRANSCRIPT
#vmworld
Virtual Volumes (VVol)Deep DivePatrick Dirks, VMware, Inc.Pete Flecha, VMware, Inc.
HCI2810BU
#HCI2810BU
VMworld 2018 Content: Not for publication or distribution
Disclaimer
2©2018 VMware, Inc.
This presentation may contain product features orfunctionality that are currently under development.
This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.
Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.
Technical feasibility and market demand will affect final delivery.
Pricing and packaging for any new features/functionality/technology discussed or presented, have not been determined.
VMworld 2018 Content: Not for publication or distribution
3©2018 VMware, Inc.
CLIFF DIVING in Cinque TerreVMworld 2018 Content: Not for publication or distribution
Agenda
7©2018 VMware, Inc.
VVols Under the Hood
VVols Compared
VVol Replication
Migration Considerations
What’s New for VVols
Q & A
VMworld 2018 Content: Not for publication or distribution
8©2018 VMware, Inc.
vCenter Deployments with Vvols VMware Analytics Cloud (VAC)
vSphere 6.0
vSphere 6.5
2017 2018
vCen
ter
Dep
loym
ents
with
VV
ols
vSphere 6.7
VMworld 2018 Content: Not for publication or distribution
10©2018 VMware, Inc.
VVol value proposition resonates with vSphere customers• VM granularity & Storage Policy Based Management• Direct and standardized integration from storage array to vCenter• Tighter control and reporting
Customer shift to vSphere versions that support VVols• vSphere 5.5 End of Support in September 2018• Many VMware customers are just now upgrading to vSphere 6.5• Most vSphere customers are not on a release that supports VVols
Success of HCI – Shift to vSphere admin centricity• VI admin now manages Compute, Network, Storage
Why are VVols Gaining Traction?
VMworld 2018 Content: Not for publication or distribution
11©2018 VMware, Inc.
Virtual Volumes is Simpler, Smarter, Faster
Simplified Management
Smarter Infrastructure
Improved Performance
VMworld 2018 Content: Not for publication or distribution
‹#› 12©2018 VMware, Inc.
VVols Under the Hood
VMworld 2018 Content: Not for publication or distribution
13©2018 VMware, Inc.
vSphere Virtual Volumes
Integration Framework for VM-Aware Storage
vSphere VVols
Storage Policy Management by SPBM
No File System; virtual disks are natively represented on arrays
Vendor Specific VASA Provider matched to storage array
Extends vSphere Storage Policy-Based Management to the storage ecosystem
SPBM uses array native capabilities on a VM granular storage
Supports existing storage I/O protocols (FC, iSCSI, NFS)
Based on T10 industry standards
Industry-wide initiative supported by all major storage vendors
Included with vSphere at no additional costVVol enabled Storage Cluster
Storage Array VASA Provider
VVol VVol VVol VVol VVol VVol
VMworld 2018 Content: Not for publication or distribution
14©2018 VMware, Inc.
VASA Provider (VP)
Management component for specific array• Presents array’s SPBM capabilities• Provides Storage Awareness services• Creates and manages Virtual Volumes
ESX and VC connect to VASA Provider (VP) using VMware-defined VASA protocol
Service can run anywhere; interface to array is private
Standard VASA
Control Path
Private array
Control PathVVol enabled Storage Cluster
Storage Container
VVol Datastore
Protocol Endpoint(s)
Standard SAN/NAS Data Path
VASA Provider
PEPE
VVol VVol VVol VVol VVol VVol
VMworld 2018 Content: Not for publication or distribution
15©2018 VMware, Inc.
• Two main types of datastores in use today:– NFS Mount Points (NAS)– IP or block based datastores (SAN)
• Datastores serve two purposes today:– Endpoints – receive SCSI or NFS I/O commands– Data repository – holding large numbers of VM
metadata and data files
• Differences between PEs and Datastores:– Concept is same for all storage: mount point for
NAS, LUN for SAN-based storage– PEs is ONLY the destination for I/O commands;
PEs don’t directly store files or VVols themselves
Protocol Endpoints (PEs)
Standard VASAControl Path
Private arrayControl PathVVol enabled Storage Cluster
Storage Container
VVol Datastore
ProtocolEndpointLUN(s)
Standard SAN/NAS Data
Path
VASA Provider
VVol VVol VVol VVol VVol VVol
PE PE PE
VMworld 2018 Content: Not for publication or distribution
16©2018 VMware, Inc.
Bindings are data path coordinating mechanisms that occurs between VASA providers and ESXi hosts for accessing virtual volume.
• Different Binding Mechanism:
– Binding – allows array create I/O channels for a virtual volume
– Unbind – destroys the I/O channel for a virtual volume to a given ESXi host
– Rebind – provides the ability to change the I/O path (i.e. choice of PE) for a given virtual volumes run time using array-generated events.
Binding Operations
create bind
unbinddelete
VM Creation VM power-onopen (2)
I/O read (2)
I/O write (2)
VM power-offclose (2)
VM destroy
VP rebalance
REBINDI/O
Virtual Volume Lifecycle
VMworld 2018 Content: Not for publication or distribution
17©2018 VMware, Inc.
Storage Container
Storage Containers
• Logical storage constructs for grouping of virtual volumes.
• Corresponds 1:1 with datastores in vSphere
• Typically defined and setup by storage administrators in order to define:
– Storage capacity limits
– Capability/policy restrictions
• Minimum one storage container per array
• Maximum depends on the array
vSphere VVols
Array Storage Cluster
VVol Partner VASA Provider
VVol Datastore
Storage Container
vSphere View
Storage View
VVolVVolVVol
VVolVVolVVol
VMworld 2018 Content: Not for publication or distribution
18©2018 VMware, Inc.
Virtual Volumes • Virtual machine objects stored natively
on array• “Owned” and operated on by array only• SLA defined by object’s storage policy
There are five different types of Virtual Volumes:• CONFIG – vmx, logs, nvram, log files, etc• DATA – VMDKs• MEM – Snapshots• SWAP – Swap files• Other – vSphere solution specific type
vSphere Virtual Volumes
vSphere Web Client View
vvol
Mem
Data
Config
VVol
Swap
VVol
VVol
VVol
VMworld 2018 Content: Not for publication or distribution
19©2018 VMware, Inc.
Differences between Storage Containers and LUNs
• Introduced where limits are desirable
• Storage containers are logical constructs that can be reconfigured without disrupting current users
• Storage containers can present multiplecapabilities, SLA/QoS directed by storage policy
Storage Containers
10PB
VVol VVol VVol VVol VVol VVol
LUN • Size is fixed, growing is painful
• All blocks are same
• Needs a filesystem
• Array is unaware of use of individual blocks:
• Filesystem metadata, vmdk blocks, VM metadata
• Managed by in-band fileSystem commands
VMworld 2018 Content: Not for publication or distribution
20©2018 VMware, Inc.
Standard VASAControl Path
Private arrayControl PathVVol enabled Storage Cluster
Storage Container
VVol Datastore
VASA Provider
VVol VVol VVol VVol VVol VVol
Seamless Data Protection of Virtual Volumes
Standard VADP Libraries work independent of datastore type
SAN Transport Mode not supported but Hot Add Mode yields equivalent performance
Arrays leverage insight into virtual disk structure to offer native data protection options
VMworld 2018 Content: Not for publication or distribution
‹#› 21©2018 VMware, Inc.
Simpler, Smarter, FasterVVols Compared
VMworld 2018 Content: Not for publication or distribution
22©2018 VMware, Inc.
Storage Container vs LUNs• Logical vs Physical• Boundaries at VI admin’s choice• VM granular policy vs. fixed, uniform blocks
Retail vs. Wholesale
Policy change vs. Storage Migration
VVols are Simpler
VMworld 2018 Content: Not for publication or distribution
23©2018 VMware, Inc.
Encryption
Deduplication
Replication
QoS
Disk Type
Array Capabilities and Storage Policies
• Storage Admin creates Storage Containers, selecting
• size limit
• capabilities
• VI Admin registers VP
• VP presents Storage Containers and associated capabilities to vSphere
• VI Admin authors VM Storage Polices based on presented capabilities to match expected VM needs
• VI Admin assigns policies as VMs are provisioned
• Storage policies determine selection of datastore
Storage Capabilities to Storage Policies
Storage Policy Management by SPBM
VVol enabled Storage Cluster
Storage Array VASA Provider
VASA Provider
“Development”Test\Dev
Availability
PerformanceData
ProtectionSecurity
“Staging”Capacity
Availability
HDDData
ProtectionEncrypted
“Production”Performance
Snapshots
All FlashData
ProtectionEncrypted
PE PE
Storage Container
vSphere VVolsVVol Datastore
VVol VVol VVol VVol VVol VVol
10PB
VMworld 2018 Content: Not for publication or distribution
24©2018 VMware, Inc.
SPBM vs Spreadsheets• Visibility into array capabilities• Filters datastores to highlight compatible
storage for a given storage policy• Makes storage requirements explicit and visible
for inspection/compliance verification
Array “owns” VVols• Array can apply its full capabilities• Efficient clones and snapshots
Policies can control array behavior• QoS• Replication• Availability
VVols Are Smarter
VMworld 2018 Content: Not for publication or distribution
25©2018 VMware, Inc.
SPBM Policy Disk Type Snapshots Encryption QoS Dedupe Replication
RAID
Production All Flash Yes Yes 10K IOPS Enabled NYC RAID-6
Staging All Flash Yes No 5K IOPS Disabled LA RAID-10
EvaluationHybrid No Yes 1K IOPS Enabled Miami RAID-6
Test/Dev HDD No Yes Best Effort Disabled No RAID-0
Storage Policy Based Management (SPBM)Storage Policies Managed within vSphere
RAID
Policy-based storage configuration, protection and replicationat the VM and application-level
vSphere VVols
Storage Policy Management by SPBM
• VASA lets arrays present their unique native capabilities
• SPBM lets you compose capabilities into storage policies assigned to VMs or virtual disks
• Policies cover- storage configuration- data protection- replication- data encryption
VMworld 2018 Content: Not for publication or distribution
26©2018 VMware, Inc.
Clones and snapshots offloaded• Exploit array’s native capabilities – instant
deletion• Inherent space efficiency
Policy change vs Storage Migration• Heterogeneous containers vs uniform LUNs• Policy changes happen behind the scenes• QoS on the fly
VVols are Faster
VMworld 2018 Content: Not for publication or distribution
27©2018 VMware, Inc.
Snapshots: VMFS vs VVols
1h
1d
1w
1h
VVolID 56
Snapshots in non-VVolstorage
• Without array offloading, vSphere emulates snapshots with its own redo log mechanism
• Redo logs are optimized for discarding changes, going back to an earlier point. Committingchanges is a slow and laborious process involving repeating all intervening I/O.
Virtual Volumes
• VVols offload snapshots to the array to create them in the most efficient way possible.
• No data is copied between array and host during creation or deletion of snapshots
• Snapshot time is bounded (and minimal)
VVolID 66
VVolID 56
VVolID 42
Creation and deletion of snapshots is faster and more efficient on VVol storage due to array offloading of management operations
VMFS VVols
VMworld 2018 Content: Not for publication or distribution
‹#› 28©2018 VMware, Inc.
Virtual Volume Replication
VMworld 2018 Content: Not for publication or distribution
29©2018 VMware, Inc.
VASA Replication Model
Overview
• Replication is modeled as relationship (“Replication Group”) between “Fault Domains”
• Topology Discovery (Fault Domains, Replication Groups) through VASA APIs
• VASA replication APIs intended to substitute for SRA interface when dealing with finer-grained replication technologies (e.g. VVol, VAIO)
• Replication APIs operate on Replication Group-granularity
Palo Alto
New York
Miami
RG-NY-Marketing
RG-MIA-Sales
RG-NY-Accounting
VMworld 2018 Content: Not for publication or distribution
30©2018 VMware, Inc.
Replication Groups Presented for Selection
Overview
• Users select1. policy containing replication
2. Compatible datastore
3. Choose from list of replication groups
• “Special” replication group presented for replication providers that support creating new RGs on the request
• The policy defines the SLA
• Replication Groups collect together VMs or disks that need to be replicated in a write-order consistent fashionVMworld 2018 Content: Not for publication or distribution
31©2018 VMware, Inc.
VASA Replication Model – APIs
SRA command VASA API PowerCLI
Synchronize syncReplicationGroup Sync-SpbmReplicationGroup
Failover failoverReplicationGroup Start-SpbmReplicationFailoverStart-SpbmReplicationPrepareFailover
TestFailover testFailoverReplicationGroupStarttestFailoverReplicationGroupStop
Start-SpbmReplicationTestFailoverStop-SpbmReplicationTestFailover
Reprotect reverseReplicateGroup Start-SpbmReplicationReverse
Replication discovery
queryReplicationPeerqueryFaultDomain
queryReplicationGroupqueryPointInTimeReplica
Get-SpbmFaultDomainGet-SpbmReplicationGroupGet-SpbmReplicationPair
VMworld 2018 Content: Not for publication or distribution
32©2018 VMware, Inc.
SRM Support for VVols with Array Based Replication
Site Recovery Manager (SRM) plans to support protection and orchestrated recovery of virtual machines that are located on Virtual Volumes and replicated by Storage Array based replication
VVol + SRM Now On SRM Roadmap!
VMworld 2018 Content: Not for publication or distribution
‹#› 33©2018 VMware, Inc.
Migration Considerations
VMworld 2018 Content: Not for publication or distribution
34©2018 VMware, Inc.
Requirements and Incompatibility for VVols
Hosts must be ESXi 6.x, must have a vCenter Server 6.x
Array firmware must support VVols (may need to upgrade)
Must use supported storage protocol (varies by vendor)
Storage array capable & licensed to support thin provisioning & snapshots
HBA driver that supports Secondary LUN IDs
• Storage I/O Control • NFS version 4.1• Storage Distributed Resource
Scheduler (SDRS) • Raw Device Mapping (RDM)
vSphere Requirements vSphere Incompatibility
VMworld 2018 Content: Not for publication or distribution
Verify Array and HBA Support for secondary LUN IDs
Virtual Volumeshas it’s own HCL
Secondary LUNID HCL is a feature under IO Devices
VMworld 2018 Content: Not for publication or distribution
36©2018 VMware, Inc.
Before migrating remove excessive content from VM home directory
For OVF, first deploy to non-VVol datastore, remove excessive content, then migrate to VVol datastore
VVol migration fails when VM home directories exceeds 4GB
Issue
Solution
VVol Migration Considerations Config VVol cannot exceed 4GB
VMworld 2018 Content: Not for publication or distribution
37©2018 VMware, Inc.
• Offline Migration• Pros: All migration done in VMware. Should work with all storage vendors that support VVols. No need
to add or remove any storage manually to the VM.• Cons: Might take the longest. Still need to delete the RDM(s). Offline process.
• Create a new VVol and Copy• Pros: Fastest option–the least downtime. Easily scriptable.• Cons: Need to delete the old RDM. Need to use the array tools and VMware tools. May not work with
all array vendors implementations of VVols.
• Convert the RDM to a VVol• Pros: Fast option. Easily scriptable. No need to delete the RDM–you can re-use it.• Cons: Your array may not have a method to assign an existing volume a UUID. You still need to
disconnect and remove the RDM from the VM. Requires use of both VMware tools and array tools.
Moving from an RDM to a VVolRDMs
VMworld 2018 Content: Not for publication or distribution
38©2018 VMware, Inc.
Migrating from RDM to VVolsCreate a new VVol and Copy
VMworld 2018 Content: Not for publication or distribution
‹#› 39©2018 VMware, Inc.
What’s New in vSphere 6.7
VMworld 2018 Content: Not for publication or distribution
40©2018 VMware, Inc.
Virtual VolumesNew Functionality in vSphere 6.7
With vSphere6.7, three features have been added to enhance the VVol functionality
WSFC
End-to-end IPv6 support for
management access to VASA providers
SCSI-3 Persistent Group Reservations
(PGRs) support
TLS 1.2 default VASA provider
security
VMworld 2018 Content: Not for publication or distribution
41©2018 VMware, Inc.
• Enables VVols to be used with business critical applications such as SQL Server and Exchange
• Simplified management by eliminating the need for RDMs
Enable SCSI-3 Persistent Reservations with Virtual Volumes to enable support for Windows Server Failover Clustering (WSFC)
Overview
Benefits
Windows Server Failover Clustering (WSFC) Support
Storage Array
VVol
Windows Server Failover Cluster
VMworld 2018 Content: Not for publication or distribution
42©2018 VMware, Inc.
vSAN - VVol Interoperability
vSAN & VVols are fully interoperable
Common policy management experience under SPBM
vMotion for non-disruptive migrations
vSAN VVol
vRealize Suite for automation, orchestration and monitoring
vSphere Replication for disaster recovery and availability
Storage vMotion for non-disruptive migrations
Per VM granular management
SPBM for per VM common policy management experience z
VMworld 2018 Content: Not for publication or distribution
43©2018 VMware, Inc.
Eliminate inefficient handoffs between VI and Storage Admin
Faster storage provisioning through automation
Simplified change management through flexible consumption
Self-service provisioning via cloud automation tools.
Leverage native array-based capabilitiesFine control at the VM levelDynamic configuration on the flyEnsure compliance through policy enforcement using automation
Increase capacity utilization.
Eliminate overprovisioning
Reduce management overhead
Eliminate stranded storage
TakeawaysA More Efficient Operational Model For External Storage
Simplifies Storage Operations
Simplifies Delivery of Service Levels
Improves Resource Utilization
VMworld 2018 Content: Not for publication or distribution
44©2018 VMware, Inc.
vSphere Virtual Volumes is an Industry-wide Initiative
Wide range of products
offered
Unique capabilities
And Many More…
29 Partners in the
Program
VMworld 2018 Content: Not for publication or distribution
45©2018 VMware, Inc.
Dive inThe water’s fineVMworld 2018 Content: Not for publication or distribution
©2018 VMware, Inc. 46
Virtually Speaking Podcast
VMworld 2018 Content: Not for publication or distribution
PLEASE FILL OUTYOUR SURVEY.Take a survey and enter a drawingfor a VMware company store gift card.
#vmworld #HCI2810BU
VMworld 2018 Content: Not for publication or distribution
Leveraging Virtual Volumes (VVol) to Simplify Storage
ManagementCustomer Panel
Bryan Young - VMware
Michael Bailess - Krispy Kreme Doughnut Corp
J.J. Seely - NuScale Power
3:30 p.m. - 4:30 p.m. Mandalay Bay H, Level 2
HCI2550PU
VMworld 2018 Content: Not for publication or distribution