vmware past, present, powermax - wordpress.com · • a set of apis using standard t10 scsi...

VMware past, present, and future with Dell EMC PowerMax

Drew Tonnesen, PowerMax Engineeringhttps://drewtonnesen.wordpress.com

© Copyright 2019 Dell Inc.3

Roadmap Information Disclaimer• Dell EMC makes no representation and undertakes no obligations with regard to

product planning information, anticipated product characteristics, performance specifications, or anticipated release dates (collectively, “Roadmap Information”).

• Roadmap Information is provided by Dell EMC as an accommodation to the recipient solely for purposes of discussion and without intending to be bound thereby.

• Roadmap information is Dell EMC Restricted Confidential and is provided under the terms, conditions and restrictions defined in the Dell EMC Non-Disclosure Agreement in place with your organization.


Agenda• VMware Integration Level Set

• Summary

• VAAI Updates• VAAI plugin • ESXi 6.7 U2 UNMAP Granularity

• Automation• Management Interfaces• vRO Plug-in for Dell EMC PowerMax

• Demo• VSI

• Demo

• VDI

• Virtual Volumes• VVols 3.0/VASA 3.0• SRM

• NVM and NVMe• Introduction• Storage Transitions• NVMe over Fabrics (NVMeoF)• Demo

• Wrap Up


VMware-PowerMax Integration Points

Array APIs/Mgmt

vCenter Vendor specific

plug-Ine.g. VSI

VI Client VM

PowerMax Vendor specific VAAI T10 SCSI command support

FC, iSCSI Futures: NVMe over Fabrics, e.g. FC-NVMe

Storage API for Data Protection

(VADP)e.g. Avamar

Vendor specific Storage API for

SRM (SRA)

Site RecoveryManager (SRM)

VM-Aware Unisphere

IO Filtere.g. RP4VM

Enterprise Storage

Analytics

vRealize SuitevROps, vRO, vRA, Log Insight

VMAX and PowerMax

Content Pack

Protocol Endpoint (VVols)

ESXi Storage Stack

Datamover

VMFS NFS

NFSclient

Network Stack

VMware LVM

HBA Drivers

Storage API for multi-pathing (PP/VE)

NMP

NIC Drivers

VAIO Framework

VVolsvRO plug-

in for PowerMax


Array Awareness (VASA)

Vendor specific Storage API for Virtual Volumes

(VVol)


Array Integration (VAAI)

REST API

Python Ansible

Playbooks PowerShell

VMware Cloud

Foundation

SYMAPI

Legend: PowerMax Integration Points VMware Products


PowerMax VMware Integration Summary• Storage API Integration (VAAI, VASA)

• Offload activities to the array• Advertise capabilities from the array

• VMware Virtual Volumes (VVols)• Storage paradigm that provides array data

services with VM level granularity to VM Admin• Removes scalability limitations of traditional ESXi

environments

• Unisphere for PowerMax• View VMware infrastructure storage from within

Unisphere

• VMware Cloud Foundation• External array for Workload Domain

• PowerMax plugin for vRealize Orchestrator• Workflows providing auto provisioning capabilities,

datastore creation, snapshotting

• Virtual Storage Integrator (VSI)• In-context storage translation• Self service provisioning of storage within vCenter

• Dell EMC Storage Analytics (ESA)• Extends vRealize Operations Manager by collection

array metrics• End to end analytics, mapping, dependencies and

planning

• PowerMax and VMAX Content Pack for Log Insight

• Provides dashboards, alerts, and queries to sort and present log information intelligently

• Dell EMC SRDF SRA for Site Recovery Manager (SRM)

• Leverage SRDF replication with SRM• Support for 2-site, 3-site, STAR, and SRDF/Metro in

VMware Metro Storage Clusters


VAAI Updates


Storage APIs for Array Integration (VAAI)• A set of APIs using standard T10 SCSI commands that allows vSphere to

offload specific operations to storage arrays• Been around since vSphere 4.1 (talk about the past)!

• UNMAP• XCOPY• WRITE SAME• ATS• Thin provisioning (reporting, STUN)


VAAI Plugin (not that kind of plugin…)• VAAI Plugin is attached to PowerMax devices presented to ESXi hosts –

ensures PowerMax receives VAAI commands

• VMAX/PowerMax has a custom plugin, like most vendors, called VMW_VAAIP_SYMM

• Since 4.1 VMware has had a generic T10 VAAI plugin called VMW_VAAIP_T10

• In vSphere 6.7 VMware requested vendors voluntarily move to the generic plugin

• In a future vSphere release it will be mandatory• VMware will initiate during upgrade


VAAI Plugin (not that kind of plugin…)


Changing to VMW_VAAIP_T10• Recommendation is to move to new plugin

• To move to new plugin, remove the existing rule and add the new one• esxcli storage core claimrule remove --rule 65430 --claimrule-class=VAAI• esxcli storage core claimrule add -r 914 -t vendor -V EMC -M SYMMETRIX -P

VMW_VAAIP_T10 -c VAAI -a -s -m 240• Reboot or unclaim/reclaim (not recommended)• Rules can be removed/added at any time, do not take effect until reboot• Process is in the VAAI WP: http://www.emc.com/collateral/hardware/white-

papers/h8115-vmware-vstorage-vmax-wp.pdf

• Fear not, if you cannot/do not wish to manually move , VMware will take care of it during vSphere upgrade

http://www.emc.com/collateral/hardware/white-papers/h8115-vmware-vstorage-vmax-wp.pdf


Changing to VMW_VAAIP_T10


UNMAP Granularity in 6.7 U2• Beginning with vSphere 6.7 changes were made to

UNMAP• Two options – modifications only available post-creation, some

CLI required• Priority – Low, Medium, High - CLI• Fixed Rate – 100 MB – 2000 MB – GUI or CLI

• UNMAP granularity changes were not supported so if array did not advertise 1 MB or lower, automated UNMAP did not work

• New in vSphere 6.7 U2 VMware will support granularity larger than 1 MB

• This does not mean you can change the granularity, it is fixed at 1 MB• No impact to PowerMax, though some other Dell arrays benefit


UNMAP Granularity in 6.7 U2


Automation


PowerMax / VMAX Management Interfaces

Uni

sphe

re +

Sol

utio

ns E

nabl

er2. SYMCLI

1. Unisphere

{ REST }

Embe

dded

or E

xter

nal

3. REST API

{ REST }4. Python LibraryPowerShell Scripts/Bash Scripts with Curl Commands, Ansible playbooks

Stateless, for provisioning & monitoring, version-specific endpoints for compatibility

https://github.com/ciarams87/PyU4V• Enhancements ongoing (v3.0)• MIT licensehttps://github.com/rawstorage/dellemcpmax_ansible

Unisphere 9.0 GUI - HTML 5

Comprehensive, interactive command line interface

{ SYMAPI }

https://github.com/ciarams87/PyU4V

https://github.com/rawstorage/dellemcpmax_ansible


vRO Plug-in for Dell EMC PowerMax• Free plugin to vRealize Orchestrator

• Plugin is delivered as a vmoapp file for vRO Control Center (7.3, 7.4, 7.5)• Utilizes Unisphere REST API 9.0• Supports PowerMaxOS (PowerMax and VMAX All Flash)• Built-in workflows and actions• No error handling/intelligence, i.e. be careful!

• Workflow categories• Plugin Settings• PowerMax Discovery• PowerMax Host Operations • PowerMax Provisioning• PowerMax Snapshot Operations• PowerMax Storage Group Operations• PowerMax Volume Operations


vRO PowerMax plugin workflows• Array connection management

– Add/Remove/Update/List Unisphere server instances

• Device management– Create/Delete/Resize volumes

• Host Management – Add/Remove/Modify Host groups/Port groups/Masking views

• Snapshot management– Create/Delete/Rename/Restore SNAPVX snapshots of Storage

groups

• VMware integrated workflows– Create/Delete/Expand VMFS datastores– Create/Delete RDMs – Present/Unpresent storage devices to cluster/ESXi– Create/Delete/Rename/Restore SNAPVX snapshots of VMFS

datastores and RDMs


vRO PowerMax plugin actions• Actions are repetitive tasks

– PowerMax specific operations› Array connection management› Device management› Snapshot management

– Helper functions› List connections› List PowerMax arrays› List hosts

• Part of existing workflows, can be utilized for custom workflows


PowerMax Workflows in vRealize Automation


PowerMax vRO Plug-In Demo


vRO PowerMax plugin futures• New plugin versions will be releases a few times a year• Plugin.Next (*GA May)

• SRDF A/S/Metro• Create/Failover/Failback/Split/Expand etc.• Storage Group level like Unisphere

• Future Planning• Zoning• Physical hosts

• Windows• Linux


vRO PowerMax plugin futures


Virtual Storage Integrator Update•VSI is a free plug-in integrated into vSphere vCenter

•Supports both VMAX and PowerMax•Around since ESX 3.5, transforming from…


Virtual Storage Integrator Update


VSI 8 and 8.1• VSI 8.0 initial release of HTML 5

• Utilizes VMware Photon and Docker Containers (IAPI and Redis DB), VSI part of filesystem• Supports in-context storage translation• Provisioning

• Lacked data services functionality

• VSI 8.1 (*GA May) • Moves VSI from FS into IAPI container to ease upgrades (and downgrades)• A redesign of provisioning, resolving 8.0 gaps

• Selection of service levels, deduplication & compression, SGs• Add RDM support

• Supports in-context storage translation for VMFS and RDMs• Best practices

• NMP RR iops=1 for existing devices, adds claim rule (latency coming for SRDF/Metro environments)• XCOPY claim rule for 240 MB extents


VSI 8.1 RDM wizard


VSI 8.1 Demo


PowerMax VDI Testing• Ongoing VDI testing to establish recommendations

• Environment– Hardware

› PowerMax 8000› Dell R740s

– Software› vSphere 6.7› VMware Horizon View 7.5.1› VMware Horizon Client 4.10

• Tests– Instant clone (~ linked/fast clones)– Full clones


PowerMax VDI Testing - Results• Instant clones

– Minimal storage footprint– Access is to the same parent VM data– Recommended for desktops with small change rate

• Full clones– Full copy of data– Independent VM data for each– Recommended for desktops with high rate of change

• Initial Results– Full clones achieve ~99% data savings with deduplication– Boot storms .2 ms read/ 1.3 ms write performance

› VAAI (ATS), PowerMax architecture– White paper/Tech Note forthcoming


Virtual Volumes


VMware Virtual Volumes (VVols) - Present• Paradigm that uses storage containers to replace traditional

datastores (and RDMs) in vSphere 6• VVols are encapsulations of VM files, virtual disks, etc.• Created automatically during VM tasks – creation, cloning, snapshots –

all parts of the VM are VVols

• Provides array data services with VM Level granularity• Allows the VM Admin to directly leverage array Data Services

• Overcomes device limitation by utilizing a Protocol Endpoint to which VVols are bound

• PowerMax supports VVols 1.0/VASA 2.0 on vSphere 6.x• Replication available via RecoverPoint for VMs


VMware Virtual Volumes (VVols) - Futures• VMware announced planned SRM support for VVols

2.0/VASA 3.0• Attempting to coordinate with vSphere.Next• VVols will not require an SRA, all code is implemented in the

VASA Provider• Still need an SRA for VMFS if hybrid environment

• PowerMax has accelerated development of VASA 3.0


NVMe and NVMeoF


NVM is non-volatile memory media

• NAND-based flash

• Storage Class Memory (SCM) media technologies

NVMe (Non-Volatile Memory Express)

• A set of standards which define a PCI Express (PCIe) interface used to access data on NVM

• Takes advantage of modern CPUs and SSDs

• Replaces older storage protocols

• NVMe unlocks up to 64k queues, up to 64k commands per queue (depends on vendor)

Standard developed by an open industry consortium

• Directed by a 13 company promoter group which includes Dell

Core designobjective to achieve high levels of the following

• Parallelism

• Concurrency

• Scalability

What is NVM and NVMe


Concept attribution: J Metz, “The Data Robot Parable”

Flash + SCSI/SAS

Flash: all data blocks available at the same seek time & latency

SCSI /SAS: pick & place robot w/single arm executing 1 command at a time, 1 queue

HDD + SCSI/SAS

Slower Faster

HDD: varying speed conveyer belts carrying data blocks (faster belts = lower seek time & latency)

SCSI / SAS: pick & place robot w/tracked, single arm executing 1 command at a time, 1 queue

Media

Protocol

Flash + NVMe

Flash: all data blocks available at the same seek time & latency

NVMe / PCIe: pick & place robot with 1000s of arms, all processing & executing commands simultaneously, over 64K queues

Conceptualizing NVMe


HDD 7K, 10K, 15K

M E D I ATransitions

D R I V E I N T E R FA C E

Transitions

H O S T I N T E R FA C E

Transitions

NAND FLASHSLC, MLC, TLC

SCM

PARALLEL SCSI & ATA

FIBRE CHANNEL SAS & SATA NVMe

SCSI FIBRE CHANNEL NVMeoF

Storage technology transitions


Simplest - an NVMe card in a server, direct access to storage on the server or directly to array

NVMe over Fabric (NVMeoF)

Fabric can be any of the following:• FC (FC-NVMe)• TCP• RoCE (RDMA over Converged Ethernet)• iWARP• Infiniband• Next Gen Fabrics SSD SSD SSD SSD SSD SSD

Controller Side Transport Abstraction

Host Side Transport Abstraction

NVMe Host Software

Fibre Channel

InfiniBand RoCE Next Gen

FabricsiWARP

APPOS

APPOS

NVMe host connectivity

SCMSCM


Late

ncy

Slide credit: Intel and NVM ExpressSource: Storage Technologies Group, Intel. Comparisons between memory technologies based on in-market product specifications and internal Intel specifications.

HDD + SAS

SERVER

NVMeoF

HOST

STORAGE NETWORK

POWERMAX

P-Brick

Back End

Disk Shelf

NVMeNAND + SAS

Drive Latency Controller Latency(i.e. SAS HBA)

Software Latency

NAND+ NVMe BE

SCM+ NVMe End to end

Flash SCM*HDDKEY

How NVMe/NVMeoF reduces latency


• Highly virtualized environments requiring high IOPS and low latency

• Large Oracle / SAP deployments• Real time data processing and analytics

– Factory control / SCADA systems– Data warehousing– High end trading applications– Oil and Gas

• Large scale web services• Less applicable

– Cache friendly workloads (higher % of read hit / write hit)

– Workloads which don’t tax backend storage such as Microsoft Exchange

Real-time analytics

Big Data

Machine Learning

Does NVMe help everywhere?


FC-NVMe Futures Demo• Single 32 GB FC network (same switch)

• vSphere.Next• 2 ESXi hosts• 2 32 GB HBAs (one for FC, one for FC-NVMe)

• PowerMax.Next• EFD and Storage Class Memory (SCM) in SRP• FA emulation for FC• FN emulation for FC-NVMe• 4 devices presented down each protocol

• Migration from FC to FC-NVMe• Storage vMotion• Unmap/Remap devices (not demonstrated)


NVMeoF Futures Demo


Wrap it up!• Futures NVMeoF and SCM will be a welcome and easy transition, offering

improved performance

• VVol SRDF replication and SRM support is on the horizon

• New vRO plugin to help on your way to automation

• Remember our many other free plugins and integrations!• Storage API Integration (VAAI, VASA, VVols)• Dell EMC VMAX and PowerMax Content Pack for Log Insight• Virtual Storage Integrator (VSI)• Unisphere for PowerMax VMware view• Dell EMC Storage Analytics (ESA) • Dell EMC SRDF SRA for Site Recovery Manager (SRM)• VMware Cloud Foundation


Resources


Where do I get more information?• Google – yes it’s a cliché, and yes it works. A couple of key words and next

thing you know you’re the expert!

• Free plugins available through Dell EMC Support:• PowerMax Plug-in for vRealize Orchestrator https://support.emc.com/products/47176• Virtual Storage Integrator (VSI) https://support.emc.com/products/32161• Enterprise Storage Analytics (ESA) https://support.emc.com/products/30680• SRDF Storage Replication Adapter https://support.emc.com/products/5596• Unisphere for PowerMax https://support.emc.com/products/44740• VMAX and PowerMax Log Insight Content Pack

https://marketplace.vmware.com/vsx/solutions/dell-emc-vmax-content-pack--2

• All topics covered here (and more) https://drewtonnesen.wordpress.com/• All current documentation on PowerMax/VMware integration:

https://drewtonnesen.wordpress.com/important-docs/

https://support.emc.com/products/47176





https://marketplace.vmware.com/vsx/solutions/dell-emc-vmax-content-pack--2

https://drewtonnesen.wordpress.com/

https://drewtonnesen.wordpress.com/important-docs/

vmware past, present, powermax - wordpress.com · • a set of apis using standard t10 scsi...

Documents