vmware vsan vision - nlvmug.com · vmware vsan vision duncan epping - @duncanyb chief technologist...

1

VMware vSAN Vision

Duncan Epping - @DuncanYB

Chief Technologist – Storage & Availability

Data: tremendous business value potential…

Information Growth50 x data growth

from 2010 to 2020

Source: IDC* and EMC *April 2014

Where is all that data going again?

The rise of software-defined storage

Why?

Today’s storage products do not meet the requirements of the

evolving IT industry

LUNs SUCK

Storage Devices Becoming Faster with Better Endurance

$5 $20$1

IOPS

250K

500K

1M

750K

$/Gig

$15$10

Tomorrow

SSD is the new Capacity Disk

High capacity NVMe

Byte-addressable NVDIMMs

Network latency >> Device Latency

Today

Low latency devices relatively

expensive for persistent storage

Device latency >> Network latency

NVDIMM

DRAM DRAM

CONFIDENTIAL

This is where we are today…

Hyper-Converged powered by VMware vSAN and vSphere

Runs on any standard x86 server

Pools HDD/SSD into a shared datastore

Delivers enterprise-grade scale and performance

Managed through per-VM storage policies

Deeply integrated with the VMware stack

vSphere vSAN

vSAN Datastore

VMware customers trust their infrastructure to vSAN

vSAN Customers

Countries Deployed

Critical Apps on All-Flash

>7,000 >100 83%

Leading HCI Vendor#1

Supporting a broad variety of use cases

vSAN

Business Critical Apps Virtual Desktops (VDI)

DR / DA

Test/DevDMZ

ROBOManagement

Staging

Really Simple Setup

Storage policies

VM / VMDK policy profile

Policy Gold

Availability

Capacity

reservation

IOPS Limits

Value

FTT = 2

Space efficient

40GB

1000

Moving to application lifecycle management through policy

Policy

Placement and configuration by

policy

Control of QoS at VM / VMDK

level

Simple, scalable automation

platform

Primary Cluster

vSAN

Virtual Volumes

VAIO

vSphere

vSAN objects and components

Component-2

Mirror Copy

Component-1

witness

Mirror Copy

RAID-1

vSAN is an Object Store

Each Object has multiple Components

This to allow you to meet Availability and

Performance requirements

Data is distributed based on VM Storage

Policy

Fault Domains, increasing availability through rack awareness

• Create fault domains to increase availability

• 8 node cluster with 4 defined fault domains (2 nodes in each)

FD1 = esxi-01, esxi-02 FD3 = esxi-05, esxi-06

FD2 = esxi-03, esxi-04 FD4 = esxi-7, esxi-08

• To protect against one rack failure only 2 replicas are required and a witness across 3 failure domains!

15

FD2 FD3 FD4

esxi-01

esxi-02

esxi-03

esxi-04

esxi-05

esxi-06

esxi-07

esxi-08

FD1

vmdk vmdk witness

RAID-1

vSAN – Stretched ClusterActive-Active data centers

• vSAN cluster split across 2 sites!

• Each site is a Fault Domain (FD)

• Site-level protection with zero data loss and near-instantaneous recovery

• Support for up to 5ms RTT latency between data sites

– 10Gbps bandwidth expectation

• Witness VM can reside anywhere

– 200ms RTT latency

– 100Mbps bandwidth required at most

• Automated failover

5ms RTT, 10GbE

Today

VMware vSphere & vSAN

vSphere

witness

vmdk vmdk

witness

Space efficiency

Nearline deduplication and compression per disk group level

– Enabled on a cluster level

– Deduplicated when de-staging from cache tier to capacity tier

– Fixed block length deduplication (4KB Blocks)

– Compression after deduplication

RAID-5 and RAID-6 (Inline Erasure coding)

– RAID-5 needs a 3+1 configuration, but only 33% overhead

– RAID-6 needs a 4+2 configuration, but only a 50% overhead

Provide block storage through vSAN iSCSI

vSAN iSCSI Target Service enables Block Storage!

– Support for Oracle RAC

– Storage for physical workloads

– Provides max ~1K LUNS per Cluster

Provides all core vSAN functionality for the iSCSI target

– Dedupe and Compression, RAID-1, RAID-5, RAID-6

iSCSI Target

iSCSI Initiator iSCSI Initiator

iSCSI Network

iSCSI Object

iSCSI Object

vSAN

Datastore

2-node Direct Connect and Witness traffic separation

vSAN Datastore

witness

10GbE vsan traffic

management & witness traffic

Ability to connect the two nodes directly using Ethernet cables

Separating the vSAN data traffic from witness traffic

Two cables between hosts for higher availability of network

Allows for Layer-2 and Layer-3 topologies and strict separation

of traffic streams

1GbE Switch

vSAN 5.5March 2014

vSAN 6.0March 2015

All Flash Configuration

64 node vSAN cluster

Health UI

Rack Awareness

vSAN 6.2March 2016

Dedupe and Compression

RAID 5/6 support

Software Checksum

IOPS Limits

Performance Service

Enhanced Capacity Views

vSAN 6.1September 2015

Stretched Cluster

Replication - 5 Minutes RPO

2-node ROBO

Health Monitoring & Remediation

vSAN 6.5November 2016

iSCSI Services

2-Node Direct Connect

This is where we are going…

Three key pillars for the future

Availability and Security

Management and Monitoring

Data Management and Mobility

Decrease operational effort through pro-active

health monitoring and analytics.

Increase availability through advanced fault domain

architectures and encryption of data at rest

Policy based data management from

on-premises to public cloud.

INFRASTRUCTURE MANAGEMENT

Site failure or outage

Cluster-level recovery

Planned migration

Monitoring

vSphere vSAN

From Infrastructure Management in the Private Cloud…

Infrastructure Management

This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.

…To Data Management In the Hybrid Cloud

vSphere vSAN

Data Repair and Recovery

File or DB Recovery

Data Archival

Data Analytics

Data Governance

Data Management

Test and Development

This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.

Stretched Cluster: Local Failure Protection

Overview

• Provide local host and disk group failure protection

• In case of site failures there will still be local availability within the surviving site

• RAID1 of RAID5/RAID6/RAID1 is supported on All-Flash

• RAID1 of RAID1 is supported on Hybrid

Benefits

• Host and disk group protection when site failure happens

• Local replication when component has failed

vSphere vSAN vSphere vSAN

Primary Cluster

vSphere vSAN

Primary Cluster

vSphere vSAN

5ms RTT, 10GbE

RAID-1

RAID-5 RAID-5

Witness Location

This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.

Increase flexibility with Site Affinity for Stretched Clusters

• User can specify single site location of VM’s components if site level protection is unnecessary

• Policy driven using SPBM

• Reduces network and storage requirements

• Ideal for solutions that already use application redundancy (Exchange DAGs, SQL Availability groups, etc.)

vSphere vSAN vSphere vSAN

Primary Cluster

vSphere vSAN

Primary Cluster

vSphere vSAN

RAID-0

RAID-6

Witness Location

This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.

vSAN Encryption

Overview

• Software Encryption for Data at Rest

• Enabled on a per cluster level

• Integration with major central key management technologies that are KMIP compliant

• Works with de-duplication, compression

• Order: 1. Checksum, 2. de-dup, 3. compress, 4. Encryption

• Cache Tier is also encrypted!

Benefits

• Enable customers who require security provided by data at rest encryption for deploying vSAN

• While still benefitting from advanced data services

• Protection against security threats with unauthorized accessing/hacking of sensitive data on drives

• Safely transferring drives to other facilities

vSphere vSAN

vSAN Datastore

This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.

Infrastructure Management in vSAN Covers Day 0-2 and DC Wide Operations

Single-click upgrade for

ESXi, vCenter Server and

vSAN including firmware,

drivers, and software

Day 0

Network usage,

re-sync/rebuild traffic,

query disk groups for

read/write latencies, etc.

PowerCLI, S.M.A.R.T

data for hardware

monitoring, and health

checks on ESXi when

vCenter Server is down

Day 2

Capacity trends

for dedupe, improved

topology view for

Stretched Cluster,

and unified alerts

Integration through SPBM

DC-wide

Auto-ConfigCluster Health

PerformanceService

vRealizeOperations

vRealizeAutomation

This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.

Intelligent, automated operations with vSAN Easy Install

• Provides easy way to bootstrap new

vCenter Server on a single host

• vSAN configuration steps built into

VCSA installer workflows

• Simplifies process of building out new

environmentvSphere vSAN

vSAN Datastore

VCSA

This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.

VUM Integration

VUM

vSphere vSAN

vSAN Datastore

Overview

• Simple upgrade / updates of drivers and firmware!

• Automatic new version check and notification

• Built-in baseline/baseline group creation for vSAN clusters

• Automatic downloading of ESXi image

• Support online mode with internet access and offline mode with local image repository

Benefits

• Simplified upgrade / updates for vSAN clusters

• Simplified driver and firmware management

This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.

Using Analytics for Effortless Infrastructure Management

VMware Support

Customer

VMware Product Team

VMware Cloud

Telemetry Data Collection

Systems Modeling Predictive Algorithm Statistical Analysis

vSphere vSAN

vSphere vSAN

Customer 1

Customer N

On Premises

data

answers answers

This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.

Extending policies to data management

Data policies

All protected files matching search criteria

Policy Gold

Replication

Backup

Domain

Anti-Affinity

Zone

RPO = 10 mins

7 Day Retention

US

EMEA

Data Management policy per App,

VM, VMDK, File or Object

Policies across Private and Public

Clouds

Global monitoring and compliance

reportingSecurity Encryption

NFS DD BOOSTS3 API

Google AWS Data Domain

Secondary Storage

Primary Cluster

vSphere vSAN

NAS

VMDK Object File

ValuePolicy

This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.

Scalable, Efficient Native vSAN Snapshots

Feature Benefit

High

Scalability

• Natively built into vSAN storage level

• Designed to scale up to limits of vSAN

Highly

Performant

• Low impact to overall vSAN I/O even when large

number of snapshots are created

• Performance impact is fixed, regardless of

snapshot depth

Unified

solution

• Foundational technology for a unified approach –

Ad-hoc snapshots, local protection, cross cluster

replication, and archival to external storage

This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.

vSAN Native Archival Protection

Archivecopies

Localcopies

Feature Benefit

Archival of older snapshots to an

external NFS storage array

• Retain a longer history of

snapshots cost-effectively

Isolated copies of snapshots

available in case of total vSAN

failure

• Rebuild VMs onto a vSAN

using the snapshots stored on

external NAS array

Policy-based configuration of

archival

• Archival frequency and

retention policies are an

extension of vSAN policiesPrimary Cluster

vSphere vSAN Secondary Storage

NAS

This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.

End goal: Full data mobility

Cloud

Primary Cluster

vSphere vSAN

DR Site

vSphere vSAN

ArchiveSnapshots

LocalSnapshots

LocalSnapshots

DRSnapshots

5 minutes RPO5 minutes RPO1 Day retention

1 year

Object

Space efficient native

snapshots

Flexible scheduling and

retention policies

Replication of snapshots to

secondary storage and DR site

Snapshot based Local

Recovery, DR and Long

Term archival

This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.

So what’s left?

Future: File system abstraction for CNA

Native Distributed File System

Photon Machine

vSphere vSAN vSphere vSAN

Efficient image distribution

Scalable data volume sharing

Consistent snapshots / clonesmulti-container stateful applications

Primary Cluster

vSphere vSAN

Primary Cluster

vSphere vSAN

Photon Machine Photon Machine Photon MachinePhoton Machine

This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.

Native vSAN NFS/SMB File System on vSAN

Potential Approach• Highly integrated vSAN file services solution

• Supports large number of clients

Management• All management including installation, configuration, share

management via vCenter Server

Protocol Support• NFS

• SMB

• ??

vSphere vSAN

vSAN Datastore

Distributed File System

File system protocol Server

Client Client Client Client Client

This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.

Wrapping up…

VMware vSAN enabling hyper-converged for all workloads!

vSphere vSAN

vSAN Datastore

vmfs block file restObject