brief history microsoft sds : past, present and future dell’s journey with storage spaces...
TRANSCRIPT
Spark the future.
May 4 – 8, 2015Chicago, IL
Spark the future.
May 4 – 8, 2015Chicago, IL
Deploying Private Cloud Storage with Dell PowerEdge Servers and Windows Server vNextSyama Poluri, Claus Joergensen , Shai Ofek
BRK3496
• Brief History • Microsoft SDS : Past , Present and Future• Dell’s Journey with Storage Spaces• Learnings from field and Cloud Platform System (CPS)
• Dell Storage with Microsoft Storage Spaces• Storage Spaces Direct Technology Overview
• Physical View• Logical View • I/O Stack• ReFS
• Storage Spaces Direct Deployment• PowerEdge R730xd Platform Details• Managing , Monitoring and Troubleshooting
• Dell & Microsoft Partnership
Agenda
Microsoft SDS : Past , Present and Future
Dependent on Traditional Storage • Relied on 3rd
party SAN vendors
1st wave SDS• Introduction to
Storage Spaces • “Remote File solution
for Hyper-V using shared JBOD
• Introducied Scale Out File Server ( SOFS)
• RDMA and Multichannel
• SAN independence
Storage Spaces Direct • Remote File solution for
Hyper-V using shared nothing hardware
• Unlock use of DAS• Reduce Cost Software Storage Bus• SATA and NVMe• Node Fault Tolerance• Health Services Storage Spaces Enhancements• Scalable pools• Data Rebalancing
Pre Windows 2012
Windows 2012
Windows 2012 R2
Windows 2016
2nd wave SDS• Storage Tiering • WriteBack Cache
(WBC) • Scale Out File
Server improvements
• SMB Direct Improvements
• Volume Rebalancing
• Multiple instances
• JBOD monitoring
Dell’s Journey With Storage SpacesSept 2012: Windows 2012 Release:Understanding the technology and feedback to Microsoft
Oct 2013: Windows 2012 R2 ReleaseInternal testing and active feedback to Microsoft
Nov 2013:Worked closely with Microsoft in developing JBOD monitoring feature
Feb 2014:Support Storage Spaces using PowerVault MD1200 and MD1220
May 2014:Support Storage Spaces using PowerVault MD3060eEnd-End VDI solution using PowerVault MD1220
Sept 2014:Additional Hardware Support with 12Gbps SAS . Select PowerEdge servers , PowerVault MD1400 and MD1420
Oct 2014:Launch Cloud Platform System (CPS) Powered by Dell
Journey Continues…
Dell PowerEdge servers
Dell Storage dense enclosures
Dell Networking switches
Optimized racking & cabling for high density, servicing, reliability & redundancy
Microsoft Cloud Platform SystemAzure-consistent Cloud in a Box
Windows Server 2012 R2, System Center 2012 R2, Windows Azure Pack
Microsoft-designed architecture based on Public Cloud learning
Optimized run-books for Microsoft applications
Microsoft-led support & orchestrated updates
Visit Microsoft Cloud Platform
booth to see full demo and hardware
Learnings from the field and CPS • Planning real world deployments
• Capacity planning• Performance planning• Need guidance more than simple deployment guides
• Selecting the correct hardware• Some HDD scale better than others• HBA firmware , Network cards and driver information that are tested and qualified at
scale • Configuration complexities
• How many columns, HDDs and SSDs• SAS and network cabling • Pool configuration and RDMA configurations
• Validation with Windows features• Optimal configuration of windows features • Latest Hotfix / QFE information
• Change management for Storage Spaces clusters
Announcing Dell Storage with Microsoft Storage
Spaces
2
1
2
1
DC AC
DC AC
PSU A
PSU B
SBB A
SBB B
SBB B
SBB A
PSU B
PSU A PSU A
PSU B
SBB A
SBB B
SBB B
SBB A
PSU B
PSU A
1 2
LnkLnkLnkLnk
ID/Diag
Lnk Lnk
1 2
LnkLnkLnkLnk
ID/Diag
Lnk Lnk
2
1
2
1
DC AC
DC AC
PSU A
PSU B
SBB A
SBB B
SBB B
SBB A
PSU B
PSU A PSU A
PSU B
SBB A
SBB B
SBB B
SBB A
PSU B
PSU A
1 2
LnkLnkLnkLnk
ID/Diag
Lnk Lnk
1 2
LnkLnkLnkLnk
ID/Diag
Lnk Lnk
1
2
7
2
1
3
64
5
1 2 3 4iDRAC
750W750W
1
2
7
2
1
3
64
5
1 2 3 4iDRAC
750W750W
1
2
7
2
1
3
64
5
1 2 3 4iDRAC
750W750W
QSFP+
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
QSFP+
RS-232
ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56
52 60
SFP+
Force10 S4810P
1
2
7
2
1
3
64
5
1 2 3 4iDRAC
750W750W
1
2
7
2
1
3
64
5
1 2 3 4iDRAC
750W750W
Supported Client ServersAny Dell PowerEdge Server
6 Supported Dell Switches(S4810, S5000, N4064, N4064F, N4032, N4023F)
10Gb iWARP or ROCE RDMAClient Communication
2, 3 or 4 Storage NodesR630 or R730
SAS Direct Attach(HA cabling)
2 to 4 Storage Enclosures(MD1400, MD1420, MD3060e)
Compute
Networking
Client I/O
Controllers
Storage I/O
Storage
Dell Storage with Microsoft Storage Spaces (DSMS)
More information at http://www.dell.com/microsoft
Private Cloud
Backup & archival
Validated & tested
solution blocks
VDI
Hyper-V
Worldwide installation
services and total solution support
SQL
SMB3 & RDMA
Sizing & performance
guidance
Standard managemen
t with MS System Center, iDRAC,
OpenManage
Validated and tested solution blocks for reliable SOFS operation
Pre-defined SKUs for 2x2, 2x3, 2x4, 3x3 and 4x4 SOFS configurations
Award winning, industry standard 13th Generation Dell Enterprise PowerEdge servers and PowerVault JBODs
Low cost compute and networking with C-servers, FX, Blades, PowerConnect & Force 10
RDMA solutions using iWarp and RoCE
Workload optimized turnkey solutions & Reference Architectures (Private Cloud & VDI)
Lau
nch
ing
Jun
e
20
15
Dell Storage with Microsoft Storage Spaces (DSMS)
Sizing• Performance and Capacity sizing• SKUs for pre-defined configurations• Optimized solution SKUs using standard Dell servers and JBODs
Deployment• Installation and deployment option Dell services• Validation assistance for deployment on-site• Comprehensive Deployment Guide
Management• Leverages Dell & Microsoft management tools• Solution updates with single payload for storage nodes and JBODs
through Microsoft Cluster Aware Updating (CAU)
Support• Full solution-level support for all components• Dell worldwide support services
Pre-Sales engagement & sizing for workload
Fulfillment & Deployment Worldwide
On Premise Management
& Support
End-to-End Dell Engagement for lifecycle of entire solution
Dell Storage with Microsoft Storage Spaces AdvantageVisit Dell booth on
Expo floor for demo and solution details
DSMS as a Building Block for Storage Solutions Customer- a la carte
• Sizing guidance & Best Practices
• Deployment Guide• Best-in-class components
• System Center integration
Fast Track Solutions
• Validated configuration• Simple setup• High availability• Easy scalability
Reference Architectures
• Verified configuration• Deployment guide• Scalable & Customizable
Partner & Dell Services
• Best-in-class components
• Professional expertise and custom solution development
• Microsoft compatibility
• System Center integration
Dell Storage with Microsoft Storage Spaces
Introducing Storage Spaces Direct
Storage Spaces DirectEnabling cloud hardware designsSupport for DAS (shared nothing) storage hardwarePrescriptive configurations
Scale for Technical Preview 2Minimum of four serversMaximum of twelve servers Maximum of 240 disk devices in a single pool
Fault toleranceFault tolerance to disk, enclosure and node failures3-copy mirror and dual parityResilient up to two node failures
ManagementSystem Center and PowerShell
Simple expansionAdd node(s) and expand poolRebalance for capacity
Scale-Out File Server
Hyper-V Clusters
SMB3 Storage Network Fabric
Shared JBODStorage
InternalDisks
Spaces Direct Logical View Storage Access (Scale-Out File Server)Remote data access for Hyper-VData access resiliency
File System (CSVFS/ReFS)System-wide data accessFast VHDX creation, grow and mergeData Repair
Storage SpacesStorage poolingVirtual disksData storage resiliency
Software Storage BusSpans all storage nodesVirtualizes physical disksLeverages SMB3 and SMB Direct
Storage nodes with internal disksSATA, NVMe, SAS
Software Defined Storage System
Storage Node
Storage Node
Storage Node
Storage Node
Storage Pool
Storage Space Virtual Disks
Scale-Out File Server \\FileServer\Share
Cluster Shared Volumes C:\ClusterStorage
SMB3 Storage Network Fabric
Software Storage Bus
File Server
File System
Clusport Virtual HBA
CSVFS
VolMgr
PartMgr
Disk
SpacePort
PartMgr
Disk
HBA Miniport
ClusBT Volume Mgr
1. Write arrives from compute node
2. SMB sends it downstream
Server 1
Server 2
\\fileserver\share1
C:\clusterstorage\volume1
Virtual Disks
SBL Disks
Physical Disks
3. Spaceport sends write to each copy on disk
4. SBL forwards write to block target on destination node with shortest queue
5. SBL on each node sends data to physical stack
Acc
ess
Layer
Vir
tualiz
ati
on
La
yer
Sto
rag
e B
us
Layer
Ph
ysi
cal
Layer
File System
VolMgr
PartMgr
Disk
SpacePort
PartMgr
Disk
HBA Miniport
Spaces Direct I/O Stack
Clusport Virtual HBA
ClusBT Volume Mgr
ReFS is the Primary File System
• VM checkpoints have parent-child VHDs
• After backup completes, Hyper-V historically moved all data from child to parent (slow VHD “merge”)
• Using ReFS, Hyper-V now quickly reassigns blocks from the child to the parent
EfficientVM checkpoint clean-up after backup
• Available per-file integrity streams validate file contents
• Upon error detection, ReFS retrieves a correct copy from Storage Spaces, if available
• ReFS uses the correct copy to automatically repair the corruption
ResilientOptional automatic error correction
• HDDs are optimized for capacity
• SSDs are optimized for performance
• ReFS keeps hot data in SSDs and cold data in HDDs, providing both performance and capacity
AffordablePerformance of SSDs, capacity of HDDs
SSD
• Does not need to pre-fill new files and growing files with zeros
• Quickly creates large fixed VHDs
• Quickly grows dynamic VHDs
AcceleratedVHD creation and growth
ReFS: ResiliencyMetadata ChecksumsChecksums protect all filesystem metadata
User Data ChecksumsOptional checksums protect file data
Checksum Verification• Occurs on every read of checksum-protected
data• And during background scrubbing
Healing of Detected Corruption• Healing occurs as soon as corruption is
detected• Healthy version searched for in Spaces
alternate copies (i.e. mirrors or parity data)• Corrupt version overwritten by healthy version
Disk 1
Disk 2
Disk 3
Spaces 3-Way Mirror with ReFS
• Reads striped across all three disks• Checksums verified on reads• On checksum mismatch, mirrors are
consulted• Good copies used to heal bad mirror
BRK3474: Enabling Private Cloud Storage Using Servers with Local Disks
Storage Spaces DirectHardware Deployment
Options
Visit Microsoft Cloud Platform
booth to see full demo and hardware
PowerEdge R730xd
Up to 2 Intel Xeon E5-2600 v3 (18
core) CPUs
24DIMM slots(DDR4, 2133)
Up to 16x3.5 “+2 x2.5“ Drive Slots
64 TB for Data with 4TB disk
96 TB for Data with 6TB disk
2U Height Up to 6 PCIe slots
Network Select Adapter
(choice of onboard NIC)
Internal Storage ( SAS ,
SATA, NVMe
Storage Spaces
Certified HBA
PowerEdge R730xd Overview (Rear)
2 x 2.5” Drive BaysvFlash Media Card Slot
Dedicated Management Port
Serial Connector
Video Connector
USB 3.0 / 2.0 Ports
Network Daughter Card (choice of 4 x 1GbE or 2 x 10GbE + 2 x 1GbE); SFP+ Option Available
Up to 2 x Redundant Power Supplies – 495W, 750W and 1100W (AC or DC) options
Half Length, Low Profile SlotsPCIe3 x8 (x16 connector)
Full Length, Full Height SlotsPCIe3 x16 (x16 connector)
Full Length, Full Height SlotPCIe3 x8 (x16 connector)
PowerEdge R730xd Overview (Internal Bay)
“Dell has hit a long home run with the PowerEdge R730xd, delivering a powerful 2U server with lots of room to grow.”
Exceptional performance and versatility for dynamic workloadsDell PowerEdge R730xd and R730 industry reviews
“The storage-centricR730xd offers new degrees of flexibility… to keep storage as close as possible to the compute layer.”
“The results are in, and the Dell PowerEdge R730 kicks some serious butt.”
PowerEdge R730xd offers an “incredible range of configurability.”
Intel Technology Innovation Accelerated Award
DEMO @ Microsoft Cloud Platform Booth
PowerEdge R730xd Spaces Direct ClusterStorage : 13x 4TB HDD+4x900 GB SSD Network : Intel 520 10GB+i350 GBRDMA : Mellanox Connectx-3 10GB SPF
PowerEdge R630 Hyper-V Cluster
Dell S4810 Networking Switch
Deployment Process
Install H/W
Validate H/W (validation
tests)
Install OS, Configure Roles
, Features
and Network, Cluster, Set
Cluster
Property
Validate
Cluster Storage & File Share
Run workload
Ongoing Monitoring &
Troubleshooting
<-New
Simple deployment using PowerShell#Create cluster
New-Cluster –Name cluster01 –Node machine1, machines2, machine3 –NoStorage
# Enable Storage Spaces Shared Nothing (Get-Cluster).DASModeEnabled = 1
#Create file serverAdd-ClusterScaleOutFileServerRole -Name sofs1
#Create storage pool and define storage tiersNew-StoragePool -StorageSubSystemFriendlyName *clustered* -FriendlyName DAS -ProvisioningTypeDefault Fixed -ResiliencySettingNameDefault Mirror -PhysicalDisk (Get-PhysicalDisk | ? CanPool -eq $true)
#Create virtual diskNew-Volume -StoragePoolFriendlyName DAS -FriendlyName vdisk01 -PhysicalDiskRedundancy 2 -FileSystem CSVFS_REFS –Size 4000GB
#Create file sharemd c:\clusterstorage\volume1\share1 new-smbshare -name share1 -path c:\clusterstorage\volume1\share1 -fullaccess HyperV01, Hyper-V02set-smbpathacl -sharename share1
Monitoring and Troubleshooting
Dell OpenManage Integration Suite for Microsoft System Center
Dell Server and Client
Deployment Packs
Dell Server and Client
Update Catalogs
Dell Server PRO
Management Pack
Dell Hardware
Management Pack Suites
Deploy Update Monitor Virtual Management
Dell Lifecycle Controller Integration
Storage Diagnostic Logs Cmdlets
#Starts storage tracing sessions in the given subsystemStart-StorageDiagnosticLog -StorageSubSystemFriendlyName “RemoteSQLCluster”
#Repro the issue
#Stops currently running storage tracing sessions in the given subsystemStop-StorageDiagnosticLog -StorageSubSystemFriendlyName “RemoteSQLCluster”
#Gathers storage event logs, trace logs & live dumps from each node in the clusterGet-StorageDiagnosticInfo -StorageSubSystemFriendlyName “RemoteSQLCluster” –DestinationPath D:\StorageLogs\
SCOM Storage MonitoringStorage subsystem determines healthFocus on relevant objects- Storage subsystem, volumes and file shares- Automatic remediation when possibleActionable alerts- Alert specifies urgency- Alert specifies remediation action- Alerts automatically resolve when issue is
addressed- Easy to determine affected objectsConsumable through PowerShell or 3rd parties- SMAPI
Storage System
Actions
System Center vNext
Health Logic
Operations Manager
Dashboard
User notifications
3rd PartyMonitoring
PowerShell
Virtual Machine Manager
Discovery
Actions
Data Center Networking
Dell M
an
ag
em
en
t Softw
are
an
d P
lug
-ins
for S
yste
m C
en
ter
Scale-out file
server clusters
Storage Space
Storage Space
Storage Space
Hyper-V clusters
Dell PowerVault JBOD Storage
Dell PowerEdge Servers
Dell Network Switches
Dell PowerEdge Servers
Dell & Microsoft Key Value
Dell-M
icro
soft E
nd
–En
d S
olu
tion
s (V
DI,P
rivate
C
lou
d) Tested and Validated for
Scalable and Reliable Deployment
Storage Spaces (Windows Server 2012 R2) Microsoft and Dell validated Hardware and solutions Dell Storage with Microsoft Storage Spaces
Cloud Platform System Built on Dell Hardware Best Practices, Driver optimization, etc.
Storage Spaces Direct (Windows Server vNext) Co-Engineer Solution Joined TAP
Dedicated Engineering teams: Microsoft and Dell
Dell and Microsoft PartnershipContinuous and Co-Engineering
Leverage the power of Dell and Microsoft alliance
Joint IP/development
Engineering labs, technology centers, centers of excellence
US Windows Deployment Partner of the Year2 years running
Most complete
Full, end-to-end capabilities – from tablets to the data center
Global Solutions CentersDemonstrate the full Microsoft stack running on Dell
Single Point of ContactDell provides a single contact for total solution
TAP participation
Broad participation in Technology Adoption Programs
US Windows Partner of the Year
2013
“Dell is Microsoft’s largest OEM partner to have implemented support for Storage Spaces. Dell provides the most complete Storage Spaces implementation – server, controller, JBOD and disks form an end-to-end solution, and this support includes new features such as JBOD monitoring.”
Microsoft Server and Cloud Platform Team
TechNet Server & Cloud Blog
Storage Spaces (Windows Server 2012 R2) Scalable , Resilient and Efficient SDS solution Dell and Microsoft collaborate , innovate and optimize for real
worked deployments For today’s Storage Spaces needs deploy Dell DSMS solutions Visit Dell’s Booth on Expo Floor
Storage Spaces Direct (Windows Server vNext) Join TAP Visit Microsoft Cloud Platform Dell Booth in Expo Floor Visit www.delltechcenter.com for updated information
Leverage the power of Dell and Microsoft alliance
Summary
Q & A
Visit Myignite at http://myignite.microsoft.com or download and use the Ignite Mobile App with the QR code above.
Please evaluate this sessionYour feedback is important to us!
© 2015 Microsoft Corporation. All rights reserved. © 2015 Microsoft Corporation. All rights reserved.