application acceleration from the data storage perspective
DESCRIPTION
TRANSCRIPT
© Copyright 2009-2010, Cambridge Computer Services, Inc. – All Rights Reserved
www.CambridgeComputer.com – 781-250-3000
Application Acceleration From the Data Storage
Perspective
New Advances in Caching and Solid State Storage
Lecturer:
Jacob Farmer, CTO
Cambridge Computer
2www.CambridgeComputer.comApplication Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
The Pain: Random Disk I/O Performance is not Easily Scalable
Random Disk I/O performance is not keeping pace with other
advancements such as processor and network I/O
• 15K drives are not that much faster than 10K!
• Traditional RAID arrays make it difficult to aggregate disk spindles
without host-based LVM software
Applications running in virtual servers are particularly
constrained by random I/O
• The virtualization environment randomizes traditionally sequential
workloads
• Booting and page file activity tend to share the same disk pools as
applications and thus compete with apps for disk I/O
Random v. Sequential I/O
3www.CambridgeComputer.comApplication Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
The Relative Performance of Spinning Disk to CPU, Cache and RAM
1 lb
10 lb
100 lb
10,000,000 lb
CPU L2 Cache RAM Spinning Disk
4www.CambridgeComputer.com
What’s Changing?
The price of DRAM has declined to the point where a
meaningful capacity is affordable
• Traditional DRAM SSDs were very pricey, niche technologies
– 5 years ago – 20GB of SSD sold for about $500K
The emergence of enterprise-class flash technology
• Recent innovations allow flash-based SSD to achieve meaningful
performance with enterprise-class reliability.
• Flash offers new tiers in between DRAM SSD and 15K disk
We are nearing the tipping point where SSD goes viral.
• Higher demand accelerates the decline in price
• Commoditization inspires new software technologies that leverage
commodity hardware
Application Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
5www.CambridgeComputer.com
SSD Media Choices
• Equal performance for read and write
• No concern for duty cycle
• Requires uninterruptable power sourceDRAM
• Faster write speeds than MLC
• Lower power consumption than MLC
• Higher endurance than MLC
• Higher cost than MLC, but still cheaper than DRAMS
SLC FlashSingle-Level Cell
• Multiple bits occupy a flash memory cell, reducing costs
• Higher bit-error rate than SLC
• Lesser performance SLC
• Lesser endurance than SLC
• Software compensates for shortcomings in the media
MLC FlashMulti-Level Cell
Application Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
6www.CambridgeComputer.com
Flash Memory Durability and Reliability
Flash memory does wears out over time
• MLC has a lower duty cycle than SLC
“Wear leveling”
• Firmware in the flash device keeps track of I/O requests to
each logical block and distributes I/O across flash cells
• As the device ages the wear leveling mechanism might
reduce performance.
– Especially true of devices designed for desktop and laptop usage
Confirm duty cycles with product manufacturer
• Most enterprise flash devices will outlive traditional hard
drives under specified use-cases
Application Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
7www.CambridgeComputer.comApplication Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
SSD Packaging: SSDs Come in Many Shapes and Sizes
Hard Drive Form-Factor with SATA, SAS, or FC interface
• Can be installed in server drive slots
• Can be installed in enterprise SAN enclosures
– Add a new high performance tier (“Tier Zero”) to your SAN array
PCIe Cards and PCIe External Appliances
• Looks like a hard drive to the local OS
• Lower latency than devices that use storage channels (FC, SATA, SAS)
• Suitable for applications that can use direct-attached storage
– PCIe switches are slowly being adopted by enterprise applications
SAN-Attached Appliances
• Great for traditional enterprise apps, especially when high availability clustering is involved
– Failover Clusters, Server Virtualization Platforms, Oracle RAC
DIMM Modules Installed on the Motherboard on the RAM Bus
• Extremely low latency
w w w . C a m b r I d g e C o m p u t e r . c o m 8
How do you size your RAID groups or disk arrays?
By Capacity?
By Performance?
By Guess Work?
9www.CambridgeComputer.comApplication Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
Hard Drives = Units of Performance
Each hard drive (spindle) is like a little worker capable of
doing only so much work in a given day.
• Work is measured in IOPS – input/outputs per second.
• Faster hard drives deliver more IOPS than slower hard drives.
• Note: there is a diminishing return with increasing rotational speed.
Drive Type Typical Real-World IOPS
10www.CambridgeComputer.comApplication Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
SSD Performance Comparison Approximate – 4 KB Random I/O
Drive Type Typical Real-World IOPS
11www.CambridgeComputer.com
Cost Per Gigabyte versusCost Per Disk I/O
Application Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
Drive Type Cost per GB
Drive Type Cost per IOP
Notes: 320GB SSD PCIe Card 128GB SSD Hard Drive 450GB 15k Hard Disk
IOPS Assumptions 100,000 20,000 180
Costs $7,500 $10,000 $1,000
w w w . C a m b r I d g e C o m p u t e r . c o m 12
Taking Advantage of Solid State Storage
13www.CambridgeComputer.comApplication Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
Applications for SSD
Any application or data set where the demand for random I/O
performance is high relative to the demand for capacity
• Databases: logs, temps, indexes, whole tables
– MS Exchange: logs and data
• Operating systems: boot images, page files
– Next Generation Virtual Desktop Solutions (VDI)
– Multi-host Platforms: VMware, Citrix
• Out-of-band file systems for metadata catalog
– Any file system where metadata can be isolated from file data
Modern advances enable cache and SSD for general purpose
computing
• Cache and SSD can be inserted anywhere along the storage I/O path
– Dynamic tiered storage, life cycle management, caching
14www.CambridgeComputer.com
How Do You Take Advantage of These New Storage Tiers?
These new SSD tiers offer performance and cost
characteristics that are wildly different from
traditional storage.
New tiers drive the demand for automated tiered
storage.
Automated Tiered StorageThe ability to scale performance and
capacity independently of each other
15www.CambridgeComputer.com
The Storage I/O Path – Plenty of Places to Insert Acceleration Logic
Application Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
Out-of-band File System
Host-based storagemanagement software
Server Virtualization
SAN-based VolumeVirtualization Appliance
Disk Array Controller
Out-of-band Volume Level
Switch-based Virtualization
LAN-BasedAppliance
Ap plicatio n Ope ra ting System File System Volum e Ma nag er
Fabric
Hypervisor
16www.CambridgeComputer.com
Inserting Cache and SSD into the Storage I/O Path
Application Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
LAN/WAN
w w w . C a m b r I d g e C o m p u t e r . c o m 17
Solid State Storage with MS Exchange
18www.CambridgeComputer.com
SSD in an Application: MS Exchange
Sizing disk I/O for MS Exchange
• ¼ to ½ I/O per active user
• Multiply by 4 for Blackberry/iPhone (any ActiveSync)
What do put on SSD
• The entire data set if it will fit
• EDBs holding mail for select power users
• Log files
– Big win for write-intensive mail systems
Exchange 2007 and 2010 offers HA clustering without the need for a shared storage SAN
• CCR – Continuous Cluster Replication
• DAG – Database Availability Groups
• Leverage the low-cost of direct-attached SSDs
Application Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
19www.CambridgeComputer.com
MS Exchange : Clustering with Direct-Attached SSDs
Replication and HA clustering without the need for shared SAN storage • CCR – Cluster Continuous Replication (2007)
• DAG – Database Availability Groups (2010)
Application Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
SSD SSD
LAN
Exchange CCR Cluster
20www.CambridgeComputer.com
MS Exchange: Direct-Attached SSDs for Log Files Only
Exchange logs are stored on
internal SSD
Exchange databases are
stored on traditional
enterprise SAN
• Snapshots
• Replication
• Provisioning
Application Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
SAN
SSD
w w w . C a m b r I d g e C o m p u t e r . c o m 21
Solid State Storage for Databases
22www.CambridgeComputer.com
SSD for Databases
For small databases, put the entire database on SSD
Typical best practice is to determine which elements
of the database will benefit from SSD
• Usual suspects: Logs, Indexes, Temps, select tables
• Performance monitoring tools are available for all major
databases
– Example: Oracle StatPack
Use volume manager, intelligent SAN, or database
tools to migrate hotspots to SSD
• Oracle ASM allows seamless migration from one storage
device to another.
Application Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
23www.CambridgeComputer.com
Adding SSD Device for Database Hotspots
Application Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
SAN Fabric
RAM-Based
SSD
Appliance
Flash-Based
Array
Disk Array with
Big Cache or
Flash SSD Drives
RAM
Database Cluster
24www.CambridgeComputer.com
Database QoS with Operating System Resource Partitioning
Quality of service can be enforced by manipulating
prioritization of operating system resources
Specific queries or patterns of queries are given
priorities or assigned performance threshold
• Prioritization policies can be set for disk I/O as well as CPU
utilization
Renegade queries are throttled
• As opposed to running wild or being killed
Allows you to size hardware around normal
conditions rather than worst scenario
Application Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
w w w . C a m b r I d g e C o m p u t e r . c o m 25
Hyper-V, vSphere and XenServer
Better Performance for Virtual
Environments
26www.CambridgeComputer.comApplication Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
SSD with Clustered DAS Logical Volume Manager and VMWare
Add a redundant tier of
SSD to your VM
environment by installing
flash SSD cards into VM
hosts.
Server Virtualization Hosts with Internal SSD
SANSAN
“Clustered DAS” Software
Conventional
SAN Storage
SSD SSD
w w w . C a m b r I d g e C o m p u t e r . c o m 27
File Systems and NAS Acceleration
28www.CambridgeComputer.com
File System Acceleration
If metadata can be isolated from data, put the metadata on
SSD drives
• Especially useful for SAN-enabled file systems that are heavily
constrained by metadata
Some file systems (notably ZFS) can cache recently accessed
files in SSD drives.
• This is especially useful when creating very large ZFS stripe groups
Some file systems allow the journal to be stored separately
Maybe your whole file system will fit on SSD drives!
• Use SSDs with hard drive form factor in place of a hard drive.
A caching appliance in front of your NAS
• Any place where NFS/CIFS improvement is needed
Application Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
29www.CambridgeComputer.com
Clustered Scale-Out NAS: Living the Dream!!!
Redundant SAN Fabric
RAM-Based
SSD
for Metadata
Tier 0
MLC Flash
Tier 1
15K Disk
Tier 2
SATA Disk
Clustered NAS with Distributed Coherent Cache
RAM RAM RAM RAM
30www.CambridgeComputer.comApplication Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
A Clustered File System on IB with Internal SSD Drives
Infiniband Fabric
Application Servers and/or Compute Nodes
Servers with Flash SSD cards
and lots of RAM
31www.CambridgeComputer.com
Clustered In-Line NAS Cache with SSD and Tiered Storage
Application Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
Local Area Network (1Gb, 10Gb, Infiniband, etc.)
File Server (NAS)
Clustered Caching Appliance with SSD and Tiered Storage
Use all SATA in your NAS
Retain snapshots, replication, backup, etc.
w w w . C a m b r I d g e C o m p u t e r . c o m 32
SAN Acceleration
33www.CambridgeComputer.com
Hot-spotting with SSD or DRAM Using SAN Volume Virtualization
Application Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
NOTE: Virtualization layer is not drawn as redundant.
SAN
DISK CONTROLLERS
VIRTUALIZATION DEVICE
Internal SSD
External SSD
34www.CambridgeComputer.com
Transparent In-band SAN Cache
Application Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
SAN
35www.CambridgeComputer.com
SAN Topology View of Inline Cache
Application Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
SAN
36www.CambridgeComputer.comApplication Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
SSD in Dynamically Tiered Array
SSD can be leveraged across the board with dynamic block-level tiering.
Hottest blocks or fresh new writes can stored on SSD and migrate to lower tiers when not in active use.
Makes it easy to deploy SSD without having yet one more tier to worry about.
• Secondary goal is to squeeze the
most I/O out of your disk pool.
37www.CambridgeComputer.comApplication Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.
SSD with Deduplication
Deduplication gets its greatest value when
• There is a lot of duplicate data
• The media is relatively expensive
New technologies allow block-level dedupe for
primary storage
Killer applications:
• Boot appliance for booting servers over the SAN
• Virtual desktop storage
• Source code libraries
• Oracle, Exchange
38www.CambridgeComputer.com
Conclusions
Solid state storage is the big game changer in the storage industry right now
• The potential for increased random disk I/O performance changes traditional assumptions about application performance
• The wide range of SSDs create more tiers and inspire demand for automating tiered storage
New advances allow SSD to be inserted into the existing storage infrastructure
• The challenge is to figure out the best way to insert SSD for any given application
The end-goal is to purchase capacity and performance independently of each other
• Grow capacity with larger, less expensive drives and grow performance with various types of SSDs
Application Acceleration: New Advances in Caching and Solid State Storage
© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.