christian.bandulet@oracle - doag.org · christian.bandulet@oracle.com the following is intended to...
Post on 17-Sep-2018
218 Views
Preview:
TRANSCRIPT
christian.bandulet@oracle.com
christian.bandulet@oracle.com
<Insert Picture Here>
Evolution und Trends der Storage TechnologienChristian BanduletPrincipal Engineer
christian.bandulet@oracle.com
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
christian.bandulet@oracle.com
<Insert Picture Here>
Agenda
•Data•Storage Systems•Data Preservation•Storage Grids•Storage Networks•Storage Media
christian.bandulet@oracle.com
<Insert Picture Here>
Evolution of Data
christian.bandulet@oracle.com
IT Challenges Have Not SlowedIncreasing Demand for Storage Capacity and Performance
2006 New Digital Data(Created, Captured, Replicated)
2010 New Digital Data(¼ Created, ¾ Replicated)
6X Growthemerging economies
growing 30-40% faster than mature economies
161 Exabytes
988 Exabytes
70% created by Individuals
Enterprises responsible for85% of this new data
(Security, privacy, reliability, compliance)
95% U
nstructured
Source: IDC
christian.bandulet@oracle.com
Challenge: Storage Needs Are GrowingGreater Demand For Storage Capacity And Performance
2009 New Digital Data 2020 New Digital Data
44X Growth
800 Exabytes35,000 Exabytes
Source: IDC
christian.bandulet@oracle.com
Storage Market Segmentation
Media production,eCAD, mCAD, Office docs
Media-archive, DAM, Broadcast, medical imaging, Media-Internet
Transactional systems, ERP, CRM BI, data warehousing, scientific, transaction archive
Fixed DataDynamic Data
Stru
ctur
edU
nstru
ctur
ed
Time
christian.bandulet@oracle.com
Storage Market Segmentation
Media production,eCAD, mCAD, Office docs
Media-archive, DAM, Broadcast, medical imaging, Media-Internet
Transactional systems, ERP, CRM BI, data warehousing, scientific, transaction archive
Fixed DataDynamic Data
Stru
ctur
edU
nstru
ctur
ed
SemiStructurede.g. email
Time
christian.bandulet@oracle.com
Data
Metadata Owner, permissions, type, last mod. date, ...
Data For Clouds – File Objects
christian.bandulet@oracle.com
Data vs. Information
Data + Semantics = Information
+
christian.bandulet@oracle.com
Object ID
Data
Metadata Owner, permissions, type, last mod. date, ...
Attributes user/application defined
policies e.g. replication
methods e.g.encryption
Data For Clouds – File Objects
christian.bandulet@oracle.com
Object Attribute 1
Object Model
Object Attribute 2
Object Attribute 3
Object Attribute 4
Object Attribute n
ObjectDescriptor
OID
many:1 1:many
ObjectSub-Component 1
ObjectSub-Component 2
ObjectSub-Component 3
ObjectSub-Component 4
ObjectSub-Component n
christian.bandulet@oracle.com
Object ID
Data
Metadata Owner, permissions, type, last mod. date, ...
Attributes user/application defined
Policies e.g. replication
Methods e.g.encryption
Data For Clouds – File Objects
Store Retrieve
ObjectObject
Data OID OID Data
christian.bandulet@oracle.com
Object ID
Data
Metadata Owner, permissions, type, last mod. date, ...
Attributes user/application defined
Policies e.g. replication
Methods e.g.encryption
Store Retrieve
Data Blocks
Inode
name OID Object
name OID Object
name OID Object
name OID Object
name OID Object
Data For Clouds – File Objects
ObjectObject
Data OID OID Data
christian.bandulet@oracle.com
<Insert Picture Here>
Evolution of Storage Systems
christian.bandulet@oracle.com
Traditional Storage Architecture
SAN
IP Clients
ServerServer CPU& Memory Complex
SCSI Port to OSiSCSITCP/IP
Adapter Driver
HW RAIDController
HW RAIDController
christian.bandulet@oracle.com
Server
Storage
The Vertical Consolidation
Storage
Server
StorageStorage
ServerServer
Storver
christian.bandulet@oracle.com
Programmable Storage
Data OID
Store
Object
Input Procedure 1
Input Procedure 2
Input Procedure x
...
Asyn
chro
nous
Pro
cedu
re C
Asyn
chro
nous
Pro
cedu
re D
Asyn
chro
nous
Pro
cedu
re A
Asyn
chro
nous
Pro
cedu
re B
OID Data
Object
Output Procedure x
Output Procedure 2
Output Procedure 1
...As
ynch
rono
us P
roce
dure
C
Asyn
chro
nous
Pro
cedu
re D
Asyn
chro
nous
Pro
cedu
re A
Asyn
chro
nous
Pro
cedu
re B
• User-defined/developed storage based apps– Synchronous:
• modify the behavior of store/retrieve/query/delete
• e.g. transcode, downsample, filter, watermark, extract metadata from file, headers, encrypt, audit log...
– Asynchronous:• process data at rest• e.g. capacity optimization, de-
dup, scrubbing, migration, sanity check...
Retrieve
christian.bandulet@oracle.com
Application-Awareness
Server
Application Context
Application
Storage Device
• Application Context:– I/O patterns, data layout, intrusion detection, data
relationships, consumer information, spatial, etc. – e.g. which blocks are stale/empty or belong to a file– e.g. relationships among data, users and applications
Data
christian.bandulet@oracle.com
Storage Device
Application Context
Application-Aware Storage
Server
Application Context
Application
• Data layout, caching/pre-fetching strategies, B&R, ILM, DR, security, power mgmt, device-resident de-mapping functions, transport optimization, resource management
• Appropriate protocol required to exchange context information
• device-resident de-mapping / devirtualization (virtualization interferes with performance tuning and problem diagnosis
Data
christian.bandulet@oracle.com
Active Storage – Distributed Data ManagementApplication Decompositioning
Storage Device
Application Context
Server
Application Context
Application
• Runs application-specific code– e.g. indexing, search, caching
• Enables adaptive-computing
Application
Data
christian.bandulet@oracle.com
The Intelligent Storage Node
Presentation Layer (Storage Personality)Block I/F File I/F Object I/F
SCSI, FCP, iSCSI, SAS, SRP, SATA, VTL ...
FTP, NFS, CIFS, WebDAV, HTTP...
API, ANSI T10 OSD, CAS, XAM...
Storage Application Layer ( Data Services)Data Management Compliance Services Scheduled Services
Backup, Classification,Data Mining, Application
Acceleration, Aggregation, Virtualization,
Sort, Views, Search,Indexing, Data Movers,
Distributed Namespace...
WORM, Audit, Retention, Secure Deletion...
De-Duplication,Hash Generation,
Verification,Snapshot, Replication...
Operating SystemFiles systems, HSM, Device Drivers, Clustering, Security....
Persistent StorageDisk, Tap, Library...
External ServicesLDAP
NIS
Active Directory
NTP
MDSGlobal Namespace
others
christian.bandulet@oracle.com
<Insert Picture Here>
Evolution of Data Preservation
christian.bandulet@oracle.com
Information Lifecycle Management
Publish
Create
Index
Store
Retain
Cleanse
Search
Distribute
VersionCapture
Manage
Secure
Destruction
Archive
Capture data from any entry pointWeb-based submissionIngest directly from applicationsScan, Fax directly to repository
christian.bandulet@oracle.com
Green...
SATA, MAID...
christian.bandulet@oracle.com
Greener...
christian.bandulet@oracle.com
1956 first spinning hard drive (IBM RAMAC)
1956: 5MB – 2000bits/in2
2010: 2000GB ~ 400Gb/in2
200,000,000x areal density
400,000x capacitytimes improvement in 50+ years!
Disk Evolution
christian.bandulet@oracle.com
Even More Green...
Data De-DuplicationData ClassificationData Compression
christian.bandulet@oracle.com
Greenest...
Data Deletion
christian.bandulet@oracle.com
<Insert Picture Here>
Evolution of Storage Grids
christian.bandulet@oracle.com
Storage EvolutionFrom Silos to Storage Networks and Shared Storage
Application ServerApplication Server Application Server Application Server Application Server Application Server Application Server
Storage Network
christian.bandulet@oracle.com
Storage EvolutionFrom Physical Storage to Virtualized Storage and Storage Pooling
Application Server Application Server Application Server Application Server
Storage Pool
Shared FS Shared FS Shared FS
Application Server Application Server Application Server
Storage Network
Device SharingShared Storage
Device
Shared FS Shared FS Shared FS
LUN LUN LUN LUN
VLUN VLUN VLUN
Storage Network
christian.bandulet@oracle.com
Storage EvolutionHSM: from Single-Tier Storage Pools to Multi-Tier Storage Pools
Application Server Application Server Application Server Application Server
Tier 1
Storage Pool
Tier 2 Tier 3
VLUN VLUN VLUN
Application Server Application Server Application Server
Storage Pool
VLUN VLUN VLUN
TAPE
Storage-Based:Dynamic policy-based migration & Stubbing
Storage Network Storage Network
Host/Network-Based:Dynamic policy-based migration & Stubbing
LUN LUN LUN LUN LUN LUN
christian.bandulet@oracle.com
Storage EvolutionFrom HSM to ILM
Tier 1
Storage Pool
Tier 2 Tier 3
LUN LUN LUN
VLUN VLUN VLUN
TAPE
Storage-BasedDynamic policy-based migration
+ backup/recovery+ regulatory compliance (WORM, retention)+ archiving (classification, indexing, search)
Tier 1
Storage Pool
Tier 2 Tier 3
LUN LUN LUN
VLUN VLUN VLUN
TAPE
Application Server Application Server Application Server Application ServerApplication Server Application Server Application Server
FCP/iSCSI Storage Network FCP/iSCSI Storage Network
Host/Network-Based:Dynamic policy-based migration & Stubbing
Storage-Based:Dynamic policy-based migration & Stubbing
Host/Network-BasedDynamic policy-based migration
+ backup/recovery+ regulatory compliance (WORM, retention)+ archiving (classification, indexing, search)
christian.bandulet@oracle.com
Storage EvolutionFrom Block Storage to Object Archive Storage (CAS)
Tier 1
CAS
Tier 2 Tier 3
LUN LUN LUN TAPE
Dynamic policy-based migration+ backup/recovery
+ regulatory,compliance (WORM, retention)+ archiving (classification, indexing, search)
Application Server Application Server Application Server Application Server
Tier 1
Storage Pool
Tier 2 Tier 3
LUN LUN LUN
VLUN VLUN VLUN
TAPE
Dynamic policy-based migration+ backup/recovery
+ regulatory,compliance (WORM, retention)+ archiving (classification, indexing, search)
Application Server Application Server Application Server
XAM WebDAV Object ... FTP
Object Object Object Object Object Object
FCP/iSCSI Storage Network FCP/iSCSI/NFS/CIFS
Semantics/Attributes
christian.bandulet@oracle.com
Storage EvolutionFrom RAID to RAIN (homogeneous)
Application ServerApplication Server Application Server Application Server
CAS
CAS
CAS
CAS
CAS
CAS
CAS
CAS
CAS
CAS
CAS
CAS
CAS
CAS
CAS
CAS
CAS
CAS
RAIN
Application ServerApplication Server Application Server Application Server
FCP/iSCSI/NFS/CIFS Storage Network
christian.bandulet@oracle.com
Storage Grid & Data AvailabilityData Placement
Storage Node Storage Node Storage Node Storage Node
Storage Node Storage Node Storage Node Storage Node
Storage Node Storage Node Storage Node Storage Node
Storage Node Storage Node Storage Node Storage Node
Object 1
= Data
= Parity
Object 2
Object 3
Storage Grid
christian.bandulet@oracle.com
Resource Lifecycle Management
Storage Grid
Svc1 Svc1 Svc1Svc1 Svc2Svc2 Spare
Storage Grid - expand
Svc1 Svc1 Svc1Svc1
SpareNew
Storage Grid - repurposed/new services
Svc1 Svc1Svc1
Old
Svc3 SpareNew
New
Minimal application impactExpand a Storage Grid
Install an elementAuto-discover, bindProvisionRedistribute workload
Decommision elementsDrain dataDe-presentUnbindRetire and de-install
Also applies to replacements/upgrade
christian.bandulet@oracle.com
Host Host Host Host Host Host Host Host
IB FC IP
I/O RoutingI/O Redirection
Single Storage ImageISD ISD
ISDISD
RemoteIB
Global Namespace
christian.bandulet@oracle.com
NIST Definition of Cloud Computing
christian.bandulet@oracle.com
<Insert Picture Here>
Evolution of Storage Networks
christian.bandulet@oracle.com
Today’s Infrastructure
FC-SAN Network
Ethernet Management
Ethernet Data Network
StorageServers
christian.bandulet@oracle.com
Ethernet 1, 10 Gbps
IPTCP
iSCSI
InfiniBand
SRP
IPTCPFCIP
FCP*
IPTCPiFCP
FCP*
Fibrechannel
FCP*
1, 2, 4, (8), 10 Gbps 10, 20 Gbps
iSCSI FCoE InfiniBand
iSER
iSCSI
Data Center Ethernet
FCoE
FCP*
SCSI Layer
Operating System / Applications
(1), 10 Gbps
* Includes FC Layer
Encapsulation Technologies
Source: Cisco
christian.bandulet@oracle.com
<Insert Picture Here>
Evolution of Storage Media
christian.bandulet@oracle.com
1952 first commercial tape (IBM 726)
1952: 2MB - 1400bits/in2
2010: 1500GB* - 1.21Gb/in2
865,000x areal density
750,000x capacitytimes improvement in 50+ years!
Tape Evolution
*LTO-4 0.8TB, ~ 0.8 Gb/in2
*LTO-5 1.5TB, ~ 1.21 Gb/in2
christian.bandulet@oracle.com
• Moore’s Law has out-stripped improvements to disk drive technology by two orders of magnitude during the last decade
• Today’s servers are hopelessly unbalanced between the CPUs need for data and the HDDs ability to keep up
HDD vs. CPU Relative Performance Improvement
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
Server and Storage Disconnect
christian.bandulet@oracle.com
Where to Deploy Flash Storage?External Controller
Application
OS
Memory
IO
HBA/NIC/HCA/CNA
PCIe CTR
L
StorageNetwork
DR
AMD
RAM
DR
AMD
RAM
DRAM
Fmod
SSDFmod
SSD
Fmod DRAM
Fmod
christian.bandulet@oracle.com
Where to Deploy Flash Storage?External Controller
christian.bandulet@oracle.com
Where to Deploy Flash Storage?PCI Controller
HBA/NIC/HCA/CNA
Fmod
Fmod
Fmod
Fmod
Fmod
Fmod
Fmod
Fmod
Application
OS
Memory
IO
PCIe CTR
L
StorageNetwork
DR
AMD
RAM
DR
AMD
RAM
DRAM
Fmod
SSDFmod
SSD
Fmod DRAM
Fmod
Fmod
christian.bandulet@oracle.com
Where to Deploy Flash Storage?PCI Controller
christian.bandulet@oracle.com
Where to Deploy Flash Storage?Memory Controller
CTR
L FmodFmodFmodFmod
DDR2/3
Application
OS
Memory
IO
HBA/NIC/HCA/CNA
PCIe CTR
L
StorageNetwork
DR
AMD
RAM
DR
AMD
RAM
DRAM
Fmod
SSDFmod
SSD
Fmod DRAM
Fmod
Fmod
christian.bandulet@oracle.com
Where to Deploy Flash Storage?Application-Managed Hybrid/Secondary Cache
HBA/NIC/HCA/CNA
Fmod
Fmod
Fmod
Fmod
Fmod
Fmod
Fmod
Fmod
Application
OS
Memory
IO
PCIe CTR
L
StorageNetwork
DR
AMD
RAM
DR
AMD
RAM
DRAM
Fmod
SSDFmod
SSD
Fmod DRAM
Fmod
Fmod
christian.bandulet@oracle.com
Emerging SCM Technologies
christian.bandulet@oracle.com
True Dynamic Tiering
What ?: LUN, Volume, File, Page, Block
When ?: Weeks, Day, Hours, On-The-Fly
How ?: Manual Disruptive, Manual Non- Disruptive, Policy-Based, Automatic
christian.bandulet@oracle.com
Sub-Object/File/LUN Tiering
File Page TablePhysical Address Virtual Address
X'ff1234ea X'a74efa11Tier 1
Storage Pool
Tier 2
Tier 3
christian.bandulet@oracle.com
Tiered Storage Reduces IT CostsData center best practices demand tiered storage efficiencies
Single Tier of Disk
Storage
ModernMulti-tiered
Storage
DiskMulti-tiered
Storage
100%
38%
62%
32%
6%
60%
Source: Horizon Information Strategies, 2009
Flash
PerformanceDisk
CapacityDisk
TapeDrive
<Insert Picture Here>
Evolution und Trends der Storage TechnologienChristian BanduletPrincipal Engineer
christian.bandulet@oracle.com
christian.bandulet@oracle.com
top related