storage is not virtualized enough - part 1
TRANSCRIPT
Storage is not virtualized enough
Zhipeng (Howard) HuangHuawei
Bio Standard Engineer
• ETSI NFV• SNIA Object Drive TWG• OCI/Spec
Open Source Community Operation Manager• OpenStack (Tricircle, Kingbird, Dragonflow, …)• OPNFV (Parser, Multisite, DPACC, …)• OpenVswitch
Like Heavy Metal Music and father of twin girls !
Why storage is not virtualized enough
Converged Data Plane( Support File, Block, Object)
NoSQL1( Graph)NoSQL2( kv) OpenSwift Lustre …
app1
App admin
Interface block file object Kv db Graph db Mpp sql …
Performance Ops xxx Ops xxx Ops xxx Ops xxx …
Capacity 10PB 100PB 1000PB …
Reliability 5个 9 6个 9 7个 9 8个 9 …
Storage admin
Cost $1/GB $0.1/GB Free Service
Open Source …
Functionality Data Replica
Clone Backup Dedup …
Maintainability …
app2 app3 app4 app5 app…
What is Storage Function Virtualization (SFV)
SFVNFV
OS/Hypervisor
VM1
VNF1
VMn
VNFn…
OS/Hypervisor
VMx
VNFx
VMy
VNFy…
…
Cloud OS (NFV Enhanced)OS/Hypervisor
VM1
VSF1
VMn
VSFn…
OS/Hypervisor
VMx
VSFx
VMy
VSFy…
…
Cloud OS (SFV Enhanced)
FW DPI Router LB Blk/EBS MPP DB NAS Hadoop
OpenStack SFV
OpenStack Components
Third Party Storage Management Software
Third Party virtual storage apps
OpenStack SFV Enhancements
Cinder ManilaNova Glance
Heat
Murano
Virtualized Storage Funtions
OpenStack SFV
AMQP
REST
REST
REST
DriverDriver
Driver
REST
REST
REST
REST
App-Image
App-Template
DPS
REST
REST
Backup Driver
Storage Orchestration Tool
OpenStack SFV Sample Workflow
Pool’s volume、medium、 I
OPS
Nova Cinder-API
Cinder-scheduler
Cinder-volume( LVM driver)Cinder-volume( HSDP driver)
Pool/VG Pool/VG Pool/VG Pool Pool
HDD HDD HDD
LVM
SSD NVDIMM
HSDP
Create Instance( bdm)
1
2 1. filter for localized compute and storage
2. filter for volume anti-affinity rule
3. filter for capability scheduling
3 4
Pool’s volume、medium、 IO
PS
NOVAscheduler
2
OpenStack SFV related BPs Improve nova attach volume api to support disk-bus : The purpose of this spec is to add disk-bus parameter to nova attach volume api. https://review.openstack.org/#/c/185973/
Add local-to-instance to create server api : The purpose of this spec is to add "local_to_instance" to block_device_mapping parameter in nova create instance API. https://review.openstack.org/#/c/190561/
Add volume type to create server API : Currently in create server API, using block_device_mapping parameter can create cinder volume and attach to instance, but there aren't volume type parameter to create a SLA based volume, so this bp will add the volume type to create api. https://review.openstack.org/#/c/187812/
Add NVME bus to KVM hypervisor https://blueprints.launchpad.net/nova/+spec/add-nvme-bus-to-kvm-hypervisor
vHost Supporthttps://blueprints.launchpad.net/cinder/+spec/vhost-support
Notes:Some of the BPs would be resubmitted for Release MCollaboration on SFV related works are highly welcomed.
SFV Goes Beyond OpenStack
OCI
CNCF OpenStack
OCP
KVM
AddMore
…
AddMore
…
ContainerFlockerApp 1
ContainerHyperApp 2
OSv-ScyllaDB
App N
AddMore
…
AddMore
…
Kinetic
• OpenStack-dev mailinglist with [SFV] in the subject • Join and Discuz at https://groups.google.com/forum/#!forum/sfv-dev • Also hack ideas for development in various upstream projects at https://github.com/OpenSFV• More trendy communication at https://sfvdev.slack.com/