manila, an update from liberty, openstack summit - tokyo
TRANSCRIPT
Manila An Update from Liberty
October 29, 2015 OpenStack Summit - Tokyo
Akshai Parthasarathy
Technical Marketing Engineer, NetApp
@akshai
Sean Cohen
Principal Product Manager, Red Hat
@SeanCohen_RH
Thomas Bechtold
OpenStack Cloud Engineer, SUSE
Today's Presenters
Agenda
• What is Manila? • Why use Manila?
• Use Cases • Sahara • Containers
• Liberty Updates • Distributions Integration
• SUSE Demo • RedHat Demo
• Upcoming in Mitaka • Share Replication Demo • Q+A
Manila: The OpenStack Shared File Service Program Bringing self-service, shared file services
to the cloud
Manila History
§ Beginnings § Juno
§ Incubated Project § Community Inception § Puppet Support § Share Servers § Tempest Integration
§ Kilo § Driver Modes § DevStack Plug-in § Storage Pools § Default Share Type § Manage/Unmanage § Manila UI
Manila Today -
§ Production Ready § Number of Drivers: 14 § Blue Prints Completed: 51 § Major Blue Prints:
§ Share Instances § REST API Microversions § Experimental APIs § Extend & Shrink § Consistency Groups (CGs) § Share Migrations § etc.
Manila Deployment Options and Benefits
§ Single Storage Virtual Machine (SVM) / Multi SVM § driver_handles_share_servers=False/True § Benefit
§ Network Plugins
§ Standalone Network Plugin § Nova Network Plugin § Neutron Network Plugin § Benefit
Contributions/New Entrants
Welcome and Thank You: ● CloudBase Solutions ● Fujitsu ● Scality ● NEC ● NTT ● Letv Cloud Computing ● and others
Source: http://stackalytics.com/?module=manila&metric=loc&release=liberty
Code Contributions Reviews
Why Use Manila?
§ Shared File Services § Spending § Total Capacity
§ IDC: Age of Cloud File Services § Why not?
Use Cases
Standalone Share File Services Management
§ Replace Legacy Systems § Improve SLA § Integrate with Automation Frameworks
Enterprise Applications
§ Lower Cost, Heterogeneous Infrastructure § Application Porting: Move Workloads to OpenStack
DevOps/Continuous Integration
§ Pluggable Infrastructure § Storage Service Catalog
§ Analytics § Containers § Snapshots & Clones § Consistency Groups
New in Liberty
Share Instances
Needed for features like:
● share migrations
● replication
REST API Microversions
http://docs.openstack.org/developer/manila/devref/api_microversion_dev.html http://docs.openstack.org/developer/manila/devref/api_microversion_history.html
Example:
● client 1 with server 1 => version 2.3
● client 1 with server 2 => version 2.3
● client 2 with server 1 => version 2.5
● client 2 with server 2 => version 2.10
Experimental APIs
§ Expected to change at any time § can be removed without
deprecation period § Usage: needs to set header “X-
OpenStack-Manila-API-Experimental: true”
http://docs.openstack.org/developer/manila/devref/experimental_apis.html http://developer.openstack.org/api-ref-share-v2.html
@api_version(min_version='2.1', max_version='2.9') def show(self, req, id): .... stuff ....
@api_version(min_version="2.4", experimental=True) def my_api_method(self, req, id): .... stuff ....
Extend & Shrink
http://docs.openstack.org/developer/manila/devref/share_back_ends_feature_support_mapping.html
Consistency Groups (CGs)
§ grouping different shares together for the purpose of application data protection (focus of snapshots for disaster recovery)
§ example use case: database data and log files are on different shares § when doing snapshots for both shares, the data on the shares must be consistent
§ supported actions § create CG with mutliple shares § create snapshot of CG § create CG from snapshot
External CI For All Drivers
And others ...
Oversubscription
§ Tunable for setting provisioned capacity and a subscription ratio added
§ Addresses 'infinite' and 'unknown' drivers reported capacity that may lead to oversubscription
§ thin_provisoning support needed
Share Migrations
§ Share Migration allows a share to be migrated from one host pool to another hostpool through the "manila migrate <share> <host#pool>" command, as well as allowing to perform migration between different backends.
§ Basic implementation § The fallback approach for migration is rsync.
Slow, inefficient § Vendors can utilize API for optimized
migration
Availability Zones
§ The availability zones support which was inherited from Cinder was reworked this cycle:
§ Added public API extension § Allow to preserve AZ if creating a share
from a snapshot and set AZ in Share API or Share Manager.
§ AZs will benefit share replication and also give end users control of the locality of their data w.r.t. consumers of the data.
Sahara Integration
§ Use cases § Stores binaries for job templates - NFS is ideal for this case § Input and output data sources - Manila-provisioned HDFS and NFS offers more
options § Mount NFS share API
§ Binaries and data I/O from an NFS share path § New development in Sahara this cycle § Mount shares at cluster creation or auto-mount when a share is used for EDP
§ Manila-provisioned HDFS § Data sources and data processing on Sahara-external clusters § Testing and process verification of extant Manila features this cycle
§ Coming soon - NFS Hadoop driver (run jobs on your NFS shares)
Sahara Integration - Current Implementations
● Data sources on Manila-provisioned HDFS
● API to mount NFS shares to clusters (job binaries and data sources)
Images by Weiting Chen (Intel)
Manila + Containers Ceph Example § Simply mount --bind share into § container namespace § NFS re-export from host
§ mount and export fs on host § private host/guest net § avoid network hop from NFS
service VM § Host mounts CephFS § Bind Manila share/volume into
container § Further integration requires
integration with both Nova and Manila to manage the attach/detach process
HOST
M M
RADOS CLUSTER
CONTAINER
MANILA
NATIVE CEPH
CEPH.KO
NOVA
Distributions Integration
Manila in SUSE OpenStack Cloud
§ The Manila service is tech preview in SUSE Cloud 5
§ Fully supported in SUSE OpenStack Cloud 6
§ Crowbar deployment tool integration
§ Controller HA
§ NetApp driver
§ Custom driver possible
Manila in SUSE OpenStack Cloud 6 (Beta)
https://youtu.be/pp0Y87Xfy-E
Manila in RHEL OpenStack Platform 7
§ The Manila service is tech preview in RHEL OpenStack Platform 7
§ RHEL OpenStack director deployment tool § Offer integration with
§ GlusterFS native § Gluster NFS § NetApp drivers
§ Manila Certification program in RHEL OpenStack Platform 8
§ Introducing NFS-Ganesha and Gluster Automated Volume Management (based on Heketi)
Manila in RHEL OpenStack Platform 7 (Preview)
https://www.youtube.com/watch?v=lfyYh8BWH7I&feature=youtu.be
Upcoming
in Mitaka
Upcoming in Mitaka
§ Mount Automation § Rolling Updates § Export Location Metadata § Manila QoS § Capability Lists § Interaction Between New Features § Share Migration Loose-ends § Remove All Extensions § Architectural Directions For New 1st-party Drivers § Share Replication
§ Non Disruptive Operations § High Availability
§ Availability Zones
§ Failures within an AZ
§ High Availability Solution § Clustered Storage
§ Failure of an AZ
§ High Availability Solution § Share Replication
Manila State of the Art: Share Replication
Demo #3: Share Replication
GET INVOLVED WITH MANILA! § Manila Resources § https://github.com/openstack/manila § https://github.com/openstack/python-manilaclient § https://github.com/openstack/manila-ui § https://github.com/openstack/manila-image-elements § https://wiki.openstack.org/wiki/Manila § https://launchpad.net/manila
§ #openstack-manila on IRC (Freenode) § Weekly meetings @ Thursday, 15:00 UTC
§ NetApp: http://netapp.github.io § Red Hat: https://www.redhat.com/en/technologies/linux-platforms/openstack-platform § Suse: https://www.suse.com/products/suse-cloud/
MANILA RELATED SESSIONS
§ MANILA GENERAL SESSIONS § Manila and Sahara: Crossing the Desert to the Big Data Oasis: Tuesday, Oct 27 12:05pm § OpenStack Manila Hands-on Lab Session: Tuesday, Oct 27 2:00pm § The State of Ceph, Manila, and Containers in OpenStack: Wednesday, Oct 28 4:40pm
§ UPCOMING MANILA SESSION § Manila contributors meetup: Friday, Oct 30, 9:00am
Q&A
October 29, 2015 OpenStack Summit -
Tokyo