Manila An Update from Liberty

October 29, 2015 OpenStack Summit - Tokyo

Akshai Parthasarathy

Technical Marketing Engineer, NetApp

@akshai

Sean Cohen

Principal Product Manager, Red Hat

@SeanCohen_RH

Thomas Bechtold

OpenStack Cloud Engineer, SUSE

Today's Presenters

Agenda

•  What is Manila? •  Why use Manila?

•  Use Cases •  Sahara •  Containers

•  Liberty Updates •  Distributions Integration

•  SUSE Demo •  RedHat Demo

•  Upcoming in Mitaka •  Share Replication Demo •  Q+A

Manila: The OpenStack Shared File Service Program Bringing self-service, shared file services

to the cloud

Manila History

§  Beginnings §  Juno

§  Incubated Project §  Community Inception §  Puppet Support §  Share Servers §  Tempest Integration

§  Kilo §  Driver Modes §  DevStack Plug-in §  Storage Pools §  Default Share Type §  Manage/Unmanage §  Manila UI

Manila Today -

§  Production Ready §  Number of Drivers: 14 §  Blue Prints Completed: 51 §  Major Blue Prints:

§  Share Instances §  REST API Microversions §  Experimental APIs §  Extend & Shrink §  Consistency Groups (CGs) §  Share Migrations §  etc.

Manila Deployment Options and Benefits

§ Single Storage Virtual Machine (SVM) / Multi SVM §  driver_handles_share_servers=False/True §  Benefit

§ Network Plugins

§  Standalone Network Plugin §  Nova Network Plugin §  Neutron Network Plugin §  Benefit

Contributions/New Entrants

Welcome and Thank You: ●  CloudBase Solutions ●  Fujitsu ●  Scality ●  NEC ●  NTT ●  Letv Cloud Computing ●  and others

Source: http://stackalytics.com/?module=manila&metric=loc&release=liberty

Code Contributions Reviews

Why Use Manila?

§  Shared File Services §  Spending §  Total Capacity

§  IDC: Age of Cloud File Services §  Why not?

Use Cases

Standalone Share File Services Management

§  Replace Legacy Systems §  Improve SLA §  Integrate with Automation Frameworks

Enterprise Applications

§  Lower Cost, Heterogeneous Infrastructure §  Application Porting: Move Workloads to OpenStack

DevOps/Continuous Integration

§  Pluggable Infrastructure §  Storage Service Catalog

§  Analytics §  Containers §  Snapshots & Clones §  Consistency Groups

New in Liberty

Share Instances

Needed for features like:

●  share migrations

●  replication

REST API Microversions

http://docs.openstack.org/developer/manila/devref/api_microversion_dev.html http://docs.openstack.org/developer/manila/devref/api_microversion_history.html

Example:

●  client 1 with server 1 => version 2.3

●  client 1 with server 2 => version 2.3

●  client 2 with server 1 => version 2.5

●  client 2 with server 2 => version 2.10

Experimental APIs

§  Expected to change at any time §  can be removed without

deprecation period §  Usage: needs to set header “X-

OpenStack-Manila-API-Experimental: true”

http://docs.openstack.org/developer/manila/devref/experimental_apis.html http://developer.openstack.org/api-ref-share-v2.html

@api_version(min_version='2.1', max_version='2.9') def show(self, req, id): .... stuff ....

@api_version(min_version="2.4", experimental=True) def my_api_method(self, req, id): .... stuff ....

Extend & Shrink

http://docs.openstack.org/developer/manila/devref/share_back_ends_feature_support_mapping.html

Consistency Groups (CGs)

§  grouping different shares together for the purpose of application data protection (focus of snapshots for disaster recovery)

§  example use case: database data and log files are on different shares §  when doing snapshots for both shares, the data on the shares must be consistent

§  supported actions §  create CG with mutliple shares §  create snapshot of CG §  create CG from snapshot

External CI For All Drivers

And others ...

Oversubscription

§ Tunable for setting provisioned capacity and a subscription ratio added

§ Addresses 'infinite' and 'unknown' drivers reported capacity that may lead to oversubscription

§  thin_provisoning support needed

Share Migrations

§  Share Migration allows a share to be migrated from one host pool to another hostpool through the "manila migrate <share> <host#pool>" command, as well as allowing to perform migration between different backends.

§  Basic implementation §  The fallback approach for migration is rsync.

Slow, inefficient §  Vendors can utilize API for optimized

migration

Availability Zones

§  The availability zones support which was inherited from Cinder was reworked this cycle:

§  Added public API extension §  Allow to preserve AZ if creating a share

from a snapshot and set AZ in Share API or Share Manager.

§  AZs will benefit share replication and also give end users control of the locality of their data w.r.t. consumers of the data.

Sahara Integration

§  Use cases §  Stores binaries for job templates - NFS is ideal for this case §  Input and output data sources - Manila-provisioned HDFS and NFS offers more

options §  Mount NFS share API

§  Binaries and data I/O from an NFS share path §  New development in Sahara this cycle §  Mount shares at cluster creation or auto-mount when a share is used for EDP

§  Manila-provisioned HDFS §  Data sources and data processing on Sahara-external clusters §  Testing and process verification of extant Manila features this cycle

§  Coming soon - NFS Hadoop driver (run jobs on your NFS shares)

Sahara Integration - Current Implementations

●  Data sources on Manila-provisioned HDFS

●  API to mount NFS shares to clusters (job binaries and data sources)

Images by Weiting Chen (Intel)

Manila + Containers Ceph Example § Simply mount --bind share into §  container namespace § NFS re-export from host

§ mount and export fs on host §  private host/guest net §  avoid network hop from NFS

service VM §  Host mounts CephFS §  Bind Manila share/volume into

container §  Further integration requires

integration with both Nova and Manila to manage the attach/detach process

HOST

M M

RADOS CLUSTER

CONTAINER

MANILA

NATIVE CEPH

CEPH.KO

NOVA

Distributions Integration

Manila in SUSE OpenStack Cloud

§  The Manila service is tech preview in SUSE Cloud 5

§  Fully supported in SUSE OpenStack Cloud 6

§  Crowbar deployment tool integration

§  Controller HA

§  NetApp driver

§  Custom driver possible

Manila in SUSE OpenStack Cloud 6 (Beta)

https://youtu.be/pp0Y87Xfy-E

Manila in RHEL OpenStack Platform 7

§  The Manila service is tech preview in RHEL OpenStack Platform 7

§  RHEL OpenStack director deployment tool §  Offer integration with

§  GlusterFS native §  Gluster NFS §  NetApp drivers

§  Manila Certification program in RHEL OpenStack Platform 8

§  Introducing NFS-Ganesha and Gluster Automated Volume Management (based on Heketi)

Manila in RHEL OpenStack Platform 7 (Preview)

https://www.youtube.com/watch?v=lfyYh8BWH7I&feature=youtu.be

Upcoming

in Mitaka

Upcoming in Mitaka

§  Mount Automation §  Rolling Updates §  Export Location Metadata §  Manila QoS §  Capability Lists §  Interaction Between New Features §  Share Migration Loose-ends §  Remove All Extensions §  Architectural Directions For New 1st-party Drivers §  Share Replication

§ Non Disruptive Operations §  High Availability

§ Availability Zones

§ Failures within an AZ

§ High Availability Solution §  Clustered Storage

§ Failure of an AZ

§ High Availability Solution §  Share Replication

Manila State of the Art: Share Replication

Demo #3: Share Replication

GET INVOLVED WITH MANILA! §  Manila Resources §  https://github.com/openstack/manila §  https://github.com/openstack/python-manilaclient §  https://github.com/openstack/manila-ui §  https://github.com/openstack/manila-image-elements §  https://wiki.openstack.org/wiki/Manila §  https://launchpad.net/manila

§  #openstack-manila on IRC (Freenode) §  Weekly meetings @ Thursday, 15:00 UTC

§  NetApp: http://netapp.github.io §  Red Hat: https://www.redhat.com/en/technologies/linux-platforms/openstack-platform §  Suse: https://www.suse.com/products/suse-cloud/

MANILA RELATED SESSIONS

§  MANILA GENERAL SESSIONS §  Manila and Sahara: Crossing the Desert to the Big Data Oasis: Tuesday, Oct 27 12:05pm §  OpenStack Manila Hands-on Lab Session: Tuesday, Oct 27 2:00pm §  The State of Ceph, Manila, and Containers in OpenStack: Wednesday, Oct 28 4:40pm

§  UPCOMING MANILA SESSION §  Manila contributors meetup: Friday, Oct 30, 9:00am

Q&A

October 29, 2015 OpenStack Summit -

Tokyo