liberty what’s new in openstack - matrix 30122015... · what’s new in openstack liberty object...

WHAT’S NEW IN OPENSTACK LIBERTY

Sean CohenPrincipal Product Manager

October 1st, 2015

Stephen GordonSr. Technical Product Manager


INTRODUCTION

2

Stephen GordonSr. Technical Product Manager

@xsgordon

Sean CohenPrincipal Product Manager

@SeanCohen_RH


AGENDA● Liberty Overview● Project Updates● Q&A

LIBERTY OVERVIEW


October 2015

“Mitaka” - is a city located in Tokyo Metropolis, Japan.

RELEASE TIMELINE

5

“Kilo” Sèrves, France is home to the Kilogram.

April 2015

“Liberty” - a village in the Canadian province of Saskatchewan.

TBD 2016

https://wiki.openstack.org/wiki/Release_Naming

N - Austin Summit

O - Barcelona Summit

TBD

TBD




POLL QUESTIONWhich OpenStack release are you currently using, if any?

● None● Grizzly● Havana● Icehouse● Kilo● Juno● Liberty (master)


STATISTICSOpenStack “Liberty”

● 27,000+ commits● 163,000+ reviews● ~160 corporate contributors● ~2100 individual contributors

http://stackalytics.com/?release=liberty&metric=commitshttp://stackalytics.com/?release=liberty&metric=reviews

http://stackalytics.com/?release=liberty&metric=commits

http://stackalytics.com/?release=liberty&metric=commits

http://stackalytics.com/?release=liberty&metric=reviews

http://stackalytics.com/?release=liberty&metric=reviews

GOVERNANCE UPDATES


THE OPENSTACK BIG TENT● Previously projects were either in the integrated release or not,

which meant different things to different people and potentially excluding projects that were just as much a part of the OpenStack community.

● Changes toward a “big tent” model endeavor to:○ More clearly define what it means to be an OpenStack project.

○ Be more broadly inclusive that projects that meet this definition, even those which compete with each other.

○ Provide a more precise taxonomy for navigating the ecosystem of OpenStack projects.


DEFINING THE BIG TENT

● Project aligns with the OpenStack Mission● Project follows the OpenStack way (“4 opens”):

○ Open Source○ Open Community○ Open Development○ Open Design

● Project ensures basic interoperability with the rest of OpenStack● Project has an active team of one or more contributors● Project meets any policies the TC requires all projects to meet

http://governance.openstack.org/reference/new-projects-requirements.html




TAGGING THE BIG TENT

● Tag types:○ Technical Committee Managed○ Team Description○ Project Assertions○ Release Management○ Vulnerability Management

http://governance.openstack.org/reference/tags/index.html

● E.g:○ Starter kit, approved release○ Team diversity○ Follows deprecation policy○ Has stable branches, cycle-wit○ Vulnerability team managed

● Tags introduced to help operators navigate all the projects that make up OpenStack.



COMPUTE


API MICRO-VERSIONING● Compute API v2 has been in place for some time, expanded by

adding “extensions”, lots of them.● Determined that implementing new major version of API (v3) would

be too difficult:○ User impact○ Developer overhead

● Micro-versions aim to:○ Make it possible to evolve the API incrementally○ Provided backwards compatibility to REST API users○ Improve code cleanliness to make doing the “right thing” easier


API MICRO-VERSIONING (cont.)● Use a single monotonic counter of the form where X.Y where:

○ X will only be changed due to a significant backwards incompatible API change being made. Expected to rarely never be incremented.

○ Y will change when making any change to the API.

● Client will specify version it supports, e.g.:○ X-OpenStack-Nova-API-Version: 2.114

● In Kilo, v2.0 API code was still used to serve v2.0 requests, v2.0 API was frozen

● In Liberty, v2.1 API code handles all requests - implementing both the v2.0 and v2.1 API endpoints


EXPERIMENTAL ONLINE SCHEMA CHANGES● Endeavours to make database schema changes occur online (either with no

service outage or a minimized one) ● Instead of using migration scripts:

○ Compares running schema against the model defined in Nova○ Generates DDL statements to close the gap○ Optionally, executes the DDL to make the running schema match the model

● Schema changes that aren’t in use today or are difficult to automate are not catered to in the initial online schema migration implementation.

● Currently extremely experimental, requires further work in subsequent releases. Not yet a recommended path to upgrading.


INSTANCE HIGH AVAILABILITY● Provided using external tools such as Pacemaker (previously

demonstrated at Red Hat Summit with Intel, Dell).● Liberty allows for cutting down on the time to evacuate guests after

detecting node failure:○ “Mark host down”/“force down” API allows external tooling to notify

Nova that a host is down before periodic tasks would normally detect it.

○ As soon as host has been explicitly marked down evacuation can commence, triggered by the external tool (e.g. Pacemaker fence agent).


VIRTIO NETWORKING PERFORMANCE● Default behavior provides a single VirtIO-net queue:

○ Restricts scale of guest protocol stack○ Does not scale with # of vCPUs○ Guests cannot transmit or receive in parallel (only one TX/RX queue)

● Multiple queues provide greater throughput when:○ Packets are relatively large○ Guest is active on many connections at the same time○ Number of queues == number of vCPUs

● Enabling:○ hw_vif_multiqueue_enabled=true|false (default false)○ Nova will match number of queues to number of vCPUs


IMPROVED BAREMETAL DRIVER SUPPORT● Maturing and increased driver support:

○ Dell, HP, Open CloudServer, SeaMicro, Cisco UCS, Fujitsu, AMT○ Generic and virtual driver coverage

■ Wake-on-LAN power driver■ SSH driver support for VirtualBox, Parallels, Virsh

● Specific driver enhancements:○ HP (iLO)

■ UEFI secure boot○ Fujitsu Primergy

■ booting from virtual media using NFS or CIFS supported

NETWORKING


QUALITY OF SERVICE● QoS is a general term referring to capability of a network to provide

better service to select network traffic.○ Various tools exist and may be applied in different places.○ Examples include traffic classification, policing, marking, and shaping.

● OpenStack tenant administrator may want to offer different service levels based on application needs and available bandwidth.

● In Liberty OpenStack Networking (“Neutron”) provides a new extensible API for defining QoS policies.


QUALITY OF SERVICE (cont.)● Policies can be applied:

○ Per port - a specific Neutron port has the policy applied.

○ Per network - all ports plugged on the Neutron network inherit the policy.

● Initial support allows rate limiting the traffic a virtual machine can send - packets beyond the specified rate are dropped:

○ Virtual interfaces using OVS○ Virtual Functions (VFs) using SR-IOV

● Policy can be applied, modified, or removed dynamically.


PLUGGABLE IP ADDRESS MANAGEMENT● Deployers have a need to integrate OpenStack with their existing

infrastructure that uses external IPAM solutions.● Historically most OpenStack Networking (“Neutron”) plug-ins

leverage an embedded IPAM implementation.○ While DHCP provider could be changed, allocation logic could not.

● Liberty introduces pluggable IPAM framework into Neutron.○ Reference implementation remains unchanged but there is now a plug-

in system to support external IPAM offerings.


ROLE-BASED ACCESS CONTROL● Network sharing in Neutron has historically been binary, a network is

shared with every tenant or not shared at all.● Liberty introduces a RBAC table that allows finer grained sharing of

Neutron networks between tenants.○ Allow only specific tenants to attach virtual machines to the network.

○ Eliminate the ability for some tenants to create networks at all instead

only allow attachment to pre-created networks corresponding to their project.

IDENTITY


IDENTITY

● Support for tokenless authorization with X.509 SSL client certificate● Per identity provider Web Single Sign-On (WebSSO) support.● Additional attributes added to SAML assertion in order to map user

and project domains.

STORAGE


GENERIC VOLUME MIGRATION

● Volume migration allows you to migrate a volume with its data between two backends in a manner that is transparent to users and workloads.

● The original API implementation assumed that both source and destination volumes can be attached locally via iSCSI.

● The new generic API now allows more drivers such as Ceph RBD to perform the migration functionality.


IMPORT/EXPORT SNAPSHOTS● Complementing feature for the import/export volume function.● Allows to import volumes snapshot from one Cinder to another.● The imported snapshots can be used as volume templates.● Allows to import "non" openstack snapshots already on a backend-

device. Where, export snapshots should work the same way as export volumes.


VOLUME REPLICATION (v2)● Provides a new implementation for replication of volumes between

backends that leaves the bulk of the work to the driver.● Provides basic administrative API methods to perform actions such

as enable/disable and fail-over.● First iteration of this new API supports only a single remote device,

with limited tenant visibility.● Allowing to introduce new replication drivers in the Mitaka cycle.


NON-DISRUPTIVE BACKUP● Allows OpenStack administrators to perform a non-disruptive

volume backup while the volume is still attached to a guest.○ By allowing the backup driver to:

■ Take a temporary snapshot■ Attach the temporary snapshot■ Do backup from the temporary snapshot■ Detach the temporary snapshot■ Cleanup temporary snapshot

(Until now, volumes had to be detached first before they could be backed up)


GENERIC IMAGE CACHE

● With this new feature backends are able to use cached glance images when creating volumes from images.

● If enabled, and the cinder internal tenant is configured, it will create a minimal sized clone of a volume the first time it is downloaded from glance. The original will then be extended to full size.

● Any following requests to create a volume from an image will be able to do a volume clone from the image-volume cache entry instead of downloading the image data from glance again.


IMAGE SIGNING AND ENCRYPTION● Protect image integrity by ensuring no modifications occur after the

image is uploaded by a user.● Supports both signing of the image and signature validation of

bootable images when used.


OBJECT STORAGE● Increasing ring partition power

○ A new swift-ring-tool has been introduced to increase or decrease the

partition power of an OpenStack Swift ring without the need to copy all data to a new cluster.

● Symbolic links○ Swift has gained the notion of a symbolic link (“symlink”) object allowing

to do aliases or multiple names to the same object.

○ This object will reference another object, where GET, HEAD, POST, and

OPTIONS requests for a symlink object will operate on the referenced object.


DATA PROCESSING● Spark now available on YARN based (generation 2) distributions● Storm elastic data processing (EDP) support● EDP jobs can now be suspended and resumed● Can now launch multiple clusters in parallel● Hadoop generation 1 support has been deprecated ● Manila as a runtime data source


DATABASE AS A SERVICE● DB Cluster Scaling - Adds cluster grow and shrink functionality● Expanding Backup/Restore capabilities to MongoDB● Adding MariaDB support● Guestagent configuration manager functionality - provides ability to

allow custom configuration overrides● Floating IP for Vertica management master node - provides external

access to Vertica cluster management console● Database log file downloads - create API for end user to access log

files stored in swift containers

ORCHESTRATION


CONVERGENCE UPDATES● Initial steps in longer term trend towards supporting a convergence

model for Heat stack updates.● Liberty introduces an (optional) mode to persist per-resource state

during stack updates.○ Improved fault tolerance, including recovery from a failure of the Heat

engine.

○ Improved scalability, potential to spread work over engine workers in a more granular way than was previously possible.

● Lays groundwork for auto-healing style features in future.


STACK TAGS● Allow categorization of Heat stacks when they are created.● Includes ability to mark a stack as hidden based on a tag. E.g.:

○ Cloud provider wants to use a heat stack for behind the scenes

orchestration such as automating networking configuration when launching an instance


BROADER PROJECT SUPPORT● New Heat resources for emerging projects including:

○ Barbican○ Mistral○ Manila○ Monasca○ Designate

● Zaqar integration for SoftwareConfig○ SoftwareDeployment resources now support providing metadata and

signalling via Zaqar as an alternative to existing transports.

MONITORING


CEILOMETER SPLIT UP● Lack of clarity around how to

best use ceilometer for different use cases in the past.

● Split into multiple sub-projects, each aimed at one high level use and using its own storage backend:

○ Collectors○ Metrics (“Gnocchi”)○ Alarms (“AODH”)○ Events


CEILOMETER TIME SERIES METRICS

● Ceilometer’s poor performances on large scale metrics API requests was one of the biggest issues

● Gnocchi (Time Series Database as a Service) project was created to address this under the Ceilometer program umbrella.

EMERGING PROJECTS


SHARED FILE SYSTEMS SERVICE (MANILA)

● Share Extend & Shrink○ New REST API for extending and

shrinking shares

● Share migration○ Share Migration allows a share

to be migrated from one host

pool to another hostpool, as well

as allowing to perform migration between different backends.

● Big data (Sahara) support for Manila as a data source


CONTAINERS SERVICE (MAGNUM)● API service developed by the OpenStack Containers Team.● Aims to make container orchestration engines (COE) first class

resources in OpenStack.● Uses Heat to orchestrate deploying and configuring an OS image

which contains Docker and a COE on virtual machines or bare-metal.

Liberty enhancements:

● Smart(er) scale down for Kubernetes clusters.● High availability (2N) for Kubernetes master node.● Ability to act as a CA (stepping stone to TLS support etc.)


Zaqar is a multi-tenant cloud messaging service for web developers. The service enables cloud developers to send messages between various components.

Liberty enhancements include:

● Pre-Signed URLs: auth-less URLs to access resources● Persistent transport (websocket): Persistent connections and

streamed API calls.● Email push-notifications: Notify users over email on messages for

certain topics.

MESSAGE SERVICE (ZAQAR)


Which projects in the OpenStack “big tent” would you like to hear more about in the future?

● Key management (Barbican)● DNS as a Service (Designate)● Application catalog (Murano)● Governance service (Congress)● Workflow service (Mistral)● Key-value store as a Service (MagnetoDB)● Deployment (TripleO)● Benchmark service (Rally)

POLL QUESTION

WHAT’S NEW IN OPENSTACK LIBERTY49

OPENSTACK SUMMIT TOKYO

THANK YOU

plus.google.com/+RedHat

linkedin.com/company/red-hat

youtube.com/user/RedHatVideos

facebook.com/redhatinc

redhatstack.com

twitter.com/RedHatNews

liberty what’s new in openstack - matrix 30122015... · what’s new in openstack liberty object...

Documents