openstack framework introduction

OpenStack Introduction

Presenter: Jason, Tsung-Cheng, HOUAdvisor: Wanjiun Liao

June 7th, 2012 1

Motivation

• What does a cloud OS look like?• How are they building IaaS Platform?• What are current industry trend?• How will the cloud system press the network?• OpenStack

– Founded by NASA and Rackspace in 2010– Currently 178 companies and 3386 people– Growing fast now, latest release Essex, Apr. 5th

OpenStack Status

• OpenStack– Founded by NASA and Rackspace in 2010– Currently 178 companies and 3386 people– Was only 125 and 1500 in fall, 2011.– Growing fast now, latest release Essex, Apr. 5th

• Aligned release cycle with Ubuntu, Apr. / Oct.• Aim to be the “Linux” in cloud computing sys.• Open-source v.s. Amazon and vmware• Start-ups are happening around OpenStack• Still lacks big use cases and implementation

Agenda

• OpenStack Brief Overview• Some Reviews of Cloud Technology• “Keystone” Identity• “Swift” Storage• “Glance” Image• “Nova” Compute• “Quantum” Networking

Enterprises are building clouds to...

Where is your enterprise on the path to cloud?

Meet departmental resource needs & timelinesControl & monitor the entire environment

2. Cloud Data Center 3. Cloud FederationServer Virtualization1. Virtualization

Datacenters are being virtualized, Servers are firstHypervisors provide abstraction between SW and HW (Servers)

HOST 1 HOST 2 HOST 3 HOST 4, ETC.

Hypervisor:Turns 1 server into many “virtual machines” (instances or VMs)(VMWare ESX, Citrix XEN Server, KVM, Etc.)

Automation & Efficiency

Better resource utilization for each server

Hardware abstraction for each server

Compute Pool Network Pool Storage PoolVirtualized Servers Virtualized Networks Virtualized Storage

Next: Storage, Network…the building blocks

Resource pools for apps starting to form...

Flexibility, Efficiency are key drivers

But questions arise as the environment grows...“VM sprawl” can make things unmanageable very quickly

Where should you provision new VMs? How do you keep track of it all?

How do you empower employees to self-service?

USERS ADMINS

How do you make your apps cloud aware?

A Cloud Management Layer Is Missing

Solution: OpenStack, The Cloud Operating SystemA new management layer that adds automation and control

Creates Pools of Resources Automates The Network

USERS ADMINS

CLOUD OPERATING SYSTEM

Connects to apps via APIsSelf-service Portals for users

2. Cloud Data Center 3. Cloud FederationServer Virtualization1. Server Virtualization

Enterprise Private Clouds run cloud operations systems…

2. Cloud Data Center 3. Cloud FederationServer Virtualization1. Server Virtualization

What’s next?

Public Clouds run cloud operating systems…

But you can’t interoperate ifpublic clouds are built on proprietary software

Common PlatformImagine having a across clouds

Seamlessly transporting workloads

A common platform is here.OpenStack is open source software powering public and private clouds.

Public Cloud:OpenStack powers someof the worlds largest publiccloud deployments.

Private Cloud:Run OpenStack software

in your own corporatedata centers

Washington

EuropeCaliforniaTexasPrivate Cloud Private Cloud

Public Cloud

Common software platform making

Federation possible

OpenStack enables cloud federationConnecting clouds to create global resource pools

In Summary, the Cloud Operating System enables enterprises to:

1. Control and automate pools of resources2. Efficiently allocate resources3. Empower admins & users via self-service portals4. Empower developers to make apps cloud-aware

via APIsTop 3 Benefits of a Common Platform1. Easy to migrate data and applications to public

clouds when conditions are right- based on security policies, economics, and other key business criteria

2. No longer locked in with major investment in one provider

3. Disaster recovery

Core Components in Essex• Release Apr. 5th, 2012

• Dashboard: Access and control portal for admin and users, also web-based

• Identity: Unified authentication across whole system

• Object Storage: Large-scale redundant storage of static objects, not a file system

• Image Service: Store, retrieve, discover, register, and deliver VM images

• Compute: Large-scale deployment of automatically provisions VMs and related SWs

OpenStack Compute Key Features

2. Horizontally and massively scalable

1. REST-based API

3. Hardware agnostic: supports a variety of standard hardware

4. Hypervisor agnostic: support for Xen ,Citrix XenServer, Microsoft Hyper-V, KVM, UML, LXC and ESX

OpenStack Storage Key Features

4. Scalable to multiple petabytes, billions of objects

1. REST-based API

6. Account/Container/Object structure (not file system, no nesting) plus Replication (N copies of accounts, containers, objects)

5. No central database required

2. Data distributed evenly throughout system

3. Runs on standard hardware

OpenStack Image Service Key Features

2. REST-based API1. Store & retrieve VM images

3. Compatible with all common image formats

4. Storage agnostic: Store images locally, or use OpenStack Object Storage, HTTP, or S3

Agenda

Chief Reference

• CIS 607: Seminar in Cloud Computing, Spring 2012, by Dr. Allen D. Malony

• "Applied Computational Instrument for Scientific Synthesis" (ACISS)

• University of Oregon (UO), groups may configure their own VM images and carry out research work as if had dedicated clusters.

• This course delivers hands-on operations on ACISS and cloud computing knowledge to students.

VM and Hypervisor

• Virtual Machine: A software package, sometimes using hardware acceleration, that allows an isolated guest operating system to run within a host operating system.

• Stateless: Once shut down, all HW states disappear.

• Hypervisor: A software platform that is responsible for creating, running, and destroying multiple virtual machines.

• OpenStack is hypervisor agnostic.• Type I and Type II hypervisor 26

Bridged Networking• One network card acts as

many devices.• Host does not need an IP

address.• Hypervisor sets virtual

MAC address for guest machine.

• ACISS uses bridges, along with Virtual Local Area Networks (VLANs) to segment traffic and assign network addresses.

Network Block Storage

• Network Block Storage: Make data persistent by mounting a network block storage device.

• NFS Mounts: Many machines may access simultaneously. Limited permissions.

• iSCSI Mounts. Only one machine may access at any given time. Unlimited permissions.

Object Storage

• Persistent storage of objects on a network.• Generally “write once, read many.”• Durable storage with redundant copies• Access Control Lists determine visibility for

owner and authorized users.• Amazon’s S3 is an example of this.• ACISS uses OpenStack Swift.• Swift uses same API as S3.

Virtual Machine Images

• Disk images that can be booted on a virtual machine by a hypervisor.

• Can be a single image that contains boot loader, kernel and operating system.

• Boot loader and kernel can be separated.• Allows for custom kernels and resizable

images.

Image Service

• Stores and catalogs virtual machine images.– Keep track of VMs, trace and recover.

• Provides for discovery, registration, and delivery of images to hypervisors.

• Allows for many image formats and for linking of loaders and kernels to images.– There may be different types of virtualization

technologies, different kernels, etc.

• Usually built on object storage systems.• Glance on Swift.

Cloud Computing

• The course defines in the following way:– The orchestration of hypervisors, networking,

block storage, and image, and identity services to provide on demand virtual machines.

• Hence, meeting required characteristics of cloud computing.– On-demand self-service– Resource pooling– Rapid elasticity– …

Agenda

Keystone Main Functions

• Provides 4 primary services:– Identity: User information authentication– Token: After logged in, replace account-password– Service catalog: Service units registered– Policies: Enforces different user levels

• Can be backed by different databases.– LDAP– SQL– Key Value Stores (KVS)

Keystone: Identity

• User information:– username/password– Metadata (e-mail, etc.)– Tenant - organizes users into projects or group.– Role - define a user’s role and permissions in a

project.

• A user must belong to at least one tenant, and may belong to many tenants

• Roles are assigned to user/tenant pairs– Common roles: Member, Admin

Keystone: Token

• Once a user’s identity has been verified with a acc/pswd pair, a short-lived (24 hr) token is issued.

• Tokens are a stand-in for the acc/pswd.• OpenStack services hold on to tokens and

use them to query keystone during operations.

• For example, once Nova can use a token to determine if an authenticated user has authorization to delete an instance.

Keystone: Catalog

• OpenStack service endpoints are registered with Keystone to create a service catalog.

• A client for a service connects to Keystone, and determines an endpoint to call based on the returned catalog.

• Behind the scenes, services can be moved to different endpoints. A client can find online services by querying Keystone endpoint.

• Also allows for service load distribution with multiple endpoints to a single service.

Keystone: Catalog

• Every catalog entry has five elements:– region: the name given to a collection of cloud

services– service id: the service the endpoint is associated

with (Glance, Nova, Swift, Keystone)– public url: the public facing endpoint for the

service– internal url: the internal facing endpoint. Usually

the same as the public url– admin url: the endpoint for service administration

Agenda

• Object storage, objects “live” on an endpoint.– An endpoint could be any storage device

• Every object belongs to a user/account pair.– keystone tenant : swift account– keystone user : swift user– keystone role : swift group

• Proxy, Ring, and Workers• Account, Container, Object

Swift: Proxy Server

• Handles incoming requests via the OpenStack Object API or raw HTTP.

• Accepts files to upload, modifications to metadata or container creation.

• Serve files or container listing to web browsers.

• Several types of Ring files• May utilize an optional cache to improve

performance.

Swift: Workers

• Keep a distributed database of replicated objects.

• Workers are divided into reliability zones. • Copies of data are distributed across multiple

zones.• There are many types of workers:

– Account server, container server, object server– Housekeeping: Replication, updater, auditor

Swift: Ring

• Maps names to entities and locations– Stores data based on zones, devices, partitions, and replicas

• There are three types of items:– Account, container, object

• The locations are determined by a ring file• Worker IP addresses are loaded into a ring

builder.• Storage ids and locations are computed using

a hashing algorithm to evenly distribute items across the workers.

Swift: Ring• Account and container storage id has a

database, storing object metadata.• Proxy makes distributed searches across the

databases for item requests.• The ring builder can add / remove nodes, and

rebalance distribution of files across servers.

←Stores object metadata

↑Stores container / object metadata

↓Physical arrangement

↑ Logical view

← Stores real objects

Duplicated storage, load balancing

Workers can be a account server, a container server, or an object server

# of account < # of container < # of object servers

Different zones ↑

Agenda

Glance

• Image storage and indexing.• Keeps a database of metadata associated

with an image, discover, register, and retrieve.

• Built on top of Swift, images store in Swift• Two servers:

– Glance-api: public interface for uploading and managing images.

– Glance-registry: private interface to metadata database

• Support multiple image formats 57

Agenda

• Major components:– API: public facing interface– Message Queue: Broker to handle interactions

between services, currently based on RabbitMQ– Scheduler: coordinates all services, determines

placement of new resources requested– Compute Worker: hosts VMs, controls hypervisor

and VMs when receives cmds on Msg Queue– Volume: manages permanent storage

• Major components:– Network: manages networking

• Was originally a component in Nova• Default gateway, network controller• DHCP server, address mgmt• The network part in Nova will be enhanced by the

project named “Quantum”, to be released.• Will introduce Quantum later.

Nova Messaging and Data

• Messaging is managed through RabbitMQ– Server that allows messages to be posted to channels.– Subscribers to channels receive messages.– Services regularly announce availability.– Scheduler regularly reads for availability.– Scheduler makes requests to services.

• Persistent data stored in a database.– VM metadata, network topology, volume metadata, known

services

Messaging (RabbitMQ)• Get data from point A to point B• Decouple publishers and consumers• Queueing for later delivery• Load balancing and scalability• RabbitMQ is an AMQP messaging broker• Advanced Message Queueing Protocol• Network wire-level protocol• Internet protocol - like HTTP, TCP - but

ASYNCHRONOUS

Messaging (RabbitMQ)

Agenda

Without Quantum• Originally, Nova handles all networking by:

– Linux bridge networking– Virtual interfaces connecting network through the

physical interface– Assigns VM IP address– Fixed IP: Returns when VM shuts down– Floating IP: Can be reassigned online

• Network Manager provides VN to enable compute servers to interact with each other and the public network

• A Blog states currently 90% Nova bugs are network related76

Original Network Manager

• Each VM network owned by one network host – Simply a Linux running Nova-network daemon

• Nova Network node is the only gateway• Flat Network Manager:

– Linux networking bridge forms a subnet– All instances attached same bridge– Manually Configure server, controller, and IP

• Flat DHCP Network Manager:– Add DHCP server along same bridge

• Later: VLAN Network Manager77

Bridged Networking• One network card acts as

many devices.• Host does not need an IP

address.• Hypervisor sets virtual

MAC address for guest machine.

• ACISS uses bridges, along with Virtual Local Area Networks (VLANs) to segment traffic and assign network addresses.

The only gatewayWith security measures

Linux running Nova-network daemon

Network host will act as the gateway for all the NICs bridged into that network.VMs bridged in to a raw Ethernet device

• DHCP server also tracks IP leases and releases

• Re-uses and assigns IP addresses dynamically

• Sets up a routing table for outside forwarding

• Compute optionally have public IP

• Network host is a single point of failure and bottleneck

• Backup network host• A new proposed model → →• Multi-NIC→Multiple networks

VLAN Network Manager

• Current default mode for OpenStack• Nova creates a VLAN and bridge for each project.

– Requires switches with VLAN tagging (IEEE 802.1Q).– A range of private IPs, only accessible inside VLAN.

• A special VPN instance (code named cloudpipe) needs to be created. Generates (certificate, key) for users to access VPN automatically.

• Provides a private network segment for each project, accessed via dedicated VPN connection from the Internet. Each project with own VLAN, Linux networking bridge, and DHCP server.

Plugin

• The component where the ‘virtual networking’ magic happens. Fulfills API contract by implementing the ‘Plugin Interface’

• Tenants expect same behavior from Quantum API regardless of the particular plugin employed

• Available Quantum Plugins:– Open vSwitch: Builds isolated networks with OVS and L2-in-L3 tunnels.

– Cisco UCS: Isolation based on VLAN and net-profiles applied to Cisco UCS

– converged network adapters

– Linux Bridge: Build isolated networks with VLAN interfaces and linux bridge

– NTT-Data Ryu: Acts as a proxy for the NTT Ryu platform

– Nicira NVP: Acts as a proxy for the Nicira NVP platform

The Quantum Manager

• Nova’s network manager for Quantum. Forwards network related requests.

• Also, provides other network services such as IP address management, DHCP, NAT, Floating IPs…

• Virtual Networking: A label nowadays applied to too many solutions and products.– Securely partitioning the network– Defining virtual network topologies– Automating network provisioning

The Near Future

• Folsom release, Fall 2012– Become a core OpenStack project– Merge with IP Address Management service– Improve API quality and documentation– Improve GUI, i.e. Quantum Horizon plugin– Possible more plugins, Build more network

services on top of the basic building block• Each service with its own tenant-facing API • IP routing, Distributed Firewall, LB, NAT, VPN,

bridging…• Quantum is NOT SDN, but in theory can transform anything into SDN.

Reference• OpenStack Documentation

http://docs.openstack.org/

• Dr. Allen D. Malony, CIS 607: Seminar in Cloud Computing, Spring 2012, U. Oregonhttp://prodigal.nic.uoregon.edu/~hoge/cis607/

• Bret Piatt, OpenStack Overview, OpenStack Tutorialhttp://salsahpc.indiana.edu/CloudCom2010/slides/PDF/tutorials/OpenStackTutorialIEEECloudCom.pdfhttp://www.omg.org/news/meetings/tc/ca-10/special-events/pdf/5-3_Piatt.pdf

• Vishvananda Ishaya, Networking in Novahttp://unchainyourbrain.com/openstack/13-networking-in-nova

• Sandy Walsh, OpenStack 101 Technical Overviewhttp://www.slideshare.net/openstackcommgr/openstack-101-technical-overview

• Jaesuk Ahn, OpenStack, XenSummit Asiahttp://www.slideshare.net/ckpeter/openstack-at-xen-summit-asiahttp://www.slideshare.net/xen_com_mgr/2-xs-asia11kahnopenstack

• Salvatore Orlando, Quantum: Virtual Networks for Openstackhttp://qconlondon.com/dl/qcon-london-2012/slides/SalvatoreOrlando_QuantumVirtualNetworksForOpenStackClouds.pdf

• Dan Wendlandt, Openstack Quantum: Virtual Networks for OpenStackhttp://www.ovirt.org/wp-content/uploads/2011/11/Quantum_Ovirt_discussion.pdf

• Daneyon Hansen, OpenStack @ CISCOhttp://www.cisco.com/web/strategy/docs/gov/openstack_presentation.pdf

• Rick Clark, Cisco and OpenStackhttp://www.ogf.org/OGF32/materials/2310/ogf32-isod-Cisco-OpenStack-July2011.pdf

openstack framework introduction

Technology

introduction to openstack storage

openstack introduction

introduction to openstack - red...

gdl openstack community - openstack introduction

itri openstack distribution introduction

introduction to openstack architecture

introduction to openstack (2012)

introduction to openstack...introduction to openstack...

introduction openstack horizon

openstack101 - introduction to openstack

openstack kolla introduction

openstack framework iaas

openstack: an introduction

introduction à openstack

ubuntu openstack introduction

introduction openstack-horizon

introduction to openstack

|openstack | - introduction to openstack cloud|

introduction to openstack heat

an introduction to openstack