open stack ow2 conference nov10

OpenStackOpen source software to build public and private clouds.

What is OpenStack?

+

+Community

Community

+

+Community

Technology

creating open source software to build public and private clouds

Software to provision virtual machines on commodity hardware at massive scale

Software to reliably store billions of objects distributed across commodity hardware

OpenStack C ompute

OpenStack Objec t S torag e

creating open source software to build public and private clouds

OpenStack Mission

‣ “To produce the ubiquitous open source cloud computing platform that will meet

the needs of public and private cloud providers regardless of size, by being s imple to implement and mas s ively

s c a lable.”

Why is OpenStack important?

OpenStack Founding Principles

‣ Apache 2.0 license, no paid ‘enterprise’ version‣ Open design process, 2x year public Design

Summits‣ Publicly available source code repository‣ All community processes documented and

transparent‣ Commitment to drive and adopt open standards‣ Modular design for deployment flexibility via

APIs

Architect for in-hous e

Re-Architect for s ervic e provider

Architect onc eDeploy anyw here

N o StandardsWith OpenStack

OpenStack History

Rackspace Decides to Open

Source Cloud Software

March

NASA Open Sources Nebula

Platform

May June July

OpenStack formed b/w

Rackspace and NASA

Inaugural Design Summit in Austin

2010

OpenStack History

OpenStack launches with 25+ partners

July

First ‘Austin’ code release with 35+

partners

October November February

First public Design Summit in

San Antonio

Second ‘Bexar’ code release

planned

2011

N AS AFounding members operate at

massive scale

OpenStack Community Today

HOW TO: Turn Racks of Commodity Hardware

Into a Cloud with OpenStack

Start with an open, scalable platform

OpenStack C ompute OpenStack Objec t S torag e

C LOU D OS

OpenStack Imag e S ervic e

U s er C ontrol Panel

Tic ketingS ys tem

N etw orkM anag ement

M onitoringS ys tems

Hos t S erver M anag ement

E C OS Y S TE M

OpenStack C ompute OpenStack Objec t S torag e

C LOU D OS

OpenStack Imag e S ervic e

U s er C ontrol Panel

Tic ketingS ys tem

N etw orkM anag ement

M onitoringS ys tems

Hos t S erver M anag ement

Ac c ountB illing

Admin C LITools

Live C hatS upport

Ac c ountM anag ement

E C OS Y S TE M

PU B LIC C LOU D

OpenStack C ompute OpenStack Objec t S torag e

C LOU D OS

OpenStack Imag e S ervic e

U s er C ontrol Panel

Tic ketingS ys tem

N etw orkM anag ement

M onitoringS ys tems

Hos t S erver M anag ement

E C OS Y S TE M

Admin C ontrolPanel

Dept. Ac c ounting C harg ebac k

U s erM anag ement

E nterpris e S oftw areInteg ra tion S ys tems

PR IVATE C LOU D

OpenStack C ompute OpenStack Objec t S torag e

C LOU D OS

OpenStack Imag e S ervic e

OpenStack Compute DetailsSoftware to provision virtual machines on commodity hardware at massive scale.

As ync hronous eventua lly c ons is tent c ommunic a tion 

R E S T-bas ed AP I

Horizonta lly and mas s ively s c a lable

Hypervis or ag nos tic : support for Xen ,XenServer,

KVM, UML and Hyper-V is coming

H ardw are ag nos tic : commodity hardware, RAID not required

OpenStack Compute Key Features

API: Receives HTTP requests, converts commands to/from API format, and sends requests to cloud controller

Cloud Controller: Global state of system, talks to LDAP, OpenStack Object Storage, and node/storage workers through a queue

User Manager

ATAoE / iSCSI

Host Machines: workers that spawn instances

Glance: HTTP + OpenStack Object Storage for server imagesOpenStack Compute

S erver G roups1 GigE

ConnectivityDual Quad CoreRAID 10 Drives

Public N etw ork

Private N etw ork(intra data center)

M anag ement

Example OpenStack Compute Hardware

OpenStack Object Storage DetailsSoftware to reliably store billions of objects distributed across commodity hardware

REST-based AP I Data dis tributed evenly throughout system

Hardw are ag nos tic : commodity hardware, RAID not required

OpenStack Storage Key Features

N o centraldatabase

S c a lable to multiple petabytes, billions of objects

Ac c ount/C onta iner/Objec t structure (not file system, no nesting) plus R eplic a tion (N copies of accounts, containers, objects) 

System Components

‣ The R ing : Mapping of names to entities (accounts, containers, objects) on disk.‣ Stores data based on zones, devices, partitions, and replicas‣ Weights can be used to balance the distribution of partitions‣ Used by the Proxy Server for many background processes

‣ Proxy S erver: Request routing, exposes the public API‣ Objec t S erver: Blob storage server, uses xattrs, uses

binary format‣ Recommended to run on XFS‣ Object location based on path from name hash & timestamp

System Components (Cont.)

• C onta iner S erver: Handles listing of objects, stores as SQLite DB

• Ac c ount S erver: Handles listing of containers, stores as SQLite DB

• Replic ation: Keep the system consistent, handle failures

• U pdaters : Process failed or queued updates‣ Auditors : Verify integrity of objects, containers,

and accounts

Software Dependencies

Object Storage (Swift) development currently targets Ubuntu Server 10.04, but should work on most Linux platforms with the following software:‣ Python 2.6‣ rsync 3.0

And the following python libraries:‣ Eventlet 0.9.8‣ WebOb 0.9.8‣ Setuptools‣ Simplejson‣ Xattr‣ Nose‣ Sphinx

Example Network Deployment

Evolution of Object Storage ArchitectureVersion 1: Central DB

(Rackspace 2009)Version 2: Fully Dist ributed

(OpenStack Object Storage 2009)

5 Z ones2 Proxies per 25

Storage Nodes10 GigE to

Proxies1 GigE to

Storage Nodes24 x 2TB Drives

per Storage Node

Public Internet

Example OpenStack Object Storage

Hardware

Load B a lanc ers

Planning an OpenStack DeploymentRequirements & Technology Choices

Hardware Selection

‣ OpenStack is designed to run on industry standard hardware with flexible configurations

‣ C ompute

‣ X86 Server‣ Storage flexible (Local, SAN, NAS)

‣ Objec t S tora g e

‣ X86 Server (other architectures possible)‣ Do not deploy with RAID (can use controller for case)

PhysicalHardware

RemoteManagement

HostNetworking

Host SeedOS Install

Host OSInstall

Post OSConfiguration

Rack

Cable

Dell DRAC

HP iLO

IPMI

DHCP BOOTP / TFTP

GPXE

Preseed

Kickstart

YAST

Puppet

Chef

CFEngine

Static

Bootstrapping Your Physical Nodes

1 2 3 4 5 6

Server Vendor SupportFind out how much configuration your hardware can

provide‣ B as ic N eeds

‣ BIOS settings‣ Network boot‣ IP on IPMI card

‣ Advanc ed S upport

‣ Host OS installation‣ Still get management network IP via DHCP

‣ Build in a manner that requires minimal change‣ Lay out addressing in a block-based model‣ Go to Layer 3 from the top of rack uplink

‣ Keep configuration simple‣ More bandwidth is better than advanced QoS‣ Let the compute host machines create logical zones

Network Device Configuration

Host Networking

‣ DHCP for the management network‣ Infinite leases‣ Base DNS on IP

‣ Ex. nh-pod-a-10-241-61-8.example.org‣ OpenStack Compute handles IP provisioning for all

guest instances – Cloud deployment tools only need to setup management Ips

Host OS Seed Installation – Choosing a Method‣ BOOTP / TFTP – Simple to configure

‣ Security must be handled outside of TFTP‣ Host node must be able to reach management

system via broadcast request‣ Top of rack router can be configured to forward

‣ GPXE‣ Not all hardware supports‣ Better concurrent install capability than TFTP

Options to Automate Host OS Installation

‣ Building a configuration based on a scripted installation is better than a monolithic “golden image”

‣ KickPreseed for Ubuntu / Debian hosts‣ start for Fedora / CentOS / RHEL hosts‣ YaST for SUS / SLES hosts

‣ Scripted configuration allows for incremental updates with less effort

Post OS Configuration‣ Choose a configuration management solution

‣ Puppet / Chef / Cfengine‣ Create roles to scale out controller infrastructure

‣ Queue‣ Database‣ Controller

‣ Automate registration of new host machines‣ Base the configuration to run on management net IP

OpenStack Release Process: Four Phases‣ Design: Starting the day of the release to one

week after the summit (when the Blueprints are accepted and prioritized)

‣ Development: until Feature Freeze date‣ QA: until Final Freeze date‣ Release: final testing and development tasks in

the last week

OpenStack ReleasesCactus:

April/May 2011

Bexar: February

2011Austin:

October 2010

• OpenStack Object Storage production-ready• OpenStack Compute developer preview, ready for testing and proofs of concept

• OpenStack Compute ready for enterprise private cloud deployments and mid-size service provider deployments• Enhanced documentation• Easier to install and deploy

•OpenStack Compute ready for large service provider scale deployments

OpenStack Compute ‘Austin’ Release Features

‣ Multi-hypervisor support: KVM, QEMU, User-Mode Linux, Xen and XenServer

‣ Introduces official OpenStack API, while maintaining EC2 API option

‣ New image registry and delivery service, called the Glance project

‣ Support for two network models on compute nodes: VLANs with DHCP and flat with either static IP pools or DHCP

‣ Addition of base scheduling service

‣ Implements WSGI to create a standard API layer with reusable components

‣ Support for user-friendly naming

‣ Refactored ORM and networking code for simpler code that is easier to understand

‣ Addition of SQLAlchemy Database toolkit so users can leverage existing SQL infrastructure

Object Storage ‘Austin’ Release Features‣ Addition of a stats system that produces per-account

hourly summaries of system usage‣ Ability for users to set ACL’s and grant public access to

containers‣ Support for API access to account and container

metadata‣ Rate limiting was extended to allow requests to be

slowed down and support stair stepped rate limits based on container size

‣ WSGI support was improved and pulled into middleware

Join Us‣ General Information: http://openstack.org

‣ Developers & Testers

‣ http://launchpad.net/openstack

‣ http://wiki.openstack.org

‣ Writers: http://wiki.openstack.org/Documentation

‣ Blog: http://openstack.org/blog

‣ Twitter: http://twitter.com/openstack

‣ Jobs: http://openstack.org/jobs