openstack : cloud computing platform tutorial
TRANSCRIPT
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
1/55
Community StackerBret Piatt
OpenStack TutorialIEEE CloudCom 2010
Twitter: @bpiatt
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
2/55
70s 80sMainframe Era
90s-2000sClient Server Era
2010-beyondCloud Era
[Based on a Gartner Study]
Application Platforms Undergoing A Major Shift
2010 IT budgets arent getting cut....but CIOs expect their spend to go further.
#1 Priority is Virtualization#2 is Cloud Computing
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
3/55
Founded in 1998Publicly traded on NYSE: RAX120,000+ customers
$628m revenue in 2009 across two major businessesDedicated Managed Hosting
Cloud Infrastructure & Apps (Servers, Files, Sites, Email)
Primary focus on customer service ("Fanatical Support")3,000+ employees
9 datacenters in the US, UK and Hong Kong65,000+ physical servers
Overview of Rackspace
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
4/55
Rackspace Cloud: 3 Products with Solid TractionCompute: Cloud Servers
Virtualized, API-accessible servers with root accessWindows & Linux (many distros)Sold by the hour (CPU/RAM/HDD) with persistent storageLaunched 2009Based on SlicehostXen & XenServer HVs
Storage: Cloud FilesLaunched 2008Object file storev2.0 in May 2010
PaaS: Cloud SitesLaunched 2006Formally MossoCode it & Load it: .Net, PHP, Python apps autoscaled
Source: Guy Rosen (http://www.jackofallclouds.com)
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
5/55
Open ReST APIs released July 2009 (Creative Commons License)
Included in major API bindings: Libcloud, Simple Cloud, jclouds, -cloudSupported by key cloud vendors and SaaS servicesMarketplace: http://tools.rackspacecloud.com
Active Ecosystem on Rackspace APIs
http://tools.rackspacecloud.com/ -
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
6/55
What is OpenStack?Overview of the project
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
7/55
OpenStack: The Mission
"To produce the ubiquitous Open Sourcecloud computing platform that will meet
the needs ofpublic and private cloudproviders regardless of size, by beingsimple to implement and massively
scalable."
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
8/55
OpenStack History
Rackspace Decidesto Open SourceCloud Software
March
NASA OpenSources Nebula
Platform
May June July
OpenStack formedb/w Rackspace and
NASA
Inaugural DesignSummit in Austin
20102005
RackspaceCloud
developed
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
9/55
OpenStack History
OpenStacklaunches with 25+
partners
July
First Austin coderelease with 35+
partners
October November February
First public DesignSummit in San
Antonio
Second Bexar coderelease planned
2011
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
10/55
OpenStack Founding Principles
Apache 2.0 license (OSI), open development process
Open design process, 2x year public Design Summits
Publicly available open source code repository
Open community processes documented andtransparent
Commitment to drive and adopt open standards
Modular design for deployment flexibility via APIs
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
11/55
Community with Broad CommercialSupport
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
12/55
OpenStack Isn't Everything
ConsultantsBusiness Process Automation
Database EngineersOperating SystemTechniciansSystems SecurityProfessionalsNetwork Experts
Servers, Firewalls, Load BalancersOperating SystemsStorageManagement Tools
Virtualization
Data CenterNetworkingPower
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
13/55
Software to provision virtual machines onstandard hardware at massive scale
Software to reliably store billions of objects
distributed across standard hardware
OpenStack Compute
OpenStackObject Storage
creating open source software to buildpublic and private clouds
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
14/55
OpenStack Release Schedule
Design Summit:April TBA 2011
Cactus:April 15, 2011
Bexar:February 3, 2011
OpenStack Compute readyfor enterprise private clouddeployments and mid-sizeservice providerdeploymentsEnhanced documentation
Easier to install and deploy
Community gathers toplan for next release,likely Fall 2011
OpenStack Compute ready forlarge service provider scaledeployments
This is the Rackspace-readyrelease; need to communicateRackspace support and plansfor deployment
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
15/55
Building an OpenStack CloudDatacenter, Hardware, and Process
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
16/55
Business Prerequisites
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
17/55
Technical Prerequisites
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
18/55
Cloud Ready Datacenter Requirements
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
19/55
Bootstrapping Your Physical NodesBootstrapping the Host Machines
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
20/55
Building an OpenStack CloudObject Storage
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
21/55
Zettabyte
1,000 Exabytes1,000,000 Petabytes
All of the data on Earth today(150GB of data per person)
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
22/55
Zettabyte2% OF THE DATA ON EARTH IN 2020
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
23/55
If we stored all of the global data as an average enterprise..
..it would take..
..38.5% of the World GDP!
Data Must Be Stored Efficiently
ITEM MONTHLY FIGURES
ENTERPRISE AVGERAGE STORAGE COST $1.98 PER GIGABYTE
WORLD GDP $5.13 TRILLION
COST TO STORE A ZETTABYTE $1.98 TRILLION
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
24/55
Object Storage Summary
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
25/55
ReST-based API Data distributed evenlythroughout system
Hardware agnostic: standardhardware, RAID not required
Object Storage Key Features
No centraldatabase
Scalable to multiplepetabytes, billions ofobjects
Account/Container/Object structure(not file system, no nesting) plus
Replication (N copies of accounts,containers, objects)
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
26/55
System Components
The Ring: Mapping of names to entities (accounts,containers, objects) on disk.
Stores data based on zones, devices, partitions, and replicas
Weights can be used to balance the distribution of partitionsUsed by the Proxy Server for many background processes
Proxy Server: Request routing, exposes the public API
Replication: Keep the system consistent, handle failures
Updaters: Process failed or queued updates
Auditors: Verify integrity of objects, containers, and accounts
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
27/55
System Components (Cont.)
Account Server: Handles listing of containers, stores as SQLite DB
Container Server: Handles listing of objects, stores as SQLite DB
Object Server: Blob storage server, metadata kept in xattrs, data inbinary format
Recommended to run on XFS
Object location based on hash of name & timestamp
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
28/55
Software DependenciesObject Storage should work on most Linux platforms with the followingsoftware (main build target for Austin release is Ubuntu 10.04):
Python 2.6
rsync 3.0
And the following python libraries:
Eventlet 0.9.8
WebOb 0.9.8
Setuptools
SimplejsonXattr
Nose
Sphinx
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
29/55
Evolution of Object StorageArchitecture
Version 1: Central DB(Rackspace Cloud Files 2008)
Version 2: Fully Distributed(OpenStack Object Storage 2010)
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
30/55
Example Small Scale Deployment
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
31/55
5 Zones2 Proxies per 25
Storage Nodes10 GigE to Proxies
1 GigE to
Storage Nodes24 x 2TB Drivesper Storage Node
Public Internet
Load Balancers (SW)
Example Large Scale Deployment -- Many Configs Possible
Example OpenStackObject Storage Hardware
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
32/55
Building an OpenStack CloudCompute
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
33/55
Asynchronouseventually consistent
communication
ReST-based API
Horizontally andmassively scalable
Hypervisor agnostic:support for Xen ,XenServer, Hyper-V,
KVM, UML and ESX is coming Hardware agnostic:standard hardware, RAID not required
OpenStack Compute Key Features
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
34/55
API: Receives HTTP requests,converts commands to/from APIformat, and sends requests to cloud
controller
Cloud Controller: Global state ofsystem, talks to LDAP, OpenStackObject Storage, and node/storageworkers through a queue
User Manager
ATAoE / iSCSI
Host Machines: workersthat spawn instances
Glance: HTTP + OpenStack ObjectStorage for server imagesOpenStack Compute
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
35/55
System ComponentsAPI Server: Interface module for command and control requestsDesigned to be modular to support multiple APIsIn current release: OpenStack API, EC2 Compatibility ModuleApproved blueprint: Open Cloud Computing Interface (OCCI)
Message Queue: Broker to handle interactions between servicesCurrently based on RabbitMQ
Metadata Storage: ORM Layer using SQLAlchemy for datastoreabstraction
In current release: MySQLIn development: PostgreSQL
User Manager: Directory service to store user identitiesIn current release: OpenLDAP, FakeLDAP (with Redis)
Scheduler: Determines the placement of a new resourcerequested via the API
Modular architecture to allow for optimizationBase schedulers included in Austin: Round-robin, Least busy
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
36/55
System Components (Cont.)Compute Worker: Manage compute hosts through commandsreceived on the Message Queue via the API
Base features: Run, Terminate, Reboot, Attach/DetachVolume, Get Console Output
Network Controller: Manage networking resources on computehosts through commands received on the Message Queue via theAPI
Support for multiple network modelsFixed (Static) IP addressesVLAN zones with NAT
Volume Worker: Interact with iSCSI Targets to manage volumesBase features: Create, Delete, Establish
Image Store: Manage and deploy VM images to host machines
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
37/55
Hypervisor Independence
Cloud applications should be designed and packaged abstracted fromthe hypervisor, deploy and test for best fit for your workloadManage application definition and workload, not the machine image
Configuration managementAbstract virtual machine definition
Open Virtualization Format
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
38/55
Network Models
Private VMs on Project VLANs or Public VMs on flat networks
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
39/55
Network Details
Security Group: Named collection of network access rulesAccess rules specify which incoming network traffic should bedelivered to all VM instances in the groupUsers can modify rules for a group at any time
New rules are automatically enforced for all running
instances and instances launched from then onCloudpipe: Per project VPN tunnel to connect users to the cloudCertificate Authority: Used for Project VPNs and to decryptbundled imagesCloudpipe Image: Based on Linux with OpenVPN
Example OpenStack
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
40/55
Server GroupsDual Quad CoreRAID 10 Drives1 GigE Public1 GigE Private1 GigE Management
Public Network
Private Network(intra data center)
Management
Example OpenStackCompute Hardware
(other models possible)
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
41/55
Example innovation: Simcloud
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
42/55
Questions & Answers
Thank You!
Email: [email protected] Piatt
Twitter: @bpiatt
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
43/55
Backup ContentAdditional Information
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
44/55
Project Technical DocumentationOverall: http://wiki.openstack.orgObject Storage (Swift): http://swift.openstack.orgCompute (Nova): http://nova.openstack.org
Project General DocumentationHome Page: http://openstack.orgAnnouncements: http://openstack.org/blog
OpenStack Documentation
http://openstack.org/bloghttp://openstack.org/bloghttp://openstack.org/http://openstack.org/http://nova.openstack.org/http://swift.openstack.org/http://swift.openstack.org/http://wiki.openstack.org/http://wiki.openstack.org/http://openstack.org/bloghttp://openstack.org/http://nova.openstack.org/http://swift.openstack.org/http://wiki.openstack.org/ -
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
45/55
OpenStack: Core Open Principles
Open Source: All code will be released under the Apache Licenseallowing the community to use it freely.
Open Design: Every 6 months the development community will hold adesign summit to gather requirements and write specifications for theupcoming release.
Open Development: We will maintain a publicly available source coderepository through the entire development process. This will be hostedon Launchpad, the same community used by 100s of projectsincluding the Ubuntu Linux distribution.
Open Community: Our core goal is to produce a healthy, vibrantdevelopment and user community. Most decisions will be made usinga lazy consensus model. All processes will be documented, open andtransparent.
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
46/55
Backup ContentBootstrapping a cloud
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
47/55
Hardware Selection
OpenStack is designed to run on industrystandard hardware, with flexible configurations
Computex86 Server (Hardware Virt. recommended)Storage flexible (Local, SAN, NAS)
Object Storagex86 Server (other architectures possible)Do not deploy with RAID (can use controller for cache)
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
48/55
Server Vendor Support
Find out how much configuration yourhardware vendor can provide
Basic needsBIOS settings
Network bootIP on IPMI card
Advanced supportHost OS installation
Still get management network IP via DHCP
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
49/55
Network Device Configuration
Build in a manner that requires minimal changeLay out addressing in a block based modelGo to L3 from the top of rack uplink
Keep configuration simpleMore bandwidth is better than advanced QoSLet the compute host machines create logical zones
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
50/55
Host Networking
DHCP for the management networkInfinite leasesBase DNS on IP
Ex. nh-pod-a-10-241-61-8.example.orgOpenStack Compute handles IP provisioningfor all guest instances Cloud deployment toolsonly need to setup management IPs
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
51/55
Host OS Seed Installation
BOOTP / TFTP Simple to configureSecurity must be handled outside of TFTPHost node must be able to reach management
system via broadcast requestTop of rack router can be configured to forward
GPXENot all hardware supports
Better concurrent install capability than TFTP
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
52/55
Host OS Installation
Building a configuration based on a scriptedinstallation is better than a monolithicgolden image
Preseed for Ubuntu / Debian hostsKickstart for Fedora / CentOS / RHEL hostsYaST for SUSE / SLES hostsRemote bootstrapping for XenServer / Hyper-V hosts
Scripted configuration allows for incrementalupdates with less effort
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
53/55
Post OS Configuration
Utilize a configuration management solutionPuppet / Chef / Cfengine
Create roles to scale out controller infrastructure
QueueDatabaseController
Automate registration of new host machinesBase the configuration to run on management net IP
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
54/55
Backup ContentCompute
-
8/3/2019 OpenStack : Cloud Computing Platform Tutorial
55/55
Component Architecture Detail