innovation in cloud computing architectures with open nebula
DESCRIPTION
TRANSCRIPT
1/24
Distributed Systems Architecture Research Group Universidad Complutense de Madrid
This presentation is provided under the terms of the a Creative Commons Attribution-Share Alike 3.0 © OpenNebula Project Leads
Innovation in Cloud Computing Architectures with OpenNebula
20 April 2010 5th International Cloud Computing Expo
New York Next Generation Data Center Summit
2/24
Position in the Cloud Ecosystem
What Who On-demand access to any application
End-user (does not care about hw or sw)
Platform for building and delivering web applications
Developer (no managing of the underlying hw & swlayers)
Raw computer infrastructure
System Administrator (complete management of the computer infrastructure)
Software as a Service
ᄎ
Platform as a Service
Infrastructure as a Service
Physical Infrastructure
Innovation in Cloud Computing Architectures with OpenNebula
Innovative open, flexible and scalable technology to configure your own
IT resources into a IaaS cloud
3/24
Commercial Cloud Provider
Innovation in Cloud Computing Architectures with OpenNebula
Transforming your IT Infrastructure into a Cloud
• Flexible and elastic capacity to meet dynamic demands of service
• Ubiquitous network access • Pay per use and on-demand access
Building your Own Cloud
• Optimize and Simplify Internal Operations
• Centralized management of all servers and services with dynamic resizing of infrastructure and dynamic allocation of capacity
• Higher utilization and operational saving of existing resources with server consolidation and removal of application silos
• Lower infrastructure expenses with combination of local and remote Cloud resources
• Support new IT, scientific, or business Cloud services
4/24
Deployment Models
Model Definition Examples of Deployment Pr
ivat
e Infrastructure is owned by a single organization and made available only to the organization
• Optimize and simplify internal operation • SaaS/PaaS support • IT consolidation within large organizations
(Goverment Clouds, University Clouds…)
Publ
ic
Infrastructure is owned by a single organization and made available to other organizations
• Commercial clouds providers • Special purpose clouds with dedicated
capabilities (HPC Clouds…) • Regional clouds to address regulatory or
latency issues • Community public clouds (Science
Clouds…) to enable scientific and educational projects to experiment with cloud computing
Hyb
rid Infrastructure is a
composition of two or more clouds
• Cloudbursting to address peak demands • Cloud Federation to share infrastructure
with partners • Cloud Aggregation to provide a larger
resource infrastructure
Innovation in Cloud Computing Architectures with OpenNebula
5/24
Innovation in Cloud Computing Architectures with OpenNebula
Contents
Building a Cloud Infrastructure OpenNebula as Cloud Enabler
Designing a Cloud Infrastructure Anatomy of a Cloud Architecture
Infrastructure Managers
User and Workloads
System Integrators
Business Managers
Challenges from Deployment and Usage Scenarios
6/24
Innovation in Cloud Computing Architectures with OpenNebula
Challenges: The User Perspective
Profile of Service Workloads • Multi-tier service as basic management entity
• Compute, storage and network capacity • Level of coupling between service instances • Security constraints • Variability of the demand • Elasticity of the services
Service as Groups of VMs • Service components in VMs • Inter-connection relationship • Placement constraints
Required Functionality • Cloud interface and CLI to manage virtual machines, network and storage • Support for popular interfaces (Amazon EC2)
7/24
Cloud Computing from an Institutional Perspective
Challenges: The Manager Perspective
Flexible, Efficient and Scalable Management of the Cloud • Administration interface for the centralized management of the cloud • Support for the definition of workload and resource-aware allocation policies
such as consolidation (energy efficiency), load balancing, affinity-aware, capacity reservation, live migration…
• Support any storage, networking and virtualization infrastructure service • Integration with existing processes and management tools in the data center • Management of several clusters to address different workloads
Physical Infrastructure
Scalable back-end • Virtualization • Storage • Networking
8/24
Innovation in Cloud Computing Architectures with OpenNebula
Challenges: The Business Perspective
Hybrid Cloud Computing and Federation • Cloudbursting at infrastructure layer, fully transparent to users • Scale-out decisions are taken by infrastructure administrators according to
business policies
Two levels of Collaboration • Extend the private cloud using both
partner and commercial clouds • Create a federation of clouds
Nimbus
9/24
Innovation in Cloud Computing Architectures with OpenNebula
Challenges: The Integrator Perspective
Open Architecture, Interfaces and Code • Integration with any product and service in the virtualization/cloud ecosystem
such as cloud providers, hypervisors, virtual image managers, service managers, management tools, schedulers…
• Support to build any type of deployment: private, public, hybrid and community clouds
• Easy to enhance to support new functionality • Easy to embed into other Cloud applications and platforms • Liberal open-source license
10/24
Innovation in Cloud Computing Architectures with OpenNebula
Designing a Cloud: A Design Driven by Requirements
Constraints from Existing Infrastructure and Processes in
the Organization
Requirements from Use Cases and Deployment Scenarios
“One solution does not fit all requirements and constraints, a
properly architectured solution should fully align with your Cloud strategy”
11/24
Designing a Cloud: Anatomy of a Cloud Architecture
Physical Storage Physical Machines Physical Networking
Innovation in Cloud Computing Architectures with OpenNebula
PHYSICAL INFRASTRUCTURE
VIRTUAL INFRASTRUCTURE
Virtual Machine Managers
Clo
ud M
anag
emen
t Sys
tem
Adm
Inte
rfac
e Sc
hedl
uer
Service End-Users Cloud User/ Service Provider
Cloud Interface
Hybrid
12/24
Component Design Parameters Requirements of Use Cases and Constraints from Infrastructure
Physical Machines
• Memory and CPU per server • Number and scalability of servers • Hardware virtualization support
• Memory/CPU requirements of service instances
• Elasticity of services
Physical Network
• Latency/bandwidth per network • Number and scalability of networks
• Level of coupling between service instances
Physical Storage
• Size, scalability, latency and throughout
• Organization: DAS/SAN/NAS
• Live migration • Scalability of the cluster • Performance
Clusters • Number of clusters • Workloads with different execution profiles • Scalability in one cluster
Virtual Machine Manager
• Hypervisors • Overhead in service instances • Licensing
Interfaces • Administration interface • Cloud interface
• Required functionality • Compatibility with existing interfaces
Scheduler • Allocation Policies • Energy efficiency, load balancing, affinity-aware, capacity reservation, live migration…
Hybrid • Cloudbursting • Federation
• Fluctuating demands • Security constraints • Level of coupling between service instances
Designing a Cloud: Design Parameters for Components Innovation in Cloud Computing Architectures with OpenNebula
13/24
Virt. Virt.
Innovation in Cloud Computing Architectures with OpenNebula
OpenNebula Core
Driver API
Compute
OpenNebula API
Virt. Virt. Storage Virt. Virt. Network
Virt. Virt. Cloud
Virt. Virt. Interfaces Virt. Virt. Schedulers
Cloud Manager to Orchestrate Cloud Infrastructure Complexity
Designing a Cloud: Flexible Cloud Manager as Enabler
• Management of network, computing, remote cloud and storage capacity • Management of virtual network, machine and storage life-cycles • Workload placement and management of VM images • Management of information, accounting and security • Interfacing with any infrastructure service
14/24
Innovation in Cloud Computing Architectures with OpenNebula
Building a Cloud: OpenNebula as Cloud Enabler
Open-source Toolkit OpenNebula v1.4
Innovations Technology challenges in cloud computing management from
business use cases
VM
VM
VM
• Open and flexible tool to fit into any datacenter and integrate with any ecosystem component
• Open-source released under Apache v2.0, and distributed in Ubuntu
• Most advanced solution to build private, public, federated and hybrid clouds
• Based on standards avoid vendor lock-in and to enable interoperability
• Efficient and scalable management of the cloud
15/24
Feature Function Internal Interface • Unix-like CLI for fully management of VM life-cycle and resources
• XML-RPC API and libvirt virtualization API
Scheduler • Requirement/rank matchmaker allowing the definition of workload and resource-aware allocation policies
• Support for advance reservation of capacity through Haizea
Virtualization Management • Xen, KVM, and VMware
Image Management • General mechanisms to transfer and clone VM images
Network Management • Definition of isolated virtual networks to interconnect VMs
Service Management and Contextualization
• Support for multi-tier services consisting of groups of inter-connected VMs, and their auto-configuration at boot time
OpenNebula
Admin Scheduler
Physical Infrastructure
Innovation in Cloud Computing Architectures with OpenNebula
Building a Cloud: Features for Private Clouds
16/24
Cloud API
Infrastructure Users
Feature Function
Cloud Interfaces for
Users • Implementation of a subset of the EC2 Query API and OGF - OCCI
Flexibility • The Cloud Service allows the implementation of new Cloud interfaces
Innovation in Cloud Computing Architectures with OpenNebula
Building a Cloud: Features for Public Clouds
17/24
Feature Function
Cloud Plugins • Amazon EC2 and ElasticHosts connectors
Federation • Support for simultaneous access to several remote clouds
Flexibility • Modular approach to develop new connectors
Innovation in Cloud Computing Architectures with OpenNebula
OpenNebula
Admin Scheduler
Physical Infrastructure
Building a Cloud: Features for Hybrid Clouds
18/24
Innovation in Cloud Computing Architectures with OpenNebula
Open Community for Cloud Computing • Haizea Lease Manager (University of Chicago): Advance reservation of
capacity and queuing of best effort requests • Cloud Management Console (SARA Computing and Networking Services):
Web interface for OpenNebula • Virtual Cluster Tool (CRS4 Distributed Computing Group): Atomic virtual
cluster management with versioning and multiple transport protocols. • DeltaCloud Driver (DSA-Research@UCM) • RESERVOIR Policy Engine (IBM Haifa/Elsag Datamat): Policy-driven
probabilistic admission control and dynamic placement optimization to satisfy site level management policies
• VM Consolidation Scheduler (DSA-Research@UCM): Periodic re-placement of VMs for server consolidation and suspension/resume of physical resources
• Claudia (Telefonica I+D): SLA-driven automatic service management • Under Development: SUN Cloud API, vCloud API, VirtualBox plugin, dashboard
for infrastructure management, new schedulers, SLA and security framework, Grid service manager, LVM and SAN support,…
Building a Cloud: Ecosystem
19/24
Innovation in Cloud Computing Architectures with OpenNebula
Users (Different Levels of Use: From Experimental to Production)
Building a Cloud: Users
20/24
Innovation in Cloud Computing Architectures with OpenNebula
Innovative Projects on Cloud Computing
EU grant agreement 215605 Service and Sw Architectures
and Infrastructures (2008-2011)
Proposal in negotiation e-Infrastructure
(2010-2012)
Proposal in negotiation New Infrastructure Paradigms
and Experimental Facilities(2010-2013)
Resources and Services Virtualization without Barriers • Open source technology to enable deployment and
management of complex IT services across different administrative domains
Enhancing Grid Infrastructures with Cloud Computing • Simplify and optimize its use and operation, providing a
more flexible, dynamic computing environment for scientists.
• Enhance existing computing infrastructures with “IaaS” paradigms
Building Service Testbeds on FIRE • Design, build and operate a multi-site cloud-based facility to
support research across applications, services and systems targeting services research community on Future Internet
Building a Cloud: Innovative Projects
21/24
Innovation in Cloud Computing Architectures with OpenNebula
Service Provider
Service Manager
VEE Manager
VEE Host
Service Provider Service Provider
VEE Host VEE Host V
HI
VM
I S
MI
VMI
Telco eGov Utility SAP
www.reservoir-fp7.eu
Commercial Infrastructure Provider
AWS
Commercial Service Managers
Source: RESERVOIR Project
Building a Cloud: Innovative Projects
22/24
IT Resources will be the Next Utility
Innovation in Cloud Computing Architectures with OpenNebula
OpenNebula as Building Block within Future Clouds
• Future enterprise datacenters will look like private Clouds supporting a flexible and agile execution of virtualized services, and combining local with public Cloud-based infrastructure to enable highly scalable hosting environments
• Growing number of domain specific and regional Cloud providers implementing a utility computing business model by offering pay per use resources on-demand
• Public Clouds will be supported by a network of geographically distributed datacenters for high availability, end-user service proximity, legal and policy issues…
• Public Clouds will be interconnected to meet fluctuating demands
• Grid sites will offer infrastructure cloud-like interfaces to address the new resource access demands from the community
23/24
Thanks
The OpenNebula Community
• The OpenNebula Team: Ignacio M. Llorente, Ruben S. Montero, Tino Vazquez, Javier Fontan, Jaime Melis, Carlos Martín, Rafael Moreno, Daniel Molina, and Borja Sotomayor
• … and many value community contributors from several organizations
Your support and contribution are very much appreciated!
Funding Agencies
• European Union: RESERVOIR 2008-2011, EU grant agreement 215605 • Ministry Science&Innovation: HPCcloud 2010-2012, MICINN
TIN2009-07146 • Community of Madrid: MEADIANET 2010-2013 CAM S2009/TIC-1468
24/24
More Information
More info, downloads, mailing lists at
Research References
• B. Rochwerger, J. Caceres, R.S. Montero,D. Breitgand,E. Elmroth,A. Galis,E. Levy,I.M. Llorente,K. Nagin,Y. Wolfsthal, “The RESERVOIR Model and Architecture for Open Federated Cloud Computing”, IBM Systems Journal, Vol. 53, No. 4. (2009)
• B. Sotomayor, R. S. Montero, I. M. Llorente and I. Foster, “Virtual Infrastructure Management in Private and Hybrid Clouds”, IEEE Internet Computing, September/October 2009 (vol. 13 no. 5)