cloud computing lecture 11 private and hybrid clouds · 2020. 4. 20. · •abiquo hybrid cloud...

Cloud Computing – Lecture 11

Private and Hybrid Clouds21 April 2020

Shivananda Poojara

Pelle Jakovits

Satish Srirama

Outline

• Private clouds – Definition and architecture– Necessity and design considerations– Tools: Openstack, Cloudstack,OpenNebula– advantages and disadvantages

• Hybrid Clouds – Definition and architecture – Necessity and design considerations.– Hybrid cloud tools: ManageIQ, scalr– Advantages and disadvantages

• Hybrid cloud case studies using Aneka

Shivananda Poojara 2/34

Private Cloud

– Private clouds are on demand resources managed under single organization hosted in a dedicated physical infrastructure.

– Requires CAPEX and OPEX

– Size: depends on the need.

– Managed inside the organizational firewall

– More control and greater security

– Requires skill set in the organization

– Managed by organization, third party deployed on premise or third party locations.


Essential characteristics of private clouds

• On demand self-service– Automated provisioning

• Broad network access:– Access through multiple devices

• Rapid elasticity :– Resources scaled quickly and transparent manner.

• Measured service– Monitor, control and reporting

• Location-transparent resource pooling for multiple tenants– Sense of location independence

Source: https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-146.pdf


https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-146.pdf

Private cloud architecture

DataSynapse, Zimory Pools, Elastra CloudServer, Aneka, ….

Platform as a Service Solutions

Paa

S

IaaS

IaaS

(M)

Datacenters, Clusters, Desktop Grids

Physical Infrastructure

KVM, Xen, VMWare, …

Virtual Machine Technology

Eucalyptus, OpenNebula, VMWarevCloud, OpenPEX, InterGrid, …

Infrastructure Management Software

Source:buyya.com/masteringcloudcomputing


Necessity of deploying private clouds

• Workload and Infrastructure integration

• Security / Regulation/Compliance

• Latency

• User experience

• Cost


Private cloud design considerations

• Capacity planning – Hardware Considerations– Compute, Storage and network

• Software considerations – COS- Cloud Operating system , Hypervisor.

• Physical location of data canter– On premise, third party location or near to public

clouds

• Implementation process – Design, automate, monitor and test

• Cloud design considerations- HA


Private cloud tools-COS

• A cloud operating system(COS) manages the operation, execution and processes of virtual machines, virtual servers and virtual infrastructure, as well as the back-end hardware and software resources.

• Tools-– Openstack

– Cloudstack

– Eucalyptus

– OpenNebula


OpenStack Private Cloud Platform

• Opensource• Written in : Python• Collection of open-source software tools, components and services,

called OpenStack Services• Service-Service Interaction through public APIs*• provides an Infrastructure-as-a-Service (IaaS) solution• Access:

– via Web-based user interface implemented by Horizon Dashboard– via Command-line client– REST API

• IBM Cloud Orchestrator works with the open source OpenStack software

Shivananda Poojara 9

* Except where privileged administrator commands are necessary

ppt 9-25 prepared by Chinmaya Dehury

CloudStack Private Cloud Platform [6]

• Developed by: Apache software foundation• Open-source cloud computing software

– creating, managing, and deploying infrastructure cloud services

• Written in : Java, Python• Access:

– via Web-based user interface – via Command-line client– REST API

• Support communicating with AWS EC2 and S3, for hybrid deployment.


CloudStack Private Cloud Platform [6]

• Supported Hypervisor:

– KVM-RHEL, LXC, XenServer, HyperV, etc.

• Users:

– Apple, China Telecom, British Telecom, Dell, Disney etc.


OpenNebula

• Open source, enterprise-ready solution to build your Elastic Private Cloud

• Written In: C++, Ruby, Shell Script, etc…

• Two primary uses:

– Data center virtualization solutions

– Cloud infrastructure solutions

• Cloud Management on VMware vCenter


https://github.com/OpenNebula/one

OpenNebula: Architectural Overview


Basic components of an OpenNebula system :• Front-end that executes the OpenNebula services.• Hypervisor-enabled hosts that provide the resources needed by

the VMs.• Datastores that hold the base images of the VMs.• Physical networks provide communication service.



Memory 8 GB

CPU1 CPU (4 cores)

Disk Size 100 GB

Network 2 NICs

• Single OpenNebula instance can manage a maximum of 2,500 servers and 10,000 VMs.

• OpenNebula front-end specification



Storage:• Image datastore: to stores all the images• System datastore: to store the disks of VM’s• File Datastore: a special datastore used to store plain files, not

disk images. • Default database: sqlite



Networking• At least two different physical networks are needed

• Service Network: used by front-end daemons• Instance Network: offers network connectivity to the

VMs across the different hosts.

Private cloud- Advantages

• Cost – TCO will be less for prolonged workloads

• Efficiency and control– Managed and tuned in on premise and have higher control

• Customize – Customize based on your business requirements and full

flexibility

• Security and privacy– Within company’s firewall and greater security

• Compliance– National and international laws and policies . Ex- data protection

• Location independence


Private cloud- Disadvantages

• Cloud bursting, scalability issues

– Ability to handle unpredicted workloads or new workloads

• Expertise

– Skillset and other logistics

• Scaling of hardware

• Initial investments and future pricing

– Higher CAPEX


Hybrid Clouds


Hybrid clouds

• Hybrid clouds is combination of on premise infrastructure with off premise resource or public clouds such as AWS, IBM,GCP or Microsoft Azure.

• Cloud bursting, Shrink by provisioning external resources when required and unleased when not needed.

• Perspective and people:

• Flexibility• Speed• Capacity• Consistent Agility

• Cost• SLA• Easy invoicing• Software leciening

• Visibility• Control• Manage• Access• Monitor

Hybrid Cloud

Solution designer team Infra TeamBusiness Team


Necessity of hybrid cloud

• Use Case 1: Untested workloads with uncertain demand.

• Use case 2: Cloud bursting

• Use 3: Disaster recovery and high availability

Private CloudPublic Cloud

Database servers

AS1 AS2

AS: Application Server

Load balancer

End point

AS3 AS4


Hybrid cloud architecture

Desktop Grids / NOWs

Private Cloud

DynamicProvisioning

Datacenter Cluster Desktop Grids / NOWs

Public Cloud

Enterprise dad center

Applications

PaaS

IaaS Middleware

Ref: Rajkumar et.el Mastering Cloud Coputing

Third party services


Hybrid Cloud: VmWare and Microsoft Azure Integration

ESXi Hosts

Storage Network

VM VM VM VM

Vmware vCenter

On Premise Cloud

ESXi Hosts

Storage Network

VM VM VM VM

Vmware vCenter

Azure bare metal servers

Azure Portal

Azure Management

Azure services

Azure Integration service

Source: https://www.harbourit.com.au/blog/everything-you-need-to-know-about-hybrid-cloud/


https://www.harbourit.com.au/blog/everything-you-need-to-know-about-hybrid-cloud/

Key Strategies for Hybrid Cloud Success

• The cloud operating system that gives you the ability to monitor, manage, and orchestrate across all environments using a single set of tools.

• Choosing a framework a cloud Operating systems like opens tack, cloud stack that will allow to manage workloads on-premises and in the cloud.

• Modernize your on-premises environments in accordance with that frame‐ work.

• Choose only public clouds and CSPs compatible with that framework.

Sourcehttps://cdw-prod.adobecqms.net/content/dam/cdw/on-domain-cdw/brands/nutanix/o-reilly-ebook-designing-and-building-a-hybrid-cloud.pdf


https://cdw-prod.adobecqms.net/content/dam/cdw/on-domain-cdw/brands/nutanix/o-reilly-ebook-designing-and-building-a-hybrid-cloud.pdfhttps://cdw-prod.adobecqms.net/content/dam/cdw/on-domain-cdw/brands/nutanix/o-reilly-ebook-designing-and-building-a-hybrid-cloud.pdf

Capabilities required -hybrid cloud

• Integration: Integration of data and processes needs to be seamless across the deployments,

• Data localization: Security and compliance• Operational visibility and management: Unified

management • Security services: Policies and business rules are needed to

understand and monitor information about workloads • DevOps: Single unified way to manage and synchronize

applications• Integration services: Approaches that allows application

code to be easily moved from one environment to another.

Source: https://www.ibm.com/downloads/cas/WEWZGM5J


https://www.ibm.com/downloads/cas/WEWZGM5J

Possible Workloads in hybrid cloud

• Workload- is an independent service or collection of code that can be executed.

• Workloads need to be executed based on: Criticality, Level of security, interaction with other environments.

• Types of workloads– Batch - customer bills and account statements

– Analytic – high volume, complexity, and importance

– Transactional- automation of business processes

– Database- high amount of Input/Out (I/O) cycles

Source: https://www.ibm.com/downloads/cas/WEWZGM5J


https://www.ibm.com/downloads/cas/WEWZGM5J

Assessing the needs of hybrid cloud

• Assessing the current state of operations– Identify the current IT infra locations.– Identify the status of infra like utilization, cost, physical space.– Identify the current cloud provider impact on your business.– Identify workload characteristics of your business.

• Assessing the future workloads:– Identify new applications, new products planning in a year.– Planning any updates for business applications.– Prioritize the workloads and assess type of application, resource

requirements, data locations.

• Create decision matrix – Decision matrix can help to determine where to locate each

workload going forward and how to assess anticipated costs.


Use cases

• Health care sector : Security and compliance in storing and movement of health data.

• Finance sector: Flexibility, power, scale, and seamless connectivity.

• Government: Use cloud email and collaboration tools and a mixture of public and private cloud infrastructure.

• Retail: Always in (99.999%) uptime support in ecommerce operations.

• ecommerce: Enormous web traffic and scalability.

Source: https://www.harbourit.com.au/blog/everything-you-need-to-know-about-hybrid-cloud/


https://www.harbourit.com.au/blog/everything-you-need-to-know-about-hybrid-cloud/

Hybrid cloud offerings and tools

• Microsoft Azure – Azure public cloud and on-premise Azure

Stack

• Amazon Web Services– AWS Outposts

• VMWare

• Google Cloud Platform

• RackConnect

• VMware-based private cloud

• Dell EMC

• IBM Cloud

• CloudBolt

• Cisco Cloud Center

• DivvyCloud

• RedHat Hybrid Cloud

• Abiquo Hybrid Cloud

• Cloudability

• Scalr


ManageIQ

• Its an open source project started in 2014 and managed by Redhat Inc.

• Other companies such as Xlab, Google,Lenovoare also part of the community.

• 43965+ commits, 256+ developers.

• Discover, Optimize, and Control hybrid cloud resources

• Manage containers, virtual machines, networks, and storage from a single platform


ManageIQ

It has the following feature sets:

• Insight: Discovery, Monitoring, Utilization, Performance, Reporting, Analytics, Chargeback, and Trending.

• Control: Security, Compliance, Alerting, Policy-Based Resource and Configuration Management.

• Automate: IT Process, Task and Event, Provisioning, Workload Management and Orchestration.

• Integrate: Systems Management, Tools and Processes, Event Consoles, CMDB, RBA, and Web Services.


High level architecture


Features: Self service delivery

• Service delivery catalog ,Multiple service at one time ,Approval routing


Features: Automated provisioning

• Automatic configuration and deploy

• Ansible playbooks or native coding

• Integration to external IT systems


Features: Transparent Operational Watch

• Continues discovery

• Relationship view

• Monitoring and tracking

• In depth analysis

• Analytics


Life cycle management

• Ongoing tracking, Scheduled retirement and archiving, root cause analysis


Other features

• Root cause analysis

• Performance and capacity management

• Policy enforcement

• Quotas and chargeback

• Virtualization management

• Container management


Scalr

• Its also hybrid cloud management software for enterprises

• It has highly efficient role and policy based control for security and cost management.

• It also have self service portal, power console, Service catalog, policy engine, multiple service provider support .

• Varity of deployment support such as HA,scaling


Advantages of hybrid clouds

• Flexibility : The ability to distribute workloads across public and private environments based on security, efficiency, and cost.

• Scalability: Dynamically the resources based on the demand spike and release when not required.

• Reliability: Less possible downtime even when demand spikes.

• Security: Specific workload distribution of sensitive data on private clouds and non sensitive data on public clouds.

• Affordability: Need not to purchase and manage new resources to handle short spike in demand.


Advantages of hybrid clouds

• Continuity - Business continuity is not interrupted during a failure or disaster, as data is still accessible with little or no downtime.

• Opportunity: New prototypes or new applications can be tested and deployed rapidly.

• Accessibility: Any time, any where


Disadvantages of hybrid cloud

• Possibility of vendor lock in

• Service integration issues

• Interoperability issues

• Transparency

• Security and privacy issues

• Data locality issues


Benefits of hybrid clouds

• Flexibility and agility

• Elasticity

• Self-service

• Faster delivery of new products and services

• Cost control

• Avoidance of lock-in

• Access to the latest technology


Hybrid cloud- Case study with Aneka


Hybrid cloud research challenges

• Resource management and scheduling in hybrid clouds

– Metrics: QoS, SLA, Failure, profit

• Interoperability challenges

• Data replication and sharing

• Security and privacy

• Disaster recovery and high availability


Aneka – Tool for Cloud Computing research

• Aneka is a .net based cloud application development platform developed by Manjrasoft Pty Ltd Melbourne Australia(Spin-off company of University of Melbourne)

• It supports deployment of three types of application

– High Throughput Computing(Task programming API)

– High Performance Computing (Thread Programming API)

– Data intensive computing(Map Reduce Programming API)

• Aneka can de deployed in heterogeneous infrastructure

– Private Clouds, Hybrid Cloud, Local desktop clutters, Virtualization clusters

• Develop custom resource scheduling algorithms

• It has ability to connect to multicloud environments

– AWS, Azure, openstack


High level view of Aneka


Aneka API’s and its working

• Aneka Thread API

• Aneka Task API

• Aneka Map-Reduce API


Aneka job submission

public DumbTask: ITask

{

…

public void Execute()

{

……

}

}

for(int i=0; i

Aneka support for hybrid cloud

• XenServer Pool

– Provisioning over private Cloud managed by Xen Server

• VMWare Pool

– Provisioning over private Cloud managed by VMWare

• Amazon EC2 Pool

– Provisioning over public Cloud provider: Amazon EC2

Shivananda Poojara

Executors/Schedulers

Executor

Client Libraries

Executors

Scheduler

Public Cloud

publicly available resources(physical and virtual)

Scheduler

internet

Private Cloud

private enterprise network

VPN(virtual resources)

Executor

49/34

Hybrid Cloud Case study: Aneka’s dynamic provisioning with Amazon EC2

Source:buyya.com

Use Case : Scheduling problem based on cost and time


Hybrid Cloud Case study: Aneka’s dynamic provisioning with Amazon EC2

Scheduling Service

Master Node

Tasks +

QoSProvisioning

Service

Public Clouds

Private Clouds

Static Resources

Worker Nodes

Dynamic Resources

Deadline+

Budget

Deadline Execution time Cost

80 min 70 min U$ 0,06

70 min 60 min U$ 0,12

60 min 55 min U$ 0,15

50 min 50 min U$ 0,15

40 min 35 min U$ 0,30

AWS Spot Instances,Small

Source:buyya.com


Practical session

• We will work Aneka programming in the lab

– Aneka deployed in open stack cluster consisting of 3 VM’s.

– You will work Aneka Task and Thread API.

– Focus of lab is to deploy application on anekacluster such as image processing and some custom thread programming application.

– Bit knowledge on .net but not compulsory.


References

• ppt 9-25 prepared by Chinmaya kumar Dehury

• https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-146.pdf

• www.buyya.com

• https://www.harbourit.com.au/blog/everything-you-need-to-know-about-hybrid-cloud/

• Buyya, R., Srirama, S. N., Casale, G., Calheiros, R., Simmhan, Y., Varghese, B., ... & Toosi, A. N. (2018). A manifesto for future generation cloud computing: Research directions for the next decade. ACM computing surveys (CSUR), 51(5), 1-38.

http://www.buyya.com/

cloud computing lecture 11 private and hybrid clouds · 2020. 4. 20. · •abiquo hybrid cloud...

Documents