Diane MuellerOpenShift Origin Community ManagerJanuary 2014OpenStack Meetup Seattle

Deploying OpenShiftOn OpenStack

@pythondj

noun \p-thn, -thn\+\d-j\

Pythona widely used general-purpose, high-level programming ............languageSnakea very large snake that kills the animals it eats by wrapping itself around them

+ short for Djangoa high-level Python Web framework that encourages rapid development & clean design+ Disk Jockeya person who plays popular recorded music on the radio or at a party or nightclub

Red Hat's Cloud Ecosystem Evangelist

A Snake Charmer

Agenda

Learn a little about PaaS & OpenShift

Learn a little about underlying PaaS architecture

Learn how to deploy OpenShift on OpenStack

Talk about the Future of PaaS on OpenStack

Find out where to learn more

origin

Public Cloud ServiceOn-premise or Private Cloud SoftwareOpen Source Project

FLAVORS OF OPENSHIFT

What if you could use existing infractructure (bare metal or virtualized), private cloud, and public cloud in one, unified environment?

Open hybrid cloud lets you take advantage of your existing resources, and be open to new advances. Build a cloud infrastructure thats easy to integrate, agile, and future-proof.

UNIVERSAL (UNIFIED?) ENVIRONMENT: An environment that is ready any time, all the time. Because its built with technology that is standardized, interoperable, and modular.

This allows you to reuse the infrastructure you have to build more agile enterprise I.T.

OpenStack

OpenShiftPutting the PaaS in OpenStack

Cross Community Collaboration

A bit about Heat: The Heat API implements the AWS Cloud Formations API. This API provides a rest interface for creating composite VMs called Stacks from template files. The goal of the software is to be able to accurately launch AWS CloudFormation Stacks on OpenStack. We will also enable good quality high availability based upon the technologies we created in Pacemaker Cloud including escalation.

http://www.ohloh.net/

Why I love PaaS: It's Magic

Infrastructure LayerSaaS/Applications LayerInfrastructure LayerInfrastructure Layer

IBM 1401 the year I was born http://www.computerhistory.org/

First Job http://www.computerhistory.org/

SOFTWARE IS EATING THE WORLD.

Circa 1989 Predator vs. Prey

1989Prey: Meg Ryan boutique book store Predator: Tom Hanks - Fox as a Stand in for Barnes & Noble

That's Disruptive Technology at Work

http://www.theguardian.com/media-network/media-network-blog/2013/nov/18/werner-vogels-cloud-trends-amazon

4 trends according to Werner

Cloud will enable your content to follow you wherever you goCloud has changed how we interact with mobile devices. In the past content would be moved to the device, now devices are just a window to content and services that live in the cloud. This started with our smartphones and tablets, where regardless of which device we use, or the location, we have access to our content and subscriptions.

Now this approach is migrating to non-mobile devices such as Samsung smart TV's. The devices are beautifully designed and beautifully built, but the core functionally of these television sets is software connected to services running in the cloud.

This is also moving beyond traditional devices, for example my car is already connected to my Amazon Cloudplayer giving me music everywhere I go. I have seen the first treadmills where the moment I step on them they reconfigure to give access to my music and videos, my newspaper subscriptions and books, but also my documents in services like Dropbox. I no longer need to bring my content; cloud enables my content to follow me wherever I go.

Cloud based analytics enhances the offline worldThe cloud is already the place where researchers collaborate on data that flows in real-time from devices such as the Mars rover or the Ilumina DNA sequencer into cloud storage. In 2014 expect an explosion in data generation by real-world devices and where that data is stored, analysed and shared in the cloud.

For example we will see a rise in the industrial cloud where industrial environments are equipped with sensors producing data to improve efficiency and reliability. An example is the project we run with GE on instrumenting their gas turbines or with Shell where they are going to drop sensors in their oil wells that generate petabytes of data.

Also in our daily lives we will see the rise of cloud connected sensors and devices such as the Nest Thermostat or the home control applications built by energy companies like Essent. Around the world public transport companies are instrumenting their busses and trams with sensors that feed into platforms like OneBusAway that can give real-time updates to travellers.

Passengers themselves can also become sensors: services like Mooveit use the information from an application on passengers' phones to give real time transport information in the same way that Waze does for cars.

The cloud allows everyone to become a media companyIn 2014 expect a great rise in organisations that are adding media capabilities to their offerings. A good example is sports clubs; all are looking for ways to establish an engagement with their fan base beyond the two hours on a weekend. A successful way to achieve a weeklong engagement is by daily distribution or fresh, exclusive media content. The subscription revenues for clubs that often have millions of fans around the world are substantial.

Cloud based services for pre and post production, as well as distribution, are readily available such that anyone can become an internet broadcaster operating worldwide without any capital investment. A well-known case is that of the Amazon Web Services (AWS) powered LiverpoolTV, but every football club worldwide is following their example.

Another very popular case is that of performing arts organisations, from orchestras to theatre companies, which give exclusive access to their performances through cloud-based media production. This way they are able to reach a much larger audience, which would often not be able to attend their performances in person. It extends their revenue potential, which is needed in times where arts subsidies are disappearing. A good example is Berliner Philharmoniker, the world famous orchestra that gives access to their live performances through the digitalconcerthall.com that makes use of all AWS regions around the world to provide a high quality media experience.

Faster and faster, cloud moves data processing to real-timeUp until this point big data has very much focused on looking historically - people who brought product X also brought product Y, the market moved in this direction last week so is likely to move in that direction now. There has always been a close relationship between big data and cloud computing as it requires no limits in terms of compute and storage but by adding real-time processing capabilities, we will see a rise in data analytics that are able to produce results for in real-time, radically changing the products companies can build.

For example we see companies with real time recommendations, in the form of 'other people in your network are reading X'. Some of the frontrunners here are the companies working on second-screen technologies, such as Channel 4, that make use of real-time data to power the information they present to augment TV watching. A company like Netflix that processes over 40bn events a day uses real-time analytics to power their operations, their customer engagement and their business metrics.

We see almost every industry taking advantage of the cloud to radically improve the speed at which they can process their data.

Werner Vogels is vice president and CTO at Amazon.com

App Development Got Complicated

How to Build an App
Old School

For either the Entrepreneurial or the Enterprise Developer, PaaS is the way of the future.

Let's take a quick look at the before-and-after of the application development process.

In the old days, when you wanted to build a new app (or were assigned a project to build a new app), you had to jump through a million hoops to get it up and running. Everything from ordering hardware, to installing middleware, to tuning and testing every facet of the development environment.

With PaaS, life is much easier. You have an idea for an application? You just start writing the code and let OpenShift PaaS handle the rest.

Write your code, Push to OpenShift, Test with Jenkins, and Deploy when ready!

Even Gartner knows that PaaS will be the way of the future for application development.

Unhappy Developers

Then along came Cloud

Developer Expectations

Platform Flexibility

Application Portability

Increased Productivity

Infrastructure is not enough

Infrastructure as a Service gives you

Network, storage & compute as an on-demand service

Basically, servers in the cloud

Youre still on the hook to configure & manage the cloud & stack

How do I use this?

So, what you need is the ease of use and access of a SaaS application, but you need it with your purpose-built, mission-critical, applications.

PaaS gives you just that. It allows you to quickly and easily build the application that YOU need. Whether this is for your group, your enterprise, or your next BIG IDEA, you can build it and launch your specific code on a PaaS and not have to deal with the underlying infrastructure, middleware, and management headaches.

Because of the built-in auto-scaling and elasticity provided by the PaaS infrastructure, PaaS's are ideal for modern data-hungry Big Data, Mobile, and Social applications.

With a PaaS, you can focus on what you should be focused on... your application code.

And let the Cloud provide what it is suppose to: Ease, Scale and Power

Software as a Service gives you

An on-demand application

Nothing to install or configure

This is all my customers and users care about!

So, what you need is the ease of use and access of a SaaS application, but you need it with your purpose-built, mission-critical, applications.

PaaS gives you just that. It allows you to quickly and easily build the application that YOU need. Whether this is for your group, your enterprise, or your next BIG IDEA, you can build it and launch your specific code on a PaaS and not have to deal with the underlying infrastructure, middleware, and management headaches.

Because of the built-in auto-scaling and elasticity provided by the PaaS infrastructure, PaaS's are ideal for modern data-hungry Big Data, Mobile, and Social applications.

With a PaaS, you can focus on what you should be focused on... your application code.

And let the Cloud provide what it is suppose to: Ease, Scale and Power

Platform as a Service delivers

Application run-time environment in the cloud

Configures & manages both the cloud & stack for your application

The cloud is now useful!

So, what you need is the ease of use and access of a SaaS application, but you need it with your purpose-built, mission-critical, applications.

PaaS gives you just that. It allows you to quickly and easily build the application that YOU need. Whether this is for your group, your enterprise, or your next BIG IDEA, you can build it and launch your specific code on a PaaS and not have to deal with the underlying infrastructure, middleware, and management headaches.

Because of the built-in auto-scaling and elasticity provided by the PaaS infrastructure, PaaS's are ideal for modern data-hungry Big Data, Mobile, and Social applications.

With a PaaS, you can focus on what you should be focused on... your application code.

And let the Cloud provide what it is suppose to: Ease, Scale and Power

Infrastructure LayerSaaS/Applications LayerInfrastructure LayerInfrastructure Layer

Key Terms for Developers

Applications your web code and any data store. Has to be on 1 or more gears

Gears a resource-constrained container that runs one or more user-specified software stacks, also known as cartridges. Each gear has a limited amount of RAM and disk space. If an application needs more resources, it can be configured to automatically use more gears.

Cartridges A technology/framework (PHP, Perl, Java/JEE, Ruby, Python, MySQL, etc.) to build applications

OpenShift Online (http://openshift.com)

OpenShift Origin on Github

http://openshift.github.io/

OpenShift Enterprise

ENTERPRISE IT MANAGEDLet developers have the benefits of self-service PaaS while IT retains control for governance and compliance in a Private or Hybrid Cloud

OpenShift Enterprise is designed to be deployed on top of and run on Red Hat Enterprise Linux (RHEL).

What makes OpenShift different?

RHEL Platform Support

SELinux-based Secure Containers for multi-tenancy

Extensible Architecture

System Component Redundancy for High Availability

Automatic Application Stack Provisioning

Support for Java EE 6

Choice of Cloud Infrastructure, Bare Metal, or Desktop

Configurable Deployment to Support Enterprise Requirements

Automatic Application Scaling

Empowering Developersto test, launch, fail, iterate, scale

RAPIDLY

https://install.openshift.com/

Deploying OpenShift

Origin Release 3

Fedora 19 or RHEL 6.x or CentOS 6.5

Get up and running

Vagrant

Puppet

Comprehensive guide

Ansible

Heat

http://openshift.github.io

Amazon EC2

Rackspace

Bare Metal

OpenStack

RHEV

CloudStack

OpenShift PaaS

OpenShift

Runs on IaaS

Platform agnosticSELinux

OpenShift Machines

Node

Broker

Broker

Broker

Node

Nodes are where User applications live.

An OpenShift Broker can manage multiple nodes

HIgh level - 2 types of machines

OpenShift Architecture

Broker

OpenShift Machines

Node

Broker

Broker

Broker

Node

Nodes are where User applications live.

An OpenShift Broker can manage multiple nodes

Applications and Gears

Gear

Node

Gear

Jboss

MariaDB

Jboss

Gear

Node

Gear

User application

Key OpenShift Terms

Broker Management host, orchestration of Nodes

Node Compute host containing Gears

Gear Allocation of fixed memory, compute, and storage resources for running applications

Cartridge A technology/framework (PHP, Perl, Java/JEE, Ruby, Python, MySQL, etc.) to build applications

Architecture

https://install.openshift.com/

Deploying on OpenStack with Heat

Heat Overview

Entering OpenStack Integrated status in November 2013Active code base3048 commits as of September 2013

56 contributors

Cross Project functionality with OpenStack projects Keystone, Nova, Neutron, Cinder, Ceilometer, Swift, Glance, Horizon, TripleO and Tempest

OpenStack Heat provides application autoscaling today with a stable workflow model

OpenShift on OpenStack OpenShift Enterprise Templates for RHEL ready

OpenShift Origin Templates for CentOS & Fedora in progress

Heat's Mission: Orchestration

To explicitly model the relationships between OpenStack resources of all kinds; and to harness those models, expressed in forms accessible to both humans and machines, to manage infrastructure resources throughout the life-cycle of applications.

Heat Overview

Nova

ComputeNodeGlance

ImageServiceSwift

ObjectStoreCinder

VolumeServiceKeystone

IdentityService

HorizonDashboardQuantum

NetworkingHeatOrchestrationProvides AWS Cloudformation and native ReST API

Abstract configuration of services to single-template

HA/Autoscaling/Monitoring features

Openstack integrated project

KEY POINTS Heat provides a way of defining all your openstack resources / services from a single template

Ties together the underlying IAAS services, so you can provide a consolidated infrastructure service, single integration point

Compatible with AWS Cloudformation (templates and API)

Has Autoscaling and HA features

Recently became an incubated project

OpenStack Heat Architecture

Heat API

KEY POINTSLifecycle operationsCreate creates a stack from a template

Delete deletes an existing instantiated stack

Update updates an existing instantiated stack

Introspection operations List obtains a list of existing stacks

Describe obtains details of existing stacks

Events List obtains events that have occurred for a stack

List resources lists resources associated with a stack

Template describes a stack

OpenStack Heat Engine Architecture

Autoscaling

Metrics or user events drive scaling

Metrics can include CPU utilization, memory utilization, many more as well as custom dimensions

Dynamically add and reduce OS::Nova::Server resources to meet demand

Front end Neutron LBAAS or Heat provided HA-Proxy Load Balancer distributes load to server resources

Autoscaling Workflow Internal View

User instantiates template with Heat's CLI

Heat registers with Ceilometer for callbacks on Alarm events

Ceilometer tells Heat about Alarm events and Heat scales a Group based upon a Policy decision to scale up or down

OS::Nova::Server instances can also call Alarms internally

OpenShift on OpenStack Autoscaling Workflow

http://github.com/openstack/heat-templates

Reasons to precreate images:

Performance of scaling availability

Reliability of not loading at runtime image contents

Demo

Deploying OpenShift Enterprise With Heat Templates OnOpenStack

Learn more about Heat & OpenShift:

Users, testers and developers wanted!Connect via IRC on #openshift-dev@freenode

Check out the repositories:https://github.com/openstack/heathttps://github.com/openstack/heat-templateshttps://github.com/openshift

Read the Heat Documentation:http://docs.openstack.org/developer/heat

Read the OpenShift Documentation:http://openshift.github.io

Future of PaaS

https://github.com/openshift/openshift-pep/blob/master/openshift-pep-010-docker-cartridges.mdhttp://docs.docker.io/en/latest/installation/kernel/

Future of PaaS on OpenStack

Solum initiativealso http://solum.io

Current effortsConnecting Git to SolumVia project zuul (current OS tool for testing infrastructure)

Building Images that can be runDocker (normal image w/ docker pre-installed)

Normal Nova Images

OpenShift
AutoScaling Workflow

Step 1
Create DIB elements
Elements directory structure

elements/openshift-origin-broker:-rw-rw-r--. 1 sdake sdake 37 Jun 2 12:14 element-depsdrwxrwxr-x. 2 sdake sdake 4096 Jun 2 12:14 install.d-rw-rw-r--. 1 sdake sdake 176 Jun 2 12:14 README.md

elements/openshift-origin-broker/install.d:-rwxrwxr-x. 1 sdake sdake 1598 Jun 2 12:14 30-openshift-origin-broker

elements/openshift-origin-node:-rw-rw-r--. 1 sdake sdake 37 Jun 2 12:14 element-depsdrwxrwxr-x. 2 sdake sdake 4096 Jun 2 12:14 install.d-rw-rw-r--. 1 sdake sdake 172 Jun 2 12:14 README.md

elements/openshift-origin-node/install.d:-rwxrwxr-x. 1 sdake sdake 1610 Jun 2 12:14 30-openshift-origin-node

elements/openshift-origin-repos:-rw-rw-r--. 1 sdake sdake 23 Jun 2 12:14 element-depsdrwxrwxr-x. 2 sdake sdake 4096 Jun 2 12:14 pre-install.d-rw-rw-r--. 1 sdake sdake 176 Jun 2 12:14 README.md

elements/openshift-origin-repos/pre-install.d:-rwxrwxr-x. 1 sdake sdake 286 Jun 2 12:14 29-puppetlabs-release-rwxrwxr-x. 1 sdake sdake 648 Jun 2 12:14 30-openshift-origin-repos

OpenShift Autoscaling Workflow
Step 1: Create DIB Elements
Building the broker image

[sdake@freedom openshift-origin-broker]$ more element-depsopenshift-origin-repos

[sdake@freedom openshift-origin-repos]$ ls -l pre-install.d-rwxrwxr-x. 1 sdake sdake 286 Jun 2 12:14 29-puppetlabs-release-rwxrwxr-x. 1 sdake sdake 648 Jun 2 12:14 30-openshift-origin-repos

Part 1: Parse Dependencies

Part 2: Load Dependencies

Part 3: Configure Broker

[sdake@freedom openshift-origin-broker]$ ls -l install.d-rwxrwxr-x. 1 sdake sdake 1598 Jun 2 12:14 30-openshift-origin-broker

OpenShift Autoscaling Workflow
Step 1: Create DIB elements
Contents of 30-openshift-origin-broker

[sdake@freedom install.d]$ more 30-openshift-origin-broker#!/bin/bash

set -uex

install-packages \ openshift-origin-broker \ rubygem-openshift-origin-msg-broker-mcollective \ rubygem-openshift-origin-dns-nsupdate \ rubygem-openshift-origin-dns-bind \ rubygem-openshift-origin-controller \ openshift-origin-broker-util \ rubygem-passenger \ mod_passenger \ openssh \ rubygem-openshift-origin-auth-mongo \ rubygem-openshift-origin-remote-user \ rubygem-openshift-origin-console \ openshift-origin-console \ mongodb \ mongodb-server \ bind \ bind-utils \ ntpdate \ policycoreutils \ mcollective \ httpd \ openssh-server \ rhc \ activemq \ activemq-client \ git \ puppet \ ruby \ ruby-devel \ ruby-irb \

ruby-libs \ tar \ yum-plugin-priorities \ mysql-devel \ mongodb-devel \ system-config-firewall-base \ rubygem-execjs \ rubygem-uglifier \ rubygem-listen \ rubygem-sass \ rubygem-sass-rails \ autogen-libopts \ ntp \ rubygem-coffee-script-source \ rubygem-coffee-script \ rubygem-coffee-rails \ rubygem-idn \ rubygem-addressable \ rubygem-crack \ rubygem-webmock \ rubygem-fakefs \ rubygem-chunky_png \ rubygem-hpricot \ rubygem-haml \ rubygem-fssm \ rubygem-compass \ rubygem-compass-rails \ rubygem-mongo \ rubygem-jquery-rails \ rubygem-openshift-origin-dns-avahi \ rubygem-ref \ rubygem-therubyracer

sed --in-place -e \ s/Type=oneshot/"Type=oneshot\nTimeoutSec=0"/ /lib/systemd/system/cloud-final.service

OpenShift Autoscaling Workflow
Step 2: Create Heat Template - Policy

resources: OpenshiftUser: Type: AWS::IAM::User OpenshiftOriginKeys: Type: AWS::IAM::AccessKey Properties: UserName: Ref: OpenshiftUser OpenshiftOriginNodeGroup: Type: AWS::AutoScaling::AutoScalingGroup DependsOn: BrokerWaitCondition Properties: AvailabilityZones: [] LaunchConfigurationName: Ref: NodeLaunchConfig MinSize: Ref: NodeCountMinimum MaxSize: Ref: NodeCountMaximum LoadBalancerNames: [] OpenshiftOriginScaleUpPolicy: Type: AWS::AutoScaling::ScalingPolicy Properties: AdjustmentType: ChangeInCapacity AutoScalingGroupName: Ref: OpenshiftOriginNodeGroup Cooldown: '120' ScalingAdjustment: '1'

OpenshiftOriginScaleDownPolicy: Type: AWS::AutoScaling::ScalingPolicy Properties: AdjustmentType: ChangeInCapacity AutoScalingGroupName: Ref: OpenshiftOriginNodeGroup Cooldown: '60' ScalingAdjustment: '-1'

AlarmPolicyGroup

OpenShift Autoscaling Workflow
Step 2: Create Heat Template - Alarms

NodeScaleUp: Type: AWS::CloudWatch::Alarm Properties: AlarmDescription: Scale-up if event received from broker MetricName: Heartbeat Namespace: system/linux Statistic: SampleCount Period: '60' EvaluationPeriods: '1' Threshold: '0' AlarmActions: [{Ref: OpenshiftOriginScaleUpPolicy}] Dimensions: - Name: AutoScalingGroupName Value: Ref: OpenshiftOriginNodeGroup ComparisonOperator: GreaterThanThreshold

AlarmPolicyGroup

NodeScaleDown: Type: AWS::CloudWatch::Alarm Properties: AlarmDescription: Scale-down if event received from broker MetricName: Heartbeat Namespace: system/linux Statistic: SampleCount Period: '60' EvaluationPeriods: '1' Threshold: '0' AlarmActions: [{Ref: OpenshiftOriginScaleDownPolicy}] Dimensions: - Name: AutoScalingGroupName Value: Ref: OpenshiftOriginNodeGroup ComparisonOperator: GreaterThanThreshold

OpenShift Autoscaling Workflow
Step 2:Create Heat Template
Optionally Trigger Alarms

cat /etc/heat/notify-scale-up#!/bin/bash/opt/aws/bin/cfn-push-stats --credential-file /etc/heat/heat-credentials --heartbeat watch {Ref: NodeScaleUp}EOFchmod 0700 /etc/heat/notify-scale-upcat /etc/heat/notify-scale-down#!/bin/bash/opt/aws/bin/cfn-push-stats --credential-file /etc/heat/heat-credentials --heartbeat --watch{Ref: NodeScaleDown}

AlarmPolicyGroup

UserData commands:

OpenShift Autoscaling Workflow
Step 3: Register images with glance

[sdake@freedom heat-templates] glance image-create
name=openshift-origin-broker --disk-format=qcow2 --container-format=bare < openshift-origin-broker.qcow2

[sdake@freedom heat-templates] glance image-create
name=openshift-origin-node --disk-format=qcow2 --container-format=bare < openshift-origin-node.qcow2

Autoscaling Lifecycle Example
Step 4: Launch Heat Template

[sdake@freedom heat-templates] heat create
template-file OpenShiftAutoScaling.yaml parameters KeyName=sdake;Prefix=broked.org;NodeCountMaximum=20

Click to edit the title text format