fuel, puppet and openstack

www.mirantis.com / fuel-infra.org

A Technical Overview

Fuel, Puppet and OpenStack

March 2, 2015

Introductions

Christopher Aedo — Product Architect

Christopher is an IT veteran for consulting, design and technology companies. He is also an outspoken public advocate for OpenStack, cloud computing, software defined networking and software defined storage. He often speaks at OpenStack and Open Source related conferences around the world.

About MirantisMirantis is the number one pure play OpenStack Company. We deliver all the technology, integration, training and support required for companies to succeed with production-grade open source cloud. More customers rely on Mirantis than any other company to scale out OpenStack without the compromises of vendor lock-in. Our bench of 400+ open source infrastructure experts helped make us one of top 5 contributors to OpenStack’s upstream codebase.

Agenda

● Deploying OpenStack● Fuel Overview & Architecture● Beginning, challenges, Improvements● Where Fuel is Headed● Q&A

Deploying OpenStack

● DevStack● Deployment tools

● Fuel, Crowbar, Foreman, Helion

● Config/Orchestration● Puppet, Chef, Ansible, Salt

● Distro-based tools● Packstack, Juju, RHEL OSP

Deploying OpenStack is easy!

What is Fuel?

● OpenStack deployment and management tool● Open source, growing community● GUI and CLI driven● Visit http://fuel-infra.org to learn more and download

http://fuel-infra.org

Fuel Overview & Architecture

Fuel Components: Nailgun

● Written in Python● Exposes internal data via REST API● Concentrates up to 90% of orchestration logic● Transforms user input (UI, CLI) into deployment data

(JSON)● Keeps tasks status and all information in DB● Uses SQLAlchemy, Alembic for DB access and migrations

Fuel Components: Nailgun

Fuel Components: Astute

image deployer

● Astute consumes messages sent by Nailgun● Astute and its agents are written in Ruby● Astute uses MCollective RPC● Has bindings to provisioning (cobble or Ironic-based)● Tracks progress of deployment, provisioning● Supports operations (adding/removing nodes)● Small codebase in comparison with Nailgun

Fuel Components: Astute

Fuel Components: OSTF (Health Check)

● Health Check● Network Checker

● DHCP● VLANs

● Diagnostic Snapshot● Logs collection & calculation of operations progress

Fuel: Additional Functions

Fuel Architecture

image deployer

Beginning, Challenges, Improvements

● Using Puppet Master● issues w/agent signing key and deploying a node● catalogs grew too large

● Big catalog, long compilation times● 5+ minutes with <10 nodes● time increase between linear and exponential as nodes added● sending results back to master taking too long● nginx, separate DB, other optimizations not helping much

Early days and challenges

● Very complex catalogs● We pull in 1501 resoures and 150 are classes for the controller

● Minor modifications cause order to change● Sequence errors show up● Loops we thought we’d accounted for pop up● 20% goes to fixing this stuff

● Drop the master● Rsync the manifests● Run puppet-apply!● Wait, what about orchestration?

● had astute.yaml as data entry point● already using it for cobbler, extend to puppet!

Improvements

● Hiera didn’t exist yet, so we extended Astute● $::fuel_settings all over the place

● Worked but not perfect● Lacks merge/replace elements● No eyes on issues here outside Fuel devs

● As of v6.1 Fuel uses Hiera● All puppet manifests have easy access to namespaced data● Helps with granular deployments● Soon will use it for populating diff for patches

Improvements

Where Fuel Is Headed

● Granular deployments● breaking controller role into discrete tasks● resolves most of the sequence/dependency issues● improves Fuel plug-in functionality

● New graphing system (task-lib)● feeds astute● will eventually move to Mistral● allows for more complicated catalogs of manifests● (which then allows use of other peoples composite layers)

What’s coming up

● Upstream manifests● forked because we couldn’t keep up

● pulling in upstream, deferring changes with an upstream source until there’s a commit upstream

● still behind (from 0 to 4 months), but catching up● by EOY will be on tip of upstream, plan to provide CI testing

What’s coming up

● Server spec (or equivalent) CI integration tests● tests in the manifests can confirm we end with requested state

● feed in configuration data source, ensure resulting configuration matches

● few upstream libraries providing this (mostly just using rspec tests)

● want these tests upstream for gating in the community

What’s coming up

(we’re hiring, ask me about working at Mirantis!)caedo@mirantis.com - twitter/IRC: @docaedo

Thank you

fuel, puppet and openstack

rights reservedfuelcopyright

rights reservedagenda

rights reservedhttp

rights reservedwhat

rights reservedfuel

openstack deployment

nailgunfuel components

rhel ospdeploying openstack

Internet

mirantis超簡単fuel openstack インストール

building a deployment pipeline - cloudbees...ruby developer...

enterprise grade platform for linux - confex · pdf...

best practices building · ganeti, openstack, zabbix,...

building a dev ops paas with puppet, docker, openstack and...

openstack and puppet

openstack red hat rdo - cisco.com · cisco reference...

openstack - ness-tech.co.ilירועים 2015/devops... ·...

bringing new experience with openstack and fuel (ihor...

puppet in the open source openstack infrastructure in the...

using openstack and puppet to deliver iaas at cern

experiments with abe schemes on openstack fuel iaas...

case study: utilizing mirantis fuel to install openstack ......

世界での実績no.1のopenstackインストーラ –...

puppet camp tokyo 2014: kickstack: a pure-puppet rapid...

building a devops paas with puppet, docker, openstack, and

barbican nova chef openstack openstack charms cinder...

using puppet for deploying hyper-v openstack compute nodes -...

fuel for openstack 3.1 userguide

a one stop solution for puppet and openstack · openstack...