fuel, puppet and openstack

Copyright © 2014 Mirantis, Inc. All rights reserved

www.mirantis.com / fuel-infra.org

A Technical Overview

Fuel, Puppet and OpenStack

March 2, 2015


Introductions

Christopher Aedo — Product Architect

Christopher is an IT veteran for consulting, design and technology companies. He is also an outspoken public advocate for OpenStack, cloud computing, software defined networking and software defined storage. He often speaks at OpenStack and Open Source related conferences around the world.

About MirantisMirantis is the number one pure play OpenStack Company. We deliver all the technology, integration, training and support required for companies to succeed with production-grade open source cloud. More customers rely on Mirantis than any other company to scale out OpenStack without the compromises of vendor lock-in. Our bench of 400+ open source infrastructure experts helped make us one of top 5 contributors to OpenStack’s upstream codebase.


Agenda

● Deploying OpenStack● Fuel Overview & Architecture● Beginning, challenges, Improvements● Where Fuel is Headed● Q&A


Deploying OpenStack


● DevStack● Deployment tools

● Fuel, Crowbar, Foreman, Helion

● Config/Orchestration● Puppet, Chef, Ansible, Salt

● Distro-based tools● Packstack, Juju, RHEL OSP

Deploying OpenStack is easy!


What is Fuel?

● OpenStack deployment and management tool● Open source, growing community● GUI and CLI driven● Visit http://fuel-infra.org to learn more and download

http://fuel-infra.org


Fuel


http://fuel-infra.org

Fuel Overview & Architecture


Fuel Components: Nailgun


● Written in Python● Exposes internal data via REST API● Concentrates up to 90% of orchestration logic● Transforms user input (UI, CLI) into deployment data

(JSON)● Keeps tasks status and all information in DB● Uses SQLAlchemy, Alembic for DB access and migrations

Fuel Components: Nailgun


Fuel Components: Astute

image deployer


● Astute consumes messages sent by Nailgun● Astute and its agents are written in Ruby● Astute uses MCollective RPC● Has bindings to provisioning (cobble or Ironic-based)● Tracks progress of deployment, provisioning● Supports operations (adding/removing nodes)● Small codebase in comparison with Nailgun

Fuel Components: Astute


Fuel Components: OSTF (Health Check)


● Health Check● Network Checker

● DHCP● VLANs

● Diagnostic Snapshot● Logs collection & calculation of operations progress

Fuel: Additional Functions


Fuel Architecture

image deployer


Beginning, Challenges, Improvements


● Using Puppet Master● issues w/agent signing key and deploying a node● catalogs grew too large

● Big catalog, long compilation times● 5+ minutes with <10 nodes● time increase between linear and exponential as nodes added● sending results back to master taking too long● nginx, separate DB, other optimizations not helping much

Early days and challenges


● Very complex catalogs● We pull in 1501 resoures and 150 are classes for the controller

role

● Minor modifications cause order to change● Sequence errors show up● Loops we thought we’d accounted for pop up● 20% goes to fixing this stuff



● Drop the master● Rsync the manifests● Run puppet-apply!● Wait, what about orchestration?

● had astute.yaml as data entry point● already using it for cobbler, extend to puppet!

Improvements


● Hiera didn’t exist yet, so we extended Astute● $::fuel_settings all over the place

● Worked but not perfect● Lacks merge/replace elements● No eyes on issues here outside Fuel devs

● As of v6.1 Fuel uses Hiera● All puppet manifests have easy access to namespaced data● Helps with granular deployments● Soon will use it for populating diff for patches

Improvements


Where Fuel Is Headed


● Granular deployments● breaking controller role into discrete tasks● resolves most of the sequence/dependency issues● improves Fuel plug-in functionality

● New graphing system (task-lib)● feeds astute● will eventually move to Mistral● allows for more complicated catalogs of manifests● (which then allows use of other peoples composite layers)

What’s coming up


What’s coming up


● Upstream manifests● forked because we couldn’t keep up

● pulling in upstream, deferring changes with an upstream source until there’s a commit upstream

● still behind (from 0 to 4 months), but catching up● by EOY will be on tip of upstream, plan to provide CI testing

What’s coming up


● Server spec (or equivalent) CI integration tests● tests in the manifests can confirm we end with requested state

● feed in configuration data source, ensure resulting configuration matches

● few upstream libraries providing this (mostly just using rspec tests)

● want these tests upstream for gating in the community

What’s coming up


(we’re hiring, ask me about working at Mirantis!)[email protected] - twitter/IRC: @docaedo

Thank you

mailto:[email protected]

mailto:[email protected]

fuel, puppet and openstack

Internet

rights reservedfuelcopyright

rights reservedagenda

rights reservedhttp

rights reservedwhat

rights reservedfuel

openstack deployment

nailgunfuel components

rhel ospdeploying openstack