infrastructure automation with chef

Infrastructure Automation with Chef

Jonathan Weiss

12.11.2010

1

Who am I?

Working for Peritor in Berlin, Germany

Written, maintain, or involved in

  Webistrano

  Capistrano

  SimplyStored

  Happening

  The great fire of London

http://github.com/jweiss

@jweiss

2

Scalarium

Amazon EC2 Cluster Management

  Auto-Config

  Self-Healing

  Auto-Scaling

  One-click-deployment

www.scalarium.com

3

4

DevOps Dev who? Ops what?

Developers

5

It works on my machine…

We need Node.js ���in version x.y.z

BTW the deployment is due tomorrow

DEV

SysAdmin We don’t

support that version

The architecture doesn’t match our network,

deployment, or security model

We weren’t consulted about ���

security / monitoring / backup / provisioning

OPS

The blame game

OPS DEV

WE SHOULD TALK!

OPS

WE SHOULD TALK!

DEV

DevOps

Combined interest of developers and operations in

  Processes

  Communication

  Automation

Close cooperation based on common tools

9

Automation

• Infrastructure

• Provisioning

• Deployment

flickr.com/photos/torek/3788181603

Chef

Open Source Provisioning Tool

  Configures your servers

  Cookbooks & recipes

  Ruby DSL

11

Chef Solo

12

Chef

13

14

Chef Single Server View Chef-Solo or Chef-Client

Cookbooks

15

Cookbooks

apache2

memcached

mysql

rails

Memcached Cookbook

16

Cookbooks

memcachd

attributes

recipes

templates

Memcached Cookbook

17

memcached

attributes

default.rb

recipes

service.rb

install.rb

templates

default

memcached.conf.erb

Attributes

18

cookbooks/memcached/attributes/default.rb

Set default values for variables to use in recipes & templates

Recipes

19

cookbooks/memcached/recipes/service.rb

Actual instructions to execute

Recipes

20

cookbooks/memcached/recipes/install.rb

Templates

21

cookbooks/memcached/templates/default/memcached.conf.erb

Chef Solo Run

22

Configuration JSON: /etc/chef/solo.json

Run Chef Solo

Ohai

Detecs data about your local machine & OS   Kernel stats

  Modules & drivers

  Programming Languages

  Users

  Filesystems

  Network configuration

  ...

Populates attributes and makes them available for Chef

Extensible with plugins

23

Ohai

24

$ ohai

Ohai

25

recipes

attributes

Resources

26

The steps that make up a recipe

Implemented by providers, depending on your platform (in this case apt-get, rpm, zypper or MacPorts)

Available Resources

27

Cookbook File

Cron

Deploy

Directory

Erlang Call

Execute

File

Git

Group

HTTP Request

Ifconfig

Link

Log

Mdadm

Mount

Package

Remote Directory

Remote File

Route

Ruby Block

SCM

Script

Service

Subversion

Template

User

User & Group Resource

28

Cron Resource

29

Execute Resource

30

Package Resource

31

Script Resource

32

Supports bash, csh, Ruby, Python, and Perl

Deploying

Deploy resource   Compatible with Capistrano layout

  Supports Capistrano hooks via deploy/hook_name.rb

33

Everything is Ruby

Very easy to extend

34

Available in all Resources

Common methods:

  action

  provider

  Supports

  ignore_failure

  not_if / only_if

  notifies

  subscribes

35

Notifications

Notifiy other resources to execute an action

Default notification happens at the end of the Chef run, optionally run it

36

Roles

  Composites of recipes and attributes

 Nodes can have multiple roles

37

Role Definition

38

$ cat roles/webserver.rb

39

Chef Server Setup Multiple Chef-Clients talking to Chef-Server

Architecture

40

Web UI

Sinatra Web application to manage nodes, cookbooks & roles

41

Knife

CLI tool for Chef-Server:   List nodes, cookbooks & attributes

  Assign roles

  Upload new cookbooks

42

Solr

Search for nodes by attributes:

in recipes/templates

with knife

43

44

Chef Annoyances Things that could be nicer

Idempotent

Bug or feature?

45

46

Will never happen, even if /data/logs/redhat will be created!

Two Phases of Chef

1) Compile

Load all attributes&recipes and build list of actions/dependencies

Compute attribute tree

2) Run

Actually execute the resources

47

Random Ruby Code runs here

Two Phases of Chef

 Use only_if and not_if

 Tell Chef to execute your ressource during compile phase

48

Chef Experiences

Great way to automate provisioning of servers

  Infrastructure as code

  Understandable by dev & ops

  Easy to extend

  Easy to integrate in management framework (SSH, mcollective, Nanite, ..)

  Ignore Chef-Server when starting to play with Chef

49

50

DEV OPS

© Peritor GmbH - Alle Rechte vorbehalten

Peritor GmbH Blücherstr. 22, Hof III Aufgang 6 10961 Berlin

Tel.: +49 (0)30 69 20 09 84 0 Fax: +49 (0)30 69 20 09 84 9

Internet: www.peritor.com E-Mail: info@peritor.com