1

v1.1.0

Building Testable Infrastructure with Chef, Test Kitchen, and Dockermandi walls mandi@getchef.com CodeMesh.io 5 November 2014

2

whoami• Mandi Walls • Professional Services at Chef • @lnxchk

3

v1.1.0

What is Chef

4

Automation Platform• Creates a dependable view of your entire network’s

state. • Can handle complex dependencies among the

nodes of your network. • Is fault tolerant. • Is secure. • Can handle multiple platforms • Can manage cloud resources • Provides a foundation for innovation

5

6

Infrastructure As Code• Programmatically provision and configure components

• Treat like any other code base

• Reconstruct business from code repository, data backup, and compute resources

7

Policy-Based•You capture the policy for your infrastructure in code

•Chef ensures each node in your infrastructure complies with the policy

8

v1.1.0

Our Goals

9

Reliable, Repeatable Infrastructure• Everyone on the team gets the same config • Minimize surprises at deploy time • Test app code against real config • Test config with the app code

10

Testing Chef• Chef is built on ruby • Test chef code with ruby tools like rspec • Integrate with Test Kitchen, ServerSpec, foodcritic

11

v1.1.0

My Tools

12

Components of Workflow• My workstation - 3 year old mba

• git, github repo • chefdk from downloads.getchef.com

• Cloud box - CentOS 6.something • docker-io installed from peel • chefdk from downloads.getchef.com • kitchen-docker gem • git

13

How I’m Working• Edit recipe code on my machine

• my nice editor settings, not much RAM • Run local tests - syntax, style, lint, chef spec • Check into git, upload to repo • git pull on test box • kitchen converge, kitchen verify, kitchen test

14

Simple Chef Recipe

15

package "httpd" !

service "httpd" do action :start end !

file "/var/www/html/index.html" do content "<h1>hello world</h1>\n" end

v1.1.0

What is Test Kitchen

16

Test Kitchen• If Chef is code, we should be able to test it

17

Test Kitchen•Test harness to execute code on one or more platforms

•Driver plugins to allow your code to run on various cloud and virtualization providers

• Includes support for many testing frameworks

•Included with ChefDK

18

.kitchen.yml•The configuration file for your Test Kitchen •driver – virtualization or cloud provider •provisioner – application to configure the node •platforms – target operating systems •suites – target configurations

19

.kitchen.yml

20

--- driver: name: docker !provisioner: name: chef_zero !platforms: - name: centos-6.4 driver_config: forward: - 80:80 !suites: - name: default run_list: - recipe[apache::default] attributes:

.kitchen.yml • Create a docker container • With centos-6.4 • I’m testing a webapp, so forward port 80 • Provisions our environment with an apache

webserver as configured by our team in a Chef recipe

21

kitchen create-----> Creating <default-centos-64>...!build context to Docker daemon 2.048 kB!Sending build context to Docker daemon!Step 0 : FROM centos:centos6!Pulling repository centos!---> 70441cac1ed5!Step 1 : RUN yum clean all!---> Running in b4ed54c86150!Loaded plugins: fastestmirror!Cleaning repos: base extras updates!Cleaning up Everything!---> 1c333241ae96!Removing intermediate container b4ed54c86150!Step 2 : RUN yum install -y sudo openssh-server openssh-clients which curl

22

docker ps[chef@CentOS63 codemesh]$ sudo docker ps!

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES!

9ddd3300de51 87f714782104 "/usr/sbin/sshd -D - 2 minutes ago Up 2 minutes 0.0.0.0:80->80/tcp, 0.0.0.0:49157->22/tcp stoic_euclid

23

kitchen converge - install version 2.2.15-39.el6.centos of package httpd! ! * service[httpd] action start [2014-11-05T00:18:29+00:00] INFO: Processing service[httpd] action start (codemesh::default line 16)! [2014-11-05T00:18:29+00:00] INFO: service[httpd] started! ! - start service service[httpd]! !* service[httpd] action enable [2014-11-05T00:18:29+00:00] INFO: Processing service[httpd] action enable (codemesh::default line 16)! [2014-11-05T00:18:29+00:00] INFO: service[httpd] enabled! ! - enable service service[httpd]! ! * file[/var/www/html/index.html] action create [2014-11-05T00:18:29+00:00] INFO: Processing file[/var/www/html/index.html] action create (codemesh::default line 20)! [2014-11-05T00:18:29+00:00] INFO: file[/var/www/html/index.html] created file /var/www/html/index.html!

24

kitchen list[chef@CentOS63 codemesh]$ kitchen list!

Instance Driver Provisioner Last Action!

default-centos-64 Docker ChefZero Converged!

25

show me[chef@CentOS63 codemesh]$ curl localhost!<h1>hello world</h1>![chef@CentOS63 codemesh]$

26

Other Plugins• kitchen-vagrant (included!)!• kitchen-docker!• kitchen-ec2!• kitchen-rackspace!• kitchen-gce!• kitchen-digitalocean!• kitchen-openstack!• kitchen-bluebox!• kitchen-joyent!• kitchen-lxc

27

Reliable Configuration• Provision onto platforms that match production • Write Chef recipes once for everyone

• Local development work • Testing systems • Production

• Customize environments as necessary • ports, user accounts, backend services

28

v1.1.0

TDI: Test-Driven Infrastructure

29

Testing Bits for Chef• Pre-testing (no running host)

• rubocop: ruby style and syntax • ChefSpec: chef-specific unit testing • foodcritic: chef-specific logic and style rules

• Post-testing (on a running host) • ServerSpec: integration testing on services • bats: unix system testing

30

rubocop$ rubocop recipes/default.rb !Inspecting 1 file!C!Offenses:!recipes/default.rb:17:11: C: Space inside square brackets detected.! action [ :start, :enable ]! ^!recipes/default.rb:17:27: C: Space inside square brackets detected.! action [ :start, :enable ]! ^!recipes/default.rb:20:6: C: Prefer single-quoted strings when you don't need string interpolation or special symbols.!file "/var/www/html/index.html" do! ^^^^^^^^^^^^^^^^^^^^^^^^^^!1 file inspected, 3 offenses detected

31

fix

$ rubocop recipes/default.rb !Inspecting 1 file!.!!

1 file inspected, no offenses detected

32

ChefSpec• Test the intention of your code • Built on rspec • Specific to Chef

33

Methods• You can TDD infrastructure code • Keep on top of regressions • Give safe configurations to the whole team

34

chefspec filerequire 'chefspec'!!describe 'codemesh::default' do! let(:chef_run) do! ChefSpec::Runner.new.converge(described_recipe)! end !! it 'installs apache' do! expect(chef_run).to install_package('httpd')! end !end!

35

Run chefspec

$ rspec spec/unit/default.rb !.!!

Finished in 0.00735 seconds (files took 1.85 seconds to load)!1 example, 0 failures!

36

ServerSpec• Tests running systems • Run from Test Kitchen at the end of kitchen

converge • Not dependent on Chef!

• Use from other config tools • Use standalone! • serverspec.org

37

Serverspec Filerequire 'serverspec'!!set :backend, :exec!!describe 'apache' do! it "is installed" do! expect(package 'httpd').to be_installed! end ! it "is running" do! expect(service 'httpd').to be_running! end ! it "is listening on port 80" do! expect(port 80).to be_listening! end !

38

it "displays a custom home page" do!

expect(command("curl localhost").stdout).to match /hello/!

end !

end!

kitchen verify apache! is installed! is running! is listening on port 80! displays a custom home page! ! Finished in 1.45 seconds (files took 0.6278 seconds to load)! 4 examples, 0 failures! Finished verifying <default-centos-64> (0m18.23s).!-----> Kitchen is finished. (0m39.50s)!

39

test vs verify• kitchen verify will run tests on an instance that is

already running • kitchen test will start a new instance, run all the

chef recipes, run all the tests, and then destroy the instance

• nice for different workflow - user watching a test vs a build server watching for a good return code • fast with Docker!

40

v1.1.0

Further Resources

41

Container Ecosystem• Chef container resources help you manage

container workflow • Create and deploy images • https://docs.getchef.com/containers.html

42

Testing Tools• ChefSpec: https://docs.getchef.com/chefspec.html • Serverspec: http://serverspec.org/ • bats: https://github.com/sstephenson/bats • Foodcritic: • More on ChefDK:

• http://foodfightshow.org/2014/08/chefdk.html • More on this workflow:

• http://www.slideshare.net/misheska/testing-yourautomationcode-docker-version-v02

43

Some URLs• http://getchef.com • http://docs.getchef.com • http://supermarket.getchef.com • http://youtube.com/getchef • http://lists.opscode.com • irc.freenode.net: #chef, #chef-hacking • Twitter: @chef #getchef, @learnchef #learnchef • Meetups in your city?!?

44

v1.1.0

Questions?

45

46