fosdem chef-101-app-deploy

Copyright © 2010 Opscode, Inc - All Rights Reserved

Speaker:

‣ [email protected]‣ @jtimberman‣ www.opscode.com

Joshua Timberman Sr. Technical Evangelist

1

Deploying Apps with Chef

Sunday, February 6, 2011

mailto:[email protected]



Speaker:

2

Chef 101

‣ [email protected]‣ @jtimberman‣ www.opscode.com

Joshua Timberman Sr. Technical Evangelist




Copyright © 2010 Opscode, Inc - All Rights Reserved 5

(They make computers?)



(Security training/certification)



http://www.flickr.com/photos/timyates/2854357446/sizes/l/

18

Developers?System administrators?“Business” people?




At a High Level...

‣ A library for configuration management

‣ A configuration management system

‣ A systems integration platform

‣ An API for your entire Infrastructure

http://www.flickr.com/photos/asten/2159525309/sizes/l/





Principles

IdempotentData-drivenSane defaultsHackabilityTMTOWTDI



Multiple applications of an operation do not change the result

22Sunday, February 6, 2011


We start with APIs, you supply data



option :json_attribs, :short => "-j JSON_ATTRIBS", :long => "--json-attributes JSON_ATTRIBS", :description => "Load attributes from a JSON file or URL", :proc => nil

option :node_name, :short => "-N NODE_NAME", :long => "--node-name NODE_NAME", :description => "The node name for this client", :proc => nilDefaults are sane, but

easily changed24



Open source and community



TIMTOWTDI is a Perl motto

27http://www.flickr.com/photos/lidarose/225156612



A Tour of Chef



Chef Client runs on your systems



Clients talk to a Chef Server



Clients authenticate with RSA keys

31http://www.flickr.com/photos/debbcollins/3401944550/


http://www.flickr.com/photos/debbcollins/3401944550/

http://www.flickr.com/photos/debbcollins/3401944550/


The Opscode Platform is a Chef Server



Command-line API utility, Knife

33http://www.flickr.com/photos/myklroventine/3474391066/


Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.flickr.com/photos/peterrosbjerg/3913766224/ 34

We call each system you configure a Node



Nodes have Attributes

35

{ "kernel": { "machine": "x86_64", "name": "Darwin", "os": "Darwin", "version": "Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT 2010; root:xnu-1504.7.4~1/RELEASE_I386", "release": "10.4.0" }, "platform_version": "10.6.4", "platform": "mac_os_x", "platform_build": "10F569", "domain": "local", "os": "darwin", "current_user": "jtimberman", "ohai_time": 1278602661.60043, "os_version": "10.4.0", "uptime": "18 days 17 hours 49 minutes 18 seconds", "ipaddress": "10.13.37.116", "hostname": "cider", "fqdn": "cider.local", "uptime_seconds": 1619358 }

Kernel info!

Platform info!

Hostname and IP!



Attributes are Searchable

36

$ knife search node ‘platform:mac_os_x’

search(:node, ‘platform:mac_os_x’)



Nodes have a Run List

37

What Roles or Recipes to applyin Order



Nodes have a Run List

38

{ "run_list": [ "role[production]", "role[webserver]" ]}



Nodes have Roles



Roles have a Run List

40

What Roles or Recipes to applyin Order



name "webserver"description "Systems that serve HTTP traffic"

run_list( "recipe[apache2]", "recipe[apache2::mod_ssl]")

default_attributes( "apache" => { "listen_ports" => [ "80", "443" ] })



Roles are Searchable

42

$ knife search role ‘listen_ports:80’

search(:role, ‘listen_ports:80’)



Chef manages Resources on Nodes


Resources...

‣ Have a type

‣ Have a name

‣ Have parameters

‣ Take action to put the resource in the declared state

package "apache2" do version "2.2.11-2ubuntu2.6" action :installend

template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode 0644 action :createend

Declare a description of the state a part of the node should be in



Resources take action through Providers


Providers...

Multiple providers per resource type.

Know how to actually perform the actions specified by a resource.

Apt, Yum, Rubygems, Portage, Macports, FreeBSD Ports, etc.


Resources

Platform

Provider



:ubuntu => { :default => { :package => Chef::Provider::Package::Apt, :service => Chef::Provider::Service::Debian, :cron => Chef::Provider::Cron, :mdadm => Chef::Provider::Mdadm }},

Chef::Platform



Recipes are lists of Resources


Recipes...Apply resources in the order they are specified

package "apache2" do version "2.2.11-2ubuntu2.6" action :installend

template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode 0644 action :createend

1

2

‣ Evaluates resources in the order they appear

‣ Adds each resource to the Resource Collection



Recipes are just Ruby!

51

extra_packages = case node[:platform] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } end

extra_packages.each do |pkg| package pkg do action :install endend



Cookbooks are packages for Recipes



Cookbooks are shareable!

cookbooks.opscode.com



Data bags store arbitrary data



% knife data bag show users jtimberman{ "comment": "Joshua Timberman", "groups": "sysadmin", "ssh_keys": "ssh-rsa SUPERSEKRATS jtimberman@cider", "files": { ".zshrc": { "mode": "0644", "source": "dot-zshrc" }, ".vimrc": { "mode": "0644", "source": "dot-vimrc" } }, "id": "jtimberman", "uid": 7004, "shell": "/usr/bin/zsh", "openid": "http://jtimberman.myopenid.com/"}

A user data bag item...



Data Bags are Searchable

56

$ knife search users ‘shell:/bin/bash’

search(:users, ‘/bin/bash’)



Data bags make recipes awesome-r (that’s

totally a word)

57

bash_users = search(:users, 'shell:/bin/bash')

bash_users.each do |u| user u['id'] do uid u['id'] shell "/usr/bin/zsh" comment u['comment'] supports :manage_home => true home "/home/#{u['id']}" end

directory "/home/#{u['id']}/.ssh" do owner u['id'] group u['id'] mode 0700 end

template "/home/#{u['id']}/.ssh/authorized_keys" do source "authorized_keys.erb" owner u['id'] group u['id'] mode 0600 variables :ssh_keys => u['ssh_keys'] endend



I can has applications?



Application Deployment




65

tar -x -C /app -f app.tarrsync ~/dev/app www:/appcap deploy



Server Configuration




Web ServersLoad BalancersDatabase Servers




68

% vi /etc/mysql/my.cnf#!/bin/bashCapfile



def install_package(pkg) if pkg.kind_of?(Array) run("apt-get -y install #{pkg.join(' ')}") else run("apt-get -y install #{pkg}") end end

packages = [ "build-essential", "ruby", "ruby1.8-dev", "libopenssl-ruby", "rake", "irb", "zlib1g-dev", "libssl-dev", "git-core" ] logger.info("Installing baseline packages: #{packages.join(' ')}") install_package(packages)

Capistrano anyone?




70

Application Repository‣ Source‣ CI / Build

Chef Repositorycider:~/dev/rails-quick-start (ruby-1.9.2-p0)master ✔ % ls -ltotal 16-rw-r--r-- 1 jtimberman staff 3521 Nov 5 13:09 README.md-rw-r--r-- 1 jtimberman staff 2171 Nov 5 13:09 Rakefiledrwxr-xr-x 3 jtimberman staff 102 Nov 5 13:09 certificates/drwxr-xr-x 3 jtimberman staff 102 Nov 5 13:09 config/drwxr-xr-x 26 jtimberman staff 884 Nov 12 08:16 cookbooks/drwxr-xr-x 4 jtimberman staff 136 Nov 5 13:25 data_bags/drwxr-xr-x 9 jtimberman staff 306 Nov 12 08:16 roles/



Chef Repository

RolesCookbooksApplication Information

‣ Data Bag!



Application Information

Data BagJSONPredefined structure



Walkthrough



knife ec2 server create 'role[production]' 'role[base]' \'role[radiant_database_master]' 'role[radiant]' \'role[radiant_run_migrations]' 'recipe[radiant::db_bootstrap]' \-S rails-quick-start -I ~/.ssh/rails-quick-start.pem -x ubuntu \-G default -i ami-a403f7cd -f m1.small



knife ec2 server create 'role[production]' 'role[base]' \ 'role[radiant_database_master]' \ -S rails-quick-start -I ~/.ssh/rails-quick-start.pem -x ubuntu \ -G default -i ami-a403f7cd -f m1.small

knife ec2 server create 'role[production]' 'role[base]' \ 'role[radiant]' 'role[radiant_run_migrations]' \ 'recipe[radiant::db_bootstrap]' \ -S rails-quick-start -I ~/.ssh/rails-quick-start.pem -x ubuntu \ -G default -i ami-a403f7cd -f m1.small

knife ec2 server create 'role[production]' 'role[base]' \ 'role[radiant]' \ -S rails-quick-start -I ~/.ssh/rails-quick-start.pem -x ubuntu \ -G default -i ami-a403f7cd -f m1.small

knife ec2 server create 'role[production]' 'role[base]' \ 'role[radiant_load_balancer]' \ -S rails-quick-start -I ~/.ssh/rails-quick-start.pem -x ubuntu \ -G default -i ami-a403f7cd -f m1.small



{ "id": "radiant", "server_roles": [ "radiant" ], "type": { "radiant": [ "rails", "unicorn" ] }, "database_master_role": [ "radiant_database_master" ], "repository": "git://github.com/radiant/radiant.git", "revision": { "production": "0.9.1" },



base.rbproduction.rbradiant.rbradiant_database_master.rbradiant_load_balancer.rbradiant_run_migrations.rb

Server Roles



Base Role

78

name "base"description "Base role applied to all nodes."run_list( "recipe[apt]", "recipe[git]", "recipe[build-essential]", "recipe[ruby]")



Production Role

79

name "production"description "Nodes in the production environment."default_attributes( "app_environment" => "production")



Radiant Role

80

name "radiant"description "radiant front end application server."run_list( "recipe[mysql::client]", "recipe[application]")



Application Recipe

81

search(:apps) do |app| (app["server_roles"] & node.run_list.roles).each do |app_role| app["type"][app_role].each do |thing| node.run_state[:current_app] = app include_recipe "application::#{thing}" end endend

node.run_state.delete(:current_app)



Application Rails Recipe

82

app['gems'].each do |gem,ver| gem_package gem do action :install version ver if ver && ver.length > 0 endend

deploy_revision app['id'] do revision app['revision'][node.app_environment] repository app['repository'] user app['owner'] group app['group'] deploy_to app['deploy_to'] environment 'RAILS_ENV' => node.app_environment action app['force'][node.app_environment] ? :force_deploy : :deploy

...



Radiant Database Master Role

83

name "radiant_database_master"description "Database master for the radiant application."run_list( "recipe[database::master]")



Database Master Recipe

84

search(:apps) do |app| (app['database_master_role'] & node.run_list.roles).each do |dbm_role| app['databases'].each do |env,db| if env =~ /#{node[:app_environment]}/ root_pw = node["mysql"]["server_root_password"] mysql_database "create #{db['database']}" do host "localhost" username "root" password root_pw database db['database'] action [:create_db] end end end endend



Radiant Load Balancer Role

85

name "radiant_load_balancer"description "radiant load balancer"run_list( "recipe[haproxy::app_lb]")override_attributes( "haproxy" => { "app_server_role" => "radiant" })



Haproxy App Load Balancer Recipe

86

pool_members = search("node", "role:#{node['haproxy']['app_server_role']} AND app_environment:#{node['app_environment']}") || []

template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members notifies :restart, resources(:service => "haproxy")end



Control Deployment

knife data bag edit apps radiant

87

"force": { "production": false },

"force": { "production": true },



Control Migrations

data bag item has migrate setting

need an attribute set as well

88

"migrate": { "production": true }

name "radiant_run_migrations"description "Run db:migrate on demand for radiant"override_attributes( "apps" => { "radiant" => { "production" => { "run_migrations" => true } } })



Add the role and migrations will be run

knife node run list add NODE ‘role[radiant_run_migrations]’

Migrations will run, and the role is removed by Chef automatically.

89

ruby_block "remove_run_migrations" do block do if node.role?("#{app['id']}_run_migrations") Chef::Log.info("Migrations were run, removing role[#{app['id']}_run_migrations]") node.run_list.remove("role[#{app['id']}_run_migrations]") end end end



Your Application

Your application is different than Radiant.

But not a unique snowflake, right?

Mostly, you will just need to modify the data and create application specific roles...

But wait, I’m using Rails 3!



Use the Gems data

Use bundler or bundler08 in the gems hash of the application data

91

"gems": { "bundler": "1.0.9" },

"gems": { "bundler08": "0.8.5" },

before_migrate do if app['gems'].has_key?('bundler') execute "bundle install" do ignore_failure true cwd release_path end elsif app['gems'].has_key?('bundler08') execute "gem bundle" do ignore_failure true cwd release_path end



How does it scale?



Quick FAQ



Testing



Reporting



vs [Other tool]



These slides will be posted



Resources/Questions

98

www.opscode.com/chefIRC and Mailing lists‣ irc.freenode.net #chef‣ lists.opscode.com

Twitter:‣ @opscode, #opschef‣ @jtimberman

Questions?


fosdem chef-101-app-deploy

Technology

rights reserved http

node copyright

opscode platformis

rights reserved18sunday

systems copyright

x copyright

order copyright

perlmotto copyright