puppet camp atlanta 2014: continuous deployment of puppet modules
TRANSCRIPT
![Page 1: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/1.jpg)
CONTINUOUSDEPLOYMENT OF PUPPET
MODULESHOW WE DO IT AT MAILCHIMP
![Page 3: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/3.jpg)
![Page 4: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/4.jpg)
EMAIL SERVICE PROVIDERDeliver ~500 million emails daily723 million emails delivered on Cyber Monday
of 97Sender Scorehttp://delivery.mailchimp.com/
![Page 5: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/5.jpg)
HISTORY
![Page 6: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/6.jpg)
Image source: https://blog.engineyard.com/2014/configure-before-you-boot
![Page 7: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/7.jpg)
CONFIGURATION MANAGEMENT IS HARD"With Chef, Puppet, and CFEngine we found anot-insignificant learning curve on setting upthe different server daemons and learning theDSL. This was particularly challenging whenwe were configuring unique software not yet
given recipes by the existing community.Given our cluster sizes, we also didn't really
need any of the advanced features thosesystems provided."
- README from internally built tool
![Page 8: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/8.jpg)
MOVE TO COLOCATIONBuy vs. Lease analogyGrow our Operations teamNeeded a tool with dry-run mode
![Page 9: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/9.jpg)
PEOPLE MAKE MISTAKES
![Page 10: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/10.jpg)
HOW DO WE CATCH THESE MISTAKES AS EARLYAS POSSIBLE?
AUTONOMATION"automation with a human touch"
1. Detect the abnormality2. Stop3. Fix or correct the immediate condition
![Page 11: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/11.jpg)
HOW DO WE CATCH THESE MISTAKES AS EARLYAS POSSIBLE?
DSL toolsEditor SupportSource Code ManagementContinuous Integration
![Page 12: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/12.jpg)
DSL TOOLSPuppetERBYAMLPuppet Style Guide
![Page 13: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/13.jpg)
PUPPETpuppet parser validate mymanifest.pp
![Page 14: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/14.jpg)
package { 'openssh-server': ensure => installed,}
file { '/etc/ssh/sshd_config': source => 'puppet:///modules/sshd/sshd_config', owner => 'root', group => 'root', mode => '640', notify => Service['sshd'] /* sshd will restart whenever you edit this file. */ require => Package['openssh-server'],}
service { 'sshd': ensure => running, enable => 'true', hasstatus => 'true', hasrestart => 'true',}
![Page 15: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/15.jpg)
$ puppet parser validate validate_1.ppError: Could not parse for environment production:Syntax error at 'require'; expected '}' at validate_1.pp:12
![Page 16: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/16.jpg)
ERBerb -P -x -T '-' mytemplate.erb | ruby -c
![Page 17: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/17.jpg)
restrict default kod nomodify notrap nopeer<% unless @service %> noqueryrestrict 127.0.0.1restrict -6 ::1
driftfile /var/lib/ntp/drift
<% @serverlist.sort.each do |server| -%>server <%= server %> iburst maxpoll 6restrict <%= server %> mask 255.255.255.255 nomodify notrap noquery<% end -%>
![Page 18: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/18.jpg)
$ erb -P -x -T '-' broken-ntp.conf.erb | ruby -c-:11: syntax error, unexpected $end, expecting kEND
![Page 19: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/19.jpg)
YAMLnpm install -g js-yaml; js-yaml hiera.yaml
ruby -e "require 'yaml'; YAML.load_file('hiera.yaml')"
![Page 20: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/20.jpg)
---ntp::servers: - 0.us.pool.ntp.org - 1.us.pool.ntp.org - 2.us.pool.ntp.org - 3.us.pool.ntp.orghp::ilo::settings: ssh_status type: global value: true ssh_port type: global value: '22' http_port type: global value: '80' https_port type: global value: '443'
![Page 21: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/21.jpg)
$ js-yaml hiera.yamlJS-YAML: bad indentation of a mapping entry at line 9, column 13: type: global ̂$ ruby -e "require 'yaml'; YAML.load_file('hiera.yaml')"yaml.rb:133:in ̀load': syntax error on line 9, col 14: ̀ value: true' (ArgumentError) from yaml.rb:133:in ̀load' from yaml.rb:144:in ̀load_file' from yaml.rb:143:in ̀open' from yaml.rb:143:in ̀load_file' from -e:1
![Page 22: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/22.jpg)
PUPPET STYLE GUIDEhttps://docs.puppetlabs.com/guides/style_guide.htmlgem install puppet-lint
puppet-lint --fix /my/puppet/code
![Page 23: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/23.jpg)
package { 'openssh-server': ensure => installed,}
file { '/etc/ssh/sshd_config': source => 'puppet:///modules/sshd/sshd_config', owner => 'root', group => 'root', mode => '640', notify => Service['sshd'], /* sshd will restart whenever you edit this file. */ require => Package['openssh-server'],}
service { 'sshd': ensure => running, enable => 'true', hasstatus => 'true', hasrestart => 'true',}
![Page 24: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/24.jpg)
$ puppet-lint validate_2.ppWARNING: quoted boolean value found on line 16WARNING: quoted boolean value found on line 17WARNING: quoted boolean value found on line 18WARNING: indentation of => is not properly aligned on line 6WARNING: indentation of => is not properly aligned on line 7WARNING: indentation of => is not properly aligned on line 8WARNING: indentation of => is not properly aligned on line 9WARNING: indentation of => is not properly aligned on line 10WARNING: mode should be represented as a 4 digit octal value or symbolic mode on line 9WARNING: /* */ comment found on line 10
![Page 25: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/25.jpg)
$ puppet-lint --fix validate_2.ppFIXED: quoted boolean value found on line 16FIXED: quoted boolean value found on line 17FIXED: quoted boolean value found on line 18FIXED: indentation of => is not properly aligned on line 6FIXED: indentation of => is not properly aligned on line 7FIXED: indentation of => is not properly aligned on line 8FIXED: indentation of => is not properly aligned on line 9FIXED: indentation of => is not properly aligned on line 10FIXED: mode should be represented as a 4 digit octal value or symbolic mode on line 9FIXED: /* */ comment found on line 10
![Page 26: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/26.jpg)
package { 'openssh-server': ensure => installed,}
file { '/etc/ssh/sshd_config': source => 'puppet:///modules/sshd/sshd_config', owner => 'root', group => 'root', mode => '0640', notify => Service['sshd'], # sshd will restart whenever you # edit this file. require => Package['openssh-server'],}
service { 'sshd': ensure => running, enable => true, hasstatus => true, hasrestart => true,}
![Page 27: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/27.jpg)
--- validate_2.pp 2014-12-08 09:43:38.000000000 -0500+++ validate_2.pp-fixed 2014-12-08 09:50:51.000000000 -0500@@ -3,18 +3,18 @@ } file { '/etc/ssh/sshd_config':- source => 'puppet:///modules/sshd/sshd_config',- owner => 'root',- group => 'root',- mode => '640',- notify => Service['sshd'], /* sshd will restart whenever you- edit this file. */+ source => 'puppet:///modules/sshd/sshd_config',+ owner => 'root',+ group => 'root',+ mode => '0640',+ notify => Service['sshd'], # sshd will restart whenever you+ # edit this file. require => Package['openssh-server'], } service { 'sshd': ensure => running,- enable => 'true',- hasstatus => 'true',- hasrestart => 'true',+ enable => true,+ hasstatus => true,+ hasrestart => true, }
![Page 28: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/28.jpg)
EDITOR SUPPORT
![Page 29: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/29.jpg)
VIMPLUGINS FTW
with
Syntasticvim-puppetUltiSnips vim-snippets
![Page 30: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/30.jpg)
EMACShttp://www.emacswiki.org/emacs/PuppetProgramming
![Page 31: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/31.jpg)
GEPPETTOhttp://puppetlabs.github.io/geppetto/index.html
![Page 32: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/32.jpg)
SOURCE CODEMANAGEMENT
![Page 33: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/33.jpg)
COMMIT HOOKSSCRIPT RUNNING THE DSL TOOLS AGAINST NEW FILES
![Page 34: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/34.jpg)
PEER REVIEW
![Page 35: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/35.jpg)
![Page 36: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/36.jpg)
TRUNK BASEDDEPLOYMENT
![Page 37: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/37.jpg)
CONTINUOUSINTEGRATION
![Page 38: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/38.jpg)
JENKINSHTTPS://GITHUB.COM/VSTONE/JENKINS-
PUPPET-SCRIPTS
![Page 39: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/39.jpg)
WHY NOT RSPEC ORBEAKER?
![Page 40: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/40.jpg)
CONTINUOUSDEPLOYMENT
![Page 41: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/41.jpg)
REMEMBER TRUNK BASEDDEPLOYMENT?
# Keep environment up-to-datevcsrepo { '/etc/puppet/environments/production': ensure => latest, provider => hg, source => 'https://localhost/mercurial/puppet-modules',}
![Page 42: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/42.jpg)
REVIEW TIME!Catch mistakes earlyAutomation with a human touchTrunk Based Deployments
![Page 43: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/43.jpg)
QUESTIONS?
![Page 44: Puppet Camp Atlanta 2014: Continuous Deployment of Puppet Modules](https://reader033.vdocuments.site/reader033/viewer/2022060202/559b68261a28ab8d7f8b458b/html5/thumbnails/44.jpg)
Slide sources at
THANKS!BILL O'NEILL
[email protected]@WONEILL
http://github.com/woneill/puppetcamp_atlanta_2014