puppetconf 2014 killer r10k workflow without notes
DESCRIPTION
Slides (without notes) from my talk at PuppetConf 2014TRANSCRIPT
Killer R10K Workflow
Automating the KillerRobots, all 10K of Them
Who Am I@phil_zimmerman
Early Days
R10K Workflow Awesomeness
Demo Time
The Journey of A Thousand Miles Begins
With A Single, Monolithic Repo
A Single Repo? What the F*@K?!!
Why Put Everything in a Single Repo?
Simplify Development
Easy Jenkins Flow
Puppet Code and Hiera Data Together
Just Starting
Out
Single CI Job • rspec-puppet • syntax check • lint
Single Release Job • create/push tag
Single Deploy Job • Capistrano tasks
• poor man’s dynamic environments • kludgy git logic w/conditionals in Capfile
For All Modules
Forge Modules
Capistrano -> Puppet Module Tool
Worked Well… Until It Didn’t
Did I mention the
Capistrano tasks?
KLUDGY
task "create_puppet_env", :roles => :puppet_master do if exists?(:branchname) run "if [ -d #{environment_basedir}/#{branchname} ]; then cd #{environment_basedir}/#{branchname} && git pull origin #{branchname} ; else \ git clone #{module_repository} #{environment_basedir}/#{branchname} --branch #{branchname} ; fi" else puts "Please provide a valid git branch name as an argument" end end
task "create_puppet_env", :roles => :puppet_master do if exists?(:branchname) run "if [ -d #{environment_basedir}/#{branchname} ]; then cd #{environment_basedir}/#{branchname} && git pull origin #{branchname} ; else \ git clone #{module_repository} #{environment_basedir}/#{branchname} --branch #{branchname} ; fi" else puts "Please provide a valid git branch name as an argument" end end
Upgrade Forge Modules
Version Management == Face Palm
Upgrading Affects all Environments!
Let Me Count the Other Ways…
This Started to Fail
Simple Comes at a Price
You mean I have to wait for tests
to run for ALL modules before I know if my changes are good?”
“I only changed one module….
I have to deploy
everything in order to get my changes on the masters? Sigh”
“I only changed one module….
everything just to get my hiera data on the masters?”
“I need to change some hiera data…. I have to deploy
I’m Losing
My Patience!
Oh - and that little problem where
upgrading forge modules can break production…
Stop the Madness !!
Recap of Early Days:Monolithic RepoLong CI CyclesAll-Or-Nothing DeploysUpgrading Forge Modules
Tools Can Be
Awesome
(we just needed a couple more)
Make Them Work For You
R10K
R10K
Deploys Puppet Code
Handles Git/Svn Fu
Is Awesome
R10K and Puppetfile
Match Made in Heaven (or Portland)
Manage Module Versions
Inventory of Puppet Environment
Puppetfile
Format
Inventory of Modules and Their Versions
Puppetfile lives in its
own repository
Puppetfile lives in its own
Repository
Inventory of Modules and Their Versions
Each module in its own
repository AND r10k
R10KDeploy
r10k deploy environment test -p
deploys all modules in Puppetfile for the test branch
r10k deploy module tomcatdeploys a single module!
CI Job Per Module • rspec-puppet • syntax check • lint
Release Job Per Module • create/push tag • select module from dropdown list
Deploy Job For Each Module And Hiera • simpler Capistrano tasks • wrap r10k calls to each master/node
desc "for specified branch in puppetfile repo, use r10k to deploy all modules for the specified environment." task "update_environment", :roles => :puppet_master do if exists?(:branchname) run "r10k -v debug deploy environment #{branchname} -p" else puts "Please provide a valid git branch name as an argument" end end
Puppetfile Manipulation and Branch Creation
mod 'tomcat', :git => '[email protected]:fylgia/tomcat.git', :ref => 'RELEASE_1.0.13'
Tomcat Example
go to tomcat dir (production branch) !
‘git checkout -b dev_change_foo’ ‘git push origin dev_change_foo’
Create the ‘dev_change_foo’ branch
in Puppetfile repo
Change the :ref for tomcat to
‘dev_change_foo’
Call r10k to deploy ‘dev_change_foo’
Notify Me When It’s Finished
Explain the Magics
Hint:
Create Branchr10k deploy environment
Modify Branchr10k deploy module
Delete Branchauto-delete Puppetfile branch
Testing Multiple Modules in the
Same Environment
go to profile dir (production branch) !
‘git checkout -b dev_change_foo’ ‘git push origin dev_change_foo’
mod 'profile', :git => '[email protected]:fylgia/profile.git', :ref => 'RELEASE_0.1.124' mod 'tomcat', :git => '[email protected]:fylgia/tomcat.git', :ref => 'RELEASE_1.0.13'
production
mod 'profile', :git => '[email protected]:fylgia/profile.git', :ref => ‘dev_change_foo’ mod 'tomcat', :git => '[email protected]:fylgia/tomcat.git', :ref => 'dev_change_foo'
dev_change_foo
Truly Dynamic Environments!
The Post-Receive Hook
reaktor
Default SetupGitHub or GitHub Enterprise
Hipchat
Just provide some config!
REAKTOR_PUPPET_MASTERS_FILE
REAKTOR_HIPCHAT_TOKENREAKTOR_HIPCHAT_ROOMREAKTOR_HIPCHAT_FROM
PUPPETFILE_GIT_URL
Other Chat ProvidersCampfire, Slack, etc
If it has an API, it’s pluggable!
Other Git ProvidersGitlab, Bitbucket, etc
Need to Determine Best Approach
Create Release
Deploy - Ship It!
Create ReleaseModulefile
RELEASE_1.0.13 -> RELEASE_1.0.14
versionfile (hieradata)
DeployParameterized Job
environment
version to deploy
Puppetfile Manipulation (again)
change :ref to selected version
r10k deploy selected environment
commit/push updated Puppetfile
One-Click Production
Deploy
Workflow Recap:R10K and PuppetfileEach Module in Own RepoPost-Receive Hook GoodnessProduction Deployments
Conclusion:Early DaysR10K Workflow AwesomenessDemos
Helpful Links and References
https://github.com/adrienthebo/r10k
http://garylarizza.com/blog/2014/02/18/puppet-workflow-part-3/
http://garylarizza.com/blog/2014/03/07/puppet-workflow-part-3b/
http://philzim.com/2014/05/02/an-automated-r10k-workflow-that-works/
https://github.com/pzim/reaktor
