setting up puppet at colruyt
DESCRIPTION
"Setting up puppet at Colruyt" by Eric Seynaeve at Puppet Camp Ghent 2013.TRANSCRIPT
![Page 2: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/2.jpg)
2
Overview
• Colruyt ?
• Why Puppet ?
• Setup
• Lessons learned
• Future
![Page 3: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/3.jpg)
3
Colruyt
• Largest retailer in Belgium
– Several shop enseignes
– Also food services
![Page 4: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/4.jpg)
4
Colruyt
• Also active in France and Luxembourg
– Food service over full France territory
– Shops
![Page 5: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/5.jpg)
5
Long history of automation
Colruyt
60’s today
![Page 6: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/6.jpg)
6
Colruyt
• Linux usage
– 430+ Colruyt shops
Colruyt, Okay, Dreamland, Dreambaby, Bioplanet, …
– 50+ SPAR shops
Migration to Linux busy
– 150+ production servers
Websites, Databases, Accounting, …
![Page 7: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/7.jpg)
7
Why Puppet ?
• Background in Linux Servers team
– System engineers
– Do shell programming but not programmers
– Long history of ‘Not Build Here’
• Background in IT Colruyt
– Specialized groups (DBA, Network, SAN, …)
– Most don’t want to package and configure
Some want to have some control
– Co-managed servers
![Page 8: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/8.jpg)
8
Why Puppet ?
• Old system configuration management
– Home build system
– Worked well
… but …
• Takes lots of effort to main
• There were important limitations
• Not ready for the future
• We’re on our own
• We’re no experts
![Page 9: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/9.jpg)
9
Why Puppet ?
• Advantages Puppet
– Noop (aka dry-run) option
– Enterprise support
– 3th party support
– Large and active community
– Flexible
– …
![Page 10: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/10.jpg)
10
Setup
• Using Puppet Enterprise
• One puppet master for each environment
Controlled PE upgrades
• Using Subversion as version control system
• Other teams: Only package scripts/apps
Linux Servers team: Also manages
configurations
![Page 11: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/11.jpg)
11
Setup
• Light-weight wrapper CLI-menu
– Easier for the teams
– Lowers learning curve
– Reduces risk for errors
Editing package *asslib_infrascript_colruyt* within group *infrascript*
1) ** back 10) ** Status with subversion
2) ** Create file or dir 11) ** View diff with subversion
3) ** Edit file 12) ** Resolve subversion conflict
4) ** Revert file or dir 13) ** Commit to subversion
5) ** Rename file or dir 14) ** Package
6) ** Delete file or dir 15) ** Promote to rese
7) ** Customize spec file 16) ** Promote right away
8) ** View logs from subversion 17) ** Ask to adjust configs
9) ** Update from subversion
Enter a number:
![Page 12: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/12.jpg)
12
Setup
• Managing resource definitions
– Putting resource definitions of one type of resource in
one module
• Easier to fine out what is done on our server park
• Using virtual resources
• Realized with tags when needed
![Page 13: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/13.jpg)
13
Setup
• Managing resource definitions
– Creating a wrapper around Puppet code
• Easier for the team
• Consistency in definitions
• Lowers learning curve
• Flexibility for the future
![Page 14: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/14.jpg)
14
Example: cron
• Main module: zz_cron_systlinux_colruyt
• init.pp
class zz_cron_systlinux_colruyt {
include setup
…
@zz_cron_systlinux_colruyt::conf {'netbackup':
user => 'root',
command => '/opt/openv/netbackup/check_bp_conf.sh >/var/tmp/check_bp.conf.log',
minute => '0',
hour => '7',
tag => 'zz_tag_netbackup_systlinux_colruyt',
}
}
![Page 15: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/15.jpg)
15
Example: cron
• setup.pp
– Installs the needed rpms
– Manages the service
– Changes OS defaults to our likings
![Page 16: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/16.jpg)
16
Example: cron
• conf.pp
– Defines the wrapper
– Using sensible defaults
define zz_cron_systlinux_colruyt::conf (
$user,
$command,
$ensure = present,
$hour = undef,
$minute = undef,
$weekday = undef,
$month = undef,
$monthday = undef
)
{
cron { "$user $command":
ensure => $ensure,
user => $user,
command => $command,
minute => $minute,
hour => $hour,
weekday => $weekday,
month => $month,
monthday => $monthday,
}
}
![Page 17: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/17.jpg)
17
Example: cron
• Realization happens in a separate module
– Groups all zz_* modules together
– Makes life easier for the team
define base_systlinux_colruyt::configure {
…
include zz_cron_systlinux_colruyt
Zz_cron_systlinux_colruyt::Conf <| tag == "zz_tag_${name}" |>
…
}
![Page 18: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/18.jpg)
18
Example: cron
• In netbackup module
– one line to add all zz_* modules
class netbackup_systlinux_colruyt {
base_systlinux_colruyt::configure {'netbackup_systlinux_colruyt':}
…
}
![Page 19: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/19.jpg)
19
Setup: params_systlinux_colruyt
• Sometimes server specific configs are needed
• Module params_systlinux_colruyt
– Defines several ‘global’ parameters
– Can be overridden in the node definition
– Using sensible defaults
– Syntax check
class params_systlinux_colruyt (
$topology = 'int',
)
if ! ( $topology in [ 'int', 'dmz', 'shop' ] ) {
fail("topology is not 'int', 'dmz' or 'shop' (${topology})")
}
![Page 20: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/20.jpg)
20
Setup: params_systlinux_colruyt
• Used in node definitions: node 'svlirc99' {
class {'params_systlinux_colruyt':
topology => 'dmz',
}
…
}
• Used in puppet configurations
– Templates
– .pp files
![Page 21: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/21.jpg)
21
Setup: base_systlinux_colruyt
When requiring … do this:
Minimal Red Hat setup and no puppet agent Create iso file with ‘none’ puppet config
Reservation server with as little of us as possible Create node and only include class
base_systlinux_colruyt::minimal
Reservation server with our own packages and
configuration changes
Create node and only include class
base_systlinux_colruyt::basic
Server with
• our basic tools (reports, netbackup agent, ITO
agent, sophos, …)
• other software (oracle, was, …)
Create node and include
• base_systlinux_colruyt
Includes basic tools
• Other required modules (oracle, was, …)
![Page 22: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/22.jpg)
22
Lessons learned
• Setup naming conventions
<bla>_<group>_colruyt
– Prevents naming clashes with others
– Easier to parse
• Humans
• Scripts
![Page 23: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/23.jpg)
23
Lessons learned
• When packaging rpms, always start from version
control system
– Forces people to check in
– Ensures you can always recreate rpms from scratch
![Page 24: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/24.jpg)
24
Lessons learned
• Make sure you can link a version on the server
back to the version in the VCS
– Include VCS revision number in rpm version number
• 1.<revision>-1
• 1.<revision>-<ISO date>
– Keep track of VCS revision number on Puppet master
![Page 25: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/25.jpg)
25
Lessons learned
• Keep loggings of all executions
– Created wrapper puppet run script
– Defaults to noop
![Page 26: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/26.jpg)
26
Puppet annoyances
• There are hidden ‘features’ with tags
– When using ‘::’ in tags, things get complicated
• abc::xyz
– Creates tag abc::xyz
– Creates tag abc
– Creates tag xyz
![Page 27: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/27.jpg)
27
Puppet annoyances
– Every resource in a module gets the module name as
tag
– Also classes get class name as tag
• Added ‘zz_tag_’ to tags
![Page 28: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/28.jpg)
28
Puppet annoyances
• Mount resource adds mount to fstab
– Even if the resource fails
– Working together with PuppetLabs
![Page 29: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/29.jpg)
29
Puppet annoyances
• Rpm updated outside of puppet ?
– No trigger at the next puppet run
– Needed
• Upgrades done by consultants
• Override permissions for Red Hat rpms
![Page 30: Setting up Puppet at Colruyt](https://reader033.vdocuments.site/reader033/viewer/2022052622/5592de251a28ab023c8b475f/html5/thumbnails/30.jpg)
30
Future
• Switch to Hiera ?
– Removes our ‘house-build’ abstraction system ?
– Speed up puppet runs ?
• Use Gepetto for more IDE-type editing
• Setup Puppet unit testing
• Look into MCollective
• …