cfengine, 4 years later
DESCRIPTION
Everything began 4 years ago, CFEngine 3 had just been released, documentation and expertise were in short supply. We had to accept the reality of a steep learning curve. As CFEngine grew so did we. We discovered bugs, submitted pull requests, designed workarounds for various pitfalls, gained advances in productivity (thanks to the knowledge and experience gained working with CFEngine 3) and evaluated design choices available to us. This journey led us to become one of the most advanced CFEngine users in Europe. I'll recount our journey, share insights on solution architecture with CFEngine and show examples of what we had to overcome and how we achieved that using less well-known features of CFEngine. Our examples will cover advanced use of CFEngine 3 code. Finally, I will present our retrospective: what we did right, what we did wrong and share where we have got to thus far in our journey.TRANSCRIPT
Normation – CC-BY-SAnormation.com
CFEngine, 4 years laterCFEngine, 4 years later______________________________________________________
A song of code and configurationA song of code and configuration
Matthieu CERDA
Normation – CC-BY-SAnormation.com 2
Who are you ?
Name Matthieu CERDA
Email : [email protected]
Web site : http://www.normation.com
Twitter : @Kegeruneku
Job System engineer at Normation
CFEngine Enthusiast, power user and trainer
Rudder Integrator, packager
Infrastructure Team member
Normation – CC-BY-SAnormation.com 3
What are we going to talk about
CFEngine 3
● How we began with it, what we gained from it
● “Funky” use cases
● The future
Normation – CC-BY-SAnormation.com 4
Why CFEngine ?
● Few dependencies (LMDB, OpenSSL, [PCRE])
● Small memory footprint
● Highly compatible
● Working “close to the OS” (Can be seen as a flaw for some people)
http://www.cfengine.com | http://www.github.com/cfengine
Normation – CC-BY-SAnormation.com 5
CFEngine 3 : 20 % cooler !
● Created in 2009
● Complete rewrite from CF2
● Promise theory
● Based on what has been learnt from Puppet and CF2
● Alive and kicking !
Normation – CC-BY-SAnormation.com 6
e
Overview
Puppet Chef
CFEngine
e
Normation – CC-BY-SAnormation.com 7
A long path 'till today !
● Learnt to use CFEngine properly
● Acquired knowledge about best practices, worked with great people
● Helped to build Rudder from the ground
● Began working on ncf with Normation's team
● Became a trainer :)
Normation – CC-BY-SAnormation.com 8
The beginning
● Back to 2009 !!!
● Sparse documentation
● Inexperience
● Advice: Start small, to manage few machines
e
Normation – CC-BY-SAnormation.com 9
Funky example 1: Rug
● Rug was SLES 10 default package manager (Now, it is Zypper)
● Problem ? Rug relies on a Mono backend (ZMD) that hangs if you stress it too much or call it repeatedly...
● Needed a way to make an exclusion for this specific kind of machines !
Normation – CC-BY-SAnormation.com 10
2 – ALWAYS modularize when you can
● Example: Package installation definition
Normation – CC-BY-SAnormation.com 11
2 – ALWAYS modularize when you can
● Example: … and the “utility” bundle that goes with it !
Normation – CC-BY-SAnormation.com 12
A word about promises layout
● Everything begins with a bootstrap cf-agent -B <my ip address>cf-agent -B <my ip address>
● Never do everything in one file, always split your promises using a hierarchical order:
● Always separate utilities, zones and services
Normation – CC-BY-SAnormation.com 13
3 – Reporting is important
● Example: When something bad happens, you'll be happy to see where the problem is (without having to go for the debug output) !
You need a “verbose” mode. =>
● As always: modularization is important !
● Static and redundant reports are a good way to make your code fat and unreadable in the long term.
Normation – CC-BY-SAnormation.com 14
3 – Reporting is important
Normation – CC-BY-SAnormation.com 15
3 – Reporting is important
Normation – CC-BY-SAnormation.com 16
Funky example 2: Internal database bloat
● CFEngine uses a database to store internal state value (BerkeleyDB for < 3.3, TokyoCabinet for 3.3 to 3.5 and LMDB for 3.6+) => https://cfengine.com/dev/issues/2560
● BDB / TokyoCabinet do bloat when using reporting with highly volatile values (reporting a date everytime with seconds)
● Result:
Normation – CC-BY-SAnormation.com 17
Funky example 2: Internal database bloat
Solutions:
● Stop reporting all the time (Only report relevant changes)
● Mount the “state” directory on a RAMdisk http://blog.normation.com/en/2013/09/09/speed-up-your-cfengine-by-using-a-ram-disk
Bonus effect: Up to 2/3 times faster during I/O on databases
● Destroy the databases regularly (every month or week)
Normation – CC-BY-SAnormation.com 18
4 – Backup your stuff
● Example: When CFEngine does something with a file, you would like to keep a backup of this file beforehand
<= You need a backup repository.
https://cfengine.com/docs/3.5/reference-promise-types-files.html#repository
● File name is preserved, along with backup timestamp
Normation – CC-BY-SAnormation.com 19
4 – Backup your stuff
Normation – CC-BY-SAnormation.com 20
5 – Tame the agent
● Example: You want to make sure CFEngine only operates in safe environments
You need way to make CFEngine only operate in =>certain conditions
● CFEngine can be told to abort if certain conditions are not met
https://cfengine.com/docs/3.5/reference-components-cfagent.html#abortclasses
Normation – CC-BY-SAnormation.com 21
5 – Tame the agent
Normation – CC-BY-SAnormation.com 22
6 – Always a damn DNS problem (tm)
● Example: Your network interfaces resolutions are not always working properly (AWS ?)
You need to make CFEngine ignore some interfaces
● CFEngine can be told to ignore some network interfaces if needed
● It is a workaround, not a solution !
Normation – CC-BY-SAnormation.com 23
6 – Always a damn DNS problem (tm)
Normation – CC-BY-SAnormation.com 26
BONUS STAGE: Beware of the allmighty Cron
● A word of warning:
Cron(d) is not a configuration management engine !!!
● Easy to “fix” things quickly with a cron job
● KISS: Let every tool do its job, do not fall in the trap
Normation – CC-BY-SAnormation.com 27
Future
● Continue to improve Rudder
● Work on the ncf framework (See Jon's presentation !)
● Work with the Debian packaging team
● Train more people !
e
Normation – CC-BY-SAnormation.com 28
Questions ? :)Questions ? :)