taking devops to the next level - max martin
TRANSCRIPT
1
Taking Devops to the Next Level!
Max Martin Developer & Manager
@maxmartin
2
So you’ve automated your infrastructure…
• Configura7on as code • Be9er insight into the structure of your systems
• More robust deployments • But how do you reach the next level?
3
Advances in the Puppet Ecosystem
• Puppet 3.0+ • Hiera • PuppetDB • Mcollec7ve 2.0+ • Geppe9o • Puppet Forge
4
PuppetDB 0.9.0 • Big Data for Puppet • Fast, scalable Stored
Configs
Mcollective 2.0 • Sub-collectives • Advanced Filter Syntax
Puppet OSS 3.0 • Performance • Data Bindings
Puppet Enterprise 2.8 • AIX
Puppet Enterprise 2.6/2.7 • RBAC and external auth • Certificate Management
Puppet Enterprise 2.5 • Windows • Puppet Forge Integration • Puppet Data Library
Puppet Enterprise 2.0 • Discovery • Cloud Provisioning • Orchestration
2011 2012 2013 You Are Here
Puppet Enterprise 1.0 • Configuration Management • Reporting
Puppet Enterprise 3.0 • Enhanced Scalability & Performance • Dynamic Discovery • Progressive Deployment • Orchestration for Windows • Module Reusability
Puppet OSS 3.1 and 3.2 • API docs • Future parser
Mcollective 2.2 • Direct addressing • Pluggable Discovery
PuppetDB 1.4 * Report Storage
5
Puppet 3.0 – Speed and Scalability
+100% 2.7x Speed-up
6
Puppet 3.0 – Hiera func7ons + data bindings
• Hiera: hierarchical key-‐value store • Keeps site-‐specific data out of Puppet code • Parameter values are now automa7cally looked up in Hiera
• Hiera func7ons in core give 7ghter integra7on
7
Puppet 3.0 – Hiera Example node ”ntp.example.com" {
class { "ntp": servers => [ '0.us.pool.ntp.org iburst’,
'1.us.pool.ntp.org iburst',
'2.us.pool.ntp.org iburst',
'3.us.pool.ntp.org iburst'], autoupdate => false, restrict => false, enable => true, }
}
8
Puppet 3.0 – Hiera Example { "ntp::restrict" : false, "ntp::autoupdate" : false, "ntp::enable" : true, "ntp::servers" : [ "0.us.pool.ntp.org iburst", "1.us.pool.ntp.org iburst", "2.us.pool.ntp.org iburst", "3.us.pool.ntp.org iburst" ]
}
9
Puppet 3.0 – Hiera Example
node "ntp.example.com" { include ntp }
10
PuppetDB
• Fast, durable data storage for data generated by Puppet, including: – Facts – Catalogs – Reports – Exported resources
• Replaces exis7ng libraries (including storeconfigs and the old Inventory Service backend), is much faster & more reliable
11
PuppetDB -‐ Technology
• Based on PostgreSQL: reliable, ba9le-‐tested • Wri9en in Clojure: fast, mul7-‐core • Based on the JVM: can be shipped as a JAR
12
PuppetDB -‐ Technology
• Thousands of PuppetDB deployments • Hundreds of threads per install • Zero deadlocks • Zero bugs involving state
13
PuppetDB – Query Syntax and APIs • AST-‐based query syntax
["and", ["=", "type", "User”], ["=", "title", ”max"]] ["and", ["=”,["fact", "operatingsystem"], "Debian"], ["<”,["fact", "uptime_seconds"], 10000]]
• API for defining your own query syntax – Erik Dalén, Spo7fy: h9ps://github.com/dalen/puppet-‐puppetdbquery
• Rich APIs make it easy to build your own tools to consume Puppet data
14
PuppetDB – Event Inspector
15
PuppetDB -‐ Puppetboard h9ps://github.com/nedap/puppetboard
16
PuppetDB -‐ Metrics
17
MCollec7ve
A powerful orchestra7on engine
18
Mcollec7ve -‐ Subcollec7ves $ mco inventory --list-collectives * [ ==================================== ] 52 / 52 Collective Nodes ========== ===== za_collective 2 us_collective 7 uk_collective 19 de_collective 24 eu_collective 45 mcollective 52 Total nodes: 52
19
Mcollec7ve – Pluggable Discovery
• Query a file: $ mco rpc service restart service=httpd --nodes=hosts.txt
• Discover using PuppetDB: $ mco rpc service restart service=httpd -W country=uk --dm=puppetdb
• Direct addressing: $ mco rpc rpcutil ping –I example.com
20
MCollec7ve – Ruby Integra7on c = rpcclient("service") c.discover :nodes => File.readline("hosts.txt").map {|i| i.chomp}
printrpc c.restart(:service => "httpd")
21
Geppe9o
• IDE for developing Puppet modules & code • Integrates with git & SVN • Linux, OS X & Windows • Debug your Puppet modules as you develop them, and publish directly to the Puppet Forge
22
Puppet Forge
• Puppet Labs’ module repository • 1,500+ community-‐contributed modules • Improved search • Team of module engineers • Standards & best prac7ces
23
Taking you to the next level of…
• Speed • Scalability • Extensibility • Reliability • Flexibility • Agility • Community