Download - Monitoring in the cloud with Puppet
![Page 1: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/1.jpg)
Monitoring in the CloudMonitoring in the Cloudwith Puppet
PuppetCamp Ghent 2013
Kris Buytaert
![Page 2: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/2.jpg)
Kris BuytaertKris Buytaert● I used to be a Dev,I used to be a Dev,● Then Became an OpThen Became an Op● Chief Trolling Officer and Open Source Chief Trolling Officer and Open Source
Consultant @inuits.euConsultant @inuits.eu● Everything is an effing DNS ProblemEverything is an effing DNS Problem● Building Clouds since before the bookstoreBuilding Clouds since before the bookstore● Some books, some papers, some blogsSome books, some papers, some blogs● Evangelizing devopsEvangelizing devops
![Page 3: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/3.jpg)
Do you know what your children do at 5 am Do you know what your children do at 5 am in the morning ?in the morning ?
● Sleeping ?Sleeping ?
● Crashing a Party ?Crashing a Party ?
● Why are the cops at your front door ?Why are the cops at your front door ?
● What happened ?What happened ?
● How long have they been gone ?How long have they been gone ?
![Page 4: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/4.jpg)
devopsdevops● CultureCulture
● (Lean)(Lean)
● AutomationAutomation
● MeasurementMeasurement
● SharingSharing
Damon Edwards and John WillisDamon Edwards and John Willis
Gene KimGene Kim
![Page 5: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/5.jpg)
Monitoring is usually an Monitoring is usually an aftertoughtaftertought
ENOBUDGET, ENOTIMEENOBUDGET, ENOTIME
![Page 6: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/6.jpg)
A sub movementA sub movement● #monitoringsucks trending#monitoringsucks trending
● https://github.com/monitoringsucks/https://github.com/monitoringsucks/
● 2008 Study :Nagios + Friends2008 Study :Nagios + Friends
● 2011 Conclusion : Nagios/Icinga are 2011 Conclusion : Nagios/Icinga are the only automatable alternatives the only automatable alternatives
![Page 7: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/7.jpg)
#monitoringsucks#monitoringsucksMonitoring is AWESOME. Metrics are AWESOME. I love it. Here's what I don't love: Monitoring is AWESOME. Metrics are AWESOME. I love it. Here's what I don't love:
● Having my hands tied with the model of host and service bindings. Having my hands tied with the model of host and service bindings.
● Having to set up "fake" hosts just to group arbitrary metrics together Having to set up "fake" hosts just to group arbitrary metrics together
● Having to either collect metrics twice - once for alerting and another for trending Having to either collect metrics twice - once for alerting and another for trending
● Only being able to see my metrics in 5 minute intervals Only being able to see my metrics in 5 minute intervals
● Having to chose between shitty interface but great monitoring or shitty monitoring Having to chose between shitty interface but great monitoring or shitty monitoring but great interface but great interface
● Dealing with a monitoring system that thinks IT is the system of truth for my Dealing with a monitoring system that thinks IT is the system of truth for my environment environment
● Not actually having any real choicesNot actually having any real choices
John Vincent (@lusis) on his blog http://lusislog.blogspot.com/2011/06/why-John Vincent (@lusis) on his blog http://lusislog.blogspot.com/2011/06/why-monitoring-sucks.htmlmonitoring-sucks.html
![Page 8: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/8.jpg)
![Page 9: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/9.jpg)
Let's forget aboutLet's forget about● GroundWorkGroundWork
● ZenossZenoss
● ZabbixZabbix
● HypericHyperic
● CactiCacti
● MuninMunin
● BigBrotherBigBrother
![Page 10: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/10.jpg)
What's different in the cloud ?What's different in the cloud ?
● ScaleScale
● VelocityVelocity
● ChangeChange
![Page 11: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/11.jpg)
![Page 12: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/12.jpg)
![Page 13: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/13.jpg)
![Page 14: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/14.jpg)
ChallengesChallenges● ReproducabilityReproducability
● SpeedSpeed
● AuditingAuditing
● Keeping stuff in sync Keeping stuff in sync
![Page 15: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/15.jpg)
Thank Thank godgod Luke for Puppet Luke for Puppet
![Page 16: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/16.jpg)
And Mark for CfengineAnd Mark for Cfengine
![Page 17: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/17.jpg)
}}#monitoringsucks #monitoringsucks 20132013
![Page 18: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/18.jpg)
What we wantWhat we want
● Small , wel suited componentsSmall , wel suited components
• CollectCollect
• Transport / MangleTransport / Mangle
• Analyse / ActAnalyse / Act
• VisualizeVisualize
![Page 19: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/19.jpg)
![Page 20: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/20.jpg)
CollectdCollectd● CollectsCollects
● Zillion PluginsZillion Plugins
• Nginx,apache, mysql, diskNginx,apache, mysql, disk
● Graphite Carbon PluginGraphite Carbon Plugin
● Send metrics to graphiteSend metrics to graphite
● https://github.com/KrisBuytaert/pupphttps://github.com/KrisBuytaert/puppet-collectdet-collectd
![Page 21: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/21.jpg)
● Collect from Collect from anywhereanywhere
● FilterFilter
● Send anywhereSend anywhere
● Queuing supportQueuing support
![Page 22: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/22.jpg)
Puppet and LogstashPuppet and Logstash● https://github.com/KrisBuytaert/vagrant-puppet-logstashhttps://github.com/KrisBuytaert/vagrant-puppet-logstash
● Includes Logstash / Kibana / Grok / ES modules as Includes Logstash / Kibana / Grok / ES modules as submodules ! submodules !
git clonegit clone
● git submodule init git submodule init
● git submodule update git submodule update
● vagrant up vagrant up
![Page 23: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/23.jpg)
Graphite Graphite ● Graphing at ScaleGraphing at Scale
● Graphing at EaseGraphing at Ease
● Any metric is a graphAny metric is a graph
● echo "somestring $somevalue $timestamp" | echo "somestring $somevalue $timestamp" | nc <%= graphitehost %> 2003nc <%= graphitehost %> 2003
![Page 24: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/24.jpg)
Graphite ComposerGraphite Composer
x
![Page 25: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/25.jpg)
Graphite APIGraphite API
![Page 26: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/26.jpg)
Gdash In actionGdash In action
![Page 27: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/27.jpg)
Puppet and GraphitePuppet and Graphite● https://github.com/KrisBuytaert/vagrant-graphite/https://github.com/KrisBuytaert/vagrant-graphite/
● Includes Graphite / Gdash / Jmxtrans / Logster / Collectd / Includes Graphite / Gdash / Jmxtrans / Logster / Collectd / Statsd / Tattle and more modules as submodules ! Statsd / Tattle and more modules as submodules !
● git clonegit clone
● git submodule init git submodule init
● git submodule update git submodule update
● vagrant up vagrant up
![Page 28: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/28.jpg)
Collectd & GraphiteCollectd & Graphite
![Page 29: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/29.jpg)
Configuring a ServiceConfiguring a Service● PackagePackage
● ConfigConfig
● Service Service
<- we pray status is correct<- we pray status is correct
![Page 30: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/30.jpg)
Apache Example:Apache Example:
![Page 31: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/31.jpg)
Definition of Done:Definition of Done:
Done means : being monitoredDone means : being monitored
![Page 32: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/32.jpg)
Icinga ?Icinga ?• Isn't nagios dead ?Isn't nagios dead ?
• Vibrant CommunityVibrant Community
• Throw great parties in NurnbergThrow great parties in Nurnberg
• Nobody can pronounce it anyhowNobody can pronounce it anyhow
• https://github.com/Inuits/puppet-icinga/https://github.com/Inuits/puppet-icinga/
![Page 33: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/33.jpg)
Monitoring BaselineMonitoring Baseline● Deploy a host,Deploy a host,
● Add it to the monitoringAdd it to the monitoring
● Add collection toolsAdd collection tools
● Add check definitionsAdd check definitions
● Update the monitoring tool configUpdate the monitoring tool config
![Page 34: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/34.jpg)
Stored ConfigsStored Configs
![Page 35: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/35.jpg)
Collection and ExportCollection and Export
Export :Export :
@@resource { @@resource {
... }... }
Collect:Collect:
Resource <<| Resource <<| query |>>query |>>
Clean out nodes that dissapearClean out nodes that dissapear
puppet node clean puppet node clean
![Page 36: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/36.jpg)
Exporting and Collecting Exporting and Collecting
![Page 37: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/37.jpg)
Monitoring a VhostMonitoring a Vhost
![Page 38: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/38.jpg)
It is not in production if it is not It is not in production if it is not monitoredmonitored
![Page 39: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/39.jpg)
Triggers on GraphsTriggers on Graphs● Export Java MetricsExport Java Metrics
● JMXTransJMXTrans
● Export JMXConfigsExport JMXConfigs
● Configure NRPE CheckConfigure NRPE Check
● Export NagiosCheckExport NagiosCheck
● Collect JMX Exports on Collect JMX Exports on JMXTransNodeJMXTransNode
● Graph EmGraph Em
Collect Nagios Configs Collect Nagios Configs on Nagios Serveron Nagios Server
![Page 40: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/40.jpg)
Triggers on GraphsTriggers on Graphs
![Page 41: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/41.jpg)
Triggers on GraphsTriggers on Graphs
![Page 42: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/42.jpg)
Conclusion:Conclusion:● Reproducable monitoring setupReproducable monitoring setup
● Dynamically generated monitoring Dynamically generated monitoring configconfig
● Code is available at Code is available at github.com/KrisBuytaert/snippetsgithub.com/KrisBuytaert/snippets
![Page 43: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/43.jpg)
But,But,(pronounced in Ozzy dialect)(pronounced in Ozzy dialect)
![Page 44: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/44.jpg)
htt
p:/
/ww
w.k
art
ar.
net/
201
3/0
1/m
onit
ori
ng
-suck
s---
a-r
ant/
![Page 45: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/45.jpg)
#monitoringlove#monitoringlove
![Page 46: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/46.jpg)
HomeworkHomeworkSensuSensu
ExtrememonExtrememon
......
RiemannRiemann
EsperEsper
![Page 47: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/47.jpg)
[email protected]@inuits.eu
Further ReadingFurther Reading@krisbuytaert @krisbuytaert http://www.krisbuytaert.be/blog/http://www.krisbuytaert.be/blog/http://www.inuits.eu/http://www.inuits.eu/
InuitsInuits
Duboistraat 50Duboistraat 502060 Antwerpen2060 AntwerpenBelgiumBelgium891.514.231891.514.231
+32 475 961221+32 475 961221
![Page 48: Monitoring in the cloud with Puppet](https://reader038.vdocuments.site/reader038/viewer/2022103000/5549b465b4c905fc7f8b4847/html5/thumbnails/48.jpg)
http://loadays.orghttp://loadays.org