keep an eye on your app with logging and graphs - phpbenelux 2014
DESCRIPTION
Logging, metrics and graphs are often an afterthought. This usually becomes painfully clear when something goes wrong and you feel like you're running in the dark with a blindfold on. You need to know what's going on in your application! We're going to have a deeper look at tools like logstash, statsd and friends to see how they can help developers and the business as a whole. You'll fix problems quicker and will gain more insights in how your application is doing.TRANSCRIPT
So, logging!
!1
http://www.egeniq.com [email protected]
@egeniq
PHPBenelux 2014Felix De Vliegher
Keep an eye on your app with logging and graphs
• Back-end developer
• Sysadmin
• <3 APIs
• Travelling
• @felixdv
About me
!3
About me
!4
About me
!5
About me
!6
Logging shows what exactly happens in your app. !
Metrics show the impact
Terminology
!7
Logging shows what exactly happens in your app. !
Metrics show the impact
Terminology
!8
Both are important (but for different reasons)
!9
What is this talk about?
Knowing what is going on in your application!
!10
Questionnaire
• Who uses logging in their app?
• Aggregation, filtering, manipulation?
• Who collects metrics?
• Who visualises this data in a dashboard?
• Metric correlations?
!11
Logging
!12
Ye olden days
• Logs per server
• Requires SSH for inspection
• Decentralised (for once not an advantage)
• Hard to search
!13
Goals
Make logs searchable
!14
Goals
!15
Goals
Collect and manipulate logs
!16
Goals
!17
Goals
Extract metrics
!18
Enter Mr. logstash (hipster mo’ included)
!19
Logstash
• Inputs
• Filters
• Outputs
!20
Inputs
• File
• Redis
• syslog
• varnishlog
• rabbitmq
• …
!21
Filters
Modify the input data !
• grok • mutate • dns • json • geoip • …
!22
Grok
!23
Grok
!24
Mutate
!25
Json
!26
Json
!27
Outputs
• email • graphite • irc • mongodb • pagerduty • sns • elasticsearch • …
!28
Setup
!29
Example shipper config
!30
Example indexer config
!31
Example indexer config (contd.)
!32
Logstash
!33
Elasticsearch
!34
Elasticsearch
• Distributed search and analytics
• Based on Apache lucene (but better)
• Document-oriented DB
• Easy scaling
• Restful API
• Great for storing logs!
!35
Elasticsearch
!36
!
Getting data in and out:
!
• Logstash defaults to using daily indexes
• Easy retention = delete indexes after x days
• Can be configured in other ways
Elasticsearch
!37
Kibana
!38
Kibana
• Search your logs
• The v3 rewrite is pretty awesome :-)
• HTML + JS only
• Elasticsearch project
• Almost no-config setup
!39
Kibana
!40
Kibana
!41
Kibana
!42
Kibana
!43
Kibana
!44
StatsD
• Metrics aggregation framework
• Made by Etsy
• Node.js
• UDP
• Dead easy to use
!46
Metrics
“Track everything that moves”
!47
Metrics
!48
Running StatsD
!49
StatsD config
!50
StatsD config
!51
Console backend
!52
StatsD Metric types
• Counters
• Timers
• Gauges
!53
Counters
!54
Timers
!55
Gauges
Arbitrary values:
!56
Sampling data
!57
Example: ZF Controller plugin
!58
StatsD naming
!59
But…
!60
StatsD is for collection and aggregation only
We want pretty graphs
!61
Coffee is important too
!62
StatsD config
!63
Graphite
• Store and visualise time-series data
• Carbon, whisper and graphite UI
• Rendering graphs
• Powerful function library
!64
Graphite UI
!65
StatsD => Graphite
!66
production.api.httpstatus.200
StatsD => Graphite
!67
The render API
!68
Json output
!69
Add &format=json to the render API
Functions
• cumulative(‘server*.api.logins’) • asPercent(‘connections.failed’, ‘connections.attempted’) • drawAsInfinite(‘app.deploys’) • summarize(‘app.errors’, ‘1hour’) • stacked() • movingAverage()
!• http://graphite.readthedocs.org/en/latest/functions.html
!70
StatsD
!71
Functions
• cumulative(‘server*.api.logins’) • asPercent(‘connections.failed’, ‘connections.attempted’) • drawAsInfinite(‘app.deploys’) • summarize(‘app.errors’, ‘1hour’) • stacked() • movingAverage()
!• http://graphite.readthedocs.org/en/latest/functions.html
!72
Alternative dashboards
!73
Dashing
!74
Tasseo
!75
Descartes
!76
Conclusion
• No excuse not to log
• Manage your logs with logstash and kibana
• Track everything using statsd
• Visualise with graphite dashboards
• Get insight in your application!
!77
Thank you! Questions?
http://www.egeniq.com [email protected]
@egeniq
Feedback: https://joind.in/10276 Contact: @felixdv