logging at databox
TRANSCRIPT
What is logging?/../ Logging, or commercial logging, involves cutting trees for sale as timber or pulp. The timber is used to build homes, furniture, etc and the pulp is used to make paper and paper products. Logging is generally categorized into two categories: selective and clear-cutting.
Selective logging is selective because loggers choose only wood that is highly valued, such as mahogany.
Clear-cutting is not selective. Loggers are interested in all types of wood and therefore cut all of the trees down, thus clearing the forest, hence the name- clear-cutting. /../
Logs
● System logs (syslog, syslog-ng,..● Services (nginx, mysql, cassandra,... )● Applications
○ Errors (php-fpm /var/log/php-fpm/… )○ “Events” (/var/log/my_app.log)
Architecture @ Databox
new
… more servers ...
*.log logstash forwarder
<-->lumberjackprotocol log.databox.
comlogstash elastic
search
Logstash Forwarder
Logstash
● input (lumberjack, redis, *MQ,..)
● filter (gork)● output (elasticsearch,
redis, …)
ElasticSearch
● Search server● RESTful● JSON documents● Lucene based● Highly scalable● Java based● Apache License w/
Vendor
Kibana
/../ Kibana is an open source data visualization platform that allows you to interact with your data through stunning, powerful graphics that can be combined into custom dashboards that help you share insights from your data far and wide. /../
Tips
● Log more!!!○ Log important steps○ Log exceptions○ Log errors○ Log events○ Log *
● Make sure crashes are recorded somewhere and that we all know about it!
