the state of logging on docker

The State of Logging on Docker March 26, 2015

1

Housekeeping

© 2015 Logentries. All Rights Reserved. Confidential. 2

•  Please ask questions via the chat pane

•  All registrants will receive a recorded copy of the webinar following the presentation.

•  Follow us @Logentries

Meet The Presenters


Trevor Parsons, PhD Logentries Co-founder

& Chief Scientist

Peter Elger NearForm Director of

Engineering

Bright Fulton Swipely Director of

Engineering Operations

Webinar Outline


•  Challenges with monitoring micro-services

•  What should you be logging?

•  The State of Logging on Docker •  Inside the container

•  Outside the container

•  Specialized containers

•  Useful APIs

•  Common Questions

•  What about CoreOS?

Monitoring Micro-Services


Containers: Ephemeral in Nature


What to Log?


Simple best practices for what to log:

•  When the event occurred: the timestamp

•  Structured Logs: JSON, KVP

•  Context: if someone bought a book… what book was it…

•  Unique Identifiers: request ID, User ID, Session ID, thread ID

•  Host Info: hostname & IP

•  Container Info: Image ID, Container ID, Process ID

Logging on Docker: No Standard Approach


Docker Logging Support includes:

•  Capturing container stdout logs

•  Access via Docker_remote_api

•  Access via Docker CLI •  $ docker logs [ID]

•  Will fetch the logs from a container

•  can be used to tail also

•  $ docker logs –tf –tail [ID]

Logging on Docker: No Standard Approach


•  Inside the Container •  Log from the application

•  Run a logging Daemon/collector within the container

•  Outside the Container (on the host) •  Containers write logs to the host

•  Collector runs on the host

•  Logging Container •  Collector runs in a specialized container

Inside the Container


SERVER

LOG MANAGEMENT

& ANALYTICS

PLATFORM HOST OS

DOCKER

LOGS LOGS

APP APP

CONTAINER 1 CONTAINER 2

Daemon Daemon

Inside the Container


•  Logging libs (log from your app code)

•  Install a collector inside a container

•  Use syslog inside a container

Outside the Container (on host)


SERVER

LOG MANAGEMENT

& ANALYTICS

PLATFORM HOST OS

DOCKER

CONTAINER 1 CONTAINER 2

APP APP

LOGS LOGS

LOGS AGENT

Outside the container (on host)


•  Configure a container to bind to a volume & write logs to the volume

•  Write logs to stdout

•  Collector runs on the host

Logging Containers


SERVER

LOG MANAGEMENT

& ANALYTICS

PLATFORM HOST OS

DOCKER

CONTAINER 1 CONTAINER 2 LOGGING CONTAINER

APP APP Agent

LOGS LOGS

LOGS

Logging Containers


•  Syslog in a container

•  Collector in a container •  E.g. Logentries container will collect Stats and Logs

•  https://github.com/nearform/docker-logentries

•  Logspout: https://github.com/gliderlabs/logspout

Common Questions


•  Do I need to rotate any logs?

•  Is there a big performance hit? •  E.g. collecting metrics from every container

•  Any security concerns?

What about CoreOS?


•  Enables warehouse-scale computing

•  Used for managing clusters of containers •  Distributed process manager, Fleet (based on systemd)

•  etcd for service discovery and global storage of your config settings

•  How you can log from your CoreOS Clusters •  Journal - a modern logging system (e.g. JSON export, forward sealing)

•  Journal2Logentries: https://github.com/kelseyhightower/journal-2-logentries

Useful Resources:


•  Docker CLI

•  https://docs.docker.com/reference/commandline/cli/

•  Docker Remote API

•  https://docs.docker.com/reference/api/docker_remote_api/

•  Stats API (From Docker 1.5)

•  https://github.com/docker/docker/pull/9984 (e.g. used by Logentries container)

•  Docker RunTime Metrics

•  Uses the native cgroup accounting metrics

•  https://docs.docker.com/articles/runmetrics/ (e.g. used by DataDog)

•  Docker Ruby Gem

•  https://github.com/swipely/docker-api

Useful Resources: Blogs


•  The State of Logging on Docker •  https://blog.logentries.com/2014/03/the-state-of-logging-on-docker/

•  What is the Docker Stats API •  https://blog.logentries.com/2015/02/what-is-the-docker-stats-api/

•  Docker Log Management Using Fluentd •  http://jasonwilder.com/blog/2014/03/17/docker-log-management-using-fluentd/

•  How to Centralize Logs from CoreOS Clusters •  https://blog.logentries.com/2015/03/how-to-centralize-logs-from-coreos-clusters/

Our Docker Community Pack


Getting Started


•  Don’t have a Logentries Free Account Yet?

•  Sign-up for a free 30-day trial: http://logentries.com/quick-start

•  Docker & Node.Js Development:

•  http://www.nearform.com/

•  http://nscale.nearform.com/

•  Contact us directly!

•  [email protected]



•  Let’s take a few questions!

Thank you!

the state of logging on docker

Technology

logentries container

container id

container log

docker logs id

container stdout logs

app app logs logs logs

container use syslog

phd logentries