centralised logging with elk stack
TRANSCRIPT
Centralised Logging
with ELK
Credit: Pasakuru76https://flic.kr/p/8K1iYi
[email protected] simonhanmer
# whoami• Simon Hanmer– IT Consultant– Sysadmin, Infrastructure architect,
server wrangler.
[email protected] simonhanmer
[email protected] simonhanmer
Why?• Lots of log files– Server– Applications–Network
• Different formats
• Multiply by many servers
[email protected] simonhanmer Credit: Kuhnmihttps://flic.kr/p/jbAnNa
What my brain feels like after trying to parse too many logs
[email protected] simonhanmer
Elasticsearch• Indexing and search engine• Near real-time• Distributed, auto-discover clustering– AWS Plugin
[email protected] simonhanmer
Logstash• Collects logs• Parses, extracts and formats data• Passes data to Elasticsearch
[email protected] simonhanmer
Logstash - examplefilter { if [file] == "/var/log/secure" and ( [syslog_message] =~ /Invalid user/ or [syslog_message] =~ /User root from/ ) { grok { add_tag => [ "LOGIN" ] match => { "syslog_message" => “user %{WORD:username} from %{IP:srcip}" } } }
}
