osmc 2014: why we do monitoring wrong | michael medin
DESCRIPTION
In many IT fields we have in the past decade seen a truly revolution in terms of changes whereas others have been slow (I am looking at you leading open source monitoring tool). I think it is high time for a revolutionary change and more importantly I think it is time for monitoring to become not only a tool for the IT support team but for the entire enterprise. This will NOT be about some fancy new monitoring tool or even about NSClient++. Instead I will show how we can change the tools we use today to make monitoring a bit more modern and take it into the new millennia (yes, the one we are already in). ...But since I have added which I think is the coolest feature ever to NSClient++ I might (if I have time) spend 5 minutes (at the end) showing the new WEB UI for NSClient++ :)TRANSCRIPT
WrongWhy we do
…frustration…
dev not ops
Please don’t be angry!
Some times I am busy
TAKE:1
check_disk -w 80 –c 90
Slack
-w 80 –c 901gb 1tb 1pb
0.2g 219g 225 179g
Better?
-w $ARG1$1gb 1tb 1pb
0.2g 22g 2 251g
80% 98% 99,8%
Magic?
0
500
1000
1500
2000
2500
3000
Value Warning Critical
The problem
The first alert
On call staff alerted
Lost time
Things went bad!
0
500
1000
1500
2000
2500
3000
Value Warning Critical
The problem
The first alert
On call staff alerted
Lost time
No Slack
-w trend-line1gb 1tb 1pb
0g 0g 0g
Works With Everything!
Magic?
TAKE:2
planningWhat aboutCapacity
Bounds?
Alarm clock
0
500
1000
1500
2000
2500
Warning Critical HDD 1 HDD 2
Full
How long?
> 80%
> 90%
0
500
1000
1500
2000
2500
Warning Critical HDD 1 HDD 2
Full
warn=full in less than x weeks
Photo Credit Howard Dickins
Alarm clock
2 hours before work
0
500
1000
1500
2000
2500
3000
Value Warning Critical
The first alert
On call staff alerted
No basic math!
Magic?
check_disk -w 80 –c 90
0
500
1000
1500
2000
2500
Value Warning Critical
Backup
check_disk check_disk_backup
0
500
1000
1500
2000
2500
Value Warning Critical
check_disk warn=usage>80% and not_backup
Backup
No it is tags
Magic?
Other
TAKE:1
check_load -w 1 –c 2
Bad CPU load?80%
90% 100%
0%
0
10
20
30
40
50
60
70
80
90
100
Value Yesterday Last Week
No, still math
Magic?
check_load -w 1 –c 2
High Load???GOOD BAD
DO WE CARE?
No, still math
Magic?
TAKE:2
check_mem -w 80 –c 90
Bad Memory?80%
90% 100%
0%
Managed…Java
JVM .net
CLR
check_mem
check_jmx check_counter
check_wmi
check_disk -w 80 –c 90
FULL DISK???GOOD BAD
DO WE CARE?
Because we can?Why do we monitor?
Because we do?Because…
Business!Technology
NOT
IT
BUSINESS
No, common sense
Magic?
TAKE:1
Nagios™ is Old
EasySimple
What we always do
bischeckAddons
Other solutions“the new stuff”
forks
Why a tool?
fast forward 15 yearsNagios™Naemon™could do this!
Why an addon?
cron*/5 * * * * wrap.sh mycheck
#!/bin/bash $* if [ $? == 1 ];then send-email.sh fi;
TAKE:2
TAKE:1
TAKE:2
Photo by Olga Berrios
Information about NSClient++ http://nsclient.org
facebook.com/nsclient
Slides, and examples http://nsclient.org/nscp/conferances
My Blog http://blog.medin.name