nagios conference 2012 - robert bolton - custom snmp oid creation

Custom SNMP OID Creation for System Monitoring

Robert V. Bolton

robert.bolton@utah.edu

2012 2

About Robert V. Bolton

Systems Administrator

Center for High Performance Computing @ University of Utah

Mac Desktops, Linux Servers, HP Networking

Infrastructure Monitoring: Nagios and Cacti

Pyton Coder

Nagios Plugins and Cacti Scrtips

Student

Computer Engineering or Electrical Engineering

Expected Graduation = ?

Geek

Board Game Enthusiast

Amateur Radio Operator (KE7ZEA)

www.robertvbolton.com

2012 3

What we’re going to cover today.

Why Bother Creating Custom OIDs

OID Tree Structure

Python Module: snmp_passpersist

Real World Example: IOStat Statistics

2012 4

Why Bother Creating Custom OIDs

• SNMP is “simple” to use to gather system statistics.

• Nagios Plugins work great with Nagios, but may not provide data to other monitoring software.

• Example: NRPE Remote Execution.

• Offload Time Consuming System Checks.

• Provides Statistics for things not normally provided by in the SNMP tree.

OID Tree Structure

• SNMP uses a hierarchical tree structure

2012

OID Tree Structure Conitued

• Numbers are used to locate information

• Each number corresponds to a specific branch of the OID tree.

• Management Information Base (MIB) files map OID numbers to human readable format

• .1.3.6.1.4 is the top of the private branch

• Vendor OIDs

• Our Custom OIDs

2012

Python Module: snmp_passpersist

• Why Python…Because I know Python!

• I believe it is possible to create OIDs with Perl as well if your prefer.

• Github: nagius/snmp_passpersist

• Requires Net-SNMP

• Installation is easy: Download source, python setup.py install, and you’re done.

• Import: import snmp_passpersist as snmp

2012

Python Module: snmp_passpersist contiued

2012

#!/usr/bin/python -u

import snmp_passpersist as snmpfrom commands import getoutput

base_oid=".1.3.6.1.4.1234.1.3"

def get_file_systems(): file_systems = getoutput("df -iP | awk '{print $1}'").split('\n') file_systems.remove('Filesystem') return file_systems

def get_inode_count(): inode_count = getoutput("df -iP | awk '{print $3}'").split('\n') inode_count.remove('IUsed') return inode_count

def update():

file_systems = get_file_systems() inode_count = get_inode_count()

counter = 0 for x in file_systems: counter += 1 pp.add_int("1.%s" % str(counter),counter) pp.add_str("2.%s" % str(counter), x)

counter = 0 for x in inode_count: counter += 1 pp.add_gau("3.%s" % str(counter), x)

pp=snmp.PassPersist(base_oid)pp.start(update,60)

Python Module: snmp_passpersist contiued

2012

Add to /etc/snmp/snmpd.conf

pass_persist .1.3.6.1.4.1234.1.3 /usr/local/bin/inodeCount.py

Results

SNMPv2-SMI::private.1234.1.3.1.1 = INTEGER: 1SNMPv2-SMI::private.1234.1.3.1.2 = INTEGER: 2SNMPv2-SMI::private.1234.1.3.1.3 = INTEGER: 3SNMPv2-SMI::private.1234.1.3.2.1 = STRING: "/dev/mapper/winkler-root"SNMPv2-SMI::private.1234.1.3.2.2 = STRING: "/dev/sda1"SNMPv2-SMI::private.1234.1.3.2.3 = STRING: "tmpfs"SNMPv2-SMI::private.1234.1.3.3.1 = Gauge32: 355793SNMPv2-SMI::private.1234.1.3.3.2 = Gauge32: 48SNMPv2-SMI::private.1234.1.3.3.3 = Gauge32: 1

Real World Example: Iostat

• Problem

• User: “The network is slow!”

• Support: To many users hammering on a file system

• Solution Requirements

• Present Users with easy to read graphs of disk statistics.

• Allow for Nagios monitoring of disk statistics

• Minimal Impact of monitoring because of Iostat

2012

Real World Example: Iostat

• Iostat

• First output from Iostat is garbage.

• Cron Job runs Iostat and stores the results in a temp file

• Python Script iostat.py

• Reads results from temp file and uses snmp_passpersist to update a custom branch of OID tree.

• SNMP OID

• Data accessible to both Nagios and Cacti.

2012

Questions?

Robert V. Boltonrobert.bolton@utah.edu

Thank you!

www.chpc.utah.edu