Curs 03Monitorizarea ret,elei
Servicii avansate pentru ISP
6 martie 2017
SAISP Curs 03, Monitorizarea ret,elei 1/49
Outline
Introducere
SNMP
Round Robin Database
Nagios
Alte solut, ii de monitorizare
Monitorizarea traficului
Concluzii
Intrebari
SAISP Curs 03, Monitorizarea ret,elei 2/49
Monitorizarea ret,elei
I Network monitoring
I Monitorizare constanta a unei ret,ele de calculatoare
I Componente care cedeaza sau care funct, ioneaza ıncet
I Utilizare curenta de resurse
I Asigurarea unui nivel de disponibilitate
I Notificarea administratorului
I Subset al network management (autentificare, autorizare, loadbalancing, accounting etc.)
SAISP Curs 03, Monitorizarea ret,elei 3/49
Monitorizarea ret,elei
I Network monitoring
I Monitorizare constanta a unei ret,ele de calculatoare
I Componente care cedeaza sau care funct, ioneaza ıncet
I Utilizare curenta de resurse
I Asigurarea unui nivel de disponibilitate
I Notificarea administratorului
I Subset al network management (autentificare, autorizare, loadbalancing, accounting etc.)
SAISP Curs 03, Monitorizarea ret,elei 3/49
Monitorizarea ret,elei
I Network monitoring
I Monitorizare constanta a unei ret,ele de calculatoare
I Componente care cedeaza sau care funct, ioneaza ıncet
I Utilizare curenta de resurse
I Asigurarea unui nivel de disponibilitate
I Notificarea administratorului
I Subset al network management (autentificare, autorizare, loadbalancing, accounting etc.)
SAISP Curs 03, Monitorizarea ret,elei 3/49
Disponibilitate ın monitorizare
I Disponibilitate (availability) – utilizatorul poate accesasistemul
I DowntimeI planificat (patching/updates, system configuration)I neplanificat (hardware faults, power outage, security breaches,
software failures)
I Uptime – procent de uptimeI 99.9%, 99.99%, 99.998%, 99.9999% (six nines) (31.5
secunde/an downtime)I folosite ın marketing – the myth of the nines
I Uptime != disponibilitate (fizic up, dar inaccesibil din ret,ea)
I SLA – Service Level Agreement
SAISP Curs 03, Monitorizarea ret,elei 4/49
Disponibilitate ın monitorizare
I Disponibilitate (availability) – utilizatorul poate accesasistemul
I DowntimeI planificat (patching/updates, system configuration)I neplanificat (hardware faults, power outage, security breaches,
software failures)
I Uptime – procent de uptimeI 99.9%, 99.99%, 99.998%, 99.9999% (six nines) (31.5
secunde/an downtime)I folosite ın marketing – the myth of the nines
I Uptime != disponibilitate (fizic up, dar inaccesibil din ret,ea)
I SLA – Service Level Agreement
SAISP Curs 03, Monitorizarea ret,elei 4/49
Disponibilitate ın monitorizare
I Disponibilitate (availability) – utilizatorul poate accesasistemul
I DowntimeI planificat (patching/updates, system configuration)I neplanificat (hardware faults, power outage, security breaches,
software failures)
I Uptime – procent de uptimeI 99.9%, 99.99%, 99.998%, 99.9999% (six nines) (31.5
secunde/an downtime)I folosite ın marketing – the myth of the nines
I Uptime != disponibilitate (fizic up, dar inaccesibil din ret,ea)
I SLA – Service Level Agreement
SAISP Curs 03, Monitorizarea ret,elei 4/49
Disponibilitate ın monitorizare
I Disponibilitate (availability) – utilizatorul poate accesasistemul
I DowntimeI planificat (patching/updates, system configuration)I neplanificat (hardware faults, power outage, security breaches,
software failures)
I Uptime – procent de uptimeI 99.9%, 99.99%, 99.998%, 99.9999% (six nines) (31.5
secunde/an downtime)I folosite ın marketing – the myth of the nines
I Uptime != disponibilitate (fizic up, dar inaccesibil din ret,ea)
I SLA – Service Level Agreement
SAISP Curs 03, Monitorizarea ret,elei 4/49
Principii de monitorizare
I “input” de la utilizatori
I “less is more”
I Abordare iterativa (ce este important? ce este critic?)
I Start high-level and work down the application stack
I Monitorizare de servicii s, i dispozitive
I MOM – Monitor of Monitors
SAISP Curs 03, Monitorizarea ret,elei 5/49
Masurarea/monitorizarea traficului
I Captura (selectiva) de trafic
I Contabilizare/accounting
I Logging, rendering
I Util pentru: masurarea ıncarcarii ret,elei, tipuri de trafic,stat, iile/ret,elele cele mai ıncarcate, QoS, detectare flood-uri,funct, ionalitatea unui protocol, debugging
SAISP Curs 03, Monitorizarea ret,elei 6/49
Outline
Introducere
SNMP
Round Robin Database
Nagios
Alte solut, ii de monitorizare
Monitorizarea traficului
Concluzii
Intrebari
SAISP Curs 03, Monitorizarea ret,elei 7/49
SNMP
I Simple Network Management Protocols
I SNMP password (community string)
I v1, v2c - necriptat (parola = community string)
I v3 - criptatI OID – object identifiers (statistici operat, ionale)
I .1.3.6.1.4.1.789.1.6.4.8.0I .iso.org.dod.internet.private.enterprises.netapp.
netapp1.raid.diskSummary.diskSpareCount.0
I fully qualified OID (.iso.org.dod.internet.private)I aproape toate sunt urmate de oid-ul enterprisesI si de vendor ID (alocat de IANA)I http://www.iana.org/assignments/enterprise-numbers
SAISP Curs 03, Monitorizarea ret,elei 8/49
SNMP
I Simple Network Management Protocols
I SNMP password (community string)
I v1, v2c - necriptat (parola = community string)
I v3 - criptatI OID – object identifiers (statistici operat, ionale)
I .1.3.6.1.4.1.789.1.6.4.8.0I .iso.org.dod.internet.private.enterprises.netapp.
netapp1.raid.diskSummary.diskSpareCount.0I fully qualified OID (.iso.org.dod.internet.private)I aproape toate sunt urmate de oid-ul enterprisesI si de vendor ID (alocat de IANA)I http://www.iana.org/assignments/enterprise-numbers
SAISP Curs 03, Monitorizarea ret,elei 8/49
Management Information Base
I MIB – Management Information BaseI Fis, iere ce definesc funct, iile OID-urilor
I .iso.org.dod.internet.private.enterprises.netapp.netapp1.raid.diskSummary.diskSpareCount.0
I NETWORK-APPLIANCE-MIB::diskSpareCount.0
I MIB_Name::Unique_Key.instance
I Ce reprezinta o instanta ın acest caz?
I numarul disk-ului
SAISP Curs 03, Monitorizarea ret,elei 9/49
Management Information Base
I MIB – Management Information BaseI Fis, iere ce definesc funct, iile OID-urilor
I .iso.org.dod.internet.private.enterprises.netapp.netapp1.raid.diskSummary.diskSpareCount.0
I NETWORK-APPLIANCE-MIB::diskSpareCount.0
I MIB_Name::Unique_Key.instance
I Ce reprezinta o instanta ın acest caz?I numarul disk-ului
SAISP Curs 03, Monitorizarea ret,elei 9/49
OID & MIB
SAISP Curs 03, Monitorizarea ret,elei 10/49
SNMP pe Linux
I apt-get install snmp snmpd
I 1 root@valhalla:/tmp# snmp2 snmpbulkget snmpdf snmpnetstat snmptest
snmpvacm
3 snmpbulkwalk snmpget snmpset
snmptranslate snmpwalk
4 snmpconf snmpgetnext snmpstatus snmptrap
5 snmpdelta snmpinform snmptable snmpusm
I 1 root@valhalla:/tmp# netstat -ulpn | grep snmpd2 udp 0 0 127.0.0.1:161 0.0.0.0:*
2983/snmpd
SAISP Curs 03, Monitorizarea ret,elei 11/49
snmpd
I /etc/default/snmpd
I /etc/init.d/snmpd
I /etc/snmpd/snmpd.conf
I man 5 snmpd.conf
I configurare community stringI com2sec readonly localhost public
I root@valhalla:/tmp# snmpwalk -Os -c public -v 1
localhost | wc -l
2678
SAISP Curs 03, Monitorizarea ret,elei 12/49
Statistici SNMP
I system uptimeI $ snmpget -v 2c -c public localhost
.1.3.6.1.2.1.1.3.0
iso.3.6.1.2.1.1.3.0 = Timeticks: (55298)
0:09:12.98
I total RAM in machineI $ snmpget -v 2c -c public localhost
.1.3.6.1.4.1.2021.4.5.0
iso.3.6.1.4.1.2021.4.5.0 = INTEGER: 1929536
I http://www.debianadmin.com/
linux-snmp-oids-for-cpumemory-and-disk-statistics.
html
I scli
SAISP Curs 03, Monitorizarea ret,elei 13/49
Outline
Introducere
SNMP
Round Robin Database
Nagios
Alte solut, ii de monitorizare
Monitorizarea traficului
Concluzii
Intrebari
SAISP Curs 03, Monitorizarea ret,elei 14/49
RRDTool
I Dezvoltat de Tobias Oetiker, system manager la Swiss FederalInstitute of Technology
I apt-get install rrdtool
I Date stocate ın fisiere .rrd cu rol de baza de date
I Genereaza grafice pe baza datelor stocateI Operatiile se realizeaza cu utilitarul rrdtool:
I createI updateI graph
SAISP Curs 03, Monitorizarea ret,elei 15/49
Caracteristici baze de date RRD
I Dimensiunea este constanta, stabilita la creare
I Configurabil sa stocheze rata de schimbare, nu valoarea ın sineI Are nevoie de valori la intervale predefinite de timp
I ın caz ca nu primeste, stocheaza valoarea speciala UNKNOWN
I Pentru fiecare valoare stocheaza si un timestampI date stocate ca serii de timp
I Defineste variabile si modul de arhivare
SAISP Curs 03, Monitorizarea ret,elei 16/49
Exemplu RRD
rrdtool create target.rrd \
--start 1023654125 \
--step 300 \
DS:mem:GAUGE:600:0:671744 \
RRA:AVERAGE:0.5:12:24 \
RRA:AVERAGE:0.5:288:31
I DS:variable_name:DST:heartbeat:min:max
I DS - Data Source
I DST - Data Source Type
I hearbeat - timpul maxim de astepare al unei noi valori
I min, max - intervalele ıntre care se accepta o noua valoare
SAISP Curs 03, Monitorizarea ret,elei 17/49
Data Source Types
I COUNTER - rata de schimbare
I DERIVE - permite valori negative
I ABSOLUTE - presupune valoarea anterioara ca fiind 0
I GAUGE - valoarea ın sine
Values = 300, 600, 900, 1200
Step = 300 seconds
COUNTER DS = 1, 1, 1, 1
DERIVE DS = 1, 1, 1, 1
ABSOLUTE DS = 1, 2, 3, 4
GAUGE DS = 300, 600, 900, 1200
SAISP Curs 03, Monitorizarea ret,elei 18/49
Data Source Types
I COUNTER - rata de schimbare
I DERIVE - permite valori negative
I ABSOLUTE - presupune valoarea anterioara ca fiind 0
I GAUGE - valoarea ın sine
Values = 300, 600, 900, 1200
Step = 300 seconds
COUNTER DS = 1, 1, 1, 1
DERIVE DS = 1, 1, 1, 1
ABSOLUTE DS = 1, 2, 3, 4
GAUGE DS = 300, 600, 900, 1200
SAISP Curs 03, Monitorizarea ret,elei 18/49
Round Robin Archive
I RRA:CF:xff:step:rows
I CF: AVERAGE, MINIMUM, MAXIMUM, LAST
I step - numarul de valori agregate
I numarul de agregari retinute
rrdtool create target.rrd \
--start 1023654125 \
--step 300 \
DS:mem:GAUGE:600:0:671744 \
RRA:AVERAGE:0.5:12:24 \
RRA:AVERAGE:0.5:288:31
SAISP Curs 03, Monitorizarea ret,elei 19/49
Outline
Introducere
SNMP
Round Robin Database
Nagios
Alte solut, ii de monitorizare
Monitorizarea traficului
Concluzii
Intrebari
SAISP Curs 03, Monitorizarea ret,elei 20/49
Nagios
I “The Industry Standard in IT Infrastructure Monitoring”
I http://www.nagios.org/
I Monitorizarea statiilorI starea resurselor (memorie, procesor, disc)I temperatura
I Monitorizarea serviciilor (protocoale de retea)I SMTPI HTTPI SSH
SAISP Curs 03, Monitorizarea ret,elei 21/49
Nagios (cont.)
I Genereaza alerte catre contactele definite
I Genereaza statistici/rapoarte
I Calculul automat al disponiblitatii
I Extensibil prin plugin-uri
I Interfata web (vizualizare status echipamente)
SAISP Curs 03, Monitorizarea ret,elei 22/49
Obiecte s, i definit, ii
I timeperiodI operational hoursI timeperiods.cfg
I commandI map to external commandsI commands.cfg
I contact, contactgroupI send notifications (to contactgroups)I contacts.cfg, contactgroups.cfg
I host, hostgroupI physical entities (servers, routers)I hosts.cfg, hostgroups.cfg
I service, servicegroupI hosts provide services (majority of configurations)I service.cfg, servicegroups.cfg
SAISP Curs 03, Monitorizarea ret,elei 23/49
Interactiuni ıntre obiecte
Service
Contact
Host
Service Group
Time Period
Host Group
Contact Group
SAISP Curs 03, Monitorizarea ret,elei 24/49
Instalare
I apt-get install nagios3 nagios-plugins
I /etc/nagios3/nagios.cfg
I /etc/nagios3/
I /etc/nagios3/apache.conf
(http://hostname/nagios3/)
I /etc/nagios-plugins/config/
I /etc/init.d/nagios3 start|stop|restart
SAISP Curs 03, Monitorizarea ret,elei 25/49
Comenzi de monitorizare
I prin pluginuri
I /usr/lib/nagios/plugins/check_*
I configurate ın /etc/nagios-plugins/config/*.cfgkoala:/etc/nagios-plugins/config# cat apt.cfg
# ’check_apt’ command definition
define command{
command_name check_apt
command_line /usr/lib/nagios/plugins/check_apt
}
# ’check_apt_distupgrade’ command definition
define command{
command_name check_apt_distupgrade
command_line /usr/lib/nagios/plugins/check_apt -d
}
SAISP Curs 03, Monitorizarea ret,elei 26/49
Funct, ionare comenzi de monitorizare / plugin-uri
I pot fi scrise de utilizatorI programe care ıntorc un cod de eroare specific
I 0 – OKI 1 – WarningI 2 – CriticalI 3 – Unknown
define service{
hostgroup_name procload
service_description Total Processes
check_command check_procs!200!400
...
SAISP Curs 03, Monitorizarea ret,elei 27/49
Template-uri
I “capture redundant definitions inside special definitions”I pentru host-uri s, i servicii
# This is my template
define host{
name hostTemplate
check_command check-host-alive
max_check_attempts 5
contact_groups admins
notification_interval 30
notification_period 24x7
notification_options d,u,r
register 0
}
# myHost is shorter now that it inherits from hostTemplate
define host{
host_name myHost
alias My Favorite Host
address 192.168.1.254
parents myotherhost
use hostTemplate
}
SAISP Curs 03, Monitorizarea ret,elei 28/49
Procesul de configurare Nagios
I creare template-uri (skeleton files) pentru host-uri s, i servicii
I creare fis, iere de configurare pentru host-uri s, i servicii
I creare contacte s, i grupuri de contacte
I configurat comenzi folosite/plugin-uri etc.
SAISP Curs 03, Monitorizarea ret,elei 29/49
Plugin-uri
I comenzi care extind funct, ionalitatea data de nagios-core
I http://nagiosplugins.org/
I ./check_* --help
SAISP Curs 03, Monitorizarea ret,elei 30/49
Scenariu de configurare
I check_ssl_certI http://exchange.nagios.org/directory/Plugins/
Network-Protocols/HTTP/check_ssl_cert/details
I dezarhivare s, i copiere ın $USER1$ (configurat ın/etc/nagios3/resource.cfg)
I ın commands.cfg se adauga comandadefine command{
command_name check_ssl_cert
command_line $USER1$/check_ssl_cert -r /etc/ssl/certs/ -H $HOSTADDRESS$
}
I se adauga un hostgroup ın hostgroups.cfgI se configureaza serviciul ın services.cfg
# define a service to check SSL certificates
define service{
hostgroup_name ssl_cert
service_description valid SSL certificates
check_command check_ssl_cert
use generic-service
notification_interval 0
}
SAISP Curs 03, Monitorizarea ret,elei 31/49
nagstamon
I http://nagstamon.sourceforge.net/
I Nagios status monitor for the desktop
I Systray sau status bar
I Informatii detaliate la mouse hover
I Acces rapid prin SSH/RDP/VNC din meniul contextual
I Poate controla ce notificari sa primeasca
I Pentru un server Nagios, trebuie completate campurileServer URL s, i Server CGI URL
SAISP Curs 03, Monitorizarea ret,elei 32/49
NRPE
I Nagios Remote Plugin ExecutorI check_nrpe ruleaza local, un daemon ruleaza pe host-urile
monitorizateI check_nrpe solicita catre daemon executarea de comenzi
I /etc/nagios/nrpe.cfg
SAISP Curs 03, Monitorizarea ret,elei 33/49
Nagios XI
I Varianta comerciala a Nagios-Core
I Simplifica administrarea
I Grafice folosind datele colectate
I Sistem de ticketingI User interface
I framework pentru configurare si administrareI customizabil pentru fiecare utilizator
SAISP Curs 03, Monitorizarea ret,elei 34/49
Outline
Introducere
SNMP
Round Robin Database
Nagios
Alte solut, ii de monitorizare
Monitorizarea traficului
Concluzii
Intrebari
SAISP Curs 03, Monitorizarea ret,elei 35/49
MRTG
I Multi Router Traffic GrapherI monitorizarea ıncarcarii traficului
I ın momentul de fat, a creeaza grafice s, i statistici din situat, iidiverse
I RRDtool (Round Robin Database Tool) – grafice
I apt-get install mrtg
SAISP Curs 03, Monitorizarea ret,elei 36/49
Cacti
I network graphing solution
I MySQL database
I SNMP support
I RRDTool
I data retrieval, data storage, data presentation
I apt-get install cacti
SAISP Curs 03, Monitorizarea ret,elei 37/49
Zabbix
I enterprise-class open source distributed monitoring solutionI server-agent
I server care colecteaza informat, iileI un agent ruleaza pe fiecare hostI datele sunt stocate ıntr-o baza de date
I apt-get install zabbix-server-mysql
I apt-get install zabbix-agent
I http://kkovacs.eu/zabbix-vs-nagios
SAISP Curs 03, Monitorizarea ret,elei 38/49
Munin
I arhitectura de tip master/node (similara Nagios)
I foloses, te RRDTool
I prezinta grafice ıntr-o interfat, a web (recunoscut pentru acestlucru)
I extensibil prin plugin-uri
SAISP Curs 03, Monitorizarea ret,elei 39/49
Monit
I system monitoring tool
I monitorizarea s, i gestiunea resurselor ıntr-un sistem Unix
I procese, fis, iere, directoare
I pornire, oprire, repornire servicii
I monitorizare + alerte
I apt-get install monit
I /etc/monit/monitrc
SAISP Curs 03, Monitorizarea ret,elei 40/49
Outline
Introducere
SNMP
Round Robin Database
Nagios
Alte solut, ii de monitorizare
Monitorizarea traficului
Concluzii
Intrebari
SAISP Curs 03, Monitorizarea ret,elei 41/49
Solut, ii de monitorizare a traficului
I Top 6 Traffic Monitoring Toolshttp://sectools.org/traffic-monitors.html
I sniffing (tcpdump, Ethereal)
I interceptare (netfilter, iptables)
I solut, ii dedicate: ntop, ngrep, iftop, vnstat, iptraf
SAISP Curs 03, Monitorizarea ret,elei 42/49
iptables
1 iptables -N my-network
2 iptables -A FORWARD -s 192.168.1.0/24 -j my-network
3 iptables -A FORWARD -d 192.168.1.0/24 -j my-network
4 iptables -L FORWARD -n -v
SAISP Curs 03, Monitorizarea ret,elei 43/49
ntop, ngrep, iftop
I ntopI statistici vizuale despre traficul de ret,eaI poate rula ca daemonI trebuie rulat prima data obis,nuit pentru configurare paroleiI un server web asculta pe portul 3000I foloses, te RRDTool
I ngrep -d lo -x GET
I iftop – statistici top-like (top, htop, iotop) pentru traficul deret,ea
SAISP Curs 03, Monitorizarea ret,elei 44/49
iptraf, vnstat
I iptrafI network statistics utilityI TUI
I sudo vnstat -u -i eth0
I vnstat
SAISP Curs 03, Monitorizarea ret,elei 45/49
Outline
Introducere
SNMP
Round Robin Database
Nagios
Alte solut, ii de monitorizare
Monitorizarea traficului
Concluzii
Intrebari
SAISP Curs 03, Monitorizarea ret,elei 46/49
Cuvinte cheie
I monitorizare
I high availability
I uptime, downtime
I SLA
I SNMP
I OID
I MIB
I snmpd
I community string
I scli
I Nagios
I pluginuri
I obiecte s, i definit, ii
I nagstamon
I NRPE
I MRTG
I RRDTool
I Cacti
I Zabbix
I Munin
I monit
I iptables
I ntop
I ngrep
I iftop
I iptraf
I vnstat
SAISP Curs 03, Monitorizarea ret,elei 47/49
Link-uri utile
I http://en.wikipedia.org/wiki/Comparison_of_network_
monitoring_systems
I http://www.slac.stanford.edu/xorg/nmtf/nmtf-tools.html
I http://www.topology.org/comms/netmon.html
I http://www.linuxhomenetworking.com/wiki/index.php/Quick_
HOWTO_:_Ch22_:_Monitoring_Server_Performance
I http://www.debianhelp.co.uk/snmp.htm
I http://www.debuntu.org/
how-to-monitor-your-servers-with-snmp-and-cacti
I http://www.debianadmin.com/
linux-snmp-oids-for-cpumemory-and-disk-statistics.html
I http://www.nagios.org/
I http://oss.oetiker.ch/mrtg/
SAISP Curs 03, Monitorizarea ret,elei 48/49
Outline
Introducere
SNMP
Round Robin Database
Nagios
Alte solut, ii de monitorizare
Monitorizarea traficului
Concluzii
Intrebari
SAISP Curs 03, Monitorizarea ret,elei 49/49