network monitoring - pipeline
TRANSCRIPT
Network MonitoringNetwork Monitoring
Review of SoftwareReview of Software
ComponentsComponentsNetwork DiscoveryNetwork DiscoveryAvailability monitoringAvailability monitoring•• Alerting systemAlerting systemService MonitoringService MonitoringNetwork PerformanceNetwork PerformanceAsset ControlAsset ControlVersion ControlVersion ControlConfiguration ManagementConfiguration ManagementHost trackingHost trackingBehaviour analysisBehaviour analysis
Network Discovery Network Discovery –– Vital FeaturesVital Features
Selectivity Selectivity •• CanCan’’t see the wood for the trees!t see the wood for the trees!
SpeedSpeed•• Network kit often in Network kit often in
huge private rangeshuge private ranges
Method of Method of automatically automatically getting results outgetting results out
Network DiscoveryNetwork Discovery
NetdiscoNetdisco•• Free network discovery package using Free network discovery package using
CDPCDP
SNMP CDP neighbour scriptSNMP CDP neighbour script
Availability MonitoringAvailability Monitoring
Scriptable configurationScriptable configurationHierarchy of the networkHierarchy of the networkNotification schedule by timeNotification schedule by timeNotification schedule by severityNotification schedule by severityNotification methodsNotification methods•• SMS, email etcSMS, email etc
Multiple viewsMultiple viewsHistorical recordHistorical record
Availability Monitoring PackagesAvailability Monitoring Packages
NagiosNagiosOpenNMSOpenNMSZabbixZabbixZenossZenossNAVNAV……etcetcMost commercial software NOT Most commercial software NOT configurable from scriptsconfigurable from scripts
Service monitoringService monitoring
Scriptable configurationScriptable configurationMore than just connect to portMore than just connect to portMany different protocolsMany different protocolsAlert scheduling by time, group and Alert scheduling by time, group and severity as availability monitoringseverity as availability monitoring
Service Monitoring PackagesService Monitoring Packages
Tests port functionTests port function•• NetcrunchNetcrunch•• OpManagerOpManager
Port up onlyPort up only•• NagiosNagios•• OpenNMSOpenNMS•• ZabbixZabbix•• ZenossZenoss
Network Performance Network Performance -- Vital Vital FeaturesFeatures
Scriptable configurationScriptable configurationTemplate systemTemplate systemScalability Scalability •• (25,000 SNMP parameters for Bangor)(25,000 SNMP parameters for Bangor)
ReliabilityReliabilityLong term supportLong term supportLong term storage Long term storage •• Data preservation across upgradesData preservation across upgrades•• Data averaging?Data averaging?
Thresholds and Automated warningsThresholds and Automated warnings
Network Performance Network Performance MeasurementMeasurement
Most MRTG inspired or Most MRTG inspired or RRDToolRRDTool based based systems sample at interval and then systems sample at interval and then progressively average.progressively average.
•• Hourly (1 minute average)Hourly (1 minute average)•• Daily (5 minute average)Daily (5 minute average)•• Weekly (30 minute average)Weekly (30 minute average)•• Monthly (2 hour average)Monthly (2 hour average)•• Yearly (1 day averageYearly (1 day average
•• Little storage required.Little storage required.•• Predefined quantity of storage required.Predefined quantity of storage required.
The effect of averagingThe effect of averaging
0
1e+08
2e+08
3e+08
4e+08
5e+08
6e+08
7e+08
8e+08
9e+08
0 20000 40000 60000 80000 100000 120000 140000
Dat
a ra
te in
Bits
/Sec
ond
(1G
b/s
= 1e
+09)
Time in Seconds
Effect of Averaging Data Rate
5 seconds30 seconds
5 minutes30 minute
2 hour1 day
Network Performance Network Performance -- StorageStorage
RRDTool
Cacti Cricket (NAV) OpenNMS Zenoss
RRDTool defaultsmimic MRTG. Most other packages takethose defaults.
Network Performance Network Performance -- StorageStorage
MRTG style RRD AdvantagesMRTG style RRD Advantages•• Very limited storage requiredVery limited storage required•• No data growthNo data growth
No maintenance requiredNo maintenance required
MRTG style RRD DisadvantagesMRTG style RRD Disadvantages•• Useless for capacity planningUseless for capacity planning•• Rapidly loses resolutionRapidly loses resolution•• Graphs cannot be directly comparedGraphs cannot be directly compared
Network Performance Network Performance -- StorageStorageSample @ 5 Sample @ 5 secsecQuantise to Quantise to nearest % nearest % loadloadCount Count occurrences occurrences of % load for of % load for periodperiodPlot as meshPlot as mesh
Network Load - Time Series
line 1
0 0.2
0.4 0.6
0.8 1
Fraction of maximum load 0 20000
40000 60000
80000 100000
120000 140000
160000
Time in Seconds
0 20 40 60 80
100 120 140 160 180
Frequency
Network Performance Network Performance -- StorageStorage
AdvantagesAdvantages•• Graphs comparable even when count Graphs comparable even when count
period and quantise level differentperiod and quantise level different•• Low data storage requirementsLow data storage requirements
DisadvantagesDisadvantages•• Nobody does it!Nobody does it!
Network Performance Network Performance -- StorageStorageProducts that meet requirementsProducts that meet requirements
•• StatseekerStatseekerSamples @ 1/min, averaged to 1/5min after 1 yearSamples @ 1/min, averaged to 1/5min after 1 yearVery efficientVery efficientConfigurable thresholdsConfigurable thresholds
•• RTGRTGWith sufficient effort should deliverWith sufficient effort should deliverThresholds should be quite easy to codeThresholds should be quite easy to code
•• CactiCactiTheoretically need not do MRTG averagingTheoretically need not do MRTG averagingNo thresholdsNo thresholdsFree, easy to use, produces pleasing graphsFree, easy to use, produces pleasing graphs
Asset ControlAsset Control
Automatic device trackingAutomatic device trackingPermanent link between serial Permanent link between serial number and purchasing detailsnumber and purchasing detailsAble to deal with multiple ownersAble to deal with multiple ownersAble to deal with parts of chassis Able to deal with parts of chassis switches individuallyswitches individually
Asset controlAsset control
NAVNAV•• Database design sufficientDatabase design sufficient•• Front end more limited?Front end more limited?
OpenNMSOpenNMS•• Database design seems inadequateDatabase design seems inadequate
Version ControlVersion Control
Scriptable configurationScriptable configurationAutomaticAutomaticEasy identification of devices needing Easy identification of devices needing upgradeupgradeVersion historyVersion history
Configuration ManagementConfiguration Management
Scriptable configurationScriptable configurationSave running configurationSave running configurationRun scripted commandsRun scripted commandsAlert on configuration changeAlert on configuration changeSave configuration historySave configuration historyAlert on improperly configured Alert on improperly configured devicesdevices•• Use templates & central Use templates & central configconfig
generationgeneration
Configuration ManagementConfiguration Management
RANCIDRANCID•• Uses CVS or subversionUses CVS or subversion•• ConfigConfig backup and change detectionbackup and change detection•• No No templatingtemplating
CheetahCheetah•• TemplatingTemplating softwaresoftware
Host TrackingHost Tracking
Scriptable configurationScriptable configurationLocate host by IP or MAC addressLocate host by IP or MAC addressLocation historyLocation history
Host trackingHost tracking
With historical recordWith historical record•• NAVNAV
On the fly onlyOn the fly only•• NetdiscoNetdisco•• NetcrunchNetcrunch
Behaviour AnalysisBehaviour Analysis
Mirroring portMirroring portUsing Using netflownetflow or or sflowsflow data data
Mirroring methods need lots of CPUMirroring methods need lots of CPUAimsAims•• IDSIDS•• User behaviour analysis and controlUser behaviour analysis and control•• Improving efficiency on expensive linksImproving efficiency on expensive links
Behaviour AnalysisBehaviour Analysis
Specialist packagesSpecialist packagesSnortSnort mirroringmirroring freefreeInMonInMon sflowsflow commercialcommercialArgusArgus netflow/sflow/mirroringnetflow/sflow/mirroring
freefreeNtopNtop mirroringmirroring freefree
CactiCacti
•RRDTool based
•Graphing package
•CLI and API (API docs?)
•Scalable
Cacti featuresCacti features
Graph HierarchyGraph Hierarchy•• Difficult to configure from the commandDifficult to configure from the command
lineline
User managementUser managementAny Any OIDsOIDsGraphical managementGraphical managementEfficient poll Efficient poll Free!Free!
RDTRDT
Fast SNMP data collectionFast SNMP data collectionStorage in SQL databaseStorage in SQL databaseNot really a complete solution even Not really a complete solution even for performance monitoring for performance monitoring –– but a but a good basis?good basis?
StatseekerStatseekerNetwork performanceNetwork performanceSome availability functionsSome availability functionsNot free Not free Scales easily to University size networksScales easily to University size networksOnly software to meet most of our Only software to meet most of our network performance specificationnetwork performance specificationUsed by many Universities Used by many Universities –– including us!including us!Highly recommendedHighly recommendedVersion 3 now outVersion 3 now out
NetcrunchNetcrunch
Service based Service based –– lots of predefined serviceslots of predefined services•• Intelligent ping Intelligent ping –– not just port innot just port in
Logical map Logical map –– graphical mapping graphical mapping Manual physical viewsManual physical viewsPerformance monitoring can be configured Performance monitoring can be configured -- not designed to be run by default on all not designed to be run by default on all ports.ports.Servers performance monitoring through Servers performance monitoring through SNMPSNMP
NetCrunchNetCrunchCannot do everything from web, but all Cannot do everything from web, but all monitoring except trafficmonitoring except traffic22--3 days consultancy on installation3 days consultancy on installationXE XE –– unlimited unlimited ££11,500 11,500 –– down to down to ££4,000 4,000 smallest limited versionsmallest limited version•• ££3,690 software maintenance 3,690 software maintenance –– major and major and
minor updates & telephone supportminor updates & telephone support
Central behaviour analysis solution Central behaviour analysis solution --NetfortNetfort•• ££25,00025,000
NAVNAV
Database centred frameworkDatabase centred frameworkUsed by all Norwegian UniversitiesUsed by all Norwegian UniversitiesGood documentationGood documentation•• Database design documented etcDatabase design documented etc
Designed for UniversitiesDesigned for UniversitiesFreeFree
NAVNAV
Availability monitoringAvailability monitoring•• Alerting systemAlerting system
Service Monitoring?Service Monitoring?Network Performance Network Performance -- cricketcricketAsset Control Asset Control Version ControlVersion ControlHost trackingHost trackingWeathermapWeathermap
NAVNAV
ButBut……Performance monitoring is poor Performance monitoring is poor (cricket)(cricket)Can be difficult to install Can be difficult to install –– best on best on debiandebianMailing list traffic lowMailing list traffic low
Open NMSOpen NMS
Service monitoringService monitoringAvailability monitoringAvailability monitoringPerformance monitoringPerformance monitoringSome asset trackingSome asset tracking
Buggy and unpredictableBuggy and unpredictablePrimarily service monitoring Primarily service monitoring –– rest rest seems to be an afterthoughtseems to be an afterthought
ZabbixZabbix
AvailabilityAvailabilityPerformance monitoringPerformance monitoringService monitoringService monitoring
Difficult to get startedDifficult to get startedDoes not seem to excel at anythingDoes not seem to excel at anything
ZenossZenoss
Commercial backedCommercial backedService monitoringService monitoringAvailability monitoringAvailability monitoringSome performanceSome performance
Buggy and erraticBuggy and erratic
OpManagerOpManager
Availability monitoringAvailability monitoringAdvanced service monitoringAdvanced service monitoringCheapish commercial productCheapish commercial product
Asset control extraAsset control extraPoor performance monitoringPoor performance monitoring
NTopNTop
FreeFreeNetwork monitoring behaviour Network monitoring behaviour analysisanalysisEasy to installEasy to installPretty graphical outputPretty graphical outputEasy to understandEasy to understand
Availab
ility Availab
ility M
onito
ring
Monito
ring
Service
Service
Monito
ring
Monito
ring
Netw
ork
Netw
ork
Perform
ance
Perform
ance
Asset C
ontro
lAsset C
ontro
l
Versio
n C
ontro
lVersio
n C
ontro
l
Config
uratio
n
Config
uratio
n
man
agem
ent
man
agem
ent
Host T
racking
Host T
racking
Netw
ork
Netw
ork
Disco
veryD
iscovery
Beh
aviour
Beh
aviour
Analysis
Analysis
CactiCacti
ZabbixZabbix
Open Open NMSNMS
NAVNAV??
RancidRancid
SnortSnort
Bangor UniversityBangor University’’s Choicess Choices
NetdiscoNetdisco or inor in--house SNMP CDP house SNMP CDP scriptscript•• Network discoveryNetwork discovery
NagiosNagios•• AvailabilityAvailability•• Service monitoringService monitoring
StatseekerStatseeker•• Network performanceNetwork performance
Bangor UniversityBangor University’’s Choicess ChoicesNAVNAV•• Core database for network managementCore database for network management•• Asset controlAsset control•• Host trackingHost tracking•• Availability?Availability?•• Version control?Version control?
RANCIDRANCID•• Configuration managementConfiguration management
CheetahCheetah•• Configuration Configuration templatingtemplating
Snort/Snort/NtopNtop•• Behaviour analysisBehaviour analysis
Any questions or observationsAny questions or observations
??????????