sig-noc tools survey results
TRANSCRIPT
SIG-NOC Tools survey results (draft)
OPEN DISCUSSION
Maria Isabel Gandia Carriedo
3rd SIG-NOC meeting
14 Apr 2016
Nordunet offices
The Second Survey: focused on tools
66 questions in 5 groups, mainly focused on NOC tools and their
functionalities:
1. Basic information (2)
2. NOC tools per functionality (60 => 15x4: Responsibility, Importance,
Rating, in-house?):
3. Standards and Training (3)
4. Closing (1)
But, for every question on group 2, we had several tools -> In fact, we
had 383 questions!
We got 76 answers:
• 52 quite complete answers
• 11 incomplete answers
• 13 nearly empty answers
Type of network managed
41
24
18
13 12
10
4 3
0
5
10
15
20
25
30
35
40
45
Nationalresearch
andeducationnetwork(NREN)
Campus,universitynetwork
Regional,metropolitan
network
Specificresearchnetwork
(any range)
Wide areanetwork,amongseveral
countries
InternetExchangeoperator
(any size)
Commercialnetwork,ISP (anyrange)
Other
Functionalities our NOCs feel responsible for
63
54 53
46 46 45 44 42 40 37 36 36 34 33
29
0
10
20
30
40
50
60
70
Answers with information about “standard” tools
50
41 40 38 38
36 34
30 29 28 28 27 24 24 23
0
10
20
30
40
50
60
NOCs use in-house tools in these functionalities
17
12
8
6 5 5
4 4 3
2 2 1 1
0 0 0
2
4
6
8
10
12
14
16
18
We have a lot information
“Statistics: the only science that enables different experts using
the same figures to draw different conclusion.” Evan Esar
Monitoring: the questions for each tool
low low-mid mid-high high
How important is this tool for your NOC?
poor fair average good excellent
How would you rate this tool?
Option 1: raw data
Importance Rating
low low-mid mid-high high Rating
Average
Response
Count poor fair average good excellent
Rating
Average
Response
Count
CACTI 4 5 3 16 3.11 28 1 0 9 11 8 3.86 29
SNMP 0 1 4 36 3.85 41 0 2 7 12 18 4.18 39
CRICKET 3 0 2 2 2.43 7 1 2 2 2 0 2.71 7
ICINGA 2 1 3 10 3.31 16 1 1 2 6 6 3.94 16
INTERMAPPER 6 0 1 0 1.29 7 1 0 1 1 0 2.67 3
LOGGING 3 2 6 12 3.17 23 1 3 6 7 3 3.40 20
LOOKING-GLASS 5 10 8 2 2.28 25 1 4 6 9 4 3.46 24
MRTG 5 3 4 11 2.91 23 0 2 6 10 4 3.73 22
NAGIOS 3 2 7 15 3.26 27 0 1 4 17 5 3.96 27
NETFLOW 1 5 10 19 3.34 35 0 1 7 14 11 4.06 33
NFDUMP 2 2 5 5 2.93 14 1 1 1 8 3 3.79 14
NFSEN 3 4 4 11 3.05 22 2 1 5 9 4 3.57 21
OBSERVIUM 3 2 1 4 2.60 10 1 0 2 4 2 3.67 9
OPENVIEW 5 1 0 3 2.11 9 0 0 2 3 2 4.00 7
PERFSONAR 5 8 5 2 2.20 20 3 5 5 3 2 2.78 18
RANCID 3 6 5 9 2.87 23 2 5 2 8 3 3.25 20
RIPE Atlas / Stats 9 4 10 2 2.20 25 1 2 6 9 3 3.52 21
RIPE RIS / BGPlay 6 3 4 2 2.13 15 1 1 2 4 3 3.64 11
SMOKEPING 8 3 5 7 2.48 23 2 2 4 5 6 3.58 19
SPECTRUM 4 1 0 2 2.00 7 1 0 2 1 1 3.20 5
SYSLOG 1 4 11 29 3.51 45 0 3 10 13 16 4.00 42
WEATHERMAP 2 6 5 11 3.04 24 1 1 3 6 11 4.14 22
ZENOSS 4 1 0 2 2.00 7 1 1 0 1 1 3.00 4
ZINO 5 1 0 2 1.88 8 0 0 0 2 1 4.33 3
Other.. 1 0 2 10 3.62 13 0 0 3 6 3 4.00 12
Option 2: Number of answers for a single tool
0
5
10
15
20
25
30
35
40
45
50S
YS
LO
G
SN
MP
NE
TF
LO
W
CA
CT
I
NA
GIO
S
LO
OK
ING
-GLA
SS
RIP
E A
tlas / S
tats
WE
AT
HE
RM
AP
LO
GG
ING
MR
TG
RA
NC
ID
SM
OK
EP
ING
NF
SE
N
PE
RF
SO
NA
R
ICIN
GA
RIP
E R
IS /
BG
Pla
y
NF
DU
MP
Oth
er.
.
OB
SE
RV
IUM
OP
EN
VIE
W
ZIN
O
CR
ICK
ET
INT
ER
MA
PP
ER
SP
EC
TR
UM
ZE
NO
SS
Option 3: Importance average of the tool for the NOC (max=4)
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
SN
MP
Oth
er.
.
SY
SLO
G
NE
TF
LO
W
ICIN
GA
NA
GIO
S
LO
GG
ING
CA
CT
I
NF
SE
N
WE
AT
HE
RM
AP
NF
DU
MP
MR
TG
RA
NC
ID
OB
SE
RV
IUM
SM
OK
EP
ING
CR
ICK
ET
LO
OK
ING
-GLA
SS
PE
RF
SO
NA
R
RIP
E A
tlas / S
tats
RIP
E R
IS /
BG
Pla
y
OP
EN
VIE
W
SP
EC
TR
UM
ZE
NO
SS
ZIN
O
INT
ER
MA
PP
ER
Option 4:The tools with more “high” ratings on importance
0
5
10
15
20
25
30
35
40S
NM
P
SY
SLO
G
NE
TF
LO
W
CA
CT
I
NA
GIO
S
LO
GG
ING
MR
TG
NF
SE
N
WE
AT
HE
RM
AP
ICIN
GA
Oth
er.
.
RA
NC
ID
SM
OK
EP
ING
NF
DU
MP
OB
SE
RV
IUM
OP
EN
VIE
W
CR
ICK
ET
LO
OK
ING
-GLA
SS
PE
RF
SO
NA
R
RIP
E A
tlas / S
tats
RIP
E R
IS /
BG
Pla
y
SP
EC
TR
UM
ZE
NO
SS
ZIN
O
INT
ER
MA
PP
ER
Option 5: The best ratings averages (max=5)
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
5.00
ZIN
O
SN
MP
WE
AT
HE
RM
AP
NE
TF
LO
W
OP
EN
VIE
W
SY
SLO
G
Oth
er.
..
NA
GIO
S
ICIN
GA
CA
CT
I
NF
DU
MP
MR
TG
OB
SE
RV
IUM
RIP
E R
IS /
BG
pla
y
SM
OK
EP
ING
NF
SE
N
RIP
E A
tlas / S
tats
LO
OK
ING
-GLA
SS
LO
GG
ING
RA
NC
ID
SP
EC
TR
UM
ZE
NO
SS
PE
RF
SO
NA
R
CR
ICK
ET
INT
ER
MA
PP
ER
Option 6: Number of answers and importance of each tool
0
5
10
15
20
25
30
35
40
45
50
SY
SLO
G
SN
MP
NE
TF
LO
W
CA
CT
I
NA
GIO
S
LO
OK
ING
-GLA
SS
RIP
E A
tlas / S
tats
WE
AT
HE
RM
AP
LO
GG
ING
MR
TG
RA
NC
ID
SM
OK
EP
ING
NF
SE
N
PE
RF
SO
NA
R
ICIN
GA
RIP
E R
IS /
BG
Pla
y
NF
DU
MP
Oth
er.
.
OB
SE
RV
IUM
OP
EN
VIE
W
ZIN
O
CR
ICK
ET
INT
ER
MA
PP
ER
SP
EC
TR
UM
ZE
NO
SS
high
mid-high
low-mid
low
Option 7: Number of answers, importance of each tool (+average, max=4)
3.51
3.85
3.34
3.11 3.26 2.28 2.20 3.04 3.17 2.91 2.87 2.48 3.05
2.20
3.31 2.13 2.93 3.62
2.60 2.11 1.88 2.43 1.29 2.00 2.00
0
10
20
30
40
50
60
SY
SLO
G
SN
MP
NE
TF
LO
W
CA
CT
I
NA
GIO
S
LO
OK
ING
-GLA
SS
RIP
E A
tlas / S
tats
WE
AT
HE
RM
AP
LO
GG
ING
MR
TG
RA
NC
ID
SM
OK
EP
ING
NF
SE
N
PE
RF
SO
NA
R
ICIN
GA
RIP
E R
IS /
BG
Pla
y
NF
DU
MP
Oth
er.
.
OB
SE
RV
IUM
OP
EN
VIE
W
ZIN
O
CR
ICK
ET
INT
ER
MA
PP
ER
SP
EC
TR
UM
ZE
NO
SS
high
mid-high
low-mid
low
Option 8: Number of answers, importance of each tool, rating average,
max=5)
4.00
4.18
4.06
3.86 3.96 3.46 3.52 4.14
3.40 3.73 3.25 3.58 3.57 2.78
3.94 3.64 3.79 4.00
3.67 4.00 4.33 2.71 2.67 3.20 3.00
0.00
10.00
20.00
30.00
40.00
50.00
60.00S
YS
LO
G
SN
MP
NE
TF
LO
W
CA
CT
I
NA
GIO
S
LO
OK
ING
-GLA
SS
RIP
E A
tla
s /
Sta
ts
WE
AT
HE
RM
AP
LO
GG
ING
MR
TG
RA
NC
ID
SM
OK
EP
ING
NF
SE
N
PE
RF
SO
NA
R
ICIN
GA
RIP
E R
IS / B
GP
lay
NF
DU
MP
Oth
er.
.
OB
SE
RV
IUM
OP
EN
VIE
W
ZIN
O
CR
ICK
ET
INT
ER
MA
PP
ER
SP
EC
TR
UM
ZE
NO
SS
high
mid-high
low-mid
low
Option 9: Importance and rating averages, sorted by number of answers
(max=10)
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
10.00S
YS
LO
G
SN
MP
NE
TF
LO
W
CA
CT
I
NA
GIO
S
LO
OK
ING
-GLA
SS
MR
TG
WE
AT
HE
RM
AP
NF
SE
N
RIP
E A
tla
s /
Sta
ts
LO
GG
ING
RA
NC
ID
SM
OK
EP
ING
PE
RF
SO
NA
R
ICIN
GA
NF
DU
MP
Oth
er.
.
RIP
E R
IS / B
GP
lay
OB
SE
RV
IUM
CR
ICK
ET
OP
EN
VIE
W
SP
EC
TR
UM
ZE
NO
SS
INT
ER
MA
PP
ER
ZIN
O
Importance average
Rating average
Option 10: One graph with the importance or rating for each tool?
0
5
10
15
20
CACTI
Low
Medium
Mid-high
High
Cacti
Low
Medium
Mid-high
High
Option 11: Heatmap (example for importance)
Answer Options low low-mid mid-high high Rating
Average
Response
Count
SYSLOG 1 4 11 29 3.51 45
SNMP 0 1 4 36 3.85 41
NETFLOW 1 5 10 19 3.34 35
CACTI 4 5 3 16 3.11 28
NAGIOS 3 2 7 15 3.26 27
LOOKING-GLASS 5 10 8 2 2.28 25
RIPE Atlas / Stats 9 4 10 2 2.20 25
WEATHERMAP 2 6 5 11 3.04 24
LOGGING 3 2 6 12 3.17 23
MRTG 5 3 4 11 2.91 23
RANCID 3 6 5 9 2.87 23
SMOKEPING 8 3 5 7 2.48 23
NFSEN 3 4 4 11 3.05 22
PERFSONAR 5 8 5 2 2.20 20
ICINGA 2 1 3 10 3.31 16
RIPE RIS / BGPlay 6 3 4 2 2.13 15
NFDUMP 2 2 5 5 2.93 14
Other.. 1 0 2 10 3.62 13
OBSERVIUM 3 2 1 4 2.60 10
OPENVIEW 5 1 0 3 2.11 9
ZINO 5 1 0 2 1.88 8
CRICKET 3 0 2 2 2.43 7
INTERMAPPER 6 0 1 0 1.29 7
SPECTRUM 4 1 0 2 2.00 7
ZENOSS 4 1 0 2 2.00 7
Which one(s) is(are) more useful (if any!)?
1. Raw data
2. Number of answers for a single tool
3. Importance average of the tool for the NOC (max=4)
4. The tools with more “high” ratings on importance
5. The best ratings averages (max=5)
6. Number of answers and importance of each tool
7. Number of answers, importance of each tool (+average, max=4)
8. Number of answers, importance of each tool, rating average, max=5)
9. Importance and rating avg, sorted by number of answers (max=10)
10. One graph with the importance or rating for each tool?
11. Heatmap
OPEN
DISCUSSION
What is the estimated level of adoption of the given standard or
methodology at your NOC?
Answer Options NONE 10-30% 5-10% 30-60% 60-100% Respons
e Count
ISO 7 1 0 1 2 11
eTOM 10 0 0 1 0 11
ITIL 5 7 1 8 4 25
NITS 8 0 1 0 0 9
FIPS 8 0 0 0 0 8
ISO 27001 8 1 3 1 4 17
ISO 20000 8 0 2 0 1 11
Other... 1 0 0 0 0 1
Other (please specify) 2
answered question 31
skipped question 45
0
5
10
15
20
25
30
ISO eTOM ITIL NITS FIPS ISO27001
ISO20000
Other...
What is the estimated level of adoption of the given standard or methodology at your NOC?
NONE
10-30%
5-10%
30-60%
60-100%
How many of your NOC people are certified or trained (in percentage)?
Answer Options 0-10% 10-30% 30-60% 60-100% Rating
Average
Respons
e Count
ISO 5 2 0 2 1.89 9
eTOM 8 0 0 0 1.00 8
ITIL 11 6 5 3 2.00 25
NITS 6 0 0 0 1.00 6
FIPS 6 0 0 0 1.00 6
ISO 27001 10 1 1 2 1.64 14
ISO 20000 7 1 0 1 1.44 9
Other... 1 1 0 0 1.50 2
answered question 29
skipped question 47
0.00 0.50 1.00 1.50 2.00 2.50
ISO
eTOM
ITIL
NITS
FIPS
ISO…
ISO…
Other...
How many of your NOC people are certified or trained (in percentage)?
What technical training(s) are provided either in-house or contracted to
your NOC personnel? Please, describe them all.
Usually training on the job from the vendor when installing new
equipment; in-house studies and workshops
Seminars, improvement courses
dwdm/ optical management (Coriant) Misc inhouse training
Transmission training Juniper training Crisis and communication
training On the job training
Vendor related training: Juniper, Fortigate, Cumulus General technical
training: Coursera MOOC
We train our NOC members by */ having generic documentation */
having specific documentation for our networks and/or customers */
going on site to work with them */ letting them come to our site for
them to work with us */ invite them when the our staff is trained for a
product / technology that is useful for the NOC
Initial training to practical NOC duties. In-house trainings of relevant
topics on random intervals (rarely)
Internal procedures walkthrough and working together with an older
member ("shadowing")
What technical training(s) are provided either in-house or contracted to
your NOC personnel? Please, describe them all.(2)
In-house. Many procedures are described in Dokuwiki and the rest is
practice.
No standard trainings. Most is learning by doing with the background
of long-year experience with most of the staff-members.
Basic, in house, NOC training. Fibre safety. Data centre
design/management. Troubleshooting.
CCNA, CCNP, MikroTik academy, different in-house trainings...
I have no idea what kind of training you refer to. We do attend
juniper/cisco/alcatel education when appropriate
ITIL foundations, Network Auditing, English, Dedicated courses on
specific equipment (for instance Alcatel, Cisco, etc), CCNA, Linux
certification LPIC, RIPE NCC trainings, ...
