![Page 1: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/1.jpg)
Connect. Communicate. Collaborate
Experiences with tools for network anomaly detection in the GÉANT2 core
Maurizio Molina, DANTE
COST TMA tech. Seminar
Samos, 23rd Sep 2008
![Page 2: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/2.jpg)
Connect. Communicate. CollaborateThe GÉANT Network
• DANTE operates GÉANT2• Backbone network for National Research and Education
Networks in Europe• 30+ NRENs, 2 global connectivity providers (Telia and
GCrossing), peerings with other research networks (Abilene, Canarie, Clara, TEIN2, SINET…)
![Page 3: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/3.jpg)
Connect. Communicate. Collaborate
The GÉANT Network (IP layer)
• 20 Juniper routers
• tenths of GBit/s of aggregated traffic
• Main accesses and the backbone 10Gbit/s
Pls see www.dante.net
![Page 4: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/4.jpg)
Connect. Communicate. CollaborateThe Services
• So…. Just a big pipe? No!• Services
– Dedicated L1-L2 circuits via multiple technologies– Performance Monitoring services (perfSONAR)– Support for federation of National AA Infrastructures
(eduGAIN) and wireless roaming (eduROAM)– Security Service Very NEW!
NEW!
![Page 5: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/5.jpg)
Connect. Communicate. Collaborate
The vision:enhance NRENs security
• NRENs have their (+ - evolved…) CERTs to deal with security
• and DANTE can filter traffic on GÉANT upon NRENs request….
! BUT !
• Can we be more proactive to NREN CERTs exploiting
the visibility of the GN2 core?
![Page 6: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/6.jpg)
Connect. Communicate. Collaborate
The vision (cont.):enhance NRENs security
• Approach: NetFlow (+ Routing data) & good processing tools
NetFlow v5 collector
• Netflow collected on all peering interfaces
• 1 / 1,000 Sampling
• ~3k flows/s
![Page 7: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/7.jpg)
Connect. Communicate. Collaborate
Proof of concept: Can we identify anomalies in the core?
• Anomalies are often “hidden”
Requirements:
High detection rate
Low false positives
Anomaly classification
Evidence collection
NfSen
![Page 8: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/8.jpg)
Connect. Communicate. Collaborate
From “volume” to “IP feature entropies”
Connect. Communicate. Collaborate
•“IP features entropies”•Simple linear filter
![Page 9: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/9.jpg)
Connect. Communicate. CollaborateDrilling down on peaks Connect. Communicate. Collaborate
-Concentration of DST IPs and DST ports receiving flows
-Dispersion of SRC IPs and SRC ports
• IRC server in Slovenia, receiving a lot of 60 bytes syn pkts on port 6667, mainly from a /16 Subnetwork of an University in the Netherlands.
• Likely a “BotNet war”?
![Page 10: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/10.jpg)
Connect. Communicate. Collaborate
Drilling down on peaks (cont.) Connect. Communicate. Collaborate
- Concentration of SRC and DST IPs and SRC ports
- Dispersion of DST ports
• Portscan of host in CARNET, from 4 hosts, 29 bytes packets
![Page 11: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/11.jpg)
Connect. Communicate. CollaborateOpen source tools
• Results:– anomalies are observable in the GÉANT2 core– Novel methodologies (IP Features entropy) for their
classifications are applicable• Limits:
– NfSen does not fuse NetFlow and Routing data– Extensions would need to be run (and tuned) on all
ingress/egress points– No support, no guaranteed development
![Page 12: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/12.jpg)
Connect. Communicate. CollaborateCommercial tools
• Test started Jun 08 (3 tools)– Tool 1
• PCA, entropy– Tool 2
• Large scale DDoS and Worm spread– Tool 3
• Per host behaviour
![Page 13: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/13.jpg)
Connect. Communicate. CollaborateTool 1 (as a security tool…)
• Two main novel elements– Principal Component Analysis (PCA)– Both Volume and IP features Entropy anomaly
detection• Address what makes anomaly detection a complex task
– PCA: single parameter to control detection sensitivity, even if anomalies are attributed to specific OD pairs
– Entropy: Detection of both low volume (scans) and high volume (DoS) anomalies
![Page 14: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/14.jpg)
Connect. Communicate. CollaborateDemo….
• …. Or Screenshots….
![Page 15: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/15.jpg)
Connect. Communicate. Collaborate
![Page 16: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/16.jpg)
Connect. Communicate. Collaborate
![Page 17: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/17.jpg)
Connect. Communicate. Collaborate
![Page 18: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/18.jpg)
Connect. Communicate. Collaborate
![Page 19: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/19.jpg)
Connect. Communicate. Collaborate
![Page 20: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/20.jpg)
Connect. Communicate. Collaborate
![Page 21: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/21.jpg)
Connect. Communicate. Collaborate
![Page 22: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/22.jpg)
Connect. Communicate. Collaborate
![Page 23: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/23.jpg)
Connect. Communicate. CollaborateTool 2
• Well-established (and expensive!) solution for detecting “large” events
• Originally based on large volume shifts only• Now enhanced to give alerts on “fingerprints” (e.g.
communication with C&C servers)– Shared by (part) of the user community (50 out of 120)
• No usage of routing data– though “zones” can be manually created via BGP
prefixes lists• Traditional threshold based detection (although adaptive)
![Page 24: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/24.jpg)
Connect. Communicate. CollaborateTool 3
• Per host behavioural analysis• rather complex “scoring” system to distinguish normal from
abnormal behaviour. Proprietary algorithms• Doesn’t use routing info
– though “zones” can be manually created via BGP prefixes lists
• Potentially attractive methodology• Concerns on scalability and accuracy with 1,000 sampling
![Page 25: Experiences with tools for network anomaly detection in the GÉANT2 core](https://reader036.vdocuments.site/reader036/viewer/2022070402/568137c0550346895d9f5f23/html5/thumbnails/25.jpg)
Connect. Communicate. Collaborate
lessons learnt and directions for research
• Manual validation is required to confirm/correct anomalies– More automatic intelligence to help this process– Fusion with other data sources (router logs?
Honeynets?)• Detection space of 3 tools often disjoint
– (Standard) anomaly injection• Operations need supported tools to support services• If choice is among published but “not a tool” or “secret but
supported and (claiming to) work” => risk to stick to those!– Fill the gap towards TOOLS!