intrusion detection testing and benchmarking methodologies
DESCRIPTION
Intrusion Detection Testing and Benchmarking Methodologies. Nicholas Athanasiades, Randal Abler, John Levine, Henry Owen, and George Riley School of Electrical and Computer Engineering Georgia Institute of Technology. 1. Introduction. Beginning of the Intrusion Detection Evaluation - PowerPoint PPT PresentationTRANSCRIPT
Information Networking Security and Assurance LabNational Chung Cheng University
Intrusion Detection Testing and Benchmarking Methodologies
Nicholas Athanasiades, Randal Abler, John Levine, Henry Owen, and George Riley
School of Electrical and Computer EngineeringGeorgia Institute of Technology
Information Networking Security and Assurance LabNational Chung Cheng University 2
1. Introduction
Beginning of the Intrusion Detection Evaluation DARPA(1998~1999) LARIAT (Lincoln Adaptable Real-time Information Ass
urance Test-bed)(2000~2001) Most common methodologies Traffic generation is one of the most difficult ones
Synthetic traffic not represent the realities of an actual network
SmartBits Scripting tools
Information Networking Security and Assurance LabNational Chung Cheng University 3
2. Existing Tools and Testing Methodologies
A. DARAPA Environment B. LARIAT Environment C. Nidsbench and IDS Wakeup D. IDSwakeup E. Flame Thrower F. WebAvalanche/WebReflector G. Tcpreplay H. Fragrouter I. Hping2 J. Iperf
Information Networking Security and Assurance LabNational Chung Cheng University 4
2. Existing Tools and Testing Methodologies
A. DARAPA Environment Approach
An off-line (Tune and optimize) and an on-line (actual testing) evaluation executed
Tcpreplay
Protocol/traffic activity HTTP, X window, SQL, SMTP, DNS, FTP, POP3, Finger,
Telnet, IRC, SNMP, and Time
Information Networking Security and Assurance LabNational Chung Cheng University 5
2. Existing Tools and Testing Methodologies
A. DARAPA EnvironmentSolaris SunOS Linux
Denial of Service (11 types, 43 instances)
Back, Neptune, Ping of death, Smurf, syslog, Land, apache2, Mailbomb, Process table, UDP storm
Back, Neptune, Ping of death, Smurf, Land, apache2, Mailbomb, Process table, UDP storm
Back, Neptune, Ping of death, Smurf, teardrop, Land, apache2, Mailbomb, Process table, UDP storm
Remote to Local (14 types, 17 instances)
Dictionary, ftp-write, guest, phf, http tunnel, xlock, xsnoop
Dictionary, ftp-write, guest, phf, http tunnel, xlock, xsnoop
Dictionary, ftp-write, guest, imap, phf, named, http tunnel, sendmail, xlock, xsnoop
User to Root (7 type, 38 instances)
Eject, ffbconfig, Fdformat, ps
Loadmodule, ps Perl, xterm
Surveillance/ Probe (6 types, 22 instances)
Eject, nmap, Port sweep, Satan, mscan, saint
Eject, nmap, Port sweep, Satan, mscan, saint
Eject, nmap, Port sweep, Satan, mscan, saint
Figure 1 Attacks in the 1998 DARPA evaluation
Information Networking Security and Assurance LabNational Chung Cheng University 6
2. Existing Tools and Testing Methodologies
A. DARAPA Environment 1999: the goals shifted to testing complete systems Changes and additions
Victim Windows NT added New stealthy attacks added Two new types of analysis performed
• An analysis of misses and high-scoring false alarms• Participants were allowed to submit information aiding in the
identification of many attacks and their appropriate response Detection of novel attacks without first training
Information Networking Security and Assurance LabNational Chung Cheng University 7
2. Existing Tools and Testing Methodologies
B. LARIAT Environment LARIAT “emulates the network traffic from a small organizat
ion connected to the Internet” Many phases
Network discovery phase Then, initializes the network and configures the hosts The test’s conditions are set up
Traffic generation is done through the use of defined service models
Modified a Linux Kernel that allow their software to generate background traffic
Part of a government project and not publicly available
Information Networking Security and Assurance LabNational Chung Cheng University 8
2. Existing Tools and Testing Methodologies
C. Nidsbench A NIDS Test Suite released in 1999 Made up of the components tcpreplay, idtest and fragrouter
D. IDSwakeup Like Nidsbench It generates false attacks, a false positive test utility Consists of IDSwakeup and utilizes hping and iwu
E. Flame Thrower Commercial load stress tool used to identify network infrastructure weaknesses Produces transaction in order to test network infrastructure and applications Supports HTTP/HTTPS 1.0, 1.1 and SSL It can emulate over two million IP address FirewallStressor measure throughput under attack conditions Flame Thrower intended for testing firewalls
Information Networking Security and Assurance LabNational Chung Cheng University 9
2. Existing Tools and Testing Methodologies
F. WebAvalanche/WebReflector Commercial network appliances used in the testing of IDS WebAvalanche is a stress-testing appliance WebReflector emulates the behavior of large Web, applicat
ion and data server environments Support such as HTTP 1.0/1.1, SSL, RTSP/RTP and FTP Measure percent dropped packets, latencies, maximum nu
mber of users and new user arrival rates G. Tcpreplay
Allows captured traffic to be played back on a network at different speeds
Tcpdump or snoop
Information Networking Security and Assurance LabNational Chung Cheng University 10
2. Existing Tools and Testing Methodologies
H. Fragrouter An attack generation tool For testing anti-evasion techniques and fragmentation queues
I. Hping2 A command-line packet assembler and analyzer Allows one to create and transmit custom ICMP, UDP, and TCP pa
ckets Fingerprint remote operating systems
J. Iperf Measures bandwidth, delay jitter and datagram loss Used as a background traffic source
Information Networking Security and Assurance LabNational Chung Cheng University 11
4. Examples of Intrusion Detection Evaluation Environments
DARPA Like EnvironmentCustom SoftwareAdvanced Security Audit Trail Analysis on
UnixVendor Independent Testing LabTrade Magazine Evaluation
Information Networking Security and Assurance LabNational Chung Cheng University 12
DARPA Like Environment
5 components Traffic generating Victim was “an anonymous FTP server running on a Sun
UltraSparc-1 using a Solaris 2.5 OS Attack Injection programs The in house reference programs counted the number of hung
connection at the victim server as a measure of attack effectiveness. They used a metric called virulence. Virulence described the intensity of an attack situation.
The evaluation method was to use 10, 15, 30, 40 and 60 attacking hosts each utilizing rates of varying rates of attacks per second.
Information Networking Security and Assurance LabNational Chung Cheng University 13
Custom Software
A software platform that simulates intrusions and tests IDS effectiveness
Criteria used included Broad Detection Range Economy in resource usage Resilience to stress
The benchmark platform was base on Expect and Tool Command Language Distributed Programming (TCL-DP) package
Information Networking Security and Assurance LabNational Chung Cheng University 14
Advanced Security audit trail Analysis on uniX
The test consisted of the following scenarios Trojan horse Attempted break-ins Masquerading Suspicious connections Black listed addresses Nosing: numerous moves through directories Privilege abuse
Information Networking Security and Assurance LabNational Chung Cheng University 15
Vendor Independent Testing Lab
NSS tests a broad range of features of IDS Convenience: ease of installation, deployment and
management UI: reporting and alerts delivered Attack signatures Accuracy Peripheral issues like licensing, documentation and
log management
Information Networking Security and Assurance LabNational Chung Cheng University 16
Vendor Independent Testing Lab
NSS’s test-bed P3 1GHz 768 MB RAM running Windows 2000 SP
2, FreeBSD 4.4 or Red Hat 6.2/7.1 Ghost image 100M Ethernet with CAT-5, Intel NetStructure 40T r
outing Switches and Intel auto-sensing 10/100 network cards
IDS installed on a dual-homed PC on each subnet No firewall used
Information Networking Security and Assurance LabNational Chung Cheng University 17
Vendor Independent Testing Lab
NSS five types of tests Attack recognition
SAN top 20 and/or ICAT top 10 vulnerability lists
Performance under load Back Orifice ping 64-byte, 1514-byte packets/25,50,75 and 100 percent of ne
twork load Adtech AX/4000 Broadband Test System and SmartBits S
MB6000
Information Networking Security and Assurance LabNational Chung Cheng University 18
Vendor Independent Testing Lab
NSS five types of tests IDS evasion techniques
Tools: Fragrouter and whisker
Stateful operation test Tools: stick and snot used to generate false alerts
Host performance Network load, CPU and memory utilizations were monitor
ed
Information Networking Security and Assurance LabNational Chung Cheng University 19
Trade Magazine Evaluation
Interesting approach IDSs in the production network of an ISP Deployed four machines
The metrics were accuracy, ease of use, and uptime
Information Networking Security and Assurance LabNational Chung Cheng University 20
Conclusion