ecs236 winter 2006: intrusion detection #1: ids architecture
DESCRIPTION
ecs236 Winter 2006: Intrusion Detection #1: IDS Architecture. Dr. S. Felix Wu Computer Science Department University of California, Davis http://www.cs.ucdavis.edu/~wu/ [email protected]. Intrusion Prevention. Prevention : This should/must never be broken in! - PowerPoint PPT PresentationTRANSCRIPT
01/04/2006 ecs236 winter 2006 1
ecs236 Winter 2006:
Intrusion DetectionIntrusion Detection#1: IDS Architecture
Dr. S. Felix Wu
Computer Science Department
University of California, Davishttp://www.cs.ucdavis.edu/~wu/
01/04/2006 ecs236 winter 2006 2
Intrusion PreventionIntrusion Prevention
Prevention: This should/must never be broken in!– “This” means a perfectly designed,
implemented, and managed/configured secure system!
01/04/2006 ecs236 winter 2006 3
Intrusion DetectionIntrusion Detection
Prevention: This should/must never be broken in!
Detection: “This” will need to face the reality check!– We had, have, will have so many “expected”
unexpected.– Industry never really serious about cyber
security – profit/market-driven
01/04/2006 ecs236 winter 2006 4
We accept it as a fact…We accept it as a fact…
01/04/2006 ecs236 winter 2006 5
And, we have to have…And, we have to have…
01/04/2006 ecs236 winter 2006 6
Intrusion DetectionIntrusion Detection
Prevention: This should/must never be broken in!
Detection: “This” will need to face the reality check!– We had, have, will have so many “expected”
unexpected.– We had, have, will have even more
“unexpected” unexpected!!
01/04/2006 ecs236 winter 2006 7
To: All Faculty, Staff and Students
On Tuesday, January 03, 2006, UC Davis implemented temporary measures to prevent the exploitation of a serious new computer vulnerability for which no patch is yet available. This vulnerability affects Windows 2000, Windows XP, Windows Server 2003, Windows 98 and ME systems and may be exploited when infected email file attachments or infected Web pages are viewed. Once a computer is infected, data may be permanently lost and/or a remote attacker could gain control of the computer. After extensive consultation with the campus leadership, the decision has been made to temporarily block wmf image attachments. These files can have a number of different extensions, but most commonly will have .wmf and .jpg extensions.
01/04/2006 ecs236 winter 2006 8
Max-Sequence # AttackMax-Sequence # Attack Block LSA updates for one hour by injecting
one bad LSA.– You can hit it once and come back in an hour.
Implementation Bug!– Two independently developed OSPF packages.
– MaxSeq# LSA Purging has not been implemented correctly!!
Announced in May, 1997.
01/04/2006 ecs236 winter 2006 9
What is Intrusion Detection?What is Intrusion Detection?
01/04/2006 ecs236 winter 2006 10
Intrusion DetectionIntrusion Detection Detecting intrusions such as
– Viruses, Worms, Spywares, Phishing, Spamming, Insider, Un-authorized activities, faults/failures, among many others
Detecting and Managing anything “unexpected”– Anomalies
Question: “Detecting what??”
01/04/2006 ecs236 winter 2006 11
Intrusion DetectionIntrusion Detection
IntrusionDetection
Model
Input eventsequence Results
01/04/2006 ecs236 winter 2006 12
Results??Results?? This email contains virus XYZ This email might be a spam with 80%
probability This email is somewhat trusted based on
your social network This email might be malicious This email might be malicious for reasons
ABC and DEF.
01/04/2006 ecs236 winter 2006 13
Intrusion DetectionIntrusion Detection
IntrusionDetection
Model
Input eventsequence Results
Pattern matching
01/04/2006 ecs236 winter 2006 14
IDS EventsIDS Events
TCPdump traces OS kernel and Host-level information BGP traces Application Logs Many others…
01/04/2006 ecs236 winter 2006 15
Anti-VirusAnti-Virus
VirusDetection
VirusDefinition
Input eventsequence Results
Pattern matching
01/04/2006 ecs236 winter 2006 16
Credit Card Fraud DetectionCredit Card Fraud Detection
FraudDetection
SpendingPatterns
Input eventsequence Results
Statistical Pattern Matching
01/04/2006 ecs236 winter 2006 17
SNORTSNORT
Rules
Input eventsequence Results
Pattern matching
01/04/2006 ecs236 winter 2006 18
Welcome to ecs236Welcome to ecs236
S. Felix Wu – [email protected], x4-7070
Office: 3057 Engineering II Office Hours:
– 2-3 p.m. on Tuesday and Friday– by appointment
01/04/2006 ecs236 winter 2006 19
Intrusion DetectionIntrusion Detection
Practical Engineering– Performance, Accuracy, Scalability,
CPU/Memory, Correlation, Deployment. Theoretical Foundation
– Detectability/Limitation, Dimensionality, Entropy, False Negative and Positive, Evaluation
01/04/2006 ecs236 winter 2006 20
In this quarter…In this quarter… The architecture of ID and IDS
– Stateful versus stateless– Signature, specification, anomaly
Analysis of ID Results– Explanation and Analysis– Event Correlation
IDS Evaluation or Attacking IDS– Attack Polymorphism and IDS Evasion
IDS Fundamental Principles
A balance between Engineering a High-Performance IDS system
Fundamentally understand our limitations
01/04/2006 ecs236 winter 2006 21
Starter: SNORTStarter: SNORT Understand the architecture and source code How to evaluate SNORT? What is the most critical performance
bottleneck of SNORT? Is SNORT stateful or stateless? Why?
– What are the pros and cons regarding SNORT versus Bro?
http://www.snort.org/
01/04/2006 ecs236 winter 2006 22
SyllabusSyllabus SNORT IDS engine Anomaly-based Approach Event Correlation and Analysis IDS Evaluation Advanced Research Topics
01/04/2006 ecs236 winter 2006 23
Course RequirementsCourse Requirements 30%: Starter 15%: Proposal 30%: Final Project 25%: Class Participation
– “develop interesting/creative research problems related to the lectures/reading assignments, and justify the reasons”
– And, you need to interact with the instructor!– 5 of them 5% each (1~2 pages)
01/04/2006 ecs236 winter 2006 24
Final ProjectsFinal Projects Polymorphic/Metamorphic Worm detection Integration of Network/Host IDSes Anomaly Detection in SNORT IDS Evaluation using TCPopera SNORT event correlation and explanation Stateful SNORT SNORT evasion
01/04/2006 ecs236 winter 2006 25
about Web siteabout Web site
http://www.cs.ucdavis.edu/~wu/ecs236/ all lectures, notes, announcements,
homework assignments, tools, papers will be there.
01/04/2006 ecs236 winter 2006 26
Let’s start it…Let’s start it…
SNORT 2.4.3– You might need to install the PCRE (Perl
Compatible Regular Expression) package. Get it compile and install
– Any platform you like…
01/04/2006 ecs236 winter 2006 27
SnortSnort Open Source, since 1998 Used by many major network security
products Signature-based (more than 3000+) Simple IP header protocol anomaly
detection Simple stateful pattern matching
01/04/2006 ecs236 winter 2006 28
The Spirit of SNORTThe Spirit of SNORT
They started with something very simple and extensible.
If we feel we need the XYZ feature (due to an attack like STICK), we will write a plug-in for XYZ!!
An evolving system– But, not sure how much in the future…
01/04/2006 ecs236 winter 2006 29
Signature-base NIDSSignature-base NIDSMartin Overton, “Anti-Malware Tools: Intrusion Detection Systems”,
European Institute for Computer Anti-Virus Research (EICAR), 2005
Signature found at W32.Netsky.p binary sample
Rules for Snort:
01/04/2006 ecs236 winter 2006 30
Signature-based RuleSignature-based Rulealert tcp $EXTERNAL_NET any -> $HOME_NET 139
flow:to_server,established
content:"|eb2f 5feb 4a5e 89fb 893e 89f2|"
msg:"EXPLOIT x86 linux samba overflow"
reference:bugtraq,1816
reference:cve,CVE-1999-0811
classtype:attempted-admin
01/04/2006 ecs236 winter 2006 31
SNORT RulesSNORT Rules
Alert Pass log
01/04/2006 ecs236 winter 2006 32
01/04/2006 ecs236 winter 2006 33
False Alarm Rate versus False Alarm Rate versus False PositiveFalse Positive
Rules
101 events100 good + 1 bad
Pattern matching
2 alerts1 good + 1 bad
False Alarm Rate = 50%False Positive = 1%
01/04/2006 ecs236 winter 2006 34
STICKSTICK
SNORTrules STICK
AttackPackets
StatelessSNORTAlerts
01/04/2006 ecs236 winter 2006 35
What Alerts do we want?What Alerts do we want?
This is an administrative/policy issue.– Do I want to know this?
Idea: How can we rank the information quantitatively (in a meaningful way)?– Maybe it is hard to rank “one particular alert”– But, it is much more useful to rank “a sequence
of alerts/events” Correlation & Anomaly Detection!!
01/04/2006 ecs236 winter 2006 36
01/04/2006 ecs236 winter 2006 37
01/04/2006 ecs236 winter 2006 38
PreprocessorPreprocessor
Stream4 Frag2 Telenet_negotiation HTTP normalization RPC_decode Portscan Back Orifice
01/04/2006 ecs236 winter 2006 39
Experimental Pre-Experimental Pre-
Arpspoof Asn1_decode Fnord (NOP detection) portscan2
01/04/2006 ecs236 winter 2006 40
01/04/2006 ecs236 winter 2006 41
01/04/2006 ecs236 winter 2006 42
RTNRTNRule Tree Node
01/04/2006 ecs236 winter 2006 43
RTNRTN//OTNOTN MatrixMatrixOptional Tree Node
alert tcp any any -> 192.168.1.0/24 111 (content:”|00 01 86 a5|”; msg:”mountd access”;)
01/04/2006 ecs236 winter 2006 44
01/04/2006 ecs236 winter 2006 45
SNORT RulesSNORT Rules
Dynamic Activation Alert Pass log
01/04/2006 ecs236 winter 2006 46
01/04/2006 ecs236 winter 2006 47
01/04/2006 ecs236 winter 2006 48
Fast Multiple PatternsFast Multiple Patterns
Wu-Manber (Bad-word shift)– Consume the least amount of memory
Aho-Corasick (FSM)– Fast, potential for parallelism and FPGA
Boyer-Moore (Bad-word shift)– For small rule sets
01/04/2006 ecs236 winter 2006 49
Example: P = {he, she, his, hers}
0
1
h
2
9
8
6
3
4
57
e
s
i h
s er
s
Initial State
Accepting State
StateTransition Function
h Sh
hh
hh
S
SS
S
S
S
i
h
r
h
•The Construction: linear time.•The search of all patterns in P: linear time
(Edges pointing back to State 0 are not shown).
01/04/2006 ecs236 winter 2006 50
DistanceDistance
content:"SITE"; nocase; content:"EXEC"; distance:0; nocase;
01/04/2006 ecs236 winter 2006 51
WithinWithin
content: "Content-type\: video/x-ms-asf"; content:"|0a|"; within:2;
01/04/2006 ecs236 winter 2006 52
Byte jumpByte jump
byte_jump:4,4, relative,align; byte_jump:4,4, relative,align; content: "|00 01 86 A5|"; within:4;
01/04/2006 ecs236 winter 2006 53
Byte testByte testbyte_test:1,>,7,1;
01/04/2006 ecs236 winter 2006 54
A few issues about A few issues about SNORT basicsSNORT basics
Performance– DoS attack against SNORT
Expressiveness– Statefulness in SNORT – is it enough?
01/04/2006 ecs236 winter 2006 55
HW#1HW#1
I will post HW#1 details on the class website this afternoon (01/11/2006)
01/04/2006 ecs236 winter 2006 56
BroBro
Packet stream
Filtered packet stream
Event stream
AlertsPolicy script
Event control
tcpdump filters
Event Engine
Network
Libpcap
Detection Engine
01/04/2006 ecs236 winter 2006 57
Be Stateful!! (Bro Be Stateful!! (Bro SNORT) SNORT) HTTP server attack
– Snort signature: simple pattern matching on MS ISS attack
– Bro rule: additional check to see if, e.g., host is running Apache ignore alarm
Error code checking– Snort signature: no checking of reply
– Bro rule: Looks at return code for HTTP/FTP/SMTP, signature match + error code = no alert
Multi-stage attacks– Easy in Bro to express “signature A but only if followed by signature B” or
“A unless followed by B”
– Easy to express “generate alarms if given host triggers N or more signatures” or “triggers against N or more local hosts”
01/04/2006 ecs236 winter 2006 58
Stateful is good?Stateful is good? How to design an IDS
– Performance– Statefully powerful??
01/04/2006 ecs236 winter 2006 59
01/04/2006 ecs236 winter 2006 60