learning sequential models for detecting anomalous protocol usage (work in progress)

Lucent Technologies – ProprietaryUse pursuant to company instruction

Learning Sequential Models for Detecting Anomalous Protocol Usage (work in progress)

Lloyd Greenwald, Lucent Bell Labs

Lucent Technologies – Proprietary

Use pursuant to company instruction

Machine Learning Algorithms for Surveillance and Event Detection

Surveillance:

Network traffic

Event Detection:

Unknown vulnerability exploits using sequences of messages

Machine Learning Algorithms:

Learning Markov models to capture recent sequential protocol usage



NIDS Monitors Traffic and Detects Events That Violate Security Policy

(from Bro user manual)



Example Attack Sequence: NIDS Evasion Attack

(from Handley et. al. 01)

Fake missing packet (to cause buffering)

Send two interspersed sequences for same connection

Even with same ttl’s there is ambiguity with how end systems will re-create sequence



Example Attack: Multi-Step

Apache/mod_ssl worm (aka Slapper)

1. Probe/scan target for vulnerability by sending HTTP GET request on tcp port 80 that violates 1.1 standard

2. Response identifies server as Apache

3. Exploit for SSLv2-enabled OpenSSL 0.9.6d vulnerability sent to tcp port 443

4. Target sends traffic back to attacker on udp port 2002

5. Target begins scanning for other vulnerable hosts



Technical Approach

DetectionPrediction

Sequential ModelGeneration

TrafficMonitoring

AutomaticDefense

ProactiveTest CaseGeneration

QueryExperimentation

ReactiveDetection

ReactiveTest CaseGeneration

ProactiveDirected Probing

ReactiveDirected Probing

Learning

Defense

Automatically build sequential models of recent protocol usage

Analyze models for common and uncommon sequences

Proactively exercise protocol implementation with uncommon sequences sampled from models

Reactively detect uncommon sequences

Build new defense policies for NIDS



Prior Work: Machine Learning Algorithms for Automated Test Case Generation

Surveillance: Web logs

Event Detection: Exercise errors in web applications

Machine Learning Algorithms: Learning Markov models to capture recent sequential web application usage

Session Data

Internet

Internet



Prior Work: Automated Test Case Generation

Leverage dynamic user information to automatically generate NEW test cases for web applications.

Session Data

Key contribution 1) sequential statistical modelsbuilt using machine learning techniques.

Key contribution 2) flexible test case generation exploitingprobabilistic sampling methods.



Web Application Studied

– Front end – JSP

– Back end - MySql

– 10K lines of code, 118 methods, 12 classes

– 123 user sessions (sequential application usage extracted from web log)

Question: Can we build models that can be used to generate new, valid user sessions?



Building Markov Models From Web Logs

Extract User Sessions from Web Log

12.3.40.65 GET index.jsp

12.3.40.65 GET login.jsp

12.3.40.65 GET /apps/bookstore/reg.jsp?member_login=hello&member_password=world&member_password2=world

12.3.40.65 GET myinfo.jsp

Control Model: possible sequences of URLS that are visited

Data Model: possible sets of parameter values (name-value pairs)



Control Models

unigram: Probability of a user visiting a given page independent of previous page

P(currentPage=X)

default

register

search

bookDetail

0.65

0.20

0.05

0.10



Control Models

bigram: Conditional probability of a user visiting a page, given the previous page

P(currentPage=X | lastPage=Y)

default

register

search

bookDetail

0.30

0.450.15

0.10



Control Modelstrigram: Conditional probability of a user visiting a page,

given the previous two pages

P(currentPage=X | lastPage1=Y1, lastPage2=Y2)

default

register

search

book Detail

0.05 0.30

0.10

0.55



Reliability vs. Discrimination

unigram bigram trigram

Greater discrimination (more context)

Greater reliability (more training data)



Data Models

advanced: P(values=X | lastPage+importantParams=Y1,currentPage=Y2)

•Books.do?category=3BookDetail.do?category=3&itemId=8

•Books.do?category=3BookDetail.do?category=3&itemId=8

•simple: P(values=X | currentPage =Y)“important parameter”



Page1: http://decide.cs/bookstore/BookDetail.do?itemId=18

Page2: http://decide.cs/bookstore/AddOrder.do?

Simple Data Model

quantity=99&itemId=36



Page1: http://decide.cs/bookstore/BookDetail.do?itemId=18

Page2: http://decide.cs/bookstore/AddOrder.do?

Advanced Data Model

quantity=1&itemId=18



Generating Test Cases by Combining Control and Data Models

Generate arbitrary queries about user sessions and use these queries to build test cases– What are the k most likely user sessions?

– What are the k least likely user sessions?

– Generate k user sessions randomly, according to the distribution represented in a web log.



0

20

40

60

80

100

120

140

160

180

200

20 40 60 80 100 120 140 160 180 200# sessions run

# o

rde

rs a

dd

ed

1gram_advanced 1gram_simple 2gram_advanced 2gram_simple

3gram_advanced 3gram_simple original

Can our models be used to generate valid user sessions?



Network Protocol Modeling Challenges

Using live network data instead of logsAccess to reconstructed traffic in both directionsCan build models using data from multiple machines (instead of web

log from single server)What are we generating?

Sequences of packetsSequence of high-level events that can be turned into packets

What is a user session?Single connectionCluster connections from subset of 5-tuple (srcIP, dstIP, srcPort, dstPort, Protocol)

What are control and data models?Can we generate valid new sequences?



Building Sequential Model to Discover NIDS Evasion Attack

(from Handley et. al. 01)

Control model: sequence numbers

Data model: TTLs and payload

How hard is it to discover that this pattern is “uncommon” ?



DiscussionAre Markov models sufficient for this task? Too propositional?

Are data models too sparse? Are state spaces too large?

How hard is anomaly detection in this framework? What is a good definition for “uncommon” traffic that doesn’t produce many false positives or false negatives? What about emerging new usage patterns? How to avoid “training attacks”?

How much protocol knowledge to use in building models?

Can signature matching events be used in data model?

Besides generating sequences, what other analyses can we perform? Entropy of models to determine level of history-dependence in traffic?



Related WorkHost-based and Network-base Intrusion Detection Systems

(NIDS)– Signature-based anomaly detection -- manual analysis

– Packet-based or with context – detect known vulnerabilities and behaviors

Formal verification of protocols – require extensive protocol knowledge; do not account for implementation variations

Scrubbers and Normalizers remove TCP/IP ambiguities – do not account for application-layer ambiguities and must make tradeoffs concerning removing ambiguities that change semantics or lead to performance loss

Fuzzing/Fault-injection – random generation of inputs for vulnerability detection – generates invalid sequences

learning sequential models for detecting anomalous protocol usage (work in progress)

Documents