1

Semantic Approach for Attack Knowledge Extraction in Intrusion

Detection Systems

Wei Yan

New Jersey Institute of Technology

NYMAN 2004 Sep 10, 2004

2

Overview

• Motivation

• Semantic scheme

• Attack scenario knowledge extraction

• Semantic query

• Conclusion

3

Current IDS problems

Manual review time consuming and difficult security staff often not available

Alert correlation lack of accepted universal alert standard

IDMEF-XML vendor-specific correlation tools Syntax-oriented approaches need semantic processing

4

Combine NLP and Semantic Web NLP-mature enough to acquire semantics

from semi-constructed texts SW- semantic information retrieval

Syntactic alerts semantic alert streams

Attack scenario knowledge extraction

Manipulate attack knowledge offline for answering the semantic queries

Semantic Solution

5

Alerts representation formalism

Raw alerts data

Alert computational formalism

Alert machine-understandableformalism

Semantic knowledgeimplementation

Snort / RealSecure alerts

Ontology / 2-AASN

PCTCG format

Attack semanticquery

Alert description attack scenario – a sequence of attack events attack event – attack action attack action – semantic role

PCTCG make raw alerts accessible to machines

Scalable and flexible lies above alert syntax layer without modifying existing alert formats

6

Attack knowledge extraction semantic scheme

IDS SensorIDS Sensor

raw data / raw logs

IDS Sensor

PCTCGconvertor

alertsauditlogs

othertype

semanticextractor

PCTCG alert stream

semantic knowledge database

securityadministrator

ontologysemanticnetwork

correlationrules

predict model

Alert contextwindow

aggregatedlogs query model

attack scenarioinstances

Attack scenarioclasses

syntax layer

semantic layer

ontology layer

Security query processorpragmatic layer

semantic knowledge database

7

Ontological semantics

Define semantic role-semantic attribute pair

attack scenario – a sequence of attack events attack event – attack action

Present behavior semantic space by WH-questions

8

Case Grammar

Deep semantic-Relations between verb and other components

Attack action more universal than alert format attack event – attack action attack action – semantic role

9

M- alert messages set with sensor nameC- set of semantic roles between alertsF- set of arguments (case fillers)S - subordinate keywords.

Principal-subordinate Consequence Tagging Case Grammar (PCTCG)

= { , , , }nG M C F S

Snort

{ , , , } =

{{FINGER redirection attempt} ,

{has object, possible cause, cause, consequence tagging},

{finger requery, +information, {DDoS, indirect connection}, lauching attack},

{FINGER

nM C F S

requery, third party}}

where "+" means gain information or priviledge

10

2-Atom Alert Semantic Network (2-AASN)

+)

+)

+)

SN (node1, node2) = {

node 1: < subordinate, node1::subordinate keyword>

node 2: < semantic attribute, node2::case filler > or

node 1: < semantic attribute, node1::case filler >

no

+)

+)de 2: < subordinate, node2::subordinate keyword> ,

node2::case filler < semantic attribute, node 1::subordinate keyword > or

node1::case filler < semantic attribute, node 2::subordinate keyw+)

ord > }

Semantic relations between two alerts node – alert edge- PCTCG semantic attribute/subordinate keyword

2-tuple slot <subordinate, subordinate keyword> <semantic attributes, case filler>

11

Generate 2-AASN

Input-two alerts and IDS sensor name alerts PCTCG stream

If semantic matching between case filler and subordinate keyword, fills the slot: Node1:case filler <semantic role, node2:subordinate keyword>

Extract semantic relation semantic operation semantic rules

NODE 1 NODE 2 enable

Node 1: FINGER 0 query

Node 2: FINGER redirection attempt

NODE 1 NODE 2

username Indirectconnection

2be

object of

cause 4

subordinate

NODE1 NODE2

FINGERdaemon

FINGERrequery

2has

object

has object 2

subordinate

Node 1: FINGER 0 queryNode 2: FINGER redirection attempt

(a) (b)

node 1 = FINGER 0 query

node 2 = FINGER redirection attempt

SN (node1, node2) = {

node 1: < subordinate, username > ,

node 1: < subordinate, FINGER daemon>

node 2: < cause, indirect connecti

on > ,

node 2: < has object, FINGER requery > ,

node 2::indirect connection < be object of, node1::username > ,

node 2::FINGER requery < has object, node1::FINGER daemon > }

FINGER 0query

FINGERdaemon

Snort, ,

Intrusionsensorname

Has object

use account,password

Possiblecause

FINGERcommand withusername '0'

by means of

makeenabling

Consequencetagging

FINGERdaemon, user

name

Subordinatekeywords

FINGERredirection

attempt

FINGERrequery

Snort, ,

Intrusionsensorname

Has object

+information

Possiblecause

DDos, indirect

connection

cause

lauchingattack

Consequencetagging

FINGERrequery, third

party

Subordinatekeywords

entity

, , attribute

case filler

case slot

One-to-one association

One-to-many association

13

Attack semantic context

( , , ) ( , , ) ( , , )x X y Y

MI X Y d p x y d I x y d

2

( , , )( , , ) log

( ) ( )

p x y dI x y d

p x p y

Generate attack scenario instances attack scenario classes-all possible

combinations of attack strategies

Alert context window size(ACW) only consider alerts within ACW

Mutual information

Attack scenario class of DARPA 2000

Set Snort home net : 172.16.112.0, and 172.16.115.0

NODE 6

NODE 4

Object rule

NODE5

NODE1 NODE 2 NODE 3

Node 1: RPC Portmap Sadmind request UDPNode 2: RPC Sadmind UDP PingNode 3: RPC Sadmind UDP NETMGT_PROC_SERVICE CLIENT_DOMAIN overflow attemptNode 4: Reservices rsh rootNode 5: Attack response directory listNode 6: TELNET access/ TELNET login incorrectNode 7: Netbios NT null sessionNode 8: Web MISC doc/accessNode 9: Bad-traffic loopback traffic

Possible Cause rule

Possible cause rule, Instrument rule

Possible cause rule

Pos sibl e c au se r ule

NODE7 NODE8

NODE9

Object ruleEnable rule

AS (DARPA 2000) = { : attack 172.16.115.20, 172.16.112.10, 172.16.115.50

RPC Portmap Sadmind request UDP, RP

enable

objective name

gather informationC Sadmind UDP Ping,

< 202.77.162.213, 172.16.115.20, 10:08:07.354091> < 202.77.162.213, 172.16.115.20, 10:08:07.359636> < 202.77.162.213, 172.16.112.10, 10:

15:10.023115> < 202.77.162.213, 172.16.112.10, 10:15:10.026586>

< 202.77.162.213, 172.16.115.50, 10:15:10.098496> < 202.77.162.213, 172.16.115.50, 10:15:10.102257> RPC Sadmind UDP NETMGT_PROC_SERVICE CLIENT_DOMAIN overflow Telnet access, < 202.77.162.213, 172.16.115.20, 10:33:10.

cause621429> < 202.77.162.213, 172.16.115.20, 10:33:14.728748>

< 202.77.162.213, 172.16.115.20, 10:33:12.652687> < 202.77.162.213, 172.16.115.20, 10:33:18.885651> < 202.77.162.213, 172.16.115.20, 10:33:20.923039> < 202.77.162.213, 172.16.115.20, 10:33:23.011892 > < 202.77.162.213, 172.16.115.20, 10:33:27.165722> < 202.77.162.213, 172.16.115.20, 10:33:32.470221>}

RPC Portmap Sadmind request get control

UDP RPC Sadmind UDP NETMGT_PROC_SERVICE CLIENT_DOMAIN overflow < 202.77.162.213, 172.16.115.20, 10:33:10.611612> < 202.77.162.

cause213, 172.16.115.20, 10:33:10.621429>

< 202.77.162.213, 172.16.115.20, 10:33:12.642958> < 202.77.162.213, 172.16.115.20, 10:33:12.652687> < 202.77.162.213, 172.16.115.20, 10:33:18.875888> < 202.77.162.213, 172.16.115.20, 10:33:18.885651> < 202.77.162.213, 172.16.115.20, 10:33:20.913357> < 202.77.162.213, 172.16.115.20, 10:33:20.923039> < 202.77.162.213, 172.16.115.20, 10:33:27.155926> < 202.77.162.213, 172.16.115.20, 10:33:27.165722> < 202.77.162.213, 172.16.115.20, 10:33:29.205551> < 202.77.162.213, 172.16.115.20, 10:33:29.223090>

Telnet access, instrument RSERVICES rsh root, < 172.16.115.20, 202.77.162.213, 10:50:01.819752> < 172.16.115.20, 202.77.162.213, 10:50:04.146207> < 172.16.112.10, 202.77.162.213, 10:50:21.064056> < 172.16.112.10, 202.77.162.213, 10:50:22.146207> < 172.16.115.50, 202.77.162.213, 10:50:37.923074> < 172.16.115.20, 202.77.162.213, 10:50:38.176538>

bad traffic loopback traffic < 202.77.162.213, 172.16.115.20, 10:33:29.223090>}

launching attacks

15

Attack knowledge semantic query

Less attention paid to attack knowledge semantic query interface.

traditional keyword search semantic content: flexible in answering

sophisticated queries

Weight mapping- attack scenario instance graph

Spread Activation given initial node & destination node return other nodes closely related to initial node

1

1

( , )

ijk

j k

ij

n

i

n

i

W C C

n

n

NODE 6

NODE 4

Object rule

NODE5

NODE1 NODE 2 NODE 3

Node 1: RPC Portmap Sadmind request UDPNode 2: RPC Sadmind UDP PingNode 3: RPC Sadmind UDP NETMGT_PROC_SERVICE CLIENT_DOMAIN overflow attemptNode 4: Reservices rsh rootNode 5: Attack response directory listNode 6: TELNET access/ TELNET login incorrectNode 7: Netbios NT null sessionNode 8: Web MISC doc/accessNode 9: Bad-traffic loopback traffic

Possible Cause rule

Possible cause rule, Instrument rule

Possible cause rule

Pos sibl e c au se r ule

NODE7 NODE8

NODE9

Object ruleEnable rule

202.77.162.213 172.16.115.20

1 2 3 4 6 9

1 0 1 0 0 0 02 1 0 0.86 0 0 03 0 0 0 0.5 0.83 14 0 0 0 0 1 16 0 0 1 0 0 09 0 0 0 0 0 0

202.77.162.213 172.16.112.10

1 2 3 4 6 9

1 0 1 0 0 0 02 1 0 0.8 0 0 03 0 0 0 0.5 0.75 14 0 0 0 0 1 16 0 0 1 0 0 09 0 0 0 0 0 0

202.77.162.213 172.16.112.50

1 2 3 4 6 9

1 0 1 0 0 0 02 1 0 0.8 0 0 03 0 0 0 0.5 0.75 14 0 0 0 0 1 16 0 0 1 0 0 09 0 0 0 0 0 0

Query 1:whether the vulnerability sadmind cause DDos attacks

initial node: vulnerability sadmind (1) destination node: DDos (9)

Query 2: what consequence the RPC Sadmind overflow event

initial node: (3) destination node: -

17

Future work

Enrich plan library

Enrich attack taxonomy

Simulate the benchmark datasets

QUESTIONS?