february 10, 2009 - ndss symposium · motivation of rb‐seeker y system architecture y overview of...

February 10, 2009

Xin Hu, Matthew Knysz, Kang G. Shin{huxin, mknysz, kgshin}@eecs.umich.edu

Computer Science & Engineering, University of Michigan, Ann Arbor

OutlineMotivation of RB‐SeekerSystem ArchitectureOverview of subsystemsEvaluation of resultsConclusion

2/10/2009 2

Motivation: the botnet problem

Financial Incentive◦

Underground market

Common uses of botnets:◦

Redirection/Proxy, Spam, ID theft, DDoS, phishing

Can cause A LOT of damage◦

Can bring down entire systems or nations

2/10/2009 3

Motivation: botnet appeal

Modular and AdaptableEvolve to overcome defenses

Distributed natureDifficult to find/stop botmaster

DiscreetPropagation, infection, and occupation

2/10/2009 4

Motivation: Redirection/Proxy BotnetRedirect users to malicious servers◦

Additional layer of misdirection◦

Protect mothership servers◦

Evade URL based detection or IP based black list

Redirect tohttp://server2

Server1 (redirection bot)

Server2 (redirection bot)

Follow linkhttp://server2

Redirect tohttp://final_Server

Final destination

“mothership”

Real nefarious

content

Forwarding servers

Forwarding servers

Issue HTTP

request

Real nefarious

content

Real nefarious

content

Server2 (proxy bot)

2/10/2009 5

Motivation: RB‐SeekerBotnet is an ideal source for redirection/proxy servers

Botnets used for multiple purposes/scams

Previous research: detection of C&C channel

2/10/2009 6

Overview: RB‐SeekerAutomatic detection of redirection/proxy botnetsUtilizes 3 cooperating subsystemsBehavior‐based detection

Quick identification of aggressive botnets (FP < 0.01%)Advertise many IPs per queryChange IPs very often (short TTL)

Accurate identification of stealthy botnetsAdvertise few IPs per queryChange IPs more slowly (very small TTL, closely monitored)2/10/2009 7

System Architecture

DNS

logs

Content

Analysis

…..Redirection

domain db

URL probing Engine

Spam url db

…….

Web Server nWeb Server 1

Spam source ( Spam trap, open relay, personal

junk mailbox)

DNS probing engine

Report

& alert

University

Core Router

Correlation

engine

Redirection

server IPs

DNS query

history

redirection

domains

Spam Source Subsystem (SSS)

NetFlow Analysis Subsystem (NAS)

Active DNS Anomaly Detection Subsystem (a-DADs)

Local DNS server

NetFlow

Exports

DNS

query

db

RBnet

classification

engine

2/10/2009 8

SSS: Spam Source Subsystem

2/10/2009 9

SSS: Spam Source Subsystem

1. Extract embedded URLs from message bodies

2.

Probe extracted URLs to identify redirection URL links3.

Domains added to redirection domain database

Content

Analysis

…..Redirection

domain db

URL probing Engine

Spam url db


junk mailbox)


…….


redirection

domains

2/10/2009 10

System Architecture

DNS

logs

Content

Analysis

…..Redirection

domain db

URL probing Engine

Spam url db

…….



junk mailbox)

DNS probing engine

Report

& alert

University

Core Router

Correlation

engine

Redirection

server IPs

DNS query

history

redirection

domains




Local DNS server

NetFlow

Exports

DNS

query

db

RBnet

classification

engine

2/10/2009 11

NAS: NetFlow Analysis SubsystemUse NetFlow because:

Inspecting packet contents incurs too much overheadPrivacy concerns

Spammers send image‐ or PDF‐based emailsEvade content‐based filtering

User redirected to RBnet by clicking on malicious webpageInspecting each email not always possible

Privacy concerns/laws

2/10/2009 12

NAS: NetFlow Analysis SubsystemNetFlow: core router on campusLooks for suspicious redirection attempts

Without analyzing packet contents

University

Core Router


2/10/2009 13

NAS: NetFlow Analysis SubsystemSequential Hypothesis testing on:

Flow size, inter‐flow duration, and flow duration

University

Core Router


NetFlow

Exports

2/10/2009 14

NAS: NetFlow Analysis SubsystemIdentifies IPs participating in redirection

Correlation engine uses DNS logs to add domains participating in redirection to redirection domain db

DNS

logs

…..Redirection

domain db

University

Core Router

Correlation

engine

Redirection

server IPs

DNS query

history

redirection

domains


Local DNS server

NetFlow

Exports

2/10/2009 15

Accept H1

PendingAccept H0

PendingAccept H0

NAS: NetFlow Analysis Subsystem

Redirection: obtained from SSS, servers identified as redirection

Normal: normal web browsing over 2 days (removing redirection)

Start

No

H.T. on

inter‐

flow

Sort all flows

chronologically

Yes

Accept H1

H.T on

flow

size

Optional H.T. on flow

duration

Inter‐flow >

threshold

H0

: NormalH1

: Redirection

H.T.

history

database

Normal

Accept H1

Redirection

Size H.T.

history

database

2/10/2009 16

Redirection Normal

System Architecture

DNS

logs

Content

Analysis

…..Redirection

domain db

URL probing Engine

Spam url db

…….



junk mailbox)

DNS probing engine

Report

& alert

University

Core Router

Correlation

engine

Redirection

server IPs

DNS query

history

redirection

domains




Local DNS server

NetFlow

Exports

DNS

query

db

RBnet

classification

engine

2/10/2009 17

a‐DADS: active DNS Anomaly Detection Subsystem

Actively performs DNS queries on domains in redirection domain dbUses CDN Filter to remove Content Delivery Networks

CDNs behave similarly to redirection/proxy botnetsRecursively removes CDNs

…..Redirection

domain db

DNS probing engine

Report

& alert


Local DNS server

DNS

query

db

RBnet

classification

engine

2/10/2009 18


IP Usage:RBnets will accrue more unique IPs over timeRBnets will have more unique IPs per valid query

Reverse DNS names with “bad words”e.g., broadband, cable, comcast, charter, etc…

AS countNumber of different ASes the IPs belong toRBnets consist of home computers scattered geographically

2/10/2009 19


Applies 2‐tier linear SVM on remaining domainsTrained: 124 valid, 18 aggressive, 10 stealth10‐fold cross validation on multiple classifiers

knn, decision tree, naïve Bayesian, various SVMs and kernel functions

2/10/2009 20


SVM‐1: detects Aggressive RBnets based on 2 valid queriesunique IPs, num ASes, DNS “bad words”

2/10/2009 21

a‐DADs: SVM‐1 Aggressive RBnets

2/10/2009 22


SVM‐2: detects Stealth RBnets using a week of DNS queries unique IPs, num ASes

2/10/2009 23

a‐DADs: SVM‐2 Stealth RBnets

2/10/2009 24

Evaluation of ResultsSSS and NAS identified 91,600+ suspicious domains over 2 month perioda‐DADS CDN Filter

Removed 5,005 CDN domainsRecursion 16.8% increase in identified CDN domains (13.1% in IPs)Similar technique for valid domains reduced this to 35,000+ domains to be monitored

2/10/2009 25

Evaluation of Results

SVM‐1: Experienced 1 FP (< 0.008%)

2/10/2009 26

Aggressive RBnets: Redirection vs. Proxy Botnets

2/10/2009 27

48.8% 51.2%

Stealth RBnets

2/10/2009 28

Evaluation of ResultsFFSN detector:

Detected 124 of the 125 Aggressive RBnets1 FP: same as ours (mozilla.org)Missed all the Stealth RBnets

2/10/2009 29

ConclusionDesigned and implemented system for detecting redirection/proxy botnetsUses network detection techniques

multiple data sources readily available to enterprise network environments

Behavior‐based detection works despite use of C&C protocol or structureCapable of detecting Aggressive and Stealthy RBnetsAutomatic detection with low false positives (< 0.01%)

2/10/2009 30

Questions?

2/10/2009 31

Evaluation of Results

2/10/2009 32

february 10, 2009 - ndss symposium · motivation of rb‐seeker y system architecture y overview of...

Documents