320 ieee transactions on network and …cfung/research/fungtnsm12.pdf320 ieee transactions on...

320 IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT, VOL. 9, NO. 3, SEPTEMBER 2012

Effective Acquaintance Management based onBayesian Learning for Distributed Intrusion

Detection NetworksCarol J Fung, Member, IEEE, Jie Zhang, Member, IEEE,

and Raouf Boutaba, Fellow, IEEE

Abstract—An effective Collaborative Intrusion Detection Net-work (CIDN) allows distributed Intrusion Detection Systems(IDSes) to collaborate and share their knowledge and opinionsabout intrusions, to enhance the overall accuracy of intrusionassessment as well as the ability of detecting new classes ofintrusions. Toward this goal, we propose a distributed Host-based IDS (HIDS) collaboration system, particularly focusingon acquaintance management where each HIDS selects andmaintains a list of collaborators from which they can consultabout intrusions. Specifically, each HIDS evaluates both thefalse positive (FP) rate and false negative (FN) rate of itsneighboring HIDSes’ opinions about intrusions using Bayesianlearning, and aggregates these opinions using a Bayesian decisionmodel. Our dynamic acquaintance management algorithm allowseach HIDS to effectively select a set of collaborators. We evaluateour system based on a simulated collaborative HIDS network.The experimental results demonstrate the convergence, stability,robustness, and incentive-compatibility of our system.

Index Terms—Host-based intrusion detection systems, ac-quaintance management, collaborative networks, computer se-curity.

I. INTRODUCTION

IN recent years, cyber attacks from the Internet are be-coming more sophisticated and harder to detect. Intrusions

can have many forms such as worms, spamware, viruses,spyware, denial-of-service attacks (DoS), malicious logins,etc. The potential damage of these intrusions can be signif-icant if they are not detected promptly. A recent exampleis the Conficker worm which infected more than 3 millionMicrosoft server systems during the year of 2008 to 2009,with the estimated economic loss of 9.1 billion dollars [1].Contemporary intrusion attacks compromise a large numberof nodes to form botnets. Attackers not only harvest privatedata and identify information from compromised nodes, butalso use those compromised nodes to launch attacks such as

Manuscript received October 11, 2011; revised March 12, 2012. Theassociate editor coordinating the review of this paper and approving it forpublication was Y. Diao.

C. J Fung is with the Department of Computer Science, University ofWaterloo, Waterloo, Ontario, Canada (e-mail: [email protected]).

R. Boutaba is with the Department of Computer Science, Univer-sity of Waterloo, Waterloo, Ontario, Canada, and the Division of ITConvergence Engineering, POSTECH, Republic of South Korea (e-mail:[email protected]).

J. Zhang is with the School of Computer Engineering, Nanyang Techno-logical University, Singapore (e-mail: [email protected]).

Digital Object Identifier 10.1109/TNSM.2012.051712.110124

distributed-denial-of-service (DDoS) attacks [2], distributionof spam messages, or organized phishing attacks [3].

To protect computer users from malicious intrusions, In-trusion Detection Systems (IDSes) are designed to monitornetwork traffic and computer activities and to raise intrusionalerts to network administrators or security officers. IDSescan be categorized into host-based Intrusion Detection Sys-tems (HIDSes) or network-based Intrusion Detection Sys-tems (NIDSes) according to their targets, and into signature-based IDS or anomaly-based IDS according to their detectionmethodologies.

A NIDS monitors the network traffic from/to one or a groupof computers and compare the data with known intrusionpatterns. Examples of NIDS are Snort [4] and Bro [5]. A HIDSidentify intrusions by comparing observable intrusion datasuch as log files and computer activities against suspiciouspatterns. Examples of HIDSes are OSSEC [6], an anti-virussoftware, and Tripwire [7]. A signature-based IDS identifiesmalicious code if a match is found with a pattern in theattack signature database. An anomaly-based IDS [8], [9],on the other hand, monitors the traffic or behavior of thecomputer and classifies that as either normal or anomalousbased on some heuristics or rules, rather than patterns orsignatures. Alerts are raised when anomalous activities arefound. Compared to NIDS, a HIDS has a deeper sight intoeach host so it can detect intrusions which are hard to identifyby observing network traffic only. However, a single HIDSlacks an overall view of the network and can be easily com-promised by new attacks. Collaboration among HIDSes caneffectively improve the overall intrusion detection efficiencyby using collective information from collaborative HIDSes.In this paper, we focus on the collaboration among HIDSes.A signature-based IDS can accurately identify intrusions andthe false positive rate is low compared to anomaly-baseddetection. However, it is not effective for zero-day attacks,polymorphic, and metamorphic malware [10]. An anomaly-based IDS may detect zero-day attacks by analyzing theirabnormal behaviors. However, an anomaly-based detectionusually generates a high false positive rate.

Traditional HIDSes work in isolation and they rely ondownloading signatures or rules from their vendors to detectintrusions. They may be easily compromised by unknownor new threats since not a single vendor has comprehensiveknowledge about all intrusions. Collaboration among HIDSes

1932-4537/12/$31.00 c© 2012 IEEE

FUNG et al.: EFFECTIVE ACQUAINTANCE MANAGEMENT BASED ON BAYESIAN LEARNING FOR DISTRIBUTED INTRUSION DETECTION NETWORKS 321

can overcome this weakness by having each peer HIDS benefitfrom the collective knowledge and experience shared by otherpeers. This enhances the overall accuracy of intrusion assess-ment as well as the ability to detect new classes of intrusions.A Collaborative Intrusion Detection Network (CIDN) is anoverlay network which provides a collaboration infrastructurefor HIDSes to share information and experience with eachother to achieve improved detection accuracy. The topology ofa CIDN can be centralized, such as DShield [11], CRIM [12],and N-version AV [13], or distributed, such as Indra [14],NetShield [15], and Host-based CIDN [16]. Note that theCIDN collaboration framework is also applicable to NIDSes.

However, in a CIDN, malicious insiders may send falseinformation to mislead other HIDSes to make incorrect in-trusion decisions, and in this way, render the collaborationsystem not useful. Furthermore, HIDSes in the collaborationnetwork may have different intrusion detection expertise levelsand capabilities. How a HIDS selects collaborators to achieveoptimal efficiency is an important problem to solve for aCIDN. We define a CIDN acquaintance management as theprocess of identifying, selecting, and maintaining collaboratorsfor each HIDS. An effective acquaintance management iscrucial to the design of a CIDN.

We provide a Bayesian learning technique that helps eachHIDS identify expert nodes and novice nodes based on pastexperience with them. Specifically, the false positive (FP) rateand false negative (FN) rate of each collaborator. Dishonestcollaborators are identified and removed from its collaboratorlist. We define feedback aggregation in CIDN as a decisionmethod whether or not to raise an alarm based on the collectedopinions (feedback) from collaborator HIDSes. We proposea Bayesian decision model for feedback aggregation. Bayestheory is used to estimate the conditional probability of intru-sions based on feedback from collaborators. A cost functionis modeled to include the false positive decision cost and falsenegative decision cost. A decision of whether to raise alarm ornot is chosen to achieve the minimal cost of false decisions.

For collaborator selection, a HIDS may add all honest HID-Ses into its collaborator list to achieve maximized detectionaccuracy. However, including a large list of collaborators mayresult in high maintenance cost. We define an acquaintanceselection as the process to find the list of collaborators tominimize false decision cost and maintenance cost. Existingapproaches for acquaintance management often set a fixednumber of collaborators [17], or a fixed accuracy thresholdto filter out less honest or low expertise collaborators [18],[19], [20]. These static approaches lack flexibility, and thefixed acquaintance length or accuracy threshold may not beoptimal when the context changes (e.g. some nodes leave thenetwork and some new nodes join the network). Our proposedacquaintance management algorithm can dynamically selectcollaborators in any context setting to obtain high efficiencyat minimum cost.

For collaborator maintenance, the HIDSes in our systemperiodically update their collaborators lists to guarantee anoptimal cost. A probation list is used to explore and learn thequality of new potential collaborators. New collaborators stayin the probation list for a certain period before their feedbackis considered for intrusion decision.

HIDS1Acq List:HIDS2HIDS4HIDS5HIDS10

LAN

LAN

LAN

Internet5

7

1

2

3

4

89

10

12

11

6

Fig. 1. Overlay network of collaborating HIDSes.

We evaluate our system using a simulated collaborationnetwork using a Java-based discrete-event simulation frame-work. The results show that the proposed Bayesian decisionmodel outperforms the threshold-based model [13], whichonly counts the number of intrusion reports, in terms offalse decision cost. The results also show that our dynamicacquaintance management algorithm outperforms the staticapproaches of setting a fixed acquaintance length or accu-racy threshold. Finally, our approach also achieves severaldesired properties, such as efficiency, stability, robustness, andincentive-compatibility.

Major contributions of our work are summarized as follows:

1) A novel Bayesian decision model is proposed for feed-back aggregation in collaborative intrusion detectionsystems to achieve minimal false decision cost;

2) An acquaintance selection algorithm is devised to opti-mally select collaborators, which leads to minimal over-all cost including false decision cost and maintenancecost;

3) A dynamic acquaintance management algorithm is pro-posed to integrate the concept of probation period andconsensus negotiation;

The rest of the paper is organized as follows. Section IIgives a brief introduction of CIDNs and some of their im-portant components. Section III describes our formalizationof HIDS learning model and feedback aggregation. Acquain-tance selection and management algorithms are presentedin Section IV. We discuss several potential threats to oursystem and corresponding defenses in Section V. We thenpresent evaluation results demonstrating the effectiveness ofour acquaintance management and its desired properties inSection VI. We discuss some related work in Section VII, andconclude this paper in Section VIII.

II. COLLABORATIVE INTRUSION DETECTION NETWORK

A Collaborative Intrusion Detection Network (CIDN) isshown in Figure 1 as an overlay network of collaboratingHIDSes. HIDSes from different vendors are connected ina peer-to-peer manner. Each peer HIDS maintains a list of


TABLE ISUMMARY OF NOTATIONS

Symbol MeaningX ∈ {0, 1} Random variable denoting whether there is an attack or notY ∈ {0, 1} Random variable of positive or negative diagnose from a HIDSy A feedback instance vector from acquaintancesY Feedback vector from acquaintancesC Set of acquaintance candidatesA Set of Acquaintancesl The acquaintance list lengthδ The decision of raising alarm or notR(.) The risk cost of false alarms and miss intrusionsM(.) The maintenance cost of acquaintancesCfp, Cfn Unit cost of false alarm and miss intrusionCa Unit cost of maintaining each acquaintanceπ0, π1 Priory probability of no-intrusion and with-intrusionTi, Fi True positive rate and false positive rate of IDS iλ Forgetting factor of the past experience

collaborators. We use the terminology Acquaintance List torepresent the list of collaborators for each HIDS and use theterms collaborator and acquaintance interchangeably in thispaper. Note that the relationship of collaboration is symmetric.i.e., if HIDS X is in HIDS Y’s acquaintance list, then HIDSY is also in HIDS X’s acquaintance list. HIDSes may havedifferent expertise levels in intrusion detection. They canbe compromised or dishonest when providing feedback tocollaborators. They may also be self-interest driven whenproviding assistance to other peers.

When a HIDS detects suspicious behavior for example froman executable file but lacks confidence to make a decisionwhether it should raise an alarm or not, it may send aconsultation request to its collaborators for diagnosis. Theconsultation request contains the description of the suspiciousactivities or the executable file, including the suspicious in/outcommunication traffic, the log entries of suspicious activities,or the binaries source of the suspicious activities. The feed-back from collaborators contains the assessment result of therequest, which is either a “yes” for under-attack or a “no”for no-attack. All feedback will be aggregated and a finaldecision is made based on the aggregation result. However, amalicious (or malfunctioning) HIDS in a CIDN may send falseintrusion assessments to its collaborators, resulting in falsepositive or false negative decisions made by the requesters. Itis thus important to evaluate peer HIDSes and choose the onesthat likely provide higher detection accuracy. Acquaintancelearning allows a HIDS to evaluate the detection accuracy ofits collaborators based on its past experience with them.

In our system, HIDSes use test messages to evaluate thedetection accuracy and truthfulness of other HIDSes. Testmessages are “bogus” consultation requests, and sent out ina way that makes them difficult to be distinguished fromreal consultation requests. The testing node needs to knowbeforehand the true diagnosis result of the test message andcompare it with the received feedback to derive detectionaccuracy of other HIDSes. For example, the samples of knownmalware and legitimate files can be used as test messages. The

usage of test messages helps with identifying inexperiencedand/or malicious nodes within the network. The idea of “testmessages” was previously introduced in [21] and [16]. Itis adopted in our CIDN for each HIDS to gain experiencewith others quickly at affordable communication cost, and toevaluate the detection accuracy of its acquaintances.

Collaborative intrusion decision is a process for a HIDS toaggregate the intrusion assessments (feedback) from its col-laborators and make an overall decision. The evaluation resultof acquaintances’ detection accuracy is used as a parameterin the feedback aggregation function to improve collaborativeintrusion detection accuracy. The acquaintance managementsystem updates acquaintance list periodically to recruit newcollaborators and/or remove unwanted ones. The collaborationrelationship is established based on mutual consensus, i.e., itis established only if both sides agree. We will elaborate onour acquaintance management for CIDN in Section IV.

III. HIDS DETECTION ACCURACY EVALUATION AND

FEEDBACK AGGREGATION

To select collaborators, a HIDS should first learn thequalification of all candidate HIDSes. In this section, we firstintroduce a Bayesian learning model to evaluate the detectionaccuracy of the candidates. A Bayesian decision model is thenused to optimally aggregate feedback from acquaintances.

A. Detection Accuracy for a Single HIDS

To better capture the qualification of a HIDS, we use bothfalse positive (FP) and true positive (TP) rates to representthe detection accuracy of a HIDS. Let A denote the set ofacquaintances and random variables Fk and Tk denote theFP and TP rates of acquaintance k ∈ A respectively. FPis the probability that the HIDS gives a positive diagnosis(under-attack) under the condition of no-attack, and TP is theprobability that the HIDS gives a correct positive diagnosisunder the condition of under-attack. Let random variableX ∈ {0, 1} represent the random event on whether thereis an attack or not, and let random variable Y ∈ {0, 1}


denote whether the HIDS makes a positive diagnosis or not.Then FP and TP can be written as P[Y = 1|X = 0] andP[Y = 1|X = 1], respectively. The list of notations issummarized in Table I.

Let Fk and Tk be the probability density functions ofFk and Tk whose support is [0, 1]. We use the notationZ0 : Yk = 1|X = 0 and Z1 : Yk = 1|X = 1 to representthe conditional variables that acquaintance k gives positivedecision under the conditions where there is no attack andthere is an attack respectively. They can be seen as twoindependent random variables satisfying Bernoulli distributionwith successful rates Fk and Tk, respectively. The past experi-ence with acquaintance k can be seen as the samples from theBernoulli distributions. According to the Bayesian probabilitytheory [22], the posterior distribution of Fk and Tk given a setof observed samples can be represented using a Beta function,written as follows:

Fk ∼ Beta(xk|α0k, β

0k) =

Γ(α0k+β0

k)

Γ(α0k)Γ(β

0i )xα0

k−1k (1 − xk)

β0k−1,(1)

Tk ∼ Beta(yk|α1k, β

1k) =

Γ(α1k+β1

k)

Γ(α1k)Γ(β

1i )yα1

k−1k (1 − yk)

β1k−1,(2)

where Γ(·) is the gamma function [23], and its parameters α0k,

α1k and β0

k , β1k are given by

α0k =

u∑j=1

λt0k,j r0k,j β0k =

u∑j=1

λt0k,j (1− r0k,j);

α1k =

v∑j=1

λt1k,j r1k,j β1k =

v∑j=1

λt1k,j (1− r1k,j), (3)

where α0k, β

0k, α

1k, β

1k are the cumulated instances of false

positive, true negative, true positive, and false negative, re-spectively, from acquaintance k. r0k,j ∈ {0, 1} is the jthdiagnosis result from acquaintance k under no-attack. r0k,j = 1means the diagnosis from k is positive while there is actuallyno attack happening. r0k,j = 0 means otherwise. Similarly,r1k,j ∈ {0, 1} is the jth diagnosis data from acquaintance kunder attack where r1k,0 = 1 means that the diagnosis fromk is positive under attack, and r1k,0 = 0 means otherwise.Parameters t0k,j and t1k,j denote the time elapsed since thejth feedback is received. λ ∈ [0, 1] is the forgetting factoron the past experience. A small λ makes old observationsquickly forgettable. We use exponential moving average toaccumulate past experience so that old experience takes lessweight than new experience. u is the total number of no-attackcases among the past records and v is the total number ofattack cases.

To make the parametric updates scalable to data storage andmemory, we can use the following recursive formula to updateα0k, α

1k and β0

k, β1k:

αmk (tj) = λ(tmk,j−tmk,j−1)αm

k (tmk,j−1) + rmk,j ;

βmk (tj) = λ(tmk,j−tmk,j−1)βm

k (tmk,j−1) + rmk,j , (4)

where l = 0, 1 and j− 1 indexes the previous data point usedfor updating αm

k or βmk . Through this way, only the previous

state and the current state are required to be recorded, whichis efficient in terms of storage compared to when all states arerecorded in Equation 3.

0

0.2

0.4

0.6

0.8

1

R[

]

E[P]

=0 =1Parameters:Cfp=1, Cfn=5

Fig. 2. Bayes risk for optimal decisions when Cfp = 1 and Cfn = 5.

B. Feedback Aggregation

When a HIDS detects suspicious activities and is not confi-dent about its decision, it sends out the description of the sus-picious activities or the related executable files to its collabora-tors for consultation. The node receives diagnosis results fromits collaborators, denoted by vector y = {y1,y2, ...,y|A|},where yi ∈ {0, 1}, for 0 < i < |A|, is the feedbackfrom acquaintance i. We use X ∈ {0, 1} to denote thescenario of “no-attack” or “under-attack”, and Y ∈ {0, 1}|A|

to denote all possible feedback from acquaintances. Theconditional probability of a HIDS being “under-attack” giventhe diagnosis results from all acquaintances can be written asP[X = 1|Y = y]. Using Bayes’ Theorem [24] and assumingthat the acquaintances provide diagnoses independently andtheir FP rate and TP rate are known, we have

P[X=1|Y=y]=P[Y=y|X=1]P[X=1]

P[Y=y|X=1]P[X=1]+P[Y=y|X=0]P[X=0]

=π1

∏|A|k=1 T

yk

k (1− Tk)1−yk

π1

∏|A|k=1 T

yk

k (1− Tk)1−yk + π0

∏|A|k=1 F

yk

k (1− Fk)1−yk

,

where π0 = P[X = 0] and π1 = P[X = 1], such that π0 +π1 = 1, are the prior probabilities of the scenarios of “no-attack” and “under-attack”, respectively. yk ∈ {0, 1} is thekth element of vector y.

Since Tk and Fk are both random variables with distribu-tions as in Equations (1) and (2), we can see that the condi-tional probability P[X = 1|Y = y] is also a random variable.We use a random variable P to denote P[X = 1|Y = y].Then P takes a continuous value over domain [0, 1]. We usefP (p) to denote the probability density function of P .

When α and β are sufficiently large, a Beta distributioncan be approximated by Gaussian distribution according to

Beta(α, β) ≈ N(

αα+β ,

√αβ

(α+β)2(α+β+1)

). Then the density

function of P can be also approximated using Gaussiandistribution. By Gauss’s approximation formula, we have,

E[P ] ≈ 1

1 +π0

∏|A|k=1 E[Fk]

yk (1−E[Fk])1−yk

π1∏|A|

k=1 E[Tk]yk (1−E[Tk])1−yk

=1

1 + π0

π1

∏|A|k=1

α1k+β1

k

α0k+β0

k

(α0

k

α1k

)yk(β0k

β1k

)1−yk

. (5)

Let Cfp and Cfn denote the marginal cost of a FP decisionand a FN decision. We assume there is no cost when a correct


decision is made. We use marginal cost because the cost of aFP may change in time depending on the current state. Cfn

largely depends on the potential damage level of the attack.For example, an intruder intending to track a user’s browsinghistory may have lower Cfn than an intruder intending tomodify a system file. We define a decision function δ(y) ∈{0, 1}, where δ = 1 means raising an alarm and δ = 0 meansno alarm. Then, the Bayes risk can be written as:

R(δ) =

∫ 1

0

(Cfp(1− x)δ + Cfnx(1− δ))fP (x)dx

= δCfp

∫ 1

0

(1 − p)fP (p)dp+ (1 − δ)Cfn

∫ 1

0

pfP (p)dp

=

∫ 1

0

CfnxfP (x)dx+ δ

(Cfp − (Cfp + Cfn)

∫ 1

0

xfP (x)dx

)

= CfnE[P ] + δ(Cfp − (Cfp + Cfn)E[P ]), (6)

where fP (p) is the density function of P . To minimize therisk R(δ), we need to minimize δ(Cfp − (Cfp +Cfn)E[P ]).Therefore, we raise an alarm (i.e. δ = 1) if

E[P ] ≥ Cfp

Cfp + Cfn. (7)

Let τ =Cfp

Cfp+Cfnbe the threshold. If E[P ] ≥ τ , we raise an

alarm, otherwise no alarm is raised. The corresponding Bayesrisk for the optimal decision is:

R(δ) =

⎧⎪⎨⎪⎩Cfp(1 − E[P ]) if E[P ] ≥ τ ,

CfnE[P ] otherwise.

(8)

An example of the Bayes risk for optimal decisions whenCfp = 1 and Cfn = 5 is illustrated in Figure 2.

IV. ACQUAINTANCE MANAGEMENT

Intuitively when a HIDS consults a larger number ofacquaintances, it can achieve higher detection accuracy andlower risk of being compromised. However, having moreacquaintances causes higher maintenance cost since the HIDSneeds to allocate resources for each node in its acquaintancelist. When a HIDS makes a decision about how many ac-quaintances to recruit, both the intrusion risk cost and themaintenance cost should be taken into account. When adding anode as an acquaintance does not lower the total cost, then thenode shall not be added into the acquaintance list. However,how to select acquaintances and how many acquaintancesto include are crucial to build an efficient CIDN. In thissection, we first define the acquaintance selection problem,then a corresponding solution is devised to find the optimalset of acquaintances. Finally, we propose an acquaintancemanagement algorithm for HIDSes to learn, recruit, update,or remove their acquaintances dynamically.

A. Problem Statement

Let Ai denote the set of acquaintances of HIDS i. LetMi(Ai) be the cost for HIDS i to maintain the acquaintanceset Ai. We use Ri(Ai) to denote the risk cost of missingintrusions and/or false alarms for HIDS i, given the feedback

of acquaintance set Ai. In the rest of this section, we dropsubscript i from our notations for the convenience of presen-tation.

Our goal is to select a set of acquaintances from a list ofcandidates so that the overall cost R(A)+M(A) is minimized.We define the problem as follows:

Given a list of acquaintance candidates C, we need to finda subset of acquaintances A ⊆ C, such that the overall costR(A) +M(A) is minimized.

In practice, maintenance cost of acquaintances maynot be negligible since acquaintances send test mes-sages/consultations periodically to ask for diagnosis. It takesresources (CPU and memory) for the HIDS to receive, analyzethe requests, and reply with corresponding answers. The selec-tion of Mi(.) can be user defined on each host. For example,a simple maximum acquaintance length restriction can bemapped to M(A) = Cmax(|A| − L, 0), where L ∈ N+

is the acquaintance length upper-bound and C ∈ [0,∞) is thepenalty of exceeding the bound.

The risk cost can be expressed as:

R(A) = CfnP [δ = 0|X = 1]P [X = 1]

+ CfpP [δ = 1|X = 0]P [X = 0]

where Cfn, Cfp denote the marginal cost of missing anintrusion and raising a false alarm, respectively. P [X = 1] =π1, P [X = 0] = π0 are the prior probabilities of under-attackand no-attack, where π0 + π1 = 1. Note that in practice π1

can be learned from the history and be updated whenever anew threat is found. A moving average method can be usedto update the estimated value.

The above equation can be further written as:

R(A) = Cfnπ1

∑∀y∈{0,1}|A||δ(y)=0

P [Y = y|X = 1] (9)

+ Cfpπ0

∑∀y∈{0,1}|A||δ(y)=1

P [Y = y|X = 0]

= Cfnπ1

∑∀y∈{0,1}|A||δ(y)=0

|A|∏i=1

(Ti)yi(1− Ti)

1−yi

+ Cfpπ0

∑∀y∈{0,1}|A||δ(y)=1

|A|∏i=1

(Fi)yi(1 − Fi)

1−yi

= Cfnπ1

∑∀y∈{0,1}|A||f(y)<1

|A|∏i=1

(Ti)yi(1− Ti)

1−yi

+ Cfpπ0

∑∀y∈{0,1}|A||f(y)≥1

|A|∏i=1

(Fi)yi(1− Fi)

1−yi

=∑

y∈{0,1}|A|min{Cfnπ1

∏i

T yi

i (1− Ti)1−yi ,

Cfpπ0

∏i

Fyi

i (1− Fi)1−yi}

where Ti, Fi are the TP rate and FP rate of acquaintance

i respectively. f(y) =Cfnπ1

∏|A|i=1 (Ti)

yi (1−Ti)1−yi

Cfpπ0

∏|A|i=1 (Fi)yi (1−Fi)1−yi

. ∀y ∈{0, 1}l|δ(y) = 1 refers to the combination of decisions whichcauses the system to raise an alarm and vice versa.


B. Acquaintance Selection Algorithm

To solve such a subset optimization problem, the bruteforce method is to examine all possible combinations ofacquaintances and select the one which has the least overallcost. However, the computation complexity is O(2n). It isnot hard to see that the order of selecting acquaintancesdoes not affect the overall cost. We propose an acquaintanceselection algorithm based on a heuristic approach to find anacquaintance set which achieves satisfactory overall cost. Inthis algorithm, the system always selects the nodes whichbring the lowest overall cost.

For the ease of demonstration, We assume the maintenancecost can be written as follow:

M(A) = Cal = Ca|A| (10)

where Ca is the unit maintenance cost of each acquain-tance, which includes the cost of communication, detectionassistance, and test messages. Note that any other form ofmaintenance cost can be easily included into the algorithm.

Algorithm 1 Acquaintance Selection (C, Lmin, Lmax)Require: A set of acquaintance candidates CEnsure: A set of selected acquaintances A with minimum

length Lmin and max length Lmax which brings theminimum overall cost

1: Quit = false //quit the loop if Quit = true2: A ⇐ ∅3: U = min(π0Cfp, π1Cfn) //initialize the overall cost

while there is no acquaintance. min(π0Cfp, π1Cfn) isthe cost when a node makes a decision without feedbackfrom collaborators

4: while Quit = false do5: //select the node that reduces cost most in each iteration6: Dmax = −MAXNUM //initialize the maximum cost

reduction to the lowest possible7: for all e ∈ C do8: A = A ∪ e9: if U − R(A) − M(A) > Dmax //see Equation (9)

and Equation (10) for R(A) and M(A) then10: Dmax = U −R(A)−M(A)11: emax = e12: end if13: A = A \ e //remove e from A14: end for15: if (Dmax > 0 and |A| < Lmax) or |A| < Lmin then16: A = A ∪ emax

17: C = C \ emax //remove emax from C18: U = U −Dmax

19: else20: Quit = true21: end if22: end while

As shown in Algorithm 1, in the beginning, the acquaintancelist is empty. The initial cost is the minimum cost of thedecision based only on the prior information (line 3). Foreach loop, the system selects a node from the acquaintancecandidate list which brings the lowest overall cost and stores

it into emax (lines 7-14), where U − R(A) − M(A) is theamount of cost reduced by adding a node into the acquaintancelist. When such a node is found, it is then moved to theacquaintance list if the current acquaintance length is less thanLmin or the cost is reduced by adding the new node and theacquaintance length does not exceed Lmax. The loop stopswhen no node can be added into A any further.

C. Acquaintance Management Algorithm

In the previous section, we devised an algorithm to selectacquaintances from a list of candidates. However, collabora-tion is usually based on mutual consensus. If node A selectsB as an acquaintance but B does not select A (non-symmetricselection), then the collaboration is not established.

Algorithm 2 Managing Acquaintance & Probation Lists1: Initialization :2: A ⇐ ∅ //Acquaintance list.3: P ⇐ ∅ //Probation list.4: lp = lini //initial Probation length5: //Fill P with randomly selected nodes6: while |P| < lp do7: e ⇐ select a random node8: P ⇐ P ∪ e9: end while

10: set new timer event(tu, “SpUpdate”)11: Periodic Maintenance:12: at timer event ev of type “SpUpdate” do13: //Merge the first mature node into the acquaintance list.14: e ⇐ selectOldestNode(P)15: C ⇐ A //C is the temporary candidate list16: if te > tp //te is the age of node e in the probation list

then17: P ⇐ P \ e18: if Te > Tmin and Fe < Fmax //Te and Fe are the true

positive rate and false positive rate of the node e then19: C ⇐ C ∪ e20: end if21: end if22: //Consensus protocol23: S =Acquaintance Selection(C, lmin,max(lmin, q

q+1 lmax))

24: //Send requests for collaboration and receive responses25: Saccp ⇐ RequestandReceiveCollaboration(S, ttimeout)26: A ⇐ Saccp //Only nodes that accept the collaboration

invitations are moved into the acquaintance list27: //Refill P with randomly selected nodes28: while |P| < max(q|A|, lmin) do29: e ⇐ Select a random node not in A30: P ⇐ P ∪ e31: end while32: set new timer event(tu, “SpUpdate”)33: end timer event

We propose a distributed approach for a HIDS in the CIDNto select and manage acquaintances and a consensus protocolto allow a HIDS to deal with the non-symmetric selectionproblem. To improve the stability of the acquaintance list,we propose to use a probation period on each new node for


the HIDS to learn about the new node before considering itas an acquaintance. For this purpose, each HIDS maintainsa probation list, where all new nodes remain during theirprobation periods. A node also communicates with nodes in itsprobation list periodically to evaluate their detection accuracy.The purpose of the probation list is thus to explore potentialcollaborators and keep introducing new qualified nodes to theacquaintance list.

Suppose that node i has two sets Ai and Pi, which arethe acquaintance list and the probation list respectively. Thecorresponding false positive rate and true positive rate of bothsets are FA

i , TAi and FP

i , TPi . To keep learning the detection

accuracy of the acquaintances, a node sends test messagesto nodes in both the acquaintance list and the probation listperiodically, and keeps updating their estimated false positiverates and true positive rates. Let lmax be the maximum numberof HIDSes in both the acquaintance and the probation list. Weset this upper-bound because the amount of resources used forcollaboration is proportional to the number of acquaintances itmanages. lmax is determined by the resource capacity of eachHIDS. Let lmin be the minimum length of a probation list andq be the parameter that controls the length of the probationlist lp compared to the length of acquaintance list la, such thatlmin ≤ lp ≤ qla. The parameters lmin and q are used to tunethe trade-off between the adaptability to the situation wherenodes join or leave the network frequently (“high churn rate”),and the overhead of resources used on testing new nodes.

The acquaintance management procedure for each node isshown in Algorithm 2. The acquaintance list A is initiallyempty and the probation list P is filled by lini randomnodes to utilize the resources in exploring new nodes. Anacquaintance list updating event is triggered every tu timeunits. A is updated by including new trusted nodes fromP . A node that stays at least tp time units in probationis called a mature node. Only mature nodes are allowedto join the acquaintance list (lines 15-21). Mature nodeswith bad qualification will be abandoned right away. Afterthat the acquaintance selection algorithm is used to find theoptimal candidate list. Collaboration requests are sent out fornodes which are selected in the optimal list. If an acceptanceis received before expiration time then the collaboration isconfirmed, otherwise the node is abandoned (lines 22-26).Then, P is refilled with new randomly chosen nodes (lines28-31).

Several properties are desirable for an effective acquain-tance management algorithm, including convergence, stabil-ity, robustness, and incentive-compatibility for collaboration.When our acquaintance management is in place, we areinterested to know with whom the HIDS nodes end up collab-orating with and how often they change their collaborators. Welist potential malicious attacks against our acquaintance man-agement system and present corresponding defense strategiesin Section V. We also expect to see cooperative nodes arerewarded and dishonest nodes penalized.

In Section VI we evaluate our acquaintance managementalgorithm, to determine whether it achieves the above proper-ties.

V. COMMON THREATS AND DEFENSE

Efficient collaborative detection can improve detection abil-ity of the whole group. However, the collaboration man-agement itself may become the target of attacks and becompromised. For example, an HIDS may be compromisedand turn to be a malicious insider, and then disseminate falseinformation to other HIDSes in the network. An effectiveCIDN design should be robust to common insider attacks.In this section, we describe common attacks to CIDN andprovide defense mechanisms against them.

1) Sybil attacks: occur when a malicious peer in the systemcreates a large amount of pseudonyms (fake identities) [25].This malicious peer uses fake identities to gain larger influenceover the false diagnosis in the network. For example, fakenodes can gain the trust of some victims and then send themfalse diagnoses to cause false negative decisions. Our defenseagainst sybil attacks relies on an authentication mechanism.Authentication makes registering fake identities difficult. Inour model, the registration of new user IDs requires puzzlesolving (such as CAPTCHA) which requires human intel-ligence to handle. In this way, creating a large number offake IDs is not practical for an attacker. In addition, ourcollaboration model requires HIDSes to allocate resourcesto answer diagnosis requests from their collaborators beforebeing able to influence the collaborators, which is costly to dowith many fake identities. This way, our CIDN protects thecollaborative network from sybil attacks.

2) Newcomer attacks: occur when a malicious peer caneasily register as a new user [26]. Such a malicious peercreates a new ID for the purpose of erasing its bad historywith other peers in the network. Our model handles this typeof attack by requiring probation period for all newcomers. Ittakes a long time (for example, more than 10 days) for a newnode to gain trust and before they pass probation period, theirfeedback is simply not considered by other peers. In this way,it takes a long time for a node to gain influence again byregistering a new ID. Therefore, our system resists newcomerattacks.

3) Betrayal attacks: occur when a trusted peer suddenlyturns into a malicious one and starts sending false diagnoses. Acollaborative detection can be degraded dramatically becauseof this type of attacks. Our adoption of forgetting factor (Equa-tion 3) enables a faster learning of the malicious behavior (seeSection VI-E Figure 14 for evaluation results). Our dynamicacquaintance selection algorithm then quickly removes themalicious node from acquaintance list. It will take quite awhile for the malicious node to gain back trust because evenit gets the chance to join the probation list again, it needs tostay in the probation list for a certain period of time. Thisdesign makes it difficult for this malicious node to continuedeceiving or to come back as a collaborator within a shortperiod of time.

4) Collusion attacks: happen when a group of maliciouspeers cooperate together by providing false diagnosis in orderto compromise the network. First, our acquaintance selectionalgorithm employs random selection of potential collaborators.It will be unlikely for multiple malicious peers to be selectedtogether by one peer as collaborators. In addition, in oursystem, peers will not be adversely affected by collusion


attacks. This is because, in our learning model, each peerrelies only on its own experience to detect dishonest peers. Notrelying on recommendation or reputation makes the systemimmune to dishonest third party opinions. This is primarily thereason why we do not consider third party recommendationsin the current work. We leave this for future investigationafter devising an effective approach for coping with collusionattacks (see Section VIII).

VI. EVALUATION

In this section, we describe the conducted experimentsto demonstrate the desirable properties of our acquaintancemanagement algorithm. We evaluate the cost efficiency of ourBayesian decision model, cost and time efficiency of the ac-quaintance selection algorithm, and several desired propertiesof the acquaintance management algorithm. Each experimentalresult presented in this section is derived from the averageof a large number of replications with an overall negligibleconfidence interval.

TABLE IISIMULATION PARAMETERS

Parameter Value DescriptionR 10/day Test message rateλ 0.95 Forgetting factor

Cfp/Cfn 20/100 Unit cost of false positive/negative decisionsCa 0.01 Maintenance cost of one acquaintancetp 10 days Probation periodtu 1 day Acquaintance list update intervallini 10 Initial probation lengthlmax 20 Maximum total number of acquaintanceslmin 2 Minimum probation list lengthTmin 0.5 Minimum acceptable true positive rateFmax 0.2 Maximum acceptable false positive rate

q 0.5 Length ratio of probation to acquaintance listπ1 0.1 Prior probability of intrusions

A. Simulation Setting

We simulate an environment of n HIDS peers collaboratingtogether by adding each other as acquaintances. We adopt twoparameters to model the detection accuracy of each HIDS,namely, false positive rate (FP) and false negative rate (FN).Notice that in reality most HIDSes have low FP (< 0.1)and FN is normally in the range of [0.1, 0.5] [13]. This isbecause false positives can severely damage the reputationof the product, so vendors strive to control their FP rate ata low level. In our experiment, we select parameters whichreflect real world properties. To test the detection accuracyof acquaintances, each peer sends test messages where theircorrect answers are known beforehand. Test messages are sentfollowing a Poisson process with average arrival rate R. R willbe determined in the next subsection. We use a simulationday as the time unit in our experiments. The diagnosis resultsgiven by a HIDS are simulated following a Bernoulli randomprocess. If a test message represents a benign activity, theHIDS i raises alarm with a probability of FPi. Similarly, if thetest message represents intrusions, an alarm will be raised witha probability of 1-FNi. All parameter settings are summarizedin Table II.

B. Determining the Test Message Rate

The goal of our first experiment is to study the relationshipbetween test message rates and FP, FN learning speed. Wesimulate two HIDSes A and B. A sends B test messages toask for diagnosis, and learns the FP and FN of B based onthe quality of B’s feedback. The learning procedure followsEquations (1), (2), and (3). We fix the FN of B to 0.1, 0.2,and 0.3 respectively. Under each case, we run the learningprocess under different test message rates, 2/day, 10/day, and50/day respectively. We observe the change of estimated FNover time, plotted in Figure 3. We see that when R is 2/day,the estimated FN converges after around 30 days in thecase of FN=0.2. The converging time is slightly longer andshorter in the cases of FN=0.3 and FN=0.1, respectively. WhenR is increased to 10/day, the converging time decreases toaround 10 days. In the case of R=50/day, the correspondingconverging time is the shortest (around 3 days) among thethree cases. Increasing the test message rate R to 50/day doesnot reduce much learning process time. Based on the aboveobservation, we choose R=10/day and the probation period tpto be 10 days as our system parameters. In this way, the testmessage rate is kept low and the learned FN and FP valuesconverge after the probation period.

The second experiment is to study the efficiency of learningresults after our chosen probation period. We fix R=10/day,tp=10/day, and randomly choose FN of node B uniformlyamong [0, 1]. We repeat the experiments 100 times withdifferent FNs. The FNs estimated using our learning processtill the end of probation period are plotted in Figure 4. We cansee that in all different settings of FNs, the estimated FN ratesare close to the actual FN rates after the probation period.

C. Efficiency of our Feedback Aggregation

In this experiment, we evaluate the effectiveness of ourBayesian decision based feedback aggregation by comparingit with a threshold based aggregation. We have describedour Bayesian decision model in Section III-B. In a simplethreshold based feedback aggregation method, if the numberof HIDSes reporting intrusions is larger than a predefinedthreshold, then the system raises an alarm. The threshold-based decision is used in N-version cloud anti-virus systems[13].

We set up eight HIDSes {HIDS0,HIDS1, ...,HIDS7} withtheir FP and FN rates randomly chosen from the range [0.1,0.5]. HIDS0 sends consultations to all other HIDSes, collectsand aggregates feedback to make intrusion decisions. Thecosts of false positive and false negative decisions are Cfp=20and Cfn=100 respectively. We compare the average falsedetection cost using the Bayesian decision model and thesimple threshold-based approach. Figure 5 shows that the costof threshold decision largely depends on the chosen thresholdvalue. An appropriate threshold can significantly decrease thecost of false decisions. In contrast, the Bayesian decisionmodel does not depend on any threshold setting and prevailsover the threshold decision under all threshold settings. This isbecause the threshold decision treats all participants equally,while the Bayesian decision method recognizes different de-tection capabilities of HIDSes and takes them into account


0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0 10 20 30 40 50

Est

imat

ed F

N R

ate

Days

FN=0.1

FN=0.2

FN=0.3

R=2/dayR=10/dayR=50/day

Fig. 3. The convergence of learning speed andthe test message rate.

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Est

imat

ed F

N R

ate

Actual FN Rate

Fig. 4. The distribution of estimated FN rate(R=10/day).

0

2

4

6

8

10

1 2 3 4 5 6 7 8

Cos

t of F

alse

Dec

isio

ns

Threshold

BayesianThreshold

Fig. 5. Comparison of cost using thresholddecision and Bayesian decision.

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

0 2 4 6 8 10 12 14

Ove

rall

Cos

t

Number of Collaborators

FN=0.1FN=0.2FN=0.3FN=0.4optimal

Fig. 6. The average cost under differentcollaborator quality.

0

1

2

3

4

5

6

2 4 6 8 10 12 14

Cos

t

Length of Candidates

GreedyBrute Force

Fig. 7. The cost using different acquaintanceselection algorithms.

0

100

200

300

400

500

600

2 4 6 8 10 12 14

Run

ning

tim

e (m

illis

econ

ds)

Length of Candidates

GreedyBrute Force

Fig. 8. The running time using differentacquaintance selection algorithms.

in the decision process. For example, if a HIDS asserts thatthere is intrusion, our Bayesian model may raise an alarmif the HIDS has a low FP rate and ignores the warning ifthe HIDS has a high FP rate. However, the threshold baseddecision model will either raise an alarm or not based on thetotal number of HIDSes which raise warnings and compare itwith a predefined threshold, irrespective of the individual thatissued the warning.

D. Cost and the Number of Collaborators

We define risk cost to be the expected cost from falsedecisions such as raising false alarms (FP) and missing thedetection of an intrusion (FN). We show that introducing morecollaborators can decrease the risk cost. In this experiment,we study the impact of the number of collaborators on therisk cost. We set up four groups with an equal number ofHIDSes. Nodes in all groups have the same FP rate of 0.03,but their FN rates vary from 0.1 to 0.4, depending on thegroup they are in. Inside each group every node collaborateswith every other node. We are interested in the risk cost aswell as the maintenance cost. The maintenance cost is thecost associated with the amount of resource that is used tomaintain the collaboration with other nodes, such as answeringdiagnosis requests from other HIDSes. Since our purpose isto capture the concept of maintenance cost but not to studyhow much it is, we assume the maintenance cost to be linearlyproportional to the number of collaborators with a unit rateCa=0.01 (see Table II).

We increase the size of all groups and observe the averagecost of nodes in each group. From Figure 6, we can see that

in all groups, the costs drop down fast in the beginning andslow down as the groups’ sizes increase. After an optimal point(marked by large solid circles), the costs slowly increase. Thisis because when the number of collaborators is large enough,the cost saving by adding more collaborators becomes small,and the increment of maintenance cost becomes significant.We find that groups with higher detection accuracy havelower optimal costs. Also they need a smaller number ofcollaborators to reach the optimal costs. For example, in thecase of FN = 0.4, 13 collaborators are needed to reach theoptimal, while the number of collaborators required is 5 in thecase of FN = 0.1.

E. Efficiency of Acquaintance Selection Algorithms

We learned in the previous section that when the number ofcollaborators is large enough, adding more collaborators doesnot decrease the overall cost because of the associated mainte-nance cost. An acquaintance selection algorithm is proposedin Algorithm 1. In this section, we compare the efficiencyof acquaintance selection using the brute force algorithm andour acquaintance selection algorithm. We create 15 HIDSesas candidate acquaintances with FP and FN rates randomlychosen from intervals [0.01, 0.1] and [0.1, 0.5], respectively.Both algorithms are implemented in Java and run on a PCwith AMD Athlon dual core processor 2.61GHZ, and with1.93 GB RAM. We start the candidate set size from 1 andgradually increase the size. We observe the cost efficiencyand running time efficiency of both algorithms.

Figure 7 shows that the brute force algorithm performsslightly better with respect to acquaintance list quality since


0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

FN

Rat

e of

Acq

uain

tanc

es

FN Rate

Fig. 9. Acquaintances distribution on day 25.

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

FN

Rat

e of

Acq

uain

tanc

es

FN Rate

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

FN

Rat

e of

Acq

uain

tanc

es

FN Rate

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

FN

Rat

e of

Acq

uain

tanc

es

FN Rate

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

FN

Rat

e of

Acq

uain

tanc

es

FN Rate

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

FN

Rat

e of

Acq

uain

tanc

es

FN Rate

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

FN

Rat

e of

Acq

uain

tanc

es

FN Rate

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

FN

Rat

e of

Acq

uain

tanc

es

FN Rate

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

FN

Rat

e of

Acq

uain

tanc

es

FN Rate

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

FN

Rat

e of

Acq

uain

tanc

es

FN Rate

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

FN

Rat

e of

Acq

uain

tanc

es

FN Rate

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

FN

Rat

e of

Acq

uain

tanc

es

FN Rate

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

FN

Rat

e of

Acq

uain

tanc

es

FN Rate

Fig. 10. Acquaintances distribution on day200.

0

1

2

3

4

5

6

7

0 50 100 150 200 250 300 350

Cos

t

Days

FN=0.1FN=0.3FN=0.5

Fig. 11. The average cost for collaboration.

0

20

40

60

80

100

120

140

160

180

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

Col

labo

ratio

n P

erio

d (d

ay)

FN Rate of Acquaintances

FN=0.1FN=0.3FN=0.5

Fig. 12. The collaboration time span.

0

1

2

3

4

5

6

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

Cos

t

FN Rate of Nodes

No CollabortionFixed Collaborators

Dynamic Collaborators

Fig. 13. The converged cost distribution.

0

0.2

0.4

0.6

0.8

1

0 20 40 60 80 100

Fal

se D

iagn

osis

Pro

babi

lity

Days

FP FN

Fig. 14. The FP and FN of betrayal node.

the overall cost using its selected list is slightly lower. How-ever, Figure 8 shows that the running time of the brute forcemethod increases significantly when the candidate set sizeexceeds 11, and continues to increase exponentially, while ouralgorithm shows much better running time efficiency. Theseexperiments suggest to use the brute force method only whenthe size of candidates list is small (≤ 11). When the candidateslist is large, our greedy algorithm should be used to selectacquaintances.

F. Evaluation of Acquaintance Management Algorithm

In this experiment, we study the effectiveness of our ac-quaintance management algorithm (Algorithm 2). We set upa simulation environment of 100 nodes. For the convenienceof observation, all nodes have fixed FP rate 0.1 and their FNrates are uniformly distributed in the range of [0.1, 0.5]. Allnodes update their acquaintance list once a day (tu=1). Weobserve several properties: convergence, stability, robustness,and incentive-compatibility.

1) Convergence: Our first finding about our acquaintancemanagement algorithm is that HIDSes converge to collaborat-ing with other HIDSes with similar detection accuracy levels.We observed through experiments that HIDSes collaboratewith random other nodes in the network in the beginning(Figure 9). After a longer period of time (200 days), all HID-Ses collaborate with others with similar detection accuracy, asshown in Figure 10. Our explanation is that the collaborationbetween pairs with high qualification discrepancy is relativelynot stable since our collaboration algorithm is based on mutualconsensus and consensus is hard to reach between those pairs.

Figure 11 plots the average overall cost in the first 365 daysof collaboration for three nodes with FN values 0.1, 0.3, and0.5 respectively. In the first 10 days, the costs for all nodesare high. This is because all collaborators are still in probationperiod. After day 10, all cost values drop down significantly.This is because collaborators pass probation period and startto contribute to intrusion decisions. The cost for high expertisenodes continues to drop while the cost for low expertise nodesincreases partially after around day 20, and stabilizes after day50. This is because the acquaintance management algorithmselects better collaborators to replace the initial random ones.We can see that the collaboration cost of nodes converges withtime and becomes stable after the initial phase.

2) Stability: Collaboration stability is an important prop-erty since the collaboration between HIDSes is expected to belong term. Frequently changing collaborators is costly becauseHIDSes need to spend considerable amount of time to learnabout new collaborators. In this experiment, we record theaverage time span of all acquaintances from the time theypass probation period till they are replaced by other acquain-tances. The result is shown in Figure 12, where the averagecollaboration time spans for three selected nodes are shownwith different point shapes. We can see that collaborationamong nodes with similar expertise levels is more stablethan that between nodes with different expertise levels. Forexample, nodes with low FN = 0.1 form stable collaborationconnections with other nodes with low FN (around 180 daysin average), while the collaboration with HIDSes with highFN is short (close to 0 day in average).

3) Incentive-compatibility: Collaboration among HIDSes isexpected to be a long term relationship. Incentive is important


for the long term sustainability of collaborations since it pro-vides motivation for peers to contribute [27], [28]. We comparethe average overall cost of all nodes with different FN ratesunder three different conditions, namely, no collaboration,fixed acquaintances collaboration (acquaintance length=8),and dynamic acquaintance management collaboration. Figure13 shows the distribution of the converged cost of all nodes.We can observe that the cost of all HIDSes is much higherwhen no collaboration is performed in the network. On theother hand, collaborating with random fixed acquaintancescan significantly reduce the cost of false decisions, however,the cost of high expertise nodes and low expertise nodes arevery close. With our dynamic acquaintance management, highexpertise nodes achieve much lower cost than nodes with lowexpertise, which reflects an incentive design of the collabo-ration system. Therefore, the system provides motivation fornodes to update their knowledge base and behave truthfullyin cooperation.

4) Robustness: Robustness is a desired property of a CIDNsince malicious users may try to attack the collaboration mech-anism to render it ineffective. Several strategies can be adoptedas we discussed in Section V. We focus on the Betrayal attackin this experiment. To study the impact from one maliciousnode, we set up a collaboration scenario where HIDS0 iscollaborating with a group of other HIDSes with FP = 0.1and FN = 0.2. Among the group, one HIDS turns to bedishonest after day 50 and gives false diagnoses. We observethe FP rate and FN rate of this malicious node perceived byHIDS0, and the impact on the risk cost of HIDS0 under variouscollaborator group sizes. Figure 14 shows the perceived FPand FN rate of the malicious node during each simulationday. We can see that the perceived FP and FN increase fastafter day 50. The malicious node is then removed from theacquaintance list of HIDS0 when its perceived FP and FN arehigher than a predefined threshold. The cost of HIDS0 underbetrayal attack is depicted by Figure 15; we notice that thebetrayal behavior introduces a spike of cost increment underall group sizes, but the magnitude of increment decreases whenthe number of collaborators increases. However, the systemcan efficiently learn the malicious behavior and recover tonormal by excluding malicious nodes from the acquaintancelist.

VII. RELATED WORK

Existing CIDNs can be divided into information-basedCIDNs and consultation-based CIDNs. In information-basedCIDNs, intrusion information such as intrusion alerts, intru-sion traffic samples, firewall logs, and system logs are sharedin the network and aggregated to achieve better network-wideintrusion decisions. Several information-based CIDNs, suchas [29], [11], [15], and [30], have been proposed in the pastfew years. They are especially effective in detecting epidemicworms or attacks, and most of them require homogeneousparticipant IDSes. In contrast, in consultation-based CIDNs,suspicious file samples or traffic samples are sent to col-laborators for diagnosis. Diagnosis results (feedback) fromcollaborators are then aggregated to help the sender IDS makeintrusion decisions. Examples of such CIDNs include [19],[20], [16], [31], and [13]. Consultation-based CIDNs may

0

1

2

3

4

5

30 40 50 60 70 80

Cos

t

Days

acqlen=4acqlen=6acqlen=8

Fig. 15. The cost of a HIDS under a betrayal attack.

involve heterogeneous IDSes and are effective in detectingmany intrusion types including worms, malware, port scan-ning, and vulnerability explorations. The CIDN proposed inthis paper is a consultation-based CIDN. With respect to theCIDN collaboration topology, many proposals are centralized,such as [29], [12], and [11]. A centralized CIDN requiresa continuously available central server and it suffers fromthe single point of failure problem. A decentralized CIDNsuch as [30], [32] can alleviate the workload of the centralserver by means of clustering where the cluster heads partiallyprocess and summarize the data they collect, and then forwardit to higher level nodes for processing. In a fully distributedCIDN [33], [16], [15], [34], all IDSes are equally responsiblefor collecting/processing information and therefore is the mostscalable and flexible topology. The CIDN proposed in thispaper is a fully distributed overlay network.

Various approaches have been proposed to evaluate HIDSes.All use a single trust value to measure whether a HIDS willprovide good feedback about intrusions based on past experi-ence with this HIDS. For example, Duma et al. [31] introducea trust-aware collaboration engine for correlating intrusionalerts. Their trust management scheme uses each peer’s pastexperience to predict others’ trustworthiness. Our previouswork [19], [20] uses Dirichlet distributions to model peer trust,but it does not investigate the conditional detection accuracysuch as false positives and false negatives. In this work, weuse both false positive and true positive rates to represent thedetection accuracy of a HIDS based on a Bayesian learningapproach. The methods for aggregating feedback provided byDuma et al. [31] and our previous work [19], [20] are alsosimplistic. They both use a weighted average approach to ag-gregate feedback. Another broadly accepted decision model inCIDNs is the threshold-based, which is used in AVCloud [13].In this model, when the total number of collaborators raisingalarms exceeds a fixed threshold, an alarm will be raised. Inthis paper, we apply the well established Bayes’ theorem forfeedback aggregation which achieves better performance. Ourprevious work [19], [20] focuses on the trust evaluation witha simple threshold-based acquaintance selection. This workfocuses on the optimal collaboration decision and optimalacquaintance selection.

Most previous approaches set a fixed length of the acquain-tance list, such as in [17]. Others use a trust threshold tofilter out less honest acquaintances [18], [19]. The advantageof the threshold based decision is its simplicity and ease


of implementation. However, it is only effective in a staticenvironment where collaborators do not change, such as thatpresented in [13]. In a dynamic environment, nodes join andleave the network and the acquaintance list changes withtime. Therefore, finding an optimal threshold is a difficulttask. Our Bayesian decision model is efficient and flexible.It can be used in both static and dynamic collaborationenvironments. Equipped with this Bayesian decision model,our acquaintance selection algorithm can find the smallestset of best acquaintances that can maximize the accuracyof intrusion detection. Based on this acquaintance selectionalgorithm, our acquaintance management method uses a pro-bation list to explore potential candidates for acquaintancesand balances the cost of exploration and the speed of updatingthe acquaintance list.

VIII. CONCLUSION AND FUTURE WORK

We proposed a statistical model to evaluate the tradeoff be-tween the maintenance cost and intrusion cost, and an effectiveacquaintance management method to minimize the overall costfor each HIDS in a CIDN. Specifically, we adopted a Bayesianlearning approach to evaluate the accuracy of each HIDS interms of its false positive and true positive rates in detectingintrusions. The Bayes’ theorem is applied for the aggregationof feedback provided by the collaborating HIDSes. Our ac-quaintance management explores a list of candidate HIDSesand selects acquaintances using an acquaintance selectionalgorithm. This algorithm is based on a greedy approach tofind the smallest number of best acquaintances and minimizethe cost of false intrusion decisions and maintenance. Theacquaintances list is updated periodically by introducing newcandidates which pass the probation period.

Through a simulated CIDN environment, we evaluatedour Bayesian decision model against threshold-based decisionmodels, and acquaintance selection algorithm against a bruteforce approach. Compared to the threshold-based model, ourBayesian decision model performs better in terms of cost offalse decisions. Compared to the brute force approach, ouralgorithm achieves similar performance but requires muchless computation time. Our acquaintance management is alsoshown to achieve the desirable properties of convergence,stability, robustness, and incentive-compatibility.

As future work, we intend to seek a theoretical methodto determine the optimal length of the acquaintance listfor a HIDS based on the detection accuracy of a set ofcandidate acquaintances. We will also investigate other moresophisticated attack models on the collaboration mechanismand integrate corresponding defense techniques. Robustness ofthe acquaintance management system is particularly critical ifextended to support HIDS peer recommendations. In this case,malicious HIDSes may provide untruthful recommendationsabout other HIDSes [35], [18], [36], or worse collide tocollaboratively bring the system down.

ACKNOWLEDGMENT

This work was supported in part by the Natural Science andEngineering Council of Canada (NSERC) under its Discoveryprogram, and in part by the WCU (World Class University)

program through the Korea Science and Engineering Foun-dation funded by the Ministry of Education, Science andTechnology (Project No. R31-2008-000-10100-0).

REFERENCES

[1] “ZDnet,” http://www.zdnet.com/blog/security/confickers-estimated-economic-cost-91-billion/3207 (last accessed Oct. 1, 20110.

[2] J. Mirkovic and P. Reiher, “A taxonomy of DDoS attack and DDoSdefense mechanisms,” SIGCOMM Comput. Commun. Rev., vol. 34,no. 2, pp. 39–53, 2004.

[3] T. Holz, C. Gorecki, K. Rieck, and F. Freiling, “Detection and mitigationof fast-flux service networks,” in Proc. 2008 Annual Network andDistributed System Security Symposium.

[4] “Snort Intrusion Detection System,” http://www.snort.org/ (last accessedOct. 1, 2011).

[5] “Bro Intrusion Detection System,” http://www.bro-ids.org/ (last accessedOct. 1, 2011).

[6] “OSSEC,” http://www.ossec.net (last accessed Oct. 1, 2011).[7] “TripWire,” http://www.tripwire.org/ (last accessed Oct. 1, 2011).[8] A. Kind, M. P. Stoecklin, and X. Dimitropoulos, “Histogram-based traf-

fic anomaly detection,” IEEE Trans. Network and Service Management,vol. 6, no. 2, pp. 110–121, 2009.

[9] N. Samaan and A. Karmouch, “Network anomaly diagnosis via statisti-cal analysis and evidential reasoning,” IEEE Trans. Network and ServiceManagement, vol. 5, no. 2, pp. 65–77, 2008.

[10] P. Li, M. Salour, and X. Su, “A survey of Internet worm detectionand containment,” IEEE Commun. Surveys & Tutorials, vol. 10, no. 1,pp. 20–35, 2008.

[11] J. Ullrich, “DShield,” http://www.dshield.org/indexd.html (last accessedOct. 1, 2011).

[12] F. Cuppens and A. Miege, “Alert correlation in a cooperative intrusiondetection framework,” in 2002 IEEE Symposium on Security and Pri-vacy.

[13] J. Oberheide, E. Cooke, and F. Jahanian, “Cloudav: N-version antivirusin the network cloud,” in Proc. 2008 USENIX Security Symposium.

[14] R. Janakiraman and M. Zhang, “Indra: a peer-to-peer approach tonetwork intrusion detection and prevention,” in Proc. 2003 IEEE In-ternational Workshops on Enabling Technologies.

[15] M. Cai, K. Hwang, Y. Kwok, S. Song, and Y. Chen, “CollaborativeInternet worm containment,” IEEE Security & Privacy, vol. 3, no. 3,pp. 25–33, 2005.

[16] C. Fung, O. Baysal, J. Zhang, I. Aib, and R. Boutaba, “Trust man-agement for host-based collaborative intrusion detection,” in 2008IFIP/IEEE International Workshop on Distributed Systems.

[17] B. Yu and M. Singh, “Detecting deception in reputation management,”in Proc. 2003 International Joint Conference on Autonomous Agentsand Multiagent Systems.

[18] J. Zhang and R. Cohen, “Evaluating the trustworthiness of advice aboutselling agents in e-marketplaces: a personalized approach,” Electron.Commerce Research and Applications, vol. 7, no. 3, pp. 330–340, 2008.

[19] C. Fung, J. Zhang, I. Aib, and R. Boutaba, “Robust and scalabletrust management for collaborative intrusion detection,” in Proc2009IFIP/IEEE International Symposium on Integrated Network Manage-ment.

[20] C. Fung, J. Zhang, I. Aib, and R. Boutaba, “Dirichlet-based trustmanagement for effective collaborative intrusion detection networks,”IEEE Trans. Network and Service Management, vol. 8, no. 2, pp. 79–91, 2011.

[21] E. Staab, V. Fusenig, and T. Engel, “Towards trust-based acquisitionof unverifiable information,” in Proc. 2008 International Workshop onCooperative Information Agents XII.

[22] A. Gelman, Bayesian Data Analysis. CRC Press, 2004.[23] A. Jøsang and R. Ismail, “The beta reputation system,” in Proc. 2002

Bled Electronic Commerce Conference.[24] S. Russell, P. Norvig, J. Canny, J. Malik, and D. Edwards, Artificial

Intelligence: A Modern Approach, vol. 74. Prentice Hall, 1995.[25] J. Douceur, “The Sybil attack,” 2002 Peer-To-Peer Systems: First

International Workshop.[26] P. Resnick, K. Kuwabara, R. Zeckhauser, and E. Friedman, “Reputation

systems,” Commun. ACM, vol. 43, no. 12, pp. 45–48, 2000.[27] E. Fehr and H. Gintis, “Human motivation and social cooperation:

experimental and analytical foundations,” Annu. Rev. Sociol., vol. 33,pp. 43–64, 2007.

[28] A. Dal Forno and U. Merlone, “Incentives and individual motivation insupervised work groups,” European J. Operational Research, vol. 207,no. 2, pp. 878–885, 2010.


[29] “SANS Internet Storm Center (ISC),” http://isc.sans.org/ (last accessedOct. 2011).

[30] V. Yegneswaran, P. Barford, and S. Jha, “Global intrusion detection inthe DOMINO overlay system,” in Proc. 2004 Network and DistributedSystem Security Symposium.

[31] C. Duma, M. Karresand, N. Shahmehri, and G. Caronni, “A trust-aware,P2P-based overlay for intrusion detection,” in 2006 DEXA Workshops.

[32] Z. Li, Y. Chen, and A. Beach, “Towards scalable and robust distributedintrusion alert fusion with good load balancing,” in Proc. 2006 SIG-COMM Workshop on Large-Scale Attack Defense.

[33] C. Zhou, S. Karunasekera, and C. Leckie, “A peer-to-peer collaborativeintrusion detection system,” in Proc. 2005 IEEE International Confer-ence on Networks.

[34] Z. Zhong, L. Ramaswamy, and K. Li, “ALPACAS: a large-scale privacy-aware collaborative anti-spam system,” in 2008 Conference on ComputerCommunications.

[35] P. B. Velloso, R. P. Laufer, D. de O. Cunha, O. C. M. B. Duarte,and G. Pujolle, “Trust management in mobile ad hoc networks usinga scalable maturity-based model,” IEEE Trans. Network and ServiceManagement, vol. 7, no. 3, pp. 172–185, 2010.

[36] L. Mekouar, Y. Iraqi, and R. Boutaba, “A recommender scheme forpeer-to-peer systems,” in 2008 IEEE International Symposium on Ap-plications and the Internet.

Carol Fung is a PhD student in University of Wa-terloo (Canada). She received her BSc and Msc fromUniversity of Manitoba, Canada. Her research topicis collaborative Intrusion Detection networks, whichincludes trust management, collaborative decision,resource management, and incentive design of sucha system. She is also interested in the securityproblems in wireless networks and social networks.

Jie Zhang joined Nanyang Technological Univer-sity, Singapore as an assistant professor, working ontrust modeling and incentive mechanism design. Hegot Ph.D. in the School of Computer Science at theUniversity of Waterloo, and was the recipient of theAlumni Gold Medal, awarded to honour the top PhDgraduate.

Raouf Boutaba Raouf Boutaba received the MScand PhD degrees in computer science from theUniversity Pierre & Marie Curie, Paris, in 1990 and1994, respectively. He is currently a professor ofcomputer science at the University of Waterloo anda Distinguished Visiting Professor at the PohangUniversity of Science and Technology (POSTECH),Korea. His research interests include network, re-source and service management in wired, and wire-less networks. He has received several best paperawards and other recognitions such as the Premiers

Research Excellence Award, the IEEE Hal Sobol Award in 2007, the Fred W.Ellersick Prize in 2008, the Joe LociCero award and the Dan Stokesbury in2009.

320 ieee transactions on network and …cfung/research/fungtnsm12.pdf320 ieee transactions on...

Documents