SCAP Based Analytics for Enterprise SecurityPolicy Verification and Risk Assessment

Mohammed Noraden Alsaleh and Ehab Al-ShaerDepartment of Software and Information Systems

University of North Carolina at CharlotteCharlotte, NC, USA

Email: {malsaleh, ealshaer}@uncc.edu

Abstract—Information systems include many heterogeneous,yet interdependent, hardware and software componenents. Thesecurity of the overall systems are directly dependent on theproper configuration of all the componenets. To verify thesecurity requirements of information systems, standards andautomated tools are required to scan and report the compli-ance of distributed devices. However, hundreds or thousands ofcompliance reports can be generated frequently to check thecompliance of the end hosts. There are no tools yet that canaggregate and analyze the collected compliance reports usinglogical and statistical analysis in a way that makes sense fordecision makers. It is also still unclear how to assess the riskand impact on the enterprise as a whole based on the compliancereports, from both mission and business aspects. Moreover, theconnectivity of hosts may amplify or eleminate the risk imposedby their vulnerabilities. Hence, a comprehensive analysis mustconsider both the network configuration and hosts’ compliancereports.

In this paper, we present two types of security configurationanalytics: top-down analytics that enable defining, verifying andenforcing security policies by correlating network-wide XCCDFreports and network configurations, and bottom-up analyticsthat enable measuring global enterprise risk based on XCCDFreports, their inter-dependencies, and network configurations.The risk scores help the administrators to prioritize mitigationactions. We take advantage of Security Content AutomationProtocol (SCAP) languages and scoring systems along with theconfiguration to study vulnerabilities and compute the systemexposure.

I. INTRODUCTION

Computer networks are broadly used in many areas, includ-ing social, economical, medical, and other human activities.Although the network infrastructure (i.e. network core devices)are of a great importance to ensure proper network operation,the network services and assets are hosted on the computerhosts. The hosts also constitute the network interface with itsusers. Hence, hosts’ proper configuration and compliance withsecurity policy is a key property to ensure overall networksecurity. To cope with increasing size and complexity ofcurrent networks, the process of validating hosts’ compliancewith security policies need to be automated. Several vul-nerability scanning tools have been developed to scan com-puter hosts against a given security checklists and report thecompliance state. The Security Content Automation Protocol(SCAP) [10] provides standard specifications to communicatesecurity information. The Extensible Configuration ChecklistDescription Format (XCCDF) [14], a SCAP specification,

specifies a language to describe security checklists and collectcompliance results of targeted systems. However, each host istypically scanned in isolation from the rest of the network. Thecompliance state alone is not enough to accurately estimatethe risk imposed by certain vulnerabilities on the hosts. Webelieve that the likelihood of exploiting a vulnerability ona particular host is highly correlated with its exposure tothe rest of the world. Hence, there is a tremendous needfor integrating the hosts’ compliance state with the networkconfiguration. For example, lets consider the policy “clientswith weak passwords are not allowed to access the financialservers remotely.”. In order to verify this policy, we need to:(i) identify the hosts whose password policy do not complywith the strong password rules, (ii) identify financial serversin which the remote access is enabled, and (iii) verify thatthe network configuration allows clients with weak passwordpolicy communicate with financial servers that are enablingremote access. We see that the first two checks depend on thecompliance state of the hosts and the last check depends onthe network configuration.

The compliance state of network hosts also plays a keyrole in assessing the risk of cyberattacks on the network.Any computing system has vulnerabilities of one kind oranother (software flaws, misconfigurations, etc.). It is notalways feasible to fix all the vulnerabilities at the same timedue to financial or business constraints. Hence, system admin-istrators need automated mechanisms to prioritize fixes andjustify trade-offs between cost, functionality, and security. Riskassessment aims to provide consistent recommendations tomaximize the protection of organizations’ assets. Risk assess-ment process include the following steps. (i) The identificationof the business critical assets and the dependencies betweenthem. (ii) The Identification of network vulnerabilities throughthe compliance checking reports. Finally, (iii) estimating thelikelihood of exploiting vulnerabilities and their impact onthe organization’s mission. The end result is an estimationof the amount and the likelihood of losses due to exploits.SCAP provides a set of measurement and scoring systems toprovide universal scores of known vulnerabilities. However,interdependency of vulnerabilities distributed in the networkand the connectivity of hosts, which is realized through thenetwork configuration, can significantly elevate the network-wide risk. Thus, it is mandatory to collectively analyze the

compliance reports of all connected systems for comprehen-sive risk assessment. The scores generated by the measurementand scoring systems report severity scores for independentvulnerabilities and they are not meant to be used as risk scores.Although they define a group of temporal and environmentalmetrics to tune the severity scores to specific environments,the process is still manual and needs to be done separately foreach vulnerability. In conclusion, there is a tremendous needfor tools that automatically combine all these factors in a waythat best correlates with the risk of cyberattacks.

In this paper, we present a top-down and bottom-up ap-proaches that utilize network-wide compliance reports alongwith global configuration for security analytics. Our contribu-tions in this work are twofold:• We present a framework that allows users to define and

verify enterprise security policy in terms of securitychecklists (top-down). The framework integrates the com-pliance reports with the complete network configurationto devise a holistic verification report.

• We design objective metrics to measure the exposure ofan organization’s assets to potential threat sources and toassess the risk of potential attacks utilizing the network-wide compliance reports (bottom-up). The metrics canassess the administrator in security hardening and prior-itizing vulnerability fixes and patching.

The presented approaches consider the major risk factors,including threat sources, vulnerabilities, and network configu-ration. The integration of network configuration provides liveinformation about the countermeasures and assets distributionin the network, which leads to precise exposure and riskestimation. Moreover, the vulnerabilities of all potential threatsources are evaluated in order to calculate the risk associatedwith a particular host. In this way, we capture the dependencybetween vulnerabilities residing on single or multiple hosts.We use the standard specifications of SCAP to communicatecompliance reports and vulnerability scoring information. XC-CDF documents are used to collect the security checklists’scanning results of all hosts in the network. We take advantageof the base metrics defined in the standard vulnerabilityscoring systems (CVSS, CMSS, and CCSS) in calculatingexposure and risk scores. Finally, we describe a validationplan to show that the metrics are accurate and repeatable sothey can be used to prioritize vulnerabilities remediation tasksand assess the impact of configuration changes.

The rest of the paper is organized as follows. In Section II,we present the system model. Section IV provides the defini-tions of our metrics. In Section ??, we describe our validationplan. The related work is provided in Section VI. Finally, weconclude and present future plans in Section VII.

II. SYSTEM MODEL

Our model depends on the hosts’ compliance reports, vul-nerability scoring systems, network configuration, and securityrequirements as depicted in Figure 1. The compliance reportsgenerated by the vulnerability scanning tools specify whichvulnerabilities exist in which hosts. The severity scores of

these vulnerabilities are provided by the standard measurementand scoring systems. The network configuration captures theconnectivity and dependency between the hosts. The securityrequirements specify the required isolation levels betweenhosts based on their compliance state. In the following sec-tions, we demonstrate (i) how the requirements are composedinto the enterprise policy in terms of compliance-based profiles(Figure 1a), and (ii) how the global risk is estimated based oncompliance reports and network connectivity (Figure 1b).

A. Network Configuration

The network configuration includes the network topologyalong with the policies of the network core devices (i.e.,routers, firewalls, intrusion detection systems, etc.). Each net-work device has a different policy format and structure fromthe others. However, the configuration of network devices canbe abstracted as shown in Table I. The Flow is determinedby the basic IP header fields. The Isolation represents theaction applied by the countermeasures through which thespecified flow passes. The Loc is a unique ID for the devicein which the isolation level is enforced on the specified flow.The isolation levels may differ from one network to anotherbased on the deployed countermeasures. A value is assignedfor each isolation level that intuitively determines the abilityof counteracting attacks (i.e., the level of protection). Table IIshows examples of isolation levels. This set may be extendedto include multiple encryption levels, for example, or to addother isolation levels that are specific to a particular network.These protection values are provided by the network operator.We are planning to implement a procedure to calculate themempirically.

We build our framework on top of a network verificationtool called ConfigChecker [4]. ConfigChecker is a modelchecker based on the Binary Decision Diagrams (BDD) thatmodels the entire network configuration as a state machine.The state space is the cross-product of the packet attributesby the locations in the network. ConfigChecker models eachdevice by encoding how it transforms the packets. For ex-ample, a firewall might only drop the packets or forwardthem to its gateway. A router might forward the packetswithout changing their headers. A NAT gateway will changethe location as well as some header information. In eithercase, the disjunction of all possible packets’ transformationsin each device encoded as Boolean formulas constitutes itstransition relation. Further, the global transition relation of theentire network is the disjunction of the transition relation of allits devices. ConfigChecker provides a language based on theComputation Tree Logic (CTL) [6] to express the enterprisepolicy rules as temporal properties. Properties can be definedin terms of the locations and the packet header fields connectedusing standard Boolean operators, as well as the CTL temporaloperators. ConfigChecker translates the queries to Booleanexpressions and executes them against the global transitionrelation.

We extended the ConfigChecker model to encode thecompliance-based profiles in the global transition relation to

Holistic Policy Verification

Network Configuration

Security Checklists

Scanning Tools

Reports

Vulnerabilities

Compliance-based Profiles

Enterprise Policy

Security Requirements

(a) Top-down approach.

Vulnerabilities

Scoring Systems

Security Checklists

Scanning Tools

Enterprise Risk Estimation

Network Configuration

Reports

Severity

Scores

Base Scores

(b) Buttom-up approach.

Fig. 1: System Overview.

Loc Flow Isolation1 (tcp): *:* → 10.10.3.0/24:80 Inspect1 (*): *:* → 10.10.4.0/24:* Forward2 (udp): *:* → 10.10.5.100/32:* Encrypt

. . . . . .TABLE I: Network Configuration

Isolation ProtectionForward 1Encrypt 8Inspect 6Tunnel 5

TABLE II: Isolation

allow for enterprise policy verification. We also extended thelanguage interface to execute the reachability queries that areessential for estimating the hosts’ exposure and to expressproperties in terms of compliance-based profiles. These ex-tensions are technically discussed in Section III.

B. Compliance State

The compliance state consists of the vulnerability scanningreports of the entire network reported using XCCDF resultdocuments, a standard XML-based specification language forwriting security checklists. Multiple security checklists maybe scanned in each host. XCCDF documents are used tocommunicate both the security checklists definitions and theirresults. The security checklists consist of a set of rules thatdirect the vulnerability scanning tools to detect common vul-nerabilities and misconfigurations. The rules may be organizedin nested groups. The XCCDF specification defines a set ofresults that can be assigned to each rule, those are {Pass,Fail, Error, Unknown, Notapplicable, notchecKed, notSelected,Informational, or fiXed}. In the following discussion, weuse the capitalized letters as abbreviations for the results.The complete details of XCCDF structures and the meaningof these results can be found in the XCCDF specificationdocument [14].

Definition We define the compliance state of the entire net-work as the three-tuple Scomp = (H, V, M), where:• H represents the set of hosts.• V represents the set of common vulnerabilities.• M is a matrix that maps each host to its vulnerabilities.M[h, v] ∈ {P, F,E, U,N,K, S, I,X} for all h ∈ H andv ∈ V.

In this model, the common vulnerabilities are referencedby their unique IDs specified in the security checklists.The XCCDF specification allows security checklist authorsto reference the standard names of common vulnerabilitiesand configuration settings defined in the public dictionaries,

such as the Common Vulnerabilities and Exposures dictio-nary (CVE) [2] and the Common Configuration Enumeration(CCE) [1]. However, if the rule’s definition is missing thereference to the standard name, we use the rule ID to referencethe vulnerability.

C. Vulnerability Measurement and Scoring

Vulnerability measurement and scoring systems providestandard scoring mechanisms for measuring the exploitabilityand impact of the common vulnerabilities. NIST InteragencyReport 7502 [13] categorizes the vulnerabilities into threecategories: (1) software flaw vulnerabilities (unintended errorsin the design or coding of software), (2) security configurationissue vulnerabilities (involve the use of security settings thatnegatively affect the security), and (3) software feature misusevulnerabilities (features in the software that provide an avenueto compromise the system). Three standard scoring specifica-tions have been created to measure the severity of the three cat-egories of vulnerabilities: the Common Vulnerability ScoringSystem (CVSS) that addresses software flaw vulnerabilities,the Common Misuse Scoring System (CMSS) that addressesmisuse vulnerabilities, and the Common Configuration ScoringSystem (CCSS) that addresses software security configurationissue vulnerabilities. Each of these specifications describesthree groups of metrics: base, temporal, and environmentalmetrics. These groups are used to compute an exploitablitysub-score, an impact sub-score, and a final severity score.

We use the base metrics in our risk assessment model.We believe that the base metrics are sufficient to measurethe impact of vulnerabilities as they measure the fundamentalattributes of vulnerabilities, while temporal and environmentalmetrics capture external factors that can potentially affect thevulnerability, such as the existence of countermeasures orthe characteristics of the system’s environment. We providean alternative for temporal and environmental metrics byintegrating the live network configuration in the risk assess-ment process. Six metrics are defined in the base metricsgroup: three to measure the exploitability (access vector,authentication, and access complexity), and another three tomeasure the impact (confidentiality impact, integrity impact,and availability impact). The standard scoring systems definethe equations to compute the exploitability and the impact sub-scores, as well as the final severity score.

Rule

ID, Weight

Group

Weight

Benchmark

(a) Benchmarks Structure.

Host Configuration

Benchmark Definition

Benchmark Score

Benchmark, Scoring Model

Compliance-based Profile

uses uses

(b) Profiles Structure.

Fig. 2: Basic Entities.

Definition We model the common vulnerabilities scores asthe three-tuple Sscore = (V, t, S), where:• V represents the set of common vulnerabilities.• t : V→ T is the type function that maps the vulnerability

to its category in T = {F,C,U}, which stands for soft-ware flaw, configuration issue, or misuse vulnerabilities,respectively.

• S is the scores matrix, with a row for each vulnerabilityand a column for each of exploitability sub-score (E),impact sub-score (I), and integrity impact base score (G).S[v, sc] ∈ [0, 10] for all v ∈ V and sc ∈ {E, I,G}.

The exploitability and impact sub-scores are reported outof 10 in the scoring systems. The integrity impact base metricis reported out of 1. We show in the following sections howthese scores are used in estimating the global risk.

III. ENTERPRISE SECURITY POLICY VERIFICATION

The enterprise security policy determines the isolation levelthat should be enforced between a pair of hosts based ontheir compliance with selective security checklists. In orderto construct a security policy in terms of compliance scores,we need to understand the relationships between the entitiesshown in Figure 2. The security checklists, referred to asBenchmarks, consist of a number of rules. As depicted inFigure 2a, the rules may be hierarically structured using thegroup construct. Each rule that normally checks for a specificvulnerability in the target system, is assigned a value (calledweight) in the interval [0, 10]. The weight is usually derivedfrom the standard vulnerability scoring systems. The weightsof the rules play a key role in calculating the total compliancescores of the benchmarks.

A. Defining Benchmarks

Multiple organizations adopted the XCCDF to define thesecurity checklists. A large number of XCCDF documentsfor the common operating systems, web server, and otherapplications are available in public repositories, such asNational Vulnerability Database (NVD) [9]. However, thesecurity checklists are not limited to the public checklists.Organizations can define their own checklists for their specificneeds. New checklists can be defined either from scratch, oras a subset of a standard checklist, or by combining differentportions from multiple standard checklists.

Definition A security benchmark B is formally defined as thetree B = (N,E, n0), such that:• N and E are the sets of nodes and edges, respectively.• n0 represents the root node.• nl ∈ V for every leaf node nl ∈ N .

Our framework allows the user to load a set of standardbenchmarks that can be used to create new customized ones.The user selects the required rules or groups from the standardbenchmarks and adds them to the appropriate location inthe tree structure of the new benchmark. The leafs of thebenchmarks’ trees should be XCCDF rules. Recall that eachXCCDF rule is associated with a single vulnerability in Vaccording to our model.

B. Defining Compliance-Based Profiles

A compliance-based profile describes a class of hosts interms of their compliance scores with single or multiplebenchmarks. Let us first discuss the scoring process of thecompliance benchmarks, then we will show how to define andvalidate the compliance-based profiles.

Benchmarks are structured lists of security rules that havedifferent weights. The scoring of a benchmark should considerthe weights of the rules as well as their positions within thechecklist structure. The XCCDF specification [14] providesfour scoring models that can be used to calculate the score ofa particular host’s compliance with a benchmark: the DefaultModel (DM), the Flat Model (FM), the Flat UnweightedModel (UM), and the Absolute Model (AM). The scanningtools may deffer in their support of these models. In ourframework, we read the raw results from the scanning tools(i.e., the list of independent rules and whether their test resultsare pass or fail). We provide a fifth model called the AbsoluteInverse Model (IM). The user can select which model he likesto use in order to calculate the benchmark score. To formallydefine the scoring models within the context of our framework,let Sc(n,M) represent the score of the node n in a benchmarktree calculated according to the scoring modelM. Further, letL represent the set of the leaf nodes in the benchmark treeand let Cn be the set of child nodes of the parent node n. Thefollowing shows the formal definitions of the scoring models.

Sc(n,DM) =

{Wn × rn n ∈ L

Wn ×∑

c∈CnSc(c,DM)∑

c∈CnWc

n /∈ L(1)

Sc(n, FM) =

∑c∈L Wc × rc∑

c∈L Wc(2)

Sc(n,UM) =

∑c∈L rc

len(L)(3)

Sc(n,AM) = 1 if Sc(n,UM) = 1 and 0 otherwise (4)

Sc(n, IM) = 1 if∑c∈L

rc 6= 0 and 0 otherwise (5)

Where Wn is the weight of the node n and rc = {0, 1} isthe scanning result of the rule c in the target system (rc = 1if the result is pass and 0 otherwise). len(L) represents thelength of the set L. Note that all the scoring models except

Profile P ::= Sc(B,M) ./ C | !P | P ∧ P | P ∨ POperator ./::= > | < | ≥ | ≤ | = | 6=

Model M ::= DM | FM | UM | AM | IMBenchmark B ::= <a standard or customized benchmark>

Action A ::= Block | Encrypt | Inspect | TunnelRule R ::= Enforce(P , P , A)

Fig. 3: Policy definition language syntax

the default model (DM) consider only the leaf nodes; hence,they can be only called for the root node n0.

A profile is defined as a Boolean relation in terms ofcompliance scores. Figure 3 shows the syntax of the profiledefinition language. The profile can be defined as a singlecondition on the score of a predefined benchmark calculatedaccording to any of the scoring models. C represents a constantvalue. Multiple conditions on multiple benchmarks’ scoresmay also be combined using binary logical operators.

C. Defining Security Policy

The security policy in the presented framework determinesthe isolation levels that should be enforced on the trafficflowing between given sources and destinations. The sourcesand destinations are defined as compliance-based profilesrather than exact addresses. A policy consists of a set of“Enforce” rules. As depicted in Figure 3, each rule consists ofa source profile, a destination profile, and an action. We showin the figure four possible actions. However, the set of actionsdepends on the network configuration and it can be extendedbased on the network hardware capabilities.

D. Integration with Network Configuration Analytics Engine

The compliance state of the network needs to be integratedwith the global network configuration in order to performcomprehensive security analysis. Since the policy languageis profile-based (i.e., the hosts are described in terms of theprofiles), there is no need to encode the complete scanningresults for each host. The host state should only include theprofiles and whether it matches them or not. Recall that inConfigChecker each state is encoded by the packet and thelocation. To encode the compliance state of the network andverify related properties, we extend the ConfigChecker asfollows:• We attach a variable for each profile defined in the system

to the location, where the mapping between the variablesand the corresponding profiles is preserved. For eachlocation l in the network, the value of the profile variablevp is set to 1 if the compliance state of l matches thecorresponding profile p. vp is set to 0 otherwise.

• We extend the CTL query language of ConfigChecker byintroducing the directive comp(p) that returns the valueof the variable that corresponds to the profile whoseindex is p. Our framework translates the policy ruleswritten in the policy definition language illustrated in

Figure 3 to the appropriate CTL queries that can be runon ConfigChecker.

At this version, we only consider the compliance statesof hosts. The transition relation of hosts should be modifiedaccordingly to encode the compliance state. Let us assumethat there are N profiles defined in the system. The profilesare indexed by the values {1, 2, . . . , N}. the following showsthe transition relation Th of the host h when the compliancestate is considered.

Th = (IPsrc ⇔ IPh) ∧ (Lc ⇔ IPh) ∧ (Ln ⇔ IPg)∧∧i∈N comp(p)⇔ match(h, p)

(6)

where match(h, p) indicates whether the host h matches theprofile whose index is p. IPsrc, Lc, and Ln represent thesource IP address, the current location, and the new locationvariables in the transition relation. IPh and IPg represent thevalues of the host IP address and its default gateway.

E. Example - Putting it Together

In this section, we present a practical example of a simpleenterprise policy and explain the process of composing therules in terms of compliance-based profiles. Let us assumethat an organization wants to enforce the following two rules:• Communication between Apache 2.2 web servers that

have cross-site scripting (XSS) vulnerabilities to MSSQL2012 servers that do not enforce encryption should beinspected.

• Traffic destined to servers hosting multiple databasesfrom web servers that do not enforce proper input vali-dation, without authentication should be blocked.

To verify rules of this kind, comprehensive analysis of boththe network configuration (reachability) and the XCCDFreports (vulnerabilities) should conducted. We show in thefollowing the detailed steps that need to be taken to verifythe first rule. We omit these details for the second rule dueto space restrictions.

Defining Benchmarks. For the first rule, we can usetwo standard benchmarks: “Apache 2.2 Security TechnicalImplementation Guide (STIG) - Windows” and “MSSQLServer 2012 STIG”. We also need to define two customizedbenchmarks. The first benchmark (“Apache XSS Checklist”)contains a set of rules to check the the absence of XSSvulnerabilities in Apache 2.2 servers. The second benchmark(“MSSQL Encryption Checklist”) includes rules to check forthe proper encryption configuration in MSSQL servers. Letus denote these four benchmarks by Bap, Bms, Bxss, andBenc, respectively.

Defining Compliance-based Profiles. We need to define thetwo profiles: “SrcProfile” and “DestProfile” as follows:

SrcProfile = [Sc(Bap, DM) < 75] ∨ [Sc(Bxss, AM) = 0]

SrcProfile = [Sc(Bms, DM) < 75] ∨ [Sc(Benc, AM) = 0]

The first profile describes Apache 2.2 servers that do notcompletely comply with the benchmark Bxss or theircompliance score with the benchmark Bap is less than athreshold (75). The second profile describes MSSQL databaseservers that do not completely comply with the benchmarkBenc or their compliance score with the benchmark Bms

is less than a threshold (75). Note that we used the defaultscoring model (DM ) with each of the benchmarks Bap andBms, while we used the absolute scoring model (AM ) withthe benchmarks Bxss and Benc.

Defining Enforce Rule. Finally, we need to define the rulethat combines the two profiles as follows:

Enforce( SrcProfile, DestProfile, Inspect )

This rule specifies that the traffic from the hosts that matchthe profile SrcProfile to any host that matches the profileDestProfile should be inspected. The policy verification enginewill verify this rule by (i) extracting all the hosts that matcheither of the profiles based on the network compliance state,and (ii) verifying that the Inspect isolation level is enforcedbetween them based on the current network configuration.

IV. RISK ASSESSMENT MODEL

In this section, we formally define our exposure and riskmetrics. We also introduce another attribute for the hosts calledAttackability that is calculated based on the scores reported bythe standard vulnerability scoring systems. The Attackabilityis related to the host’s behaviour as an attacker, while theExposure is related to hosts as victims.

A. Attackability

The host’s Attackability is a quantitative attribute that mea-sures the ability of the host to establish attacks against otherhosts. This attribute is correlated to the number and severityof the host’s vulnerabilities because exploitable vulnerabilitiesallow attackers to compromise the host, install root-kits, and,eventually, launch attacks to its environment. The Attackabil-ity is calculated based on the scores reported by the standardvulnerability scoring systems.

Definition The Attackability of a host is proportional to theexploitability sub-scores and the integrity impact base scoresof its vulnerabilities.

In this definition, we use the integrity impact base scoreinstead of the total impact sub-score. According to NISTReport 7502 [13], Integrity refers to the trustworthiness andguaranteed veracity of information. The integrity impact mea-sures the ability of an exploited vulnerability to modify systemfiles and install root-kits which can be used for spoofingand launching attacks. We believe that the integrity impactis more correlated to the Attackability than the confidentialityand availability impacts are. We define the Attackability of aparticular host formally as the normalized sum of the integrity

6

9

4

5

8 3

10

1

Target

Qu

anti

ty

FW

FW

IDS

Depth

Fig. 4: Host Exposure factors. The numbers in circles representthe Attackability (out of 10) of the potential threat sources.

impact base scores of the host’s vulnerabilities weighted bytheir exploitability sub-scores:

Attackability(h) =

∑v∈Vh

S[v,E]× S[v,G]∑v∈V S[v,E]× S[v,G]

(7)

Where Vh = {v : v ∈ V,M[h, v] ∈ {F,E}} is the set ofactive vulnerabilities in the host h. S, V, and M are defined inSection II. The Attackability is normalized by dividing on theweighted sum of the integrity impact of all the vulnerabilitiesidentified in the system.

B. Exposure

The Exposure of a host (victim) measures its attack surfacebased on the network’s configuration and compliance state.Specifically, the exposure depends on the following factors:• Quantity: how many hosts can reach the victim.• Quality (Attackability): how bad are the hosts that can

reach the victim in terms of their compliance scores.• Depth: how many attack stages are required to success-

fully attack the victim.• Attack Resistance: how strong are the attack counter-

measures deployed in between.To calculate the exposure, we need to evaluate the reachabilityto the victim from all potential threat sources. We build areachability tree for each host in the network as shown inFigure 4. A node in the reachability tree corresponds toanother host that can reach the victim. The weight of theedge connecting two nodes (i, j) is the protection value ofthe isolation level enforced on the traffic flowing from i to jas defined in the network model section.

Definition The Exposure of a victim is proportional to thenumber and the Attackability of the potential threat sources,and it is inversely proportional to their depth in the reachabilitytree and the isolation level enforced in-between.

Based on this definition, we define the Exposure of a hosth formally as follows. Let TRh be the reachability tree of thehost h and let (x → h) represent a path in the tree from thenode x to the node h. Then the exposure Exp(h) is defined as:

Exp(h) =∑

x∈TRh

Attackability(x)Resistance(x→ h)

(8)

Where Resistance(x → h) represents the resistance of thepath from x to h with Resistance(h → h) being 1. Theresistance in this definition is calculated as the summation ofthe weights of all the edges in a path (i.e., the summation ofisolation level protection values).

C. Risk Estimation

Risk models define the risk factors to be assessed and therelationships among them. Typical risk factors include threat,vulnerability, impact, likelihood of exploit, and predisposingcondition (a special condition in the system that can introducenew vulnerabilities in special situations) [3]. In this work, therisk model depends on: (i) the exploitability and impact scoresreported by the vulnerability scoring systems, (ii) the exposure,and (iii) the assets distribution in the network.

Definition The risk associated with a host is proportional toits exposure, its asset value, and the number and severity ofits vulnerabilities.

Hosts in the network may have more than one vulnerability.In this case, we calculate the total severity of these vulnerabili-ties as the normalized sum of their impact sub-scores weightedby their exploitabilities. Note that in this definition, we usethe total impact sub-score and not only the integrity impactbase score as in the case of Attackability. The asset valueof each host in the network is given as part of the networkconfiguration. The total risk associated with a given networkRisk(N) is formally defined as:

Risk(N) =∑h∈N

[A(h)× Exp(h)×

∑v∈Vh

S[v,E]× S[v, I]∑u∈V S[v,E]× S[v, I]

](9)

Where A(h) is the asset value of the host h and Vh = {v :v ∈ V,M[h, v] ∈ {F,E}} is the set of active vulnerabilitiesin the host h. S and V are defined in Section II. The globalrisk of the entire network is calculated as the total sum of therisks associated with all its hosts.

V. EVALUATION

In this section, we present the space and time requirementsof our framework under various network parameters. Thespace requirements are measured by the number of BDD nodesused to encode the full network configuration, including theXCCDF-based compliance reports. We also measure the timeto build the model and execute reachability queries. We studythe impact of network size and the number of compliance-based profiles on the space the time requirements. To createa large number or network instances, we have developed arandom network generator that allows the customization ofthe network parameters. The experiments were conducted ona standard PC with 3.33 GHz Intel Core 2 Duo processor and4 GB of RAM.

A. The impact of network size

In this experiment, we study the impact of network size,measured in the number of nodes, on the time and spacerequirements for building the model and executing the reach-ability queries. We generated networks with sizes that rangedfrom 200 to 4000 nodes. A network node may be a host, arouter, or a firewall. The hosts constitute 40-70 % of the totalnodes in the network. We repeated the experiment four timeswith different number of profiles. Figures 5a and 5b show the

memory and time requirements for building the BDD model.We can see that the memory requirement in increasing linearlywith the network size. However, the time required to build themodel is best described by a quadratic function. Based on ourexperiments, the time ans space requirements stays in saferages even for large networks. As expected, increasing thenumber of profiles also increases the memory consumption,but it does not affect the order of increase with respect to thenetwork size.

Figure 5c shows the relation between time required torun the basic reachability queries and the network size. Thereachability analysis is required to compute the exposure. Foreach network instance we ran 30 reachability queries betweenrandom hosts. The figure reports the average running time.We can see that for networks of sizes below 1200 nodes, thequery execution time was less than one second. However, itstarts increasing rapidly after this size.

B. The impact of the number of compliance-based profiles

In this experiment, we study the impact of increasing thenumber of profiles on the space and time requirements forbuilding the model and query execution time. Each profile isassociated with a variable in the BDD model. Although wedo not expect a high number of profiles in practical networks(may not exceed a couple of hundreds), we studied the impactof up to 2000 profiles. We repeated the experiment for multiplenetworks with different sizes. In Figure 6a, we can see thatthe memory consumption increases linearly with the number ofprofiles. We can also see that the increase rate is proportionalto the network size as well. Figure 6b reports the time requiredto build the model with respect to the number of profiles.This figure, as well as Figure 5b, indicate that the number ofprofiles has no direct impact on the time to build the modelof sizes less than 3000 nodes. However, for network of sizesgreater than 3000 nodes, the time to build the model startsincreasing linearly with the number of profiles. We believe thatthis behavior is acceptable for two reasons. First, the numberof profiles is not expected to be extremely high. We can seein the figure that the increase in time in the case of 4000nodes does not exceed 15% when the number of profiles wasincreased from 50-1000 profiles. Second, the model is builtonce at the beginning of analysis, then the queries are executedsequentiealy wethout the need to rebuild the model again.

In Figure 6c, we show the query execution time withrespected to the number of profiles. These represent the queriesthat need to be executed to verify the enterprise policy. Weran 30 random queries against multiple networks whose sizesvaried between 200 and 2000 nodes and reported the avereragerunning time. The number of profiles varied between 0 and2000 profiles. The results show that the query execution time isindependent from the the number of profiles. It is only affectedby the network size.

VI. RELATED WORK

Researchers proposed various strategies for systems’ riskassessment, including penetration testing, standard specifica-

0

50

100

150

200

200 600 1000 1400 1800 2200 2600 3000 3400 3800

Mem

ory

(M

B)

Network Size

50 Profiles

250 Profiles

500 Profiles

1000 Profiles

(a) Space requirements.

0

50

100

150

200

250

300

200 600 1000 1400 1800 2200 2600 3000 3400 3800

Bu

ild T

ime

(sec

)

Network Size

50 Profiles

250 Profiles

500 Profiles

1000 Profiles

(b) Build time.

0

20

40

60

80

200 700 1200 1700 2200 2700 3200 3700

Tim

e (s

ec)

Network Size

(c) Query execution time.

Fig. 5: The impact of network size.

0

50

100

150

200

250

300

350

50 350 650 950 1250 1550 1850

Me

mo

ry (

MB

)

Number of Profiles

200 Nodes 500 Nodes1000 Nodes 2000 Nodes4000 Nodes

(a) Space requirements.

0

50

100

150

200

250

300

350

400

50 350 650 950 1250 1550 1850

Bu

ild T

ime

(sec

)

Number of Profiles

200 Nodes

500 Nodes

1000 Nodes

2000 Nodes

4000 Nodes

(b) Build time.

0

0.5

1

1.5

2

2.5

3

0 500 1000 1500 2000

Tim

e (

sec)

Number of Profiles

200 Nodes

500 Nodes

1000 Nodes

2000 Nodes

(c) Query Execution Time.

Fig. 6: The impact of the number or profiles.

tions and universal scoring systems, and attack graphs. Inthis literature review, we focus on recent works that relyon standard scoring systems. Houmb et al. [7] have defineda model for the quantitative estimation of the security risklevel of a system by combining the frequency and impactof a potential threat. The frequency and impact are derivedfrom CVSS metrics. Joh and Malaiya [8] presented a formalquantitative approach for software risk evaluation which uses astochastic model for the vulnerability lifecycle and the CVSSmetrics. Elahi et al. [5] implemented a risk analysis methodthat measures the impact of risk qualitatively. Poolsappasitet al. [12] proposed a method for security risk assessmentbased on Bayesian attack graphs. They used the exploitabilitysub-scores reported by CVSS to estimate the probability of avulnerability being exploited.

Risk estimation techniques based on attack graphs [11],[12], [15] are driven by specific attack scenarios and they donot provide global quantitative risk scores. The works [5], [7],[8] provide quantitative and qualitative risk metrics, but theydo not target network systems. Our risk model considers theglobal network configuration and vulnerabilities dependencybased on hosts connectivity.

VII. CONCLUSIONS AND FUTURE WORK

This work investigates the feasibility of using security com-pliance reports along with universal vulnerability scores andnetwork configuration in assessing the risk of cyberattacks. Weidentified the key risk factors and the importance of integratingnetwork-wide compliance reports with network configuration.

We show how risk scores can be mathematically defined interms of key factors. Our next step is to complete the imple-mentation and conduct a thorough evaluation to validate thecorrectness of our hypothesis. We are also planning to comparewith existing risk assessment tools in terms of functionalityand usability.

REFERENCES

[1] Common Configuration Enumeration (CCE). http://cce.mitre.org/.[2] Common Vulnerabilities and Exposures (CVE). http://cve.mitre.org/.[3] NIST Special Publication 800-30, Guide for Conducting Risk Assess-

ments. http://csrc.nist.gov/publications/PubsSPs.html#800-30, 2012.[4] E. Al-Shaer, W. Marrero, A. El-Atawy, and K. Elbadawi. Network

configuration in a box: Towards end-to-end verification of networkreachability and security. In ICNP, pages 123–132, 2009.

[5] G. Elahi, E. Yu, and N. Zannone. Security risk management byqualitative vulnerability analysis. In Security Measurements and Metrics(Metrisec), pages 1–10, Sept 2011.

[6] E. A. Emerson. Temporal and modal logic. Handbook of Theoret-ical Computer Science, Volume B: Formal Models and Sematics (B),995:1072, 1990.

[7] S. H. Houmb, V. N. Franqueira, and E. A. Engum. Quantifying securityrisk level from CVSS estimates of frequency and impact. Journal ofSystems and Software, 83(9):1622 – 1634, 2010.

[8] H. Joh and Y. K. Malaiya. Defining and assessing quantitative securityrisk measures using vulnerability lifecycle and cvss metrics. In The 2011international conference on security and management (sam), 2011.

[9] NIST. National Vulnerability Database (NVD). http://nvd.nist.gov/.[10] NIST. The security content automation protocol (SCAP). http://scap.

nist.gov/.[11] X. Ou and A. Singhal. Security risk analysis of enterprise networks using

attack graphs. In Quantitative Security Risk Assessment of EnterpriseNetworks, pages 13–23. Springer, 2011.

[12] N. Poolsappasit, R. Dewri, and I. Ray. Dynamic security risk manage-ment using bayesian attack graphs. IEEE Transactions on Dependableand Secure Computing, 9(1):61–74, Jan 2012.

[13] K. Scarfone and P. Mell. The common configuration scoring system(CCSS): Metrics for software security configuration vulnerabilities.http://csrc.nist.gov/publications/nistir/ir7502/nistir-7502 CCSS.pdf, De-cember 2010.

[14] D. Waltermire, C. Schmidt, K. Scarfone, and N. Ziring. Specification forthe extensible configuration checklist description format (XCCDF) v1.2.

http://csrc.nist.gov/publications/nistir/ir7275-rev4/NISTIR-7275r4.pdf.[15] X. Yin, Y. Fang, and Y. Liu. Real-time risk assessment of network

security based on attack graphs. In 2013 International Conference onInformation Science and Computer Applications (ISCA 2013). AtlantisPress, 2013.