Download - Leading Indicators in Information Security
Metricon ‘06
Leading Indicators inInformation security
John NyeAugust 1, 2006
2Symantec Security Services
Leading Indicators
In Medicine Body temperature
• Elevated values indicate probable illness andseverity
• Temperature alone can not diagnose the illness
Characteristics Inexpensive to collect Accurately diagnose the presence of the
condition May or may not reveal the nature of the
condition
3Symantec Security Services
Leading Indicators in Information Security
Are there easily measured system attributes that predictan insecure configuration?For example, does having a large number of open portscorrelate to having an insecure environment?
Application
Evaluate an environment for its degree ofvulnerability/risk to determine if additional investment iswarranted (for example conducting a full vulnerabilityassessment)
4Symantec Security Services
Symantec Attack Center
5Symantec Security Services
SYMC Attack Center – The Data Set
Scans conducted between April, 2005 and July, 2006 Adoption of the tool has been increasing Most scan results are relatively recent
449 Scans ConductedMostly External Penetration TestsNessusSet Selection – We Eliminated: Suspected test scans (i.e. we were testing the AC, not a client) Scans that weren’t used to produce a report
6Symantec Security Services
Methodology - Identifying Leading Indicators
Performed initial analysis using scans as the setVulnerability Score = sum of vulnerability severitiesdivided by host count (calculated for each scan)Scans ranked into quartiles based on vulnerabilityscoresVulnerability Saturation = count of instances of aparticular vulnerability divided by host count(calculated for each quartile)Plotted each vulnerability’s saturation from quartile toquartile and examined the results
7Symantec Security Services
Eliminating Vulnerabilities as Potential Leading Indicators
Vulnerability eliminated from consideration if: Highest quartile saturation did not exceed 2% Saturation didn’t increase with environment’s vulnerability Particular to a type of environment, not generic to most
environments (i.e. Web vulnerabilities)
Real Problems with the Data Set – 11th hour
Internal Network Scans Had to eliminate most vulnerable quartile completely from the
analysis because it contained multiple (and not-easily identified)scans conducted from within an enterprise perimeter
Probably eliminated several of the most vulnerable externalscans in doing so
8Symantec Security Services
Findings (By Nessus Vuln ID)
All non-Web scanner findings with a final saturation > 2% identified during remote penetration tests.
Potential Leading Indicators
0
0.05
0.1
0.15
0.2
0.25
0.3
1 2 3
Quartile
Vu
lnera
bilit
y S
atu
rati
on
11951
11935
10092
10263
11002
11618
10114
11936
9Symantec Security Services
Top General IndicatorsLeading Indicators (Preliminary Study)
0
0.05
0.1
0.15
0.2
0.25
0.3
1 2 3
Quartile
Vu
lne
rab
ilit
y S
atu
rati
on
Host Responds to
Syn/Fin
ICMP Timestamp
Request
OS Identified
10Symantec Security Services
Top Web Indicators
Leading Web Indicators (Preliminary Study)
0
0.1
0.2
0.3
0.4
0.5
0.6
1 2 3
Quartile
Vu
lne
rab
ilit
y S
atu
rati
on
SSL2.0
Web Mirror
Possible missing IIS
Service Pack
HTTP Trace Enabled
HTTP: Does not
reply with 404
HTTP Directory
Enumeration
HTTP Server Type
and Version
HTTP Server Type
and Version
11Symantec Security Services
Correlation: Scans vs. Project ReportsLeading Indicators (Small Data Set)
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
1 2 3 4
Quartile
Vu
lne
rab
ilit
y S
atu
rati
on
FTP Banner (10092)
HTTP Server Type
and Version (10107)
ICMP Timestamp
Request (10114)
HTTP Directory
Enumeration (11032)
HTTP Trace Enabled
(11213)
Possible Missing IIS
Service Pack (11874)
•All data is from external penetration testsSmall sample spaceTop 8 general and top 8 Web vulnerabilities depicted (only 6 of the 16 were presentin this data set.
12Symantec Security Services
Next Steps
Clean up the data set Quartile ranking of project reports doesn’t match that of Scans Mix of internal and external scan data Small sample set of project reports
Upgrade the math Statistical regression Multi-vulnerability analysis
Repeat analysis for different types of environment Internal vs. External, Web vs. Generic, etc.
Implement the analysis directly in the Attack Center
13Symantec Security Services
Dangers with Leading Indicators
The leading indicator itself can not be used as adiagnosisGaming the system Administrators may attempt to resolve only those
vulnerabilities that are used as leading indicators.
14Symantec Security Services
Questions?
John NyeConsulting Services Technical Lead
T. 617-768-2737M. 617-501-3248
Thank You.