Static Code Analysis and Governance

Effectively Using Source Code Scanners

About Me

• Jonathan Carter– Principal Security Consultant @ Pure Hacking• Governance Business Unit• Application Security

– Enterprise Security Architect and Designer– Security Researcher @ Fortify• API’s, Frameworks, Threat Intelligence

Process

Technology

People

Presentation Flow

1. What do scanners do?2. How do they do it?3. What do you need to

worry about?4. How do you address

these concerns?

What do analyzers do?

Translation

Analysis

Reporting

Source Code

API RulesSecurity Intelligence

Vulnerabilities

1

2

3

Process

Technology

People

Translation Mechanics

TranslationSource Code

1

• Translation builds a model of how data flows through various layers

• Allows full interoperability of languages

Presentation Layer

Business Layer

Data Layer

Model

Process

Technology

People

Translation Example

String URLparameter = Request[“URLElement”];

Object ‘URLParameter’ Declared of Type String;Temporary Object ‘t1’ Declared;‘t1’ = Result of ‘Request’ object’s ‘GetElement’ Method Executed;‘URLParameter’ = ‘t1’;

1. Engine Reads .NET Source Code and Encounters:

2. Engine Translates Statement into Intermediate Language:

3. Engine Adds New Content to Existing Translation of Code

Process

Technology

People

Translation Pitfalls

Potential False Negatives:– Language Versions Not Supported– Translation Incorrect

• Translation step is not easy– Does the Translator Support the Language?– Are there subtle differences between different

versions of a particular language?– How will the user know when translation fails?

Process

Technology

People

Translation Solutions

Here’s What You Can Do:1. Verify that scanner supports

all languages involved inyour scan

2. Ask vendors about roadmaps forlanguages

3. Ensure you know how to detecttranslation failures.

Scan Mechanics

Analysis

Intelligence

ASP.NET Rules

ADO.NET Rules

T-SQL Rules

Java Rules

Model Vulnerabilities

Process

Technology

People

Scan Example

<% = Request[“URLElement”] %>1. Engine Translates .NET Source Code into Intermediate Language

Model

2. Engine Recognizes That ‘Request’ Object is Dangerous Source

3. Engine Recognizes Dangerous Output and Declares XSS Presence

ModelDangerous Source Rule

XSS.NET XSS Rule Model

Scan Pitfalls

• Scan step is even trickier than translation– Do rules cover a particular library, API?– Are rules accurately describing the conditions for a

vulnerability to exist?– Are the analyzers correctly applying a rule all the

time?– Are the rules good at detecting the vulnerabilities

you care about?– Are the rules being overly paranoid in describing

risk?

Scan Pitfalls

Potential False Positives:1. Engine models data flow and

control flow incorrectly• Engine applies rules incorrectly

2. Rules identify data sources as untrustworthy and your organization disagrees

3. Rules don’t take into account dynamic nature of your code

4. Old Rules Process

Technology

People

Scan Pitfalls

Potential False Negatives:5. Code is simply missing and

analyzer never applies rules to it6. Rules Don’t Recognize New

Methods, Classes

Process

Technology

People

Scan Pitfall

False Taint Promotion1. Engine lacks enough computing

resources to perform a full scan2. To compensate, engine cuts

corners during scan phase and makes broad generalizations about various data structures

3. Engine reports a large number of false positives

Scan Pitfall

1. Not Really Suited for Identifying Architectural Issues

2. Not Ideal for Finding Vulnerabilities in Dynamic Code

Philosophical Limitations in Static Analysis

Scan Solutions

Here’s What You Can Do:1. Verify that the scanner uses

the latest rules2. Verify that rules adequately

cover all of the libraries yourcode may use

3. Ensure that the engine providesdetailed evidence of everyvulnerability it reports.

Process

Technology

People

Scan Solutions

Here’s What You Can Do:4. Contact product’s technical

support when the evidence fora vulnerability is simply wrong

5. Ensure that the scanner’s rulesidentify any custom data sourcesand sinks

6. Examine Scan Logs to ensure scan failuresare not occurring.

Process

Technology

People

Scan Solutions

Here’s What You Can Do:7. Verify that the engine is including

all of its rules when performinga scan

8. Exclude any data source rulesfor data sources your organizationconsiders trustworthy

9. Gather feedback from developers about the accuracy of the results

Process

Technology

People

Reporting Mechanics

Vulnerabilities

Report

Report

Project Preferences

3

Engine produces various reportsProcess

Technology

People

Reporting Example

1. Engine Identifies XSS Vulnerability in Scan

XSS.NET XSS Rule Model

2. Previously, User Specifies Classification Scheme for Vulnerabilities

Risk and Vulnerability Grouping Scheme

2. Engine Produces PDF

XSS+ Custom Vulnerability

Reporting Pitfalls

Potential Problems:1. Report does not take into

account risk appetite of organization

2. Reports do not capture usefulsecurity metrics.

3. Vulnerability Description / Remediation advice not satisfactory Process

Technology

People

Reporting Solutions

Here’s What You Can Do:1. Demand to see sample reports

from vendors before purchasingthe scanner

2. Verify that the report’s risk assessment strategy is inline withyour organization’s risk methodology

3. Inspect the engine’s capability to customizereports based on security metrics

Process

Technology

People

Reporting Solutions

Here’s What You Can Do:4. Verify that you can produce

reports that reflect yourorganization’s security metrics

5. Ask your software developersif they find the reports usefulin identifying and fixing the issues

Process

Technology

People

Process Impacts

Design

Build

TestDeploy

Maintain

• Vendor Engagement• Code Development• Build• Code Review• QA• Security Auditing• Vulnerability

Management• Change Management• Risk Assessment

Process Impacts

• Impacts to Processes Are Profound– Where should a scan occur in the SDLC?– How should the results be managed?– Should the organization refuse to release until

scans are clean?– How does the organization aggregate the risks?– Does every project get a scan or just some?– How does the organization patch and

maintain the scanner? Process

Technology

People

People Impacts

Process

Technology

People

Design

Build

TestDeploy

Maintain

• Vendors• Software Developers• Testers• Security Auditors• Release Engineers• Project Managers• Risk Analysts• Operational Staff

People Impacts

Process

Technology

People

• Impacts to People Are Profound– Who’s responsible for running the scan?– Who do we turn to when results look suspicious?– Who verifies that things are getting fixed?– Who agrees to audit the results?– Who accepts the risks of the associated

vulnerabilities?– Who maintains the rules?– Who audits the quality of the scans?

Conclusions

• Source Code Analyzers are powerful and amazingly complex under the covers

• Anyone who tells you they are the complete solution is probably in sales ;-)

Conclusions

Developers – Education about the scanneris critical to identifying false positives and negatives

Risks Staff – Verify that scanner’s method of risk assessmentis aligned with yours.

Conclusions

Auditors – Don’t be overwhelmed bya lot of issues. Chances aregood there are a lot ofnon-issues (risk appetite).

Risk Owners – Insist that the results havebeen verified by someonewho wrote the code

Contact Info