SESSION ID:SESSION ID:

#RSAC

Abraham Kang

Avoiding the Pitfalls of Security Testing (Static Analysis) Tools

ASD-W02

Senior DirectorTelecommunications Company@KangAbraham

#RSAC

Special Thanks

Sung Min Hong (Staff Engineer on my team)

Romain Gaucher (Security Researcher at Coverity)

Alvaro Muñoz (Principle Security Researcher at HP Fortify)

Amit Ashbel (Product Marketing at Checkmarx)

Ohad Rabinovich (Engine Product Manager at Checkmarx)

#RSAC

Interaction Is Important

This is not your normal one way presentation

I am looking to get feedback from you so speak up and ask questions

#RSAC

About Me

Worked with a static analysis tool as an end user at Wells Fargo for over 4 years running scans, reviewing results, and assessing risk on web, infrastructure and desktop applications

Worked as Principle Security Researcher at HP Fortify (static analysis tool vendor) writing custom rules and doing research on security vulnerabilities to improve Fortify scan results

Working as Senior Director of security team building security solutions and successfully applying static analysis tools to mobile, infrastructure, and payment solutions

#RSAC

Who Are You

Backgrounds

QA Testers

Security Testers

Managers

How many are doing security testing?

Static Analysis

Dynamic Analysis

Reversing

#RSAC

Can You Relate?

Typical static analysis buying scenario

How much training did you get on the tool?

How much of that covered writing custom rules for your tool?

#RSAC

Overall Process

Understand What you are Scanning

Verify the Integrity of Your Scan

Add Custom Rules

Add Canaries

Understand What Static Analysis Cannot Do

Web Application, Library, Trusted App, Mobile Application

Add custom rules

Assess results

Add Canaries

Assess results

Address risk not covered by Static Analysis

Assess results

Scanning & Verification

#RSAC

What are You Scanning?

Well SupportedWeb Application— Using popular frameworks Struts, Spring MVC,etc.

Languages— Java, C/C++, Ruby

Not as well supported (your mileage may vary)Android ApplicationsSmart Card Apps (More Esoteric)Trusted Execution Environment (Extremely Esoteric)Operating System Components (Kernel, Drivers, etc.)Mixed Languages (Java to C/C++ and vice versa)Scripting Language (Javascript, PHP, Python, Ruby)Niche Languages (Closure, Erlang, Haskell, Scala, etc.)

Difficult to Scan because of DependenciesLibraries

Why is it like this?

Web Application, Trusted App, Mobile

Application

Scanning and Verification

Add custom rules

Assess results

Add Canaries

Assess results

Address risk not covered by Static Analysis

Assess results

#RSAC

Integrity of Static Analysis Scans

The type of application and framework you are scanning?

Is it well supported by the existing rules of the tool

Did all of your source and application artifacts get scanned by your static analysis tool?

Where there any errors during the translation?

Where there any errors during the analysis phase?

All these worked fine…Now What?

Web Application, Trusted App, Library, Mobile Application

Scanning and Verification

Add custom rules

Assess results

Add Canaries

Assess results

Address risk not covered by Static Analysis

Assess results

#RSAC

Important Background Information

You need to understand how static analysis tools work generally

Source files are translated into intermedia representations for eventual analysis as files/src/directory/Orginal.java => /static-analysis/engine/build_id/directory/Original.java.ast

or

In Coverity you can see which files were translated by looking for “cov-translate” in your build.log or looking in the cov-build –dir <idir> directory

In Checkmarx <Engine Install Dir>\Logs\ScanLogs\<Project name>\YYYYMMDDT######-##_#_#########_############_#-##########_####.log

What do you want to do to verify that this stage of the process worked completely?

Source Code

Intermediate Representation

Control Flow, Dataflow, etc.

Analysis

#RSAC

Understanding How Your Tool Works

You need to understand how static analysis tools work generally

Make sure that all of your source files have been translated

Source Code

Intermediate Representation

Control Flow Graph, Data Flow Graph, etc.

Analysis

#RSAC

Common Problems During Translation

Modified build scripts have errors in them

The static analysis tool is assuming the wrong version of the language parsed

Java 1.5 and 1.8 have significant changes

Make sure that your language compiler is recognized by the static analysis tool

Dependent libraries are not included

Setting $LD_PRELOAD

Using ”chroot” or “fakeroot”

Statically linking to OS

Forking (-Dmaven.compiler.fork=true) or sharing compilation (UseSharedCompilation=true)

Most of these issues can be resolved by addressing the issues in the debug log or adding the missing jar files to the classpath or *.so/*.dll to the appropriate path

Change build scripts to avoid problematic constructs

Some issues will not be resolved due to miss match in the parsers utilized by the tools being different from your build environment

Checkmarx does not have a dependency to the build representation of the code but may suffer some degradation in the fidelity of the representation.

#RSAC

Common Problems During Analysis

Identified by looking at the logs (analysis.log, sca.log, etc.) for strings like:

“analysis failed on ”, “error”, “exception”, etc.

Can be resolved by:

Calling support for your productIdentifying starved system resources (usually disk or memory)

#RSACLooks like the Translation and Analysis are Doing Well (Now What?)

Deeper understanding

Custom rules

CanariesWeb Application,

Trusted App, Mobile Application

Add custom rules

Assess results

Add Canaries

Assess results

Address risk not covered by Static Analysis

Assess results

Scanning & Verification

#RSAC

Custom Rules Applied to Vulnerable Code Flows

Sources and Entrypoints

Passthroughs

Sinks

Source

Passthrough3rd Party Library Sink

#RSAC

Sources and Entrypoints

Sources and EntrypointsWhere untrusted (possibly attacker controlled) input enters the systemSources can be identified by runtime functions calls in the application — String taintedData1 = obj.funct(taintedData2);

Entrypoints are slightly different as untrusted data is bound into the parameters of function definitions by the system or framework— Resolution funct(String taintedData1, String taintedData2) {

When do you need to create Sources or Entrypoint Rules?— Are you using a less popular framework which provides user input to the application?

Source

Passthrough3rd Party Library Sink

#RSAC

Passthroughs

PassthroughsRequired by some static analysis tools for taint propagation in functions that the static analysis tool cannot understand due to missing source code or use of a 3rd party library

— String sql= java.lang.String.format (“SELECT * FROM … %s”, taintedData2)

When do you need to create passthrough rules?Is untrusted data processed by 3rd party libraries before being passed onto backend systems

Source

Passthrough3rd Party Library Sink

#RSAC

Sinks

SinksLocations (usually parameters of function calls) where if untrusted data were to reach, the attacker would be able to cause a vulnerability

— connection.execute(continuedTaint)

When do you need to create Sink rules?Do you use any less popular or newer backend storage system?

Do you pass data to be processed by custom backend systems?

Source

Passthrough3rd Party Library Sink

#RSAC

Dataflow Source Rules

A function that returns possibly attacker controlled data via its in-out type parameters or return value.

Checkmarx

Return All.FindByMemberAccess(“ServletRequest.getParameter”)

Fortify<DataflowSourceRule language="java" formatVersion="3.8"> <RuleID>xxxx</RuleID><FunctionIdentifier> <NamespaceName><Pattern>javax\.servlet</Pattern></NamespaceName> <ClassName><Pattern>ServletRequest</Pattern></ClassName> <FunctionName><Pattern>getParameter</Pattern></FunctionName> <ApplyTo implements="true" overrides="true" extends="true"></FunctionIdentifier> <OutArguments>return</OutArguments> <TaintFlags>+WEB,+XSS</TaintFlags> </DataflowSourceRule>

#RSAC

Dataflow Source Rules (con’t)

A function that returns possibly attacker controlled data via its in-out type parameters or return value.

Coverity*{

"taint_kind": "servlet","tainted_data": {

"return_value_of": {"or": [

{// The native getParameter call"matching": "^javax\\.servlet\\.ServletRequest\\.getParameter\\("

},{

// All implementations that override the getParameter"overrides": "^javax\\.servlet\\.ServletRequest\\.getParameter\\("

}]

}}

}

#RSAC

Dataflow Entrypoint Rules

A rule used to taint parameters going into the method

Checkmarx

All.GetParameters(All.FindByShortName("main").FindByType(typeof(MethodDecl)),0)

Fortify<DataflowEntryPointRule language="java" formatVersion="3.8"> <RuleID>xxxx</RuleID> <FunctionIdentifier> <NamespaceName> <Pattern>.*</Pattern></NamespaceName> <ClassName><Pattern>.*</Pattern></ClassName> <FunctionName><Pattern>main</Pattern></FunctionName> <ApplyTo implements="true" overrides="true" extends="true"><Parameters><ParamType>java.lang.String[]</ParamType></Parameters>

<Modifiers><Modifier>static</Modifier></Modifiers></FunctionIdentifier> <InArguments>0</InArguments>

<TaintFlags>+WEB,+XSS</TaintFlags> </DataflowEntryPointRule>

#RSAC

Dataflow Entrypoint Rules (Con’t)

A rule used to taint parameters going into the method

Coverity

{"taint_kind": "servlet","tainted_data": {

"all_params_of": {"and": [

{"with_annotation": {

"matching": "^org\\.springframework\\.web\\.bind\\.annotation\\.RequestMapping$"}

},{

"in_class": {"with_super": {

"with_annotation": {"matching": "^org\\.springframework\\.stereotype\\.Controller$"

}}

}}

]}

}}

#RSAC

Dataflow Passthrough Rules

A rule used to taint parameters going through 3rd party libraries (no source)

Checkmarx

Not needed

Fortify<DataflowPassthroughRule language="java" formatVersion="3.8"> <RuleID>xxxx</RuleID> <FunctionIdentifier> <NamespaceName> <Pattern>java\.lang</Pattern></NamespaceName> <ClassName><Pattern>String</Pattern></ClassName> <FunctionName><Pattern>format</Pattern></FunctionName> <ApplyTo implements="true" overrides="true" extends="true"></FunctionIdentifier> <InArguments>1…</InArguments> <OutArguments>return</OutArguments> </DataflowPassthroughRule>

#RSAC

Dataflow Passthrough Rules (Con’t)

Coverity

public class String {public static String format(String format, Object args...) {

Dataflow.dataflow_return(data);Dataflow.dataflow_return(args);return unknown();

}}

#RSAC

Dataflow Sink Rules

A parameter of a function where a vulnerability would occur if untrusted data were passed to it

Checkmarx

All.FindByMemberAccess(“Statement.execute");

Fortify<DataflowSinkRule language="java" formatVersion="3.8"> <RuleID>xxxx</RuleID><FunctionIdentifier> <Sink><InArguments>0</InArguments></Sink><NamespaceName><Pattern>java\.sql</Pattern></NamespaceName> <ClassName><Pattern>Statement</Pattern></ClassName> <FunctionName><Pattern>execute</Pattern></FunctionName> <ApplyTo implements="true" overrides="true" extends="true"></FunctionIdentifier> </DataflowSinkRule>

#RSAC

Dataflow Sink Rules (con’t)

A parameter of a function where a vulnerability would occur if untrusted data were passed to it

Coverity{"type" : "Coverity analysis configuration","format_version" : 1,"directives" : [// Definition of the checker with name, description, remediation{"dataflow_checker_name" : "DF.FAKE_NOSQL_QUERY_INJECTION","taint_kinds" : [ "network", "servlet", "database", "filesystem", "console", "environment", "system_properties", "rpc" ],"languages": {"Java" : "Webapp-Security-Preview"

},"covlstr_sink_message" : "{CovLStrv2{{t{A tainted value {0} is evaluated by a NoSQLsink.}{\"\"}}}}","covlstr_remediation_advice" : "{CovLStrv2{{t{Avoid evaluating untrusted data.}}}}"

},

// Sinks{"sink_for_checker": "DF.FAKE_NOSQL_QUERY_INJECTION","sink" : {"param_index": 1,"methods": {"overrides": {"matching" : "com\\.coverity\\.example\\.db\\.FakeDB\\.customSelectorQuery\\(.*" }

}}}]}

#RSAC

Structural Rules

Grep on Steroids. Match based on code structural forms.

Checkmarx (finds method declarations of a method named “MyMethod”, only if it is attributed with the “Attr” attribute)

All.FindByCustomAttribute("Attr").GetAncOfType(typeof(MethodDecl)).FindByShortName("MyMethod");

Fortify<StructuralRule formatVersion="3.8" language="java"> <RuleID>yyyy</RuleID> …<Predicate><![CDATA[Variable p: p.enclosingFunction is [Function f: f.parameterscontains p] and p.enclosingFunction.annotations contains [Annotation b: b.type.name matches "org\.restlet\.resource\.(Delete|Put|Post|Get)"] and not p.type.name matches "org\.restlet\.util\.WrapperRequest|org\.restlet\.Request"]]></Predicate></StructuralRule>

#RSAC

Hybrid Rules

Mix of Structural and Dataflow rules

Checkmarx

CxList input = All.FindByName("input");CxList db = All.FindByName("execute");CxList sanitize = All.FindByName("fix"); return db.InfluencedByAndNotSanitized(input, sanitize);

Fortify<CharacterizationRule formatVersion="3.8” language="java"><RuleID>zzzz</RuleID><StructuralMatch><![CDATA[

Variable p: p.enclosingFunction is [Function f: f.parameters contains p] and p.enclosingFunction.annotations

contains [Annotation b: b.type.name matches "org\.restlet\.resource\.(Delete|Put|Post|Get)"]]]></StructuralMatch>

<Definition><![CDATA[ foreach p {

TaintEntrypoint(p, {+WEB +XSS})}

]]></Definition></ CharacterizationRule>

#RSAC

Coverity Rules

Coverity state machine based checker include matching and events#include “extend-lang.h”enum fun_state_t {

UNLOCKED = 0,LOCKED = 1

};START_EXTEND_CHECKER( block_check, int_store ); //name and flow basedANALYZE_TREE(){

Fun locking_fun(“lock”); //Define the functions you are looking forFun unlocking_fun(“unlock”);Fun blocking_fun(“fopen”);if ( MATCH(locking_fun) ) { //When function is called in path trigger event

SET_STATE(LOCKED);} else if ( MATCH(unlocking_fun) ) {

SET_STATE(UNLOCKED);} else if ( MATCH(blocking_fun) ) {

if ( GET_STATE() == LOCKED ) ) { //If calling fopen while locked trigger findingCOMMIT_ERROR(“Called fopen within a locking context.”);

}}

}END_EXTEND_CHECKER();MAKE_MAIN( block_check )

#RSAC

Custom Rules Summary

Checkmarx Fortify Coverity Veracode Whitehat

Most Flexible Very Flexible Flexible No custom rules allowed

No custom rules allowed

• Why is this important?

#RSAC

Custom Rules Sources

Fortifyhttp://community.hpe.com/hpeb/attachments/hpeb/sws-Fortifyforum/217/1/HP_Fortify_SCA_Custom_Rules_Guide_4.21.pdf

Coverityhttps://www.coverity.com/library/pdf/coverity_extend.pdf

Checkmarxhttps://checkmarx.atlassian.net/wiki/display/KC/Working+with+Queries

https://checkmarx.atlassian.net/wiki/download/attachments/5406733/CxQL%20API%20Guide%20v8.0.0.pdf?version=1&modificationDate=1458736360763&api=v2

#RSACYou Created the Rules But No Security Findings are Showing Up (What Now?)

???

#RSACYou Created the Rules But No Security Findings are Showing Up (What Now?)

You have to introduce “Canaries” into your code

Web Application, Trusted App, Mobile

Application

Add custom rules

Assess results

Add Canaries

Assess results

Address risk not covered by Static Analysis

Assess results

Scanning & Verification

#RSAC

Canaries

Does anyone know what a “canary” in the code is?

History of Canaries (if needed)

Placement is important Remember where you placed them as you will need to remove them later

Put them in ever increasing layers of your application— Right near a source of untrusted data

— In the first layer of business logic and others at deeper layers of your business logic

— Coverage is important

Does anyone know of any good security “canaries” to put into your code?

#RSAC

Sample Canaries

File path manipulation

boolean businessLogicMethod(String untrusted_param, … ) {File f = new File(untrusted_param);

}

SQL Injection

boolean businessLogicMethod(String untrusted_param, … ) {...Statement stmt = connection.createStatement();ResultSet rs = stmt.executeQuery(untrusted_param);

}

Command Injection

boolean businessLogicMethod(String untrusted_param, … ) {...Runtime.getRuntime().exec(untrusted_param);

}

#RSAC

If You are Still Not Seeing the Canaries…

Make sure that your rules do not have typos in the method or parameter identifiers

Make sure that you have defined all of the necessary passthroughrules for the 3rd party libraries.

Remember that function calls through packages like java.lang.*, java.util.*, javax.* are third party libraries.

— If the Java library function is new it may have been missed by the security researcher

#RSAC

Assuming Static Analysis Will Find Everything

Manual Code Review

Penetration Testing

Problems with 3rd Party LibrariesWeb Application,

Trusted App, Mobile Application

Add custom rules

Assess results

Add Canaries

Assess results

Address risk not covered by Static Analysis

Assess results

Scanning & Verification

#RSAC

Manual Code Review

AuthenticationPassword Handling (Reset, Storage, and Policies)

Backdoors

Authorization

Input Validation

Session Management

Proper Cryptography

Auditing

Business Logic VulnsNegative values

Information Leakage

File Upload/Download Logic

Error Handling

Languages not covered well by tool or cross language calls (JNI, Unmanaged Code, system calls, etc.)

https://www.owasp.org/index.php/Application_Threat_Modeling#Security_Controls

#RSAC

Penetration Testing

Verifies fixes and unresolved issues

Can find things outside the applicationExposed services (S3 Buckets, Github servers, etc.)

#RSAC

3rd Party Libraries (without source)

Static Analysis can only find issues in source

Use a tool like OWASP Dependency Check to ensure that you do not have vulnerabilities in your 3rd party libraries

https://www.owasp.org/index.php/OWASP_Dependency_Check

Dataflow analysis may be blocked

Source (or Entrypoint) + ??? (No sinks) = No Vulnerability Found through library

Sources provided by Application

sends untrusted data to

3rd party library with sinks

Library Sink

ProcessingApplication SinkX

X???

#RSAC

3rd Party Libraries (with source with application)

Understanding context of libraries

Source (or Entrypoint) + Sink = Vulnerability

Sources provided by Application

sends untrusted data to

3rd party library source with sinks

Library Sink

ProcessingApplication Sink

#RSACScanning 3rd Party Libraries (with source withoutapplication)

Understanding context of libraries

Empty + Sink = No Vulnerability Found

3rd party library with sinks

Library Sink

Processing

???

#RSACSuccessfully Scanning 3rd Party Libraries (withsource without application)

Understanding context of solution

Entrypoint rules on public interfaces + Sink = Library Vulnerability

3rd party library source with sinks

Library Sink

ProcessingArtifically created entrypoints on public interfaces of library

#RSAC

Conclusion, Contact Info, and Questions

You should have deeper understanding of static analysis tools and how to get more value out of your tool

Using the techniques in this presentation I have increased result output by over 10X in certain cases for security issues like buffer overflow

Twitter: @KangAbraham

Email: abraham.kang@hotmail.com

Blog: http://abrahamkang.blogspot.com/

LinkedIn: https://www.linkedin.com/in/abrahamkang

Questions: ?