what security testing tools miss - black duck …€¦ · what security testing tools miss mike...

WHAT SECURITY TESTING TOOLS MISS

Mike Pittenger

VP, Product Strategy

2 © Black Duck Software - CONFIDENTIAL

THINK YOU HAVE APPLICATION SECURITY

TESTING COVERED?

THINK AGAIN


WHAT DO THESE VULNERABILITIES HAVE IN COMMON?

Heartbleed Shellshock GhostFreak Venom

Introduced:

Discovered:

2011

2014

1989

2014

1990’s

2015

2000

2015

2004

2015

Discovered by:

Component: OpenSSL

Riku, Antti,

Matti, Mehta

Bash

Chazelas

OpenSSL

Beurdouche

GNU C library

Qualys

researchers

QEMU

Geffner

FREAK!SSL, TLS Vulnerability


ARE KNOWN VULNERABILITIES A NEW PROBLEM?

PBS interviews with Scott Charney, Richard Clarke, John Hamre, Amit Yoran and others

discussing the growing issue of known software vulnerabilities

Published April 24, 2003

http://www.pbs.org/wgbh/pages/frontline/shows/cyberwar/vulnerable/software.html

http://www.pbs.org/wgbh/pages/frontline/shows/cyberwar/vulnerable/software.html


WHY AREN’T WE FINDING THESE IN TESTING?

• Static analysis

• Testing of source code or binaries for unknown security

vulnerabilities in custom code

• Advantages in buffer overflow, some types of SQL injection

• Provides results in source code

• Dynamic analysis

• Testing of compiled application in a staging environment to detect

unknown security vulnerabilities in custom code

• Advantages in injection errors, XSS

• Provides results by URL, must be traced to source

What’s Missing?


Automated testing finds

common vulnerabilities in

the code you write

• They are good, not perfect

• Different tools work better on

different classes of bugs

• Many types of bugs are

undetectable except by

trained security researchers

THERE ARE NO SILVER BULLETS

All possible

security vulnerabilities

Identifiable

with

Static Analysis

Identifiable

with Dynamic

Analysis

FREAK!


WHAT DO SECURITY TESTING TOOLS MISS?

Static Analysis Tools and Dynamic Analysis Tools are very effective in finding bugs in the code written by internal developers.

HOWEVER…

• They are ineffective in finding known vulnerabilities in Open Source components

• They provide a point-in-time snapshot of security

What happens when the threat landscape changes?


THE THREAT LANDSCAPE CONSTANTLY

CHANGES

-

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,000

9,000

2005 2006 2007 2008 2009 2010 2011 2012 2013 2014

VULNERABILITIES DISCLOSED PER YEAR (NVD)

2014 Analysis of National Vulnerability Database• Over 7,900 new vulnerabilities disclosed

• ~4,300 in Open Source, ~3,600 in commercial

software

• Estimated < 2% were found by automated tools


OPEN SOURCE SECURITY

THINK AGAIN


SO WHY USE OPEN SOURCE?

Open source adds tremendous value

• Needed functionality w/o acquisition

costs

• Faster time to market

• Lower development costs

• Support from broad communities

Black Duck On Demand results

• > 98% - applications tested used open

source projects

• Commercial Applications ~35% open

source

• 95% of reviews find unknown open

source

• On average, clients had knowledge of <

50% of the open source used

Composition of commercial software tested by Black

Duck On Demand

Custom Code

Open Source


OPEN SOURCE HAS PASSED THE TIPPING POINT

“By 2016, Open Source

Software will be included

in mission-critical

applications within 99% of

Global 2000 enterprises.”

0.0

0.5

1.0

1.5

2.0

2007 2009 2011 2013 2015

millions

Growth of open source projects

accelerating.

Will face problems because

of no policy.

50%


WE HAVE LITTLE CONTROL OVER HOW OPEN

SOURCE ENTERS THE CODE BASE

Open Source Community

Internally Developed

Code

Outsourced Code

LegacyCode

Reused Code

Supply Chain Code

Third Party Code

Delivered Code

Open source code introduced in many ways…

…and absorbed into final code.


OPEN SOURCE: EASY TARGETS

Used everywhere Easy access to

code

Vulnerabilities are public Exploits are available


WHO’S RESPONSIBLE FOR SECURITY?

Commercial Code Open Source Code

• Dedicated security researchers

• Alerting and notification infrastructure

• Regular patch updates

• Dedicated support team with SLA

• “community”-based code analysis

• Monitor newsfeeds yourself

• No standard patching mechanism

• Ultimately, you are responsible


VULNERABILITY AWARENESS IS CRITICAL

Source: Kenna Security - The Remediation Gap: Why Companies Are Losing the Battle Against Non-Targeted Attacks


WHAT ARE ORGANIZATIONS DOING?

THINK AGAIN


HOW ARE COMPANIES ADDRESSING THIS TODAY?

NOT WELL.

Manual tabulation

• Architectural Review Board

• End of SDLC• High effort and low accuracy

• No controls

Spreadsheet-based inventory

• Dependent on developer best

effort or memory• Difficult maintenance

• Prone to inaccuracies

Tracking vulnerabilities

• No single responsible entity

• Manual effort and labor intensive• Unmanageable (11/day)

• Match applications, versions,

components, vulnerabilities

No enforcement controls• Old versions of approved

libraries may be used

• New components added as

requirements change


A SOLUTION TO SOLVING THIS PROBLEM WOULD

INCLUDE THESE COMPONENTS

Choose Open

Source

Inventory

Open Source

Map Existing

VulnerabilitiesTrack New

Vulnerabilities

Maintain accurate list of

open source

components throughout

the SDL

Identify

vulnerabilities during

development Alert on new

vulnerabilities and

map to applications

Proactively choose

secure, supported

open source

GUIDE VERIFY/ENFORCE MONITOR


BEST PRACTICES FOR OPEN SOURCE

Reqs Design Code Test Release

OSS Policies

• Application

Criticality Ranking

• OSS Risk

Parameters

• License Risk

• Security Risk

• Operational

Risk

OSS Selection

• Design Review

• License Risk

• Security Risk

• Operational

Risk

OSS Detection

• Automatically

detect and alert

on non-

conforming

components

• Correlation with

Bills of Material

OSS Enforcement

• Detect and alert

on non-

conforming

components

• Correlation with

Bills of Material

OSS Monitoring

• Timely OSS

Vulnerability

Identification &

Reporting

• Bug Severity

• Remediation

Advice

• Build and automatically enforce OSS policies

• Identify OSS components early in the SDLC

• Automatically create and maintain bills of material, and map known vulnerabilities

• Continuously monitor threat environment for new vulnerabilities


KEY TAKEAWAYS

Security testing is a good thing

• It identifies common vulnerabilities in the code companies write

• Different testing methodologies are better suited for different bug

types

Security testing is a point-in-time snapshot

• New vulnerabilities may result from…

• Changes to code can change security posture

• Changes in the threat environment, even if the code hasn’t changed

Open Source Security isn’t covered by traditional tools

• Monitor for open source with known vulnerabilities, early in the SDL

• Monitor production code for new vulnerabilities


WHAT CAN YOU DO TOMORROW?

Speak with your head of application

development and find out:

• What policies exist?

• Is there a list of components?

• How are they creating the list?

• What controls do they have to ensure

nothing gets through?

• How are they tracking vulnerabilities

for all components over time?


7 of the top 10 Software companies,

and 44 of the top 100

6 of the top 8 Mobile handset vendors

6 of the top 10 Investment Banks

24Countries

200Employees

1,600Customers

27 of the Fortune 100

ABOUT BLACK DUCK

Award for

InnovationFour Years in the “Software

500” Largest Software

Companies

Six Years in a row

for Innovation

Gartner Group

“Cool Vendor”

“Top Place to Work,”

The Boston Globe

2014

The Intelligent Management of Open Source

what security testing tools miss - black duck …€¦ · what security testing tools miss mike...

Documents