Frédéric PainchaudDRDC Valcartier / Systems of Systems

November 2010

Software Architectural Risk Analysis (SARA):SSAI Roadmap

1

Agenda

• Introduction

• Software Architectural Risk Analysis

– Linking to SSAI Activities

• Conclusion

2

Introduction

• Risk?

– “The net negative impact of the exercise of a vulnerability, considering both the probability and the impact of occurrence”.

– “A function of the likelihood of a given threat-source’s exercising a particular potential vulnerability, and the resulting impact of that adverse event on the organization”.

• Risk Management?

– “The process of identifying risk, assessing risk, and taking steps to reduce risk to an acceptable level”.

– Gary Stoneburner, Alice Goguen and Alexis Feringa, Risk Management Guide for Information Technology Systems, NIST, Special Publication 800-30.

3

Introduction

• The three processes of Risk Management:

– Risk assessment (aka Risk analysis)

– Risk mitigation

– Evaluation and assessment

• Why?

– To forecast potential problems;

– To develop and implement appropriate controls to avoid identified problems;

– To plan the actions to be taken if the controls go wrong, if uncontrolled identified problems arise (residual risk) and if unforeseen problems happen.

Risk assessmentR

isk mitigation

Evaluation and

assessment

4

Introduction

• Software Architectural Risk Analysis (SARA)?

– An adaptation of general Risk Assessment (Analysis), the first step of Risk Management.

• SARA and Risk Management in general are more an art than a science. Their processes are defined but practice and experiencetailor them to the particular organization.

5

Introduction

• Going back to Risk Management, for IT systems, it must be integrated into the Software Development Life Cycle (SDLC) to be effective.

• The risk management methodology is (roughly) the same regardless of the SDLC phase for which the assessment is being conducted.

6

Introduction

SDLC Phases Support from Risk Management activities

Phase 1: Initiation • System risk identification

• System requirements incl. security

• System CONOPSPhase 2: Development or acquisition • Architecture design

• Coding practices

– e.g., security (CERT C, ISO C)

• Testing

– e.g., security testingPhase 3: Implementation (in the environment)

• Validation w.r.t. the requirements and operational environment

Phase 4: Operation and maintenance • Continual evaluation and assessment

Phase 5: Disposal • Proper disposal and system migration

7

Introduction

• Participation in the risk management process is the business of many key players in organizations, including senior managers, business operations and IT procurement managers, IT security program managers and computer security officers, IT administrators and trainers.

8

Architectural Risk AnalysisSoftware Architectural Risk Analysis

Architecture documentation

Step 1. System characterization

One-page overview

Step 2. Threatidentification

Step 3. Vulnerabilityidentification

Step 4. Controlanalysis

History of system attacksSources of intelligence

Threat statement

Security testingresults

Vulnerability statement

Plannedcontrols

List of controls

Step 5. Attack likelihood determination

Attack likelihood rating

Step 6. Impactanalysis

Impact rating

Step 7. Risk determination

Step 8. Control recommendations

Step 9. Results documentation

Rated risks

Recommended controls or modifications

Architectural Risk Analysis Report

One-page overview

8

9

Dividing SARA Steps into Meetings

• Kick-Off Meeting (1/2 day):– Objective: First familiarization with the system to study (CSD

& CSD API)– Agenda:

• Overview of the SARA process• Overview of the meeting agenda• Introductory presentation of the CSD

– Meeting product: CD with introductory documentation and presentations on the CSD

– Post-meeting actions:• Prepare available documentation, source code and demo• Familiarize with the architecture and prepare questions

10

Step 1. System Characterization

Architecture documentation

Step 1. System characterization

One-page overview

• Mostly design diagrams• Questionnaires, on-site interviews, tools, …

• Update or create design diagrams (validate against operational system)• Merge the views and abstract the design levels to produce a one-page overview (ambiguity analysis can help)

The basis for the entire architectural risk analysis

11

Software Architecture Recovery Tools

• When your architectural documentation is incomplete, out-of-date or inexistent and when you have the source code, there are robust tools available to help.

• Refer to Philippe Charland’s presentation on the CD.

12

Software Architecture Recovery Tools

• These tools are useful when you have the source code. When you only have the binary, there are currently two choices:

1. you use decompilers to generate source code of varying quality from the binary and then you use these tools or

2. you manually analyze the binary with the help of specialized tools to accelerate the process.

• These tools are useful to accomplish step 1 of the Architectural Risk Analysis process. When it comes to managing the whole process, a solution like KDM Workbench is more appropriate.

13

Dividing SARA Steps into Meetings

• Meeting #1 (June 21-23):– Objective: Perform Step 1 – System characterization– Agenda:

• Complete the overview of the architecture and interfaces (complete global picture, CSD and CSD API)

• Demo of the system (CSD, CLIC, Webapps, TestClient)– Meeting products:

• 2 CDs with complete documentation and source code for the CSD and the CSD API

• Draft architecture– Post-meeting actions:

• Study available documentation, setup a network and deploy CSD• Refine the high-level architecture of the system• Study CAPEC in light of the understanding of the architecture and

select applicable attack patterns• Find and gather history of past attacks from operations

14

Step 2. Threat Identification

One-page overview

Step 2. Threatidentification

History of system attacksSources of intelligence

Threat statement

OPS, ASIC, Darknets/Blacknets, CAPEC,STRIDE, SANS Top Cyber Security Risks, …

• Misuse and abuse cases (add the time you take for use cases)• Attack resistance analysis

• Distributed architectures, dynamic code generation and interpretation, APIs across stateless protocols, rich internet applications, service-oriented architectures, …

Identifies threats, their level of motivation,their capacities and their likelihood

OPS: OperationsASIC: All-Source Intelligence CentreCAPEC: Common Attack Pattern Enumeration and Classification, http://capec.mitre.orgSTRIDE: Spoofing identity, Tampering with data, Repudiation, Information disclosure, Denial of service, Elevation of privilege,

http://msdn.microsoft.com/en-us/library/ee823878(CS.20).aspx

15PAGE 15

16

Dividing SARA Steps into Meetings

• Meeting #2 (November 9-10):

– Objective: Perform Step 2 – Threat identification

– Agenda:

• Presentation of past attacks

• Develop misuse and abuse cases, being inspired by high-level architecture and selected attack patterns

• Present misuse and abuse cases to lead architect and developer

– Meeting products:

• An Excel spreadsheet with the selected and prioritized attack patterns

• 9 broad misuse and abuse cases

– Post-meeting actions:

• Collect source code and prepare presentations for workshop with MARPAC/JTFP

17

Step 3. Vulnerability Identification

OPS, ASIC, Darknets/Blacknets, seven perniciouskingdoms, 19 deadly sins, OWASP Top 10, CWE,Open Source vulnerability database, CVE,questionnaires, on-site interviews, tools, …

Identifies vulnerabilities and their exploitability

One-page overview

Step 3. Vulnerabilityidentification

History of system attacksSources of intelligence

Security testingresults

Vulnerability statement

Assess evidences that controls are working properly

• Underlying framework weakness analysis• System’s dependencies

• Ambiguity analysis (in requirements and design)• Trust modeling (security zones)• Data sensitivity modeling (privacy and integrity)

Seven pernicious kingdoms: Brian Chess and Gary McGraw’s taxonomy, http://www.cigital.com/papers/download/bsi11-taxonomy.pdf19 deadly sins: Michael Howard’s list, http://blogs.msdn.com/michael_howard/archive/2005/07/11/437875.aspxOWASP: Open Web Application Security Project, http://www.owasp.org/index.php/Category:OWASP_Top_Ten_ProjectCWE: Common Weakness Enumeration, http://cwe.mitre.org/CVE: Common Vulnerabilities and Exposures, http://cve.mitre.org/

18PAGE 18

19

Step 4. Control Analysis

Must be considered for future assessments

Identifies current and future controls and their effectiveness

Assess evidences that controls are working properly

• Controls are technical or non-technical• Controls are preventive or detective• Ambiguity analysis (in requirements and design)

• Trust modeling (security zones)• Data sensitivity modeling (privacy and integrity)

One-page overview

Step 4. Controlanalysis

Security testingresults

Plannedcontrols

List of controls

20

Step 5. Attack Likelihood Determination

Identifies the likelihood of threats exercising vulnerabilities, that is, thelikelihood of attack scenarios

• No black magic: a function of threat and vulnerability likelihood, the latter being dependent on controls• Ambiguity analysis (in requirements and design)

• Threat modeling (attack surface)

Threat statement

Vulnerability statement

List of controls

Step 5. Attack likelihood determination

Attack likelihood rating

21

Step 5. Attack Likelihood Determination

Threat likelihoodHigh Medium Low

High High High Medium

Medium High Medium LowLow Medium Low Low

Vulnerability likelihood

22

Step 6. Impact Analysis

Identifies the magnitude of impacts

• Needs key players: senior management, business operations managers and IT security program managers• Interviews• Impacts can be measured quantitatively or qualitatively

• Examples: lost revenue, maintenance cost, loss of public confidence, …

One-page overview

Step 6. Impactanalysis

Impact rating

23

Step 7. Risk Determination

Identifies the risks with their associated levels

• Again, no black magic: a function of attack likelihood and impact• Associates attacks with impacts

Attack likelihood rating

Impact rating

Step 7. Risk determination

Rated risks

24

Step 7. Risk Determination

Attack likelihoodHigh Medium Low

High High High Medium

Medium High Medium LowLow Medium Low Low

Impact

25

Dividing SARA Steps into Meetings

• Meeting #3 (November 22-25, Technical SARA track: November 22-23):

– Objective (Technical SARA track): Perform Step 3 – Vulnerability identification

– Agenda:

• Test CSD with Fortify 360 SCA tool

• Analyze Fortify report with Audit Work Bench and Fortify 360 server

– Meeting product: A Fortify 360 report, tweaked and analyzed to eliminate false positives that can be eliminated automatically or quickly

– Post-meeting actions:

• Perform Step 4 – Control analysis, to find true and false positives from Fortify report

• Relate misuse and abuse cases to vulnerabilities (true positives)

• Perform Steps 5, 6 and 7 – Attack likelihood determination, impact analysis and risk determination

26

Step 8. Control Recommendations

• Risks prioritization (cost-benefit analysis)• For each risk, recommend one or more new controls or system modifications that will eliminate or mitigate that risk and are appropriate to the organization’s operations

Step 8. Control recommendations

Rated risks

Recommended controls or modifications

27

Step 9. Results Documentation

A report that describes the architecture, the identified threats, the risks with their associated levels, exploited vulnerabilities and impacts, and the final recommendations on controls and modifications to implement

Threat statement

Step 9. Results documentation

Rated risks

Recommended controls or modifications

Architectural Risk Analysis Report

One-page overview

28

Dividing SARA Steps into Meetings

• Meeting #4 (2 days):– Objectives:

• Perform Step 8 – Control recommendations• Prepare Step 9 – Results documentation• Overall debriefing

– Agenda:• Develop mitigation plans for priority misuse and abuse cases• Write final report’s table of contents• Debrief on the whole process

– Meeting products:• Mitigation plans• Final report’s table of contents

– Post-meeting actions:• Perform Step 9 – Results documentation

29

Software Architectural Risk Analysis and Risk Management in Practice

• Cigital’s SARA Methodology (mapped on the processes presented here)

• Microsoft’s

– STRIDE Threat Model (Spoofing identity, Tampering with data, Repudiation, Information disclosure, Denial of service, Elevation of privilege)

• http://msdn.microsoft.com/en-us/library/ee823878(CS.20).aspx

– Security Risk Management Guide

• http://technet.microsoft.com/en-us/library/cc163143.aspx

• Sun’s Adaptive Countermeasure Selection Mechanism/Security Adequacy Review (ACSM/SAR)

– Discussed in Secure Coding: Principles and Practices, Mark G. Graff & Kenneth R. van Wyk, 2003, ISBN 0-596-00242-4.

30

Software Architectural Risk Analysis and Risk Management in Practice• Department of Homeland Security “Build Security In” website,

https://buildsecurityin.us-cert.gov/

• NIST’s

– Recommended Security Controls for Federal Information Systems and Organizations, Special Publication 800-53 Rev. 3, August 2009.

– DRAFT Managing Risk from Information Systems: An Organizational Perspective, Special Publication 800-39, April 2008.

– DRAFT Guide for Applying the Risk Management Framework to Federal Information Systems: A Security Life Cycle Approach, Special Publication 800-37 Rev. 1, November 2009.

– Risk Management Guide for Information Technology Systems, Special Publication 800-30, July 2002.

– http://csrc.nist.gov/publications/PubsSPs.html

31

Software Architectural Risk Analysis and Risk Management in Practice

• SEI’s Operationally Critical Threat, Asset, and Vulnerability Evaluation (OCTAVE)

– http://www.cert.org/octave/

• ISACA’s Control Objectives for Information and Related Technology (COBIT)

– http://www.isaca.org/Template.cfm?Section=COBIT6&Template=/TaggedPage/TaggedPageDisplay.cfm&TPLID=55&ContentID=7981

SEI: Software Engineering InstituteISACA: Information Systems Audit and Control Association

32

Conclusion

• Do not forget:– The threats and the system change over time: SARA and risk management

are ongoing and evolving– Plan for re-evaluation and re-assessment

• Keys for success in SARA:– Senior management’s commitment

• You need time and effort from the key people in the team (lead architect & developer)

• Since SARA should be ongoing, it needs to be integrated in the development cycle

– Competence required in the risk analysis team:• Software architectural risk analysis process, with the flexibility needed

to adapt• Threats and vulnerabilities (the attacker’s perspective)• System and controls (the defender’s perspective)

33