best practices for delivering safe, secure, and dependable mission capabilities paul r. croll chair,...

Best Practices for Delivering Safe, Secure, and Dependable

Mission Capabilities

Paul R. CrollChair, IEEE Software and Systems Engineering Standards Committee

Convener, ISO/IEC JTC1/SC7 WG9, System and Software Integrity

Computer Sciences [email protected]

SSTC 2005, Track 7, Monday, April 18 2005, 3:45 PM - 4:30 PM ©2005 Paul R. Croll, All rights reserved Slide 2

Agenda

The Scope of System and Software Assurance A Best Practice Approach for Delivering Safe,

Secure, and Dependable Mission Capabilities Mission Capability Requirements for Assurance Assurance in the Context of the Life Cycle The CMMI and Assurance Standards Supporting System and Software

Assurance Implementing Assurance Processes


System and software assurance focuses on the management of risk and assurance of safety, security, and dependability within the context of system and software life cycles.Terms of Reference: ISO/IEC JTC1/SC7 WG9, System and Software Integrity

System and software assurance focuses on the management of risk and assurance of safety, security, and dependability within the context of system and software life cycles.Terms of Reference: ISO/IEC JTC1/SC7 WG9, System and Software Integrity

The Scope of System and Software Assurance


A Best Practice Approach for Delivering Safe, Secure, and Dependable Mission Capabilities

2. Look to Framework Standards for

Assurance Objectives

3. Look to the CMMI for

Assurance-Related Process Capability

Expectations

4. Look to Supporting

Standards for Assurance

Process Detail

1. Understand Your Mission Capability Requirements for

Assurance

5. Build or Refine and Execute Your

Assurance Processes



Assurance

Mission Capability Requirements for Assurance

What are your mission capability requirements for System and Software Assurance?

• Business process requirements

• Tactical doctrine requirements

• Legal and regulatory requirements

• Customer-specific requirements

• Product-specific requirements


How Does Assurance Fit in the System and Software Life Cycles?

How Does Assurance Fit in the System and Software Life Cycles?

Assurance and the Life Cycle


Life Cycle Process Framework Standards

System Life Cycle ISO/IEC 15288, Systems engineering — System life cycle processes

Software Life Cycle ISO/IEC 12207, Standard for Information Technology —Software life

cycle processes IEEE/EIA 12207.0, Industry Implementation of International Standard

ISO/IEC12207:1995 — (ISO/IEC 12207) Standard for Information Technology —Software life cycle processes

IEEE/EIA 12207.1, Industry Implementation of International Standard ISO/IEC12207:1995 — (ISO/IEC 12207) Standard for Information Technology —Software life cycle processes – Life Cycle Data

IEEE/EIA 12207.2, Industry Implementation of International Standard ISO/IEC12207:1995 — (ISO/IEC 12207) Standard for Information Technology —Software life cycle processes – Implementation considerations


Assurance in the ISO/IEC 15288 System Life Cycle Process Framework

SYSTEM LIFE CYCLE

PROJECT ASSESSMENT

PROJECT PLANNING

PROJECT CONTROLDECISION MAKING

RISK MANAGEMENTCONFIGURATION MANAGEMENT

INFORMATION MANAGEMENT

ENTERPRISE(5)

SYSTEM LIFE CYCLE MANAGEMENT

RESOURCE MANAGEMENT

QUALITY MANAGEMENT

ENTERPRISE ENVIRONMENT MANAGEMENT

INVESTMENT MANAGEMENT

TECHNICAL (11)

PROJECT (7)

ACQUISITION

SUPPLYAGREEMENT (2)

TRANSITIONSTAKEHOLDER REQUIREMENTS DEFINITION

REQUIREMENTS ANALYSISARCHITECTURAL DESIGN

IMPLEMENTATIONINTEGRATION

VERIFICATION

VALIDATIONOPERATION

MAINTENANCEDISPOSAL

(25)

Safety, Security, Integrity


Assurance in the IEEE/EIA 12207 Software Life Cycle Process Framework

SOFTWARE LIFE CYCLE

TAILORING

CONFIGURATION MANAGEMENTDOCUMENTATION

QUALITY ASSURANCEVERIFICATION

VALIDATIONJOINT REVIEW

AUDITPROBLEM RESOLUTION

PRIMARY (5)

DEVELOPMENT

OPERATION

MAINTENANCE

ACQUISITION

SUPPLY

ORGANIZATIONAL (4)MANAGEMENT

INFRASTRUCTUREIMPROVEMENT

TRAINING

SUPPORTING (8)

Adapted from: Raghu Singh, An Introduction to International Standards ISO/IEC 12207, Software Life Cycle Processes, 1997.

(17+1)

Safety, Security, Integrity

ISO/IEC 16085Risk Management


How do we know what the process objectives should be for System

and Software Assurance?

How do we know what the process objectives should be for System

and Software Assurance?

2. Look to Framework Standards for Assurance

Objectives

Life Cycle Assurance Objectives


ISO/IEC 15288 – System Assurance Objectives – 1

Project Planning Process

Establish the structure of authorities and responsibilities for project work, including, as appropriate, the legally responsible roles and individuals, e.g. safety authorization, award of certification or accreditation.

Configuration Management Process

Define a configuration management strategy, in accordance with designated levels of integrity, security and safety.

Source: ISO/IEC 15288:2002, © ISO/IEC2002.




Stakeholder Requirements Definition Process Specify health, safety, security, environment and other

stakeholder requirements and functions that relate to critical qualities; Identify safety and security risk.

Requirements Analysis Process Specify system requirements and functions that relate to

critical qualities, such as health, safety, security, reliability, availability and supportability.



Architectural Design Process Determine which system requirements are allocated

to operators, e.g., human actions critical to safety.Implementation Process Realize or adapt system elements with regard to

standards that govern applicable safety, security, privacy and environmental guidelines.




Integration Process Obtain system elements in accordance with agreed

schedules. System elements are handled in accordance with relevant health, safety, security and privacy considerations.

Verification Process Define the strategy for verifying the systems

throughout the life cycle. The nature and scope of the verification action depends on the perceived risks, e.g. safety.




Transition Process

Prepare the site of operation in accordance with installation requirements. Site preparation is conducted in accordance with applicable health, safety, security and environmental regulations.

Validation Process Define the strategy for validating the services in the

operational environment and achieving stakeholder satisfaction. The nature and scope of the validation action depends on risks, e.g. safety.




Operation Process Activities

Monitor operation to ensure that the system is operated in accordance with the operations plans, in a safe manner and compliant with legislated guidelines concerning occupational safety and environmental protection, and environmental regulations.




Maintenance Process

Prepare a maintenance strategy, including the skill and personnel levels required to effect repairs and replacements, accounting for maintenance staff requirements and any relevant legislation regarding health and safety, security and the environment.




Disposal Process

Define a disposal strategy for the system, to include each system element and any resulting waste products, including schedules, actions and resources that take account of the health, safety, security and privacy applicable to disposal actions and to the long term condition of resulting physical material and information.




Disposal Process (Cont.)

Deactivate the system to prepare it for removal from operation; Withdraw operating staff from the system and record relevant operating knowledge; Remove the system from the operational environment for reuse, recycling, reconditioning, overhaul or destruction; in accordance with relevant safety, and security, legislation and standards.




Disposal Process (Cont.)

Confirm that no detrimental health, safety, security and environmental factors exist following disposal

Archive information gathered through the lifetime of the system to permit audits and reviews in the event of long-term hazards to health, safety, security




Tailoring

Identify and document the circumstances that influence tailoring, including integrity issues such as safety, security, privacy, usability, availability.



IEEE/EIA 12207 – Software Assurance Objectives – 1

Acquisition Process

The acquirer will define and analyze the system requirements. The system requirements should include business, organizational and user as well as safety, security, and other criticality requirements along with related design, testing, and compliance standards and procedures.

Source: IEEE/EIA 12207.0-1997, © IEEE 2001.



Supply Process

The supplier shall develop and document project management plan(s) . . . including management of safety, security, and other critical requirements of the software products or services. Separate plans for safety and security may be developed.




Development Process

The developer shall develop plans for conducting the activities of the development process. The plans should include specific standards, methods, tools, actions, and responsibility associated with the development and qualification of all requirements including safety and security.




Development Process (Cont.)

The system requirements specification shall describe: functions and capabilities of the system; business, organizational and user requirements; safety, security, human-factors engineering (ergonomics), interface, operations, and maintenance requirements; design constraints and qualification requirements.




Development Process (Cont.)

The developer shall establish and document software requirements, including quality characteristics specifications such as safety specifications, including those related to methods of operation and maintenance, environmental influences, and personnel injury; and security specifications, including those related to compromise of sensitive information




Maintenance Process

The maintainer shall analyze the problem report or modification request for its impact on the organization, the existing system, and the interfacing systems with respect to criticality; e.g., impact on performance, safety, or security.




Configuration Management Process

Control and audit of all accesses to the controlled software items that handle safety or security critical functions shall be performed. The code and documentation that contain safety or security critical functions shall be handled, stored, packaged, and delivered in accordance with the policies of the organizations involved.




Verification Process

The requirements shall be verified to demonstrate that software requirements related to safety, security, and criticality are correct as shown by suitably rigorous methods.

The design and code shall be verified to demonstrate that they implement safety, security, and other critical requirements correctly as shown by suitably rigorous methods.




Infrastructure Process

The configuration of the infrastructure should be planned and documented. Functionality, performance, safety, security, availability, space requirements, equipment, costs, and time constraints should be considered.

Source: ISO/IEC 15288:2002, © ISO/IEC2002. Source: IEEE/EIA 12207.0-1997, © IEEE 2001.



Tailoring (to projects)

Determine which organizational policies are relevant and applicable, such as on computer languages, safety and security, hardware reserve requirements, and risk management

Source: ISO/IEC 15288:2002, © ISO/IEC2002. Source: IEEE/EIA 12207.0-1997, © IEEE 2001.


How does the CMMI support System and

Software Assurance?

How does the CMMI support System and

Software Assurance?

3. Look to the CMMI for Assurance-Related Process

Capability Expectations

The CMMI and Assurance


CMMI – Assurance Objectives - 1

Project Planning

Determine the technical approach for the project, including the functionality expected in the final products, such as safety and security

Estimate effort and cost using models and/or historical data including inputs related to level of security required for tasks, work products, hardware, software, personnel, and work environment.

Source: CMMI -SE/SW/IPPD/SS, V1.1, Continuous Representation, © CMU SEI,2002.


Project Planning (Cont.)

Plan for the management of project data including data supporting safety.

Establish requirements and procedures to ensure privacy and security of the data.

Project Monitoring and Control

Monitor resources provided and used, including the security environment

Collect and analyze issues and determine the corrective actions necessary to address the issues, including security issues.




Supplier Agreement Management

Evaluate the impact of candidate COTS products on the project's plans and commitments, including security requirements

Risk Management

Identify the risks associated with cost, schedule, and performance in all appropriate product life-cycle phases, including risks associated with maintaining safety and security performance.




Requirements Development

Analyze needs and requirements for each product life-cycle phase, including factors that reflect overall customer and end-user expectations and satisfaction, such as safety, security, and affordability.

Ensure that the design adheres to applicable design standards and criteria, including safety standards.




Technical Solution

Design comprehensive product-component interfaces in terms of established and maintained criteria, including safety and security.

Adhere to applicable standards and criteria, including safety standards.

Train the people performing or supporting the technical solution process as needed, including safety standards.




Product Integration

Satisfy the applicable requirements and standards for packaging and delivering the product, including those for safety and security.

Configuration Management

Perform reviews to ensure that changes have not compromised the safety and/or security of the system.




Decision Analysis and Resolution

Establish and maintain guidelines to determine which issues are subject to a formal evaluation process. For example, on design-implementation decisions when technical performance failure may cause a catastrophic failure (e.g., safety of flight item).




Organizational Environment for Integration

Plan, design, and implement an integrated work environment, including tradeoff of safety and security costs and benefits.

Causal Analysis and Resolution

Determine which defects and other problems will be analyzed further, including safety impact considerations.




Safety and Security Extensions for Integrated Capability Maturity Models

Source: United States Federal Aviation Administration, Safety and Security Extensions for Integrated Capability Maturity Models, September 2004

9. Determine Regulatory Requirements, Laws, and Standards

10. Develop and Deploy Safe and Secure Products and Services

11. Objectively Evaluate Products12. Establish Safety and Security

Assurance Arguments 13. Establish Independent Safety and

Security Reporting14. Establish a Safety and Security Plan15. Select and Manage Suppliers,

Products, and Services16. Monitor and Control Activities and

Products

1. Ensure Safety and Security Competency

2. Establish Qualified Work Environment

3. Ensure Integrity of Safety and Security Information

4. Monitor Operations and Report Incidents

5. Ensure Business Continuity6. Identify Safety and Security Risks7. Analyze and Prioritize Risks8. Determine, Implement, and Monitor

Risk Mitigation Plan


CMMI Process Areas Supporting Assurance Extensions

Process Management Organizational Process Focus Organizational Training Organizational Innovation and

Deployment

Project Management Project Planning Project Monitoring and Control Supplier Agreement Management Integrated Project Management Risk Management Integrated Supplier Management Quantitative Project Management

Engineering Requirements Management Requirements Development Technical Solution Product Integration Verification Validation

Support Configuration Management Process and Product Quality

Assurance Measurement and Analysis Organizational Environment for

Integration

Source: United States Federal Aviation Administration, Safety and Security Extensions for Integrated Capability Maturity Models, September 2004


What Standards Support System and Software Assurance?

What Standards Support System and Software Assurance?



Process Detail

Supporting Standards for Assurance


Standards Organizations Supporting Assurance

ISO IEC

JTC1TC176

SC1 SC22

Terminology Software Engineering Language, OS

SC7

TC56 SC65A

Quality Information Technology Dependability Functional Safety

SC27

IT Security Techniques

S2ESC IASC

Software and Systems Engineering

Information Assurance

IEEE CSISO

IEC

IEEE CS

NIST

FISMA Projects

U.S. Gov’t

DoD MIL-STDsPolicy Memos


Dependability Standards

Adapted from James W. Moore, Software Engineering Standards: A User's Road Map, IEEE Computer Society Press, Los Alamitos, CA, 1997

Risk Management

IEC 812Failure mode andeffects analysis

IEC 1025Fault tree analysis

IEC 300-2Programme

elements & tasks

ISO/IEC 15026Integrity levels

IEC 300-3-9Risk analysis of

technological sys

IEC 300-3-6SW aspects ofdependability

IEC 300-1Programme

management

Achieving ConfidenceRisk Analysis Risk Control

IEC 50-191Dependability

vocabulary

ISO/IEC 16085Risk Management

ISO/IEC NWI 61720Tech. & tools for

confidence

ISO/IEC 15288System life cycle

processes ISO/IEC 12207SW life cycle

processes

ISO

IEC


Safety and Security Standards

IEC 61508Functional Safety

Sector-Specific Standards

ISO/IEC 9796Digital Security

Schemes

ISO/IEC 10181Security

frameworks for open systems

ISO/IEC 15408Common Criteria for IT

Security Evaluation

ISO/IEC 21827Systems Security Engineering CMM

IEEE P1619 Standard Architecture for Encrypted Shared

Storage Media

IEEE P2200 Baseline Operating

System Security

IEEE 1228SW safety plans

Safety

Security

IEEE P1700Security Architecture for

Certification and Accreditation of

Information

Military

IEC

IEEE CS

ISO

IEEE CS

IEC 60880SW in nuclear power safety

systems

MIL-STD-882DStandard Practice for

System Safety

DO 178BSW considerations in

airborne equip certification

ISO/IEC 17799Code of Practice for Information Security

Management

RTCA

Military Standards

DEF STAN 00-56Safety Management Requirements for Defence Systems

P1667Standard Protocol for Authentication in Host

Attachments of Transient

Storage Devices

P2600Standard for Information Technology: Hardcopy

System and Device Security

IEEE CSUnder

Development

ISO/IEC 13335Management of information and communications

technology security


DHS

Harmonization Efforts Impacting Systems and Software Assurance

ISO IEC

JTC1TC176

SC1 SC22

Terminology Software Engineering Language, OS

SC7

TC56 SC65A

Quality Information Technology Dependability Functional Safety

SC27

IT Security Techniques

S2ESC IASC

Software and Systems Engineering

Information Assurance

IEEE CSISO

IEC

IEEE CS

NIST

FISMA Projects

U.S. Gov’t

DoDMIL-STDs

Policy Memos

Who’s Collaborating


Harmonization Efforts Impacting Systems and Software Assurance

What’s Being Harmonized

IEEE/EIA 12207SW life cycle

processes

IEEE 15288System life cycle

processes

IEC Dependability

and Safety Standards ISO/IEC 16085

Risk Management

ISO/IEC 15288System life cycle

processes

IEEE CS Supporting Standards

ISO/IEC 12207SW life cycle

processes

ISO/IEC 15026System and

Software Assurance

IEC Security Standards

•Requirements•Design•V&V•Test•Risk Management•Acquisition•Architecture• •


FISMA Legislation

“Each Federal agency shall develop, document, and implement an agency-wide information security program to provide information security for the information and information systems that support the operations and assets of the agency, including those provided or managed by another agency, contractor, or other source…”

- Federal Information Security Management Act of 2002

Source: FISMA Implementation Project, Dr. Ron Ross, NIST, April 2004


NIST Enterprise Risk Management Framework


In system security plan, provides a an overview of the security requirements for

the information system and documents the security controls planned or in place

SP 800-18

Security Control Documentation

Defines category of information system according to potential

impact of loss

FIPS 199 / SP 800-60

Security Categorization

Selects minimum security controls (i.e., safeguards and countermeasures) planned or

in place to protect the information system

SP 800-53 / FIPS 200

Security Control Selection

Determines extent to which the security controls are implemented correctly, operating as intended, and producing desired outcome with respect to meeting security requirements

SP 800-53A / SP 800-37

Security Control Assessment

SP 800-53 / FIPS 200 / SP 800-30

Security Control Refinement

Uses risk assessment to adjust minimum control set based on local conditions, required threat coverage, and specific agency requirements

SP 800-37

System Authorization

Determines risk to agency operations, agency assets, or individuals and, if acceptable,

authorizes information system processing

SP 800-37

Security Control Monitoring

Continuously tracks changes to the information system that may affect security controls and

assesses control effectiveness

Implements security controls in new or legacy information systems; implements security configuration

checklists

Security Control Implementation

SP 800-70

Starting Point


FISMA Implementation Project Standards and Guidelines

FIPS Publication 199 (Security Categorization) NIST Special Publication 800-37 (Certification &

Accreditation) NIST Special Publication 800-53 (Security Controls) NIST Special Publication 800-53A (Assessment) NIST Special Publication 800-59 (National Security) NIST Special Publication 800-60 (Category Mapping) FIPS Publication 200 (Minimum Security Controls)




Assurance Processes


Assurance

Implement, Execute, and Reassess:Do your assurance processes still meet your mission capability requirements for System and Software Assurance?

• Business process requirements

• Legal and regulatory requirements

• Marketplace requirements

• Customer-specific requirements

• Product-specific requirements

Use Standards-Based Best Practices to Implement Assurance Processes


A Best Practice Approach for System and Software Assurance

2. Look to Framework Standards for

Assurance Objectives

3. Look to the CMMI for

Assurance-Related Process Capability

Expectations



Process Detail


Assurance


Assurance Processes


For More Information . . .

Paul R. CrollComputer Sciences Corporation5166 Potomac DriveKing George, VA 22485-5824

Phone: +1 540.644.6224Fax: +1 540.663.0276e-mail: [email protected]

For IEEE Standards:http://computer.org/standards/sesc/http://ieeeia.org/iasc/http://computer.org/cspress/CATALOG/st01110.htm

For ISO/IEC Standards:http://saturne.info.uqam.ca/Labo_Recherche/Lrgl/sc7/

best practices for delivering safe, secure, and dependable mission capabilities paul r. croll chair,...

Documents

system assurance objectives

software life cycles

context of system

assurance of safety

ieee software

international standards

assurance fit

isoiec jtc1sc7 wg9