assurance case frameworks part of high confidence software msr t. scott ankrum mitre — software...
TRANSCRIPT
Assurance Case FrameworksPart of
High Confidence Software MSR
T. Scott Ankrum
MITRE — Software Engineering Center
March 11, 2004
March 11, 2004 T. Scott Ankrum — MITRE 2
Credits
• Part of the “High-Confidence Software Initiative” research project
• Supported by the MITRE Sponsored Research program
• Supporting cast– Chuck Howell– Alfred Kromholz – Jim Moore
Working for almost two years.
March 11, 2004 T. Scott Ankrum — MITRE 3
Agenda
• What is an Assurance Case?
• Structuring an Assurance Case
• Problems With Assurance Cases
• Choosing a Tool
• Structuring Selected Standards
• Conclusions and Follow-on
March 11, 2004 T. Scott Ankrum — MITRE 4
What Is an Assurance Case?
March 11, 2004 T. Scott Ankrum — MITRE 5
History of Assurance Cases
• Originally Only Safety Cases– Aerospace– Railways, automated passenger– Nuclear power– Off-shore oil – Defense
• Security Cases– Use compliance rules more than an assurance case
• Cases for Business Critical Systems
March 11, 2004 T. Scott Ankrum — MITRE 6
Definition of Safety Case
• From Adelard’s ASCE manual:
“A documented body of evidence that provides a convincing and valid argument that a system is adequately safe for a given application in a given environment.”
March 11, 2004 T. Scott Ankrum — MITRE 7
Definition of Assurance Case
• Generalizing that definition
A documented body of evidence that provides a convincing and valid argument that a specified set of critical claims regarding a system’s properties are adequately justified for a given application in a given environment.
March 11, 2004 T. Scott Ankrum — MITRE 8
Where is an Assurance Case Used?
– Critical systems under regulation or acquisition constraints
– Third-party certification, approval, licensing, etc.
– Documented body of evidence required
– Need a compelling case that the system satisfies certain critical properties for specific contexts
– Examples: DO-178B, Common Criteria, MIL-STD-882D
– “safety case”, “certification evidence”, “security case”…
Collectively we’ll refer to them as “assurance cases”
March 11, 2004 T. Scott Ankrum — MITRE 9
Structuring an Assurance Case
March 11, 2004 T. Scott Ankrum — MITRE 10
Elements of an Assurance Case
• Claims
• Arguments
• Evidence
• Other elements, depending on notation
March 11, 2004 T. Scott Ankrum — MITRE 11
Claims in Assurance Cases
• Assertion of compliance with key requirements and properties
• Must be in a specific context– Environment
– Services or behavior
– Threats
– “Is this brick safe?” illustrates why…
• Sub-claims may be analogous to “lemmas” in a proof – separation of concerns
– workflow
– makes overall case more manageable
March 11, 2004 T. Scott Ankrum — MITRE 12
Arguments in Assurance Cases
• Link evidence to claims via inference rules– Deterministic: defined rules => true/false assertion
– Probabilistic: quantitative, statistical, numerical threshold (MTTF)
– Qualitative: rules with an indirect link to desired properties (standards, process guides)
• No such thing as perfection: “It is quite possible to follow a faulty analytical process and
write a clear and persuasive argument in support of an erroneous judgment.” – R. Heuer, The Psychology of Intelligence Analysis
March 11, 2004 T. Scott Ankrum — MITRE 13
Evidence in Assurance Cases
• Process and people used to develop the system
• Systematic testing
• Product review and analyses
• Mathematical proofs
None of these alone provides adequate evidence
March 11, 2004 T. Scott Ankrum — MITRE 14
Problems With Assurance Cases
March 11, 2004 T. Scott Ankrum — MITRE 15
Problems with Assurance Cases
• There are problems in every aspect of assurance cases– Building them– Reviewing them– Maintaining them– Reusing them
• Problems result from:– volume of material – little structuring support– ad hoc “rules of evidence”
March 11, 2004 T. Scott Ankrum — MITRE 16
Building the Assurance Case – 1
• Most guidance is:– strong on excruciating detail for format
– weak on gathering, merging, and reviewing evidence
• Guidance often uses the “cast a wide net” tactic– Assurance costs time and money
– “Squandered diagnostic resources”
– Some work on a “portfolio management” approach
March 11, 2004 T. Scott Ankrum — MITRE 17
Building the Assurance Case – 2
• With free format text and no tool support:– coordination is hard
– tracking is hard
– workflow management is hard
• Imagine building a 500 page project plan by hand, on paper
March 11, 2004 T. Scott Ankrum — MITRE 18
Reviewing the Assurance Case – 1
• Stacks of free-format text makes review tedious– Hard to see linkages or patterns– Hides key results in sheer volume
• Weak guidance on review of arguments and evidence often results in ad hoc criteria (be very nice to your reviewer!)
• Rarely is there explicit guidance for weighing conflicting or inconsistent evidence
March 11, 2004 T. Scott Ankrum — MITRE 19
Reviewing the Assurance Case – 2
“Often viewed as irrefutable, evidence is, in fact, an interpretive science, refracted through the varying perspectives of different disciplines. ... [Judging evidence requires] reasoning based on evidence that is incomplete, inconclusive, and often imprecise.”
The Evidential Foundations of Probabilistic Reasoning, David Schum
March 11, 2004 T. Scott Ankrum — MITRE 20
Maintaining the Assurance Case – 1
• The one thing more brittle than software is – the associated assurance case
• It is difficult to understand impact of a change on assurance structure because:– volume of information is immense
– impact of a change on assurance structure is complex
March 11, 2004 T. Scott Ankrum — MITRE 21
Maintaining the Assurance Case – 2
• Reasons for change– The claims and/or evidence have changed
– Arguments no longer valid or new ones needed
– Evidence is irrelevant or new evidence needed
– “Weak link effect” of discrete systems compounds problem
• Revalidation costs are a major burden
• “Breakage” of successive dependencies
March 11, 2004 T. Scott Ankrum — MITRE 22
Reusing the Assurance Case – 1
• Assurance case frameworks are rarely the subject of study per se
• More attention for these would be useful– tool support– idioms and templates– extracting patterns for future use
March 11, 2004 T. Scott Ankrum — MITRE 23
Reusing the Assurance Case – 2
• Relationship among claims, arguments, and evidence – not often explicit – hard to distinguish the reusable from the
project specific portions of assurance case
• Compare this with building a deck with the help of a project planning tool
March 11, 2004 T. Scott Ankrum — MITRE 24
Choosing a Tool
March 11, 2004 T. Scott Ankrum — MITRE 25
What Should a Tool Provide? – 1
• Simple management of complexity and volume– MS Project-like planning and tracking of complexities– Checking simple structural properties– Browsing and report generation
• Support for multiple, geographically dispersed users– with data integrity– concurrently or asynchronously
• Useable for any domain– not specific to any one industry– not specifically for safety cases or security cases
March 11, 2004 T. Scott Ankrum — MITRE 26
What Should a Tool Provide? – 2
• Replanning as things change: (“No plan survives contact with the enemy.”)
• Templates and tailoring to – capture lessons learned – reduce wheel reinvention
• Uses and/or exchanges consistent notation for: – claims, evidence, and arguments
• Widely executable– runs under Windows 2000 or Windows XP– or has a Windows based GUI
March 11, 2004 T. Scott Ankrum — MITRE 27
Notations Considered
• Toulmin Structures– Stephen Toulmin, The Uses of Argument, 1958
• Goal Structuring Notation– Described in Tim Kelly’s dissertation, York, 1998
• ASCAD (Claims-Arguments-Data): – ESPRIT SHIP project headed by Adelard
• Proprietary
March 11, 2004 T. Scott Ankrum — MITRE 28
Selected Tool: ASCE
• Established Notations: GSN & ASCAD
• Not Industry or Safety Specific
• Extensible through a Schema
• Case is exportable to project documents
• Stable, no failures during evaluation
ASCAD Notation
Supports Supports
Is evidence for
Is evidence for
Is a subclaim of Is a subclaim ofIs a subclaim of
Is evidence forIs evidence for
SupportsSupportsSupports
Is evidence for
CLAIMRequirements Are
Correct, Complete andConsistent
ARGUMENTA formal specificationadds rigor and exposes
defects early
ARGUMENTFagan's two famouspapers make a good
argument forinspections, and theyare a proven practice.
CLAIMSoftware Has Few
Defects
ARGUMENTRequirements areonly useful if they
are used todesign the
system.
EVIDENCETest Plan Peer
Review statistics
EVIDENCERequirements
Traced to DesignComponents
EVIDENCEFagan Inspectionof each section
EVIDENCEWritten in Z
EVIDENCEFilled in test Plan
ARGUMENTTest scenarios were
marked with thecompletion date to
document that all testswere eventually
successful.
CLAIMSystem Was Well Tested
CLAIMSystem Built toRequirements
ARGUMENTPeer reviews are a
good way to increasquality and the only
available way fortest plans.
March 11, 2004 T. Scott Ankrum — MITRE 30
Structuring Selected Standards
March 11, 2004 T. Scott Ankrum — MITRE 31
Hypotheses
• Assurance is Assurance is Assurance– All assurance cases are similar enough in structure
that a distinct tool for each domain is not required
• Assurance Standard Assurance Case– There is a relationship between the actual or
implied structure of an assurance standard and the structure of an assurance case instantiated from that standard
March 11, 2004 T. Scott Ankrum — MITRE 32
Mapping Standards into ASCE
• Computer Security:– Common Criteria — Evaluation Assurance Level 4
• Aviation Safety: DO-178B – Software Considerations in Airborne Systems
• Medical Device Safety:– Discussing with FDA
Center for Devices & Radiological Health
Top-levelClaim
SupportingClaim
Needs supportingclaims
ArgumentunderParent
Evidenceto support
a claim
Argumentfor Sub-claims
Argumentfor
Evidence
ParentClaim
Start
Finished
March 11, 2004 T. Scott Ankrum — MITRE 34
Process Mechanics
• ASCAD notation: – Claims– Arguments– Evidence
• We used arguments between claims– This is a deviation from the notation
• Tried to capture all of the standard
March 11, 2004 T. Scott Ankrum — MITRE 35
Advantages of the Tool
• Carries both graphic structure and text
• Hyperlinks from node to a web page or file
• Enforces structure rules– Rules can be temporarily suspended– User-supplied rules can be added
• Can export for inclusion in a document
• User views can show parts of the structure
March 11, 2004 T. Scott Ankrum — MITRE 36
Mapping the Common Criteria
• Most hierarchical of the standards– Classes, Families, Components, Requirements– Components are atomic and cumulative
• Nearly mechanical process of mapping• Most of the structure consists of Arguments
– No sub-claims, only a top-level claim– Requirements are place-holders for evidence
• Objectives paragraphs became arguments
Supports
Supports
Supports
Supports
Supports
Supports
Supports
Supports
Supports
Supports
SupportsSupports
Supports Supports
SupportsSupports
SupportsSupports
Supports Supports
Supports Supports
Supports
Supports Supports
SupportsSupports
Supports
Supports
Supports
CLAIMEAL4
[Confidence in Securitybecause the product has
been] methodically designed,tested, and reviewed
ARGUMENTACM
ConfigurationManagement
ARGUMENTACM_AUT
CM Automation
ARGUMENTACM_CAP
CM Capabilities
ARGUMENTACM_SCPCM Scope
ARGUMENTADO
Delivery andOperation
ARGUMENTADO_DELDelivery
ARGUMENTADO_IGS
Installation,Generation and
Start-up
ARGUMENTADV
Development
ARGUMENTADV_FSP
Development
ARGUMENTADV_HLD
High-Level Design
ARGUMENTADV_IMP
ImplementationRepresentation
ARGUMENTADV_LLD
Low-Level Design
ARGUMENTADV_RCR
RepresentationCoorespondence
ARGUMENTADV_SPM
Security PolicyModeling
ARGUMENTAGD
Guidance Documents
ARGUMENTAGD_ADM
AdministratorGuidance
ARGUMENTAGD_USR
User Guidance
ARGUMENTALC
Life Cycle Support
ARGUMENTALC_DVS
Development Security
ARGUMENTALC_LCD
Life Cycle Definition
ARGUMENTALC_TATTools and
Techniques
ARGUMENTATE
Tests
ARGUMENTATE_COVCoverage
ARGUMENTATE_DPT
Depth
ARGUMENTATE_FUN
Functional Tests
ARGUMENTATE_IND
Independent Testing
ARGUMENTAVA
VulnerabilityAssessment
ARGUMENTAVA_MSUMisuse
ARGUMENTAVA_SOF
Strength of TOESecurity Functions
ARGUMENTAVA_VLA
VulnerabilityAnalysis
Supports Supports
SupportsSupports
Supports Supports
Is evidence forIs evidence for
Supports Supports
Is evidence for Is evidence for
Supports Supports
Is evidence for Is evidence for
ARGUMENTATE
Tests(50 words)
ARGUMENTATE_COVCoverage
ARGUMENTATE_COV.1Coverage(31 words)
ARGUMENTATE_COV.2Analysis ofCoverage(35 words)
EVIDENCEATE_COV.1.Coverage(53 words)
EVIDENCEATE_COV.2.2
Analysis ofCoverage(27 words)
ARGUMENTATE_DPT
Depth
ARGUMENTATE_FUN
Functional Tests(92 words)
ARGUMENTATE_DPT.1
Testing: High-LevelDesign
(43 words)
ARGUMENTATE_FUN.1
Functional Testing(27 words)
EVIDENCEATE_DPT.1.
Testing: High-LevelDesign
EVIDENCEATE_FUN.1.
Functional Testing(87 words)
ARGUMENTATE_IND
Independent Testing(53 words)
ARGUMENTATE_IND.1
IndependentTesting
(15 words)
ARGUMENTATE_IND.2
IndependentTesting
EVIDENCEATE_IND.1.Independent
Testing(51 words)
EVIDENCEATE_IND.2.Independent
Testing(41 words)
March 11, 2004 T. Scott Ankrum — MITRE 39
Mapping DO-178B
• Less structured, its title begins:– “Software Considerations …”
• Focused on system/software product lifecycle– Other standards are not time-structured– Claims, sub-claims, and evidence are laid out in
approximately their chronological order – No linkages between the generation of one
artifact and its later use
Supports
Is evidence for
Is evidence for
Supports Supports
Supports
Supports
Supports
Is a subclaim ofIs a subclaim ofIs a subclaim of
Supports
Supports
Is a subclaim ofIs a subclaim ofIs a subclaim of
Supports
Is evidence for
Supports SupportsSupports
Is evidence for
Supports Supports Supports
Is evidence for
Is evidence for
Is evidence for
Is a subclaim of
Is a subclaim ofIs a subclaim of
Is a subclaim of
Is a subclaim of
Is a subclaim of
Supports
CLAIM7.2.2
Baselines &traceability
are
CLAIM7.2.3 - 7.2.6Problem &Change
Management is
CLAIM7.2.8
Softwareload control
is
CLAIM7.2.9
Software lifecycle
environment
CLAIM7.2.1
Configurationitems areidentified
ARGUMENTnot explicit
Satisfactory SQAprocess requires
three characteristics(23 words)
CLAIM7.0
SCM process isproperly
established and
CLAIM8.0
SQA process isproperly
established and
ARGUMENTnot explicitSatisfactory
Certification LiaisonProcess comprises
three factors
CLAIM7.2.7
Archive,retrieval andrelease are
EVIDENCE11.17
ProblemReports
(0 words)
EVIDENCE11.15
Software LifecycleEnvironmentConfiguration
EVIDENCE11.18SCM
Records(0 words)
ARGUMENTnot explicit
SCM Records containevidence for the six areas
(42 words)
CLAIM8.1a
Softwaredevelopment &
integral processescomply with
CLAIM8.1b
Transition criteriafor software
lifecycleprocesses are
CLAIM8.1c, 8.3Software
conformityreview is
ARGUMENTnot explicit
SQA Records containmaterial for three areas
EVIDENCE11.19SQA
Records
CLAIM9.0
Relevantcommunication
&
CLAIM9.1
Means ofcomplianceis agreed to
CLAIM9.2
Compliancesubstantiation
is provided
EVIDENCE11.16
SoftwareConfiguration
Index(0 words)
EVIDENCE11.1
Plan for SWAspects of
Certification
EVIDENCE11.20SW
AccomplishmentSummary
ARGUMENTnot explicit
Plan for SWAspects of
Certification
ARGUMENT11.20
Substantiation ofCompliance isprovided by two
ARGUMENTnot explicit
satisfactory SCMprocess includes six
elements(37 words)
CLAIM9.0
CertificationLiaison Process isproperly defined
March 11, 2004 T. Scott Ankrum — MITRE 41
Mapping ISO 14971
• Accompanying amendment is essential for mapping into ASCE
• No structural relation between the document and the assurance case
• Claims, arguments, and evidence identified by analyzing words and phrases
• Very few arguments for evidence• For Each Identified Hazard…
Supports SupportsSupports
Is evidence for Is evidence for
SupportsIs evidence for
Supports
Supports
Is a subclaim of
SupportsSupports
Is a subclaim of
Supports
Supports
Supports
Supports
Supports
Supports
Supports
Supports
Is a subclaim of
Is evidence for
Is evidence for
Is evidence for
Is evidence for
Is evidence for
Supports
Supports
Is a subclaim of
Supports
Supports
Is a subclaim of
SupportsSupports
Is evidence for
Is evidence for
Is evidence for
ARGUMENTH.4
Rationale for clause 4,Risk analysis
ARGUMENTH.4.2
Intended use/intendedpurpose and
identification of
ARGUMENTNOTE 2
EVIDENCEAnnex B
Guidance on risk analysisfor in vitro diagnostic
medical devices
ARGUMENTNOTE 3
ARGUMENTNOTE1
EVIDENCEAnnex D
Examples ofpossible
hazards andcontributing
factorsassociated with
medicaldevices
ARGUMENTNOTE 2& H.4.3
CLAIM4.2
Intended use/intendedpurpose and
identification of
ARGUMENTH.4.3
Identification of known orforeseeable hazards
CLAIM4.3
Identification of known orforeseeable hazards
(Step 2)
CLAIM4.3
Foreseeablesequences of eventsthat may result in a
CLAIM4.4
Estimation of therisk(s) for each hazard
(Step 3)
CLAIM4.1
Risk analysisprocedure
ARGUMENTH.4.4
Estimation of therisk(s) for each
hazard
ARGUMENT4.2
Compliance is checked byinspection of the risk
management file.
EVIDENCE3.6
RiskManagement
File
ARGUMENT4.3
Compliance is checkedby inspection of the risk
management file.
ARGUMENTH.4.4
Estimation of therisk(s) for each
hazard
ARGUMENTNOTE 3
EVIDENCEAnnex F
Information on riskanalysis techniques
ARGUMENT4.4
EVIDENCEAnnec C
Guidance on riskanalysis procedure fortoxicological hazards
ARGUMENT4.4
Compliance is checked byinspection of the risk
management file.
EVIDENCE3.6
Risk Management File
EVIDENCE3.6
Risk ManagementFile
CLAIM4
Risk analysis (Steps1, 2 and 3 of Figure
2)
ARGUMENTH.4.1
Risk analysisprocedure
EVIDENCEAnnex A
Questions that can beused to identify medical
device characteristics thatcould impact on safety
ARGUMENTNOTE 1
ARGUMENT4.1
Compliance ischecked by inspection
of the risk
EVIDENCE3.6
Risk ManagementFile
EVIDENCE3.6
RiskManagement File
ARGUMENTNOTE 4
March 11, 2004 T. Scott Ankrum — MITRE 43
Validating Our Mappings
• Domain experts reviewed our mappings– Common Criteria
• System security experts within MITRE
– DO-178B• Evaluator (FAA Designated Engineering Representative)
– ISO 14971• FDA CDRH
• Varying conclusions from validations
March 11, 2004 T. Scott Ankrum — MITRE 44
Conclusions and Follow-on
March 11, 2004 T. Scott Ankrum — MITRE 45
Ada Lovelace
March 11, 2004 T. Scott Ankrum — MITRE 46
Hypotheses Revisited
• Assurance Standard Assurance Case– There does not seem to be much of a
relationship between the two structures– Experience with actual assurance supports this
• Assurance is Assurance is Assurance– Negation of the above hypothesis prevents us
from coming to any conclusion on this one
March 11, 2004 T. Scott Ankrum — MITRE 47
Standards Templates
• Mappings might be used as templates– Could be a side benefit of the study– Without structural relation, possibility looks bad
• Advantages of consistency may help drive assurance-requirements standardization– Currently, hard to “compare apples and oranges”– Evaluation of assurance claims easier if
requirements are consistent
March 11, 2004 T. Scott Ankrum — MITRE 48
Extensions to Tool
• Extend ASCE features to be more helpful
• Make ACSE more generic
• Enhance possibilities for user customization
March 11, 2004 T. Scott Ankrum — MITRE 49
Shadow a Real Project
• Activities– Document a real process
– Identify where and how to incorporate technique
• Advantages– Learning opportunity for us– Minimal impact on the project– Not in the project’s critical path
March 11, 2004 T. Scott Ankrum — MITRE 50
Develop Training
• How to use the notation and notation options
• How to develop a structured assurance case
• How changes affect the assurance case– Software, hardware– Operation, environment
• How to write a structured assurance standard
March 11, 2004 T. Scott Ankrum — MITRE 51
Use on a Real Project
• Apply methdology within a project’s schedule
• Gain experience with maintenance of assurance cases
• Update process with lessons learned
• Propagate this knowledge to other projects
March 11, 2004 T. Scott Ankrum — MITRE 52
Discussion