sehc 2012 despotou

© The University of York

Introducing Safety Cases for Medical Software

George Despotou


Overview

•  Background •  Introduction to safety cases

– What is a safety case – Graphical safety cases

•  Creating a safety case


Medical Software

•  From small medical devices – … to large scale IT systems

•  …and interconnected systems –  Electronic patient management (hospitals) – Data factories –  Electronic prescription systems –  Electronic health record

•  Benefits are often obvious – …but software systems may have safety

implications


Risk vs. Benefits

•  Benefits overshadow risks •  Risk vs. risk trade-offs

– Systems may help save lives •  Recording of risk

– System contribution to safety similar to any other domain

– …and other trade-offs •  Uncertainty

– Need to understand and explicitly capture risk


Regulatory standards in the UK

•  Information Standards Board for health and social care

•  0129 – manufacturer •  0160 – operator

•  They ask for a safety case


What is a safety case

•  Argument and evidence •  Communicates a position about the system •  Argument

–  A series of connected claims about the system •  Inferences

•  Evidence –  Testing artefacts, reviews, proofs

•  Any information that can support a claim about the system

•  Clear, comprehensible, defensible


Evolution of an Argument •  Captures statements that we are (or will be) prepared to

make to a 3rd party –  Patient management system is acceptably safe to deploy

•  Gradual elaboration process alongside system evolution –  User interfaces lists patients by criticality –  X-ray data file links to patient number –  Monitoring function X poll sensor Y every Z mS

•  Claims are goal-based requirements expressed as post-conditions –  Usually come with an acceptability qualifier

•  Allows to separate the intent from the acceptability criteria


Representing Safety Case Arguments with the Goal Structuring Notation

Purpose of a Goal Structure

To show how goals are broken down into sub-goals, and eventually supported by evidence (solutions) whilst making clear the strategies adopted, the rationale for the approach (assumptions, justifications) and the context in which goals are stated

A/J


Control Systemis Safe

All identified hazards eliminated / sufficiently mitigated

Software developed to I.L. appropriate to hazards involved

I.L. Process Guidelines defined by Ref X.

Hazards Identifiedfrom FHA (Ref Y)

Tolerability targets(Ref Z)

Fault Tree Analysis

FormalVerification

Process Evidenceof I.L. 4

Probability of H2 occurring

< 1 x 10-‐6 per annum

H1 has been eliminated



Primary Protection System developed

to I.L. 4

Secondary Protection System developed to I.L. 2

Process Evidence of

I.L. 2

J

1x10-‐6 p.a.limit for

Catastrophic Hazards

A Simple Goal Structure








Fault Tree Analysis

FormalVerification








to I.L. 4


Process Evidence of

I.L. 2

J



A Simple Goal Structure Safety Requirements & Objectives

Safety Evidence

Safety Argument








Fault Tree Analysis

FormalVerification








to I.L. 4


Process Evidence of

I.L. 2

J



A Simple Goal Structure

Without an argument we just have a ‘bag of evidence’


SafeSafetyPlanning

Prelim. Design& Safety Analysis

FurtherDesign &SafetyAnalysis

General safety objectives(e.g. standards, design conceptsafety)

Specific safety objectives(e.g. design hazards, enactedrequirements)

Verification targets(e.g. failure rate, NSPF,design properties)

Evidence Evidence Evidence

Safety Evidence(e.g. Test Results,Fault Trees, DesignInformation)

Development of Safety Case


A safety case exists

•  …regardless of whether we choose to capture it explicitly or not – Onus on the manufacturer/operator to explain

why their system is safe – Entirety of information (rationale, justification,

evidence) and reasoning about the system •  Safety case reports allow us to capture the

safety case – And GSN to visualise the argument


A C C I

D E N T S

BARRIERS

Undesirable state with potential for harm or damage

Harm to people and damage to assets

or environment

Engineering activities

Operations activities Maintenance activities

Mitigating a Hazard


Hazard Oriented Argument System is

acceptably safe

Argument over all hazards

Mitigation and/or prevention

All hazards have been completely and correctly identified

Hazard log

Hazard A has been addressed

Hazard B has been addressed

Post-conditions (however not created post development)

Further Decomposition

Evidence


F

A

I L U R

E

BARRIERS

Events and Circumstances

Harm to people and damage to assets

or environment

Engineering activities

Operations activities Maintenance activities

Preventing a Hazard

A C C I

D E N T S

Undesirable state with potential for harm or damage


Further Decomposition

Decomposing Claims

What are the conditions that we need to eliminate or be certain about? Need some bottom up analysis

Function X to be performed in Y mS

Validated data input

Human friendly colour coding

System Evidence (testing, reviews, ...)

System related requirements


Hazard Mitigation Argument •  Simple in principle

–  All identified hazards addressed but... •  All hazards have been identified (reference to process) •  Hazards prevented •  Hazards mitigated

•  Derived Safety Requirements –  Requirements set to satisfy the ‘hazards addressed’ claim –  System specification –  Architectural requirements –  Functional requirements –  Procedures


Hazard Mitigation Argument (2)

•  This step requires an understanding of... –  Contribution of the system to hazards –  Faults are not hazards and requirements are not

safety •  Need to explain how a particular hazards occurs (claim

completeness of identified possibilities as well) •  How the Derived Safety Requirements (DSRs) address this


Confidence Argument

•  Why should we believe what the hazard argument tells us?

•  Best parallelism can be found in law – Witness testimony provides evidence that

allows making a case – But is the witness credible?

•  More ‘parameters’ than evidence trustworthiness


All hazards have been completely and correctly identified

Evidence - Reviews - Analysis - Testing - Proofs

Hazard A has been addressed

System Requirement

Test Case

Correctness of argument inferences

Correctness of process

Strength of evidence

Confidence

Confidence Argument


Compliance Argument •  Not safe because of standards

–  …however standards prescribe a process that is trusted to generate all relevant information correctly (confidence argument)

•  Only the hazard argument explains why the system is safe (i.e. hazards addressed) –  Layout can sometimes obscure relation with standard

•  Supplier/operator should be given the opportunity to explain how they have complied

•  Explicit reference of the safety argument to the compliance points of guidance –  Possibly a compliance matrix


The bigger picture

Risk (Hazard management) Argument

Evidence

Confidence Argument

Evidence

Compliance Argument

Evidence


Tool Support & Further Info

•  Assurance Case Editor –  ACEdit –  GSN and OMG ARM –  Open source, Eclipse based –  You can join the development community and commit

code –  http://code.google.com/p/acedit/

•  GSN Standard and Guidance –  www.goalstructuringnotation.info –  www-users.cs.york.ac.uk/~tpk


Dependable Medical Software Workshop

•  www.dependablesos.org – Related Events

sehc 2012 despotou

Technology