1

A Systemic Model of ATM Safety: The Integrated Risk Picture (IRP)

7th USA – EUROPE ATM R&D SeminarJuly 02-05, 2007

Barcelona, SPAIN

� Eric PERRIN, Barry KIRWAN, & Ron Stroup*

EUROCONTROL Experimental Centre (EEC)*FAA ATO Safety, Washington

European Organisation for the Safety of Air Navigation

2

Outline of Presentation

� Why we need an integrated risk picture

� What it is

� How it works

� What it gives us

� What it doesn’t

� Who else is using this approach

� How it leads to a roadmap (STAR) to manage safetymanage safety

through a period of change [SESAR / NGATS]

3

Matching the safety approach to the needs…

� If your industry is stable, you have operational ongoing safety

cases and monitor accidents, incidents and precursors

� If you are making small changes, you carry out a safety

assessment of the individual Operational Improvement (OI)

� However, if you are changing the overall system, this is much

more complicated…

EEC Project

Project

OK, you’resafe - this time

4

But what if multiple parallel changes? What is the safety level of the overall system?

ATM - a set of inter-related systems

??

??

??

??

Total safety= ???

Interactionsunderstood?

Where issafety strong or weak?

5

Information Management

Demand& Capacity

Management

Traffic Management

Separation Management

Airport

ATC/FMPAOC - Military

CFMU - AOC - ATSP Military - Airport

Airspace Organization

20 minutesTake-off1 year

ATC Sector

Pilot

What is the safety

assessment of the

overall ATM system?

How might these new

elements interact?

Are there negative

interactions that can be

avoided?

Are there positive

interactions that could

lead extra safety?

Where are the strong

and weak safety areas in

the overall system?

Why do we need an Integrated Risk Picture?

6

Solution – An Integrated Risk Picture

� Look at current safety (use what we know now)

� From accidents to incidents to precursors to errors to influences

� Model how we stay safe now, and where our vulnerabilities are

� Extrapolate for the future system (predict as best we can)

� Identify safety priorities for improvements during the design phase

� Specify the safety requirements needed to keep us safe

� Consider how the transition will be managed safely

� Recognise it is not a ‘big bang’ transition

� Plot anticipated safety improvements required to keep us safe

� Measure if they are achieved, and update / modify if needed

7

So – what do we know now?

Building an Integrated Risk Picture

8

ATM Direct Contribution to Accidents –2005 (Baseline)

-1.6E-8Structural accidents

* Potential ATM contributions to these accident categories have not yet been estimated.

1.1E-082.9%3.8E-07Total aircraft accidents

--2.2E-08Fire/explosion

--6.4E-08Loss of control in landing*

--4.8E-08Loss of control in take-off*

--1.3E-07Loss of control in flight*

2.3E-106.9%3.3E-09Wake turbulence accident

8.3E-101.5%5.4E-08CFIT

3.1E-109.2%3.4E-09Taxiway collision

6.3E-0918.9%3.3E-08Runway collision

3.5E-0964.50%5.4E-09Mid-air collision

FREQUENCY OF FATAL

ACCIDENT DIRECTLY CAUSED BY

ATC

(per flight)

ATC

DIRECT

CAUSES

(%)

FATAL ACCIDENT

FREQUENCY (per

flight)ACCIDENT CATEGORY

� ATM contribution to direct causes:

� Fatal accidents – 2.9%

� ICAO accidents – 2.0% (i.e. 4.5 x 10-8 per flight)

9

Swiss Cheese Model – Runway Collisions

Visual warning

RIMCAS warning

Runway separation

Runway configuration

Intermediate runway entry

ATCO failure to recognise conflict

RIMCAS not installed

Operation in low visibility

Causal factors

Strategic conflict

Collision

Runway incursion

Precursors

Imminent runway collision

Potentially conflicting runway approach

10

Precursor Frequencies

Precursorsfrequencies

Accident frequency

Exposure frequency

RF1 Fatal runway collision involvement 3.3E-08 per flightRF2 Fatal runway collision 2.8E-08 per flightRF3 Runway collision 3.8E-08 per flightRP1 Imminent runway collision 4.3E-07 per flightRP2 Runway conflict 4.5E-07 per flightRP3A Aircraft runway incursion 2.0E-05 per flightRP4 Potentially conflicting runway approach 0.33 per flightRP5 Runway approach 1.10 per flight

RB1 Ineffective collision avoidance 0.22 per imminent collisionRB2 Ineffective conflict warning 0.95 per conflictRB4 Ineffective runway entry procedures 5.9E-05 per potentially conflicting approachRB5 Potentially conflicting runway configuration 0.30 per approach

Barrier failure probabilities

Caution: Preliminary results only. Confidence ranges being determined.

11

RB4.1 Operat ional

error in runway ent ry;

29.2%

RB6.1 Operat ional

error in t ake-of f ; 3.4%

RB4.2 Pilot deviat ion

in runway ent ry; 61.0%

RB6.2 Pilot f ailure t o

f ollow t ake-of f

inst ruct ions; 6.4%

Expanded Barrier Failure Causes

RB6.1.1 Take-off

instruction error by

ATCO; 2.0%

RB4.1.1.2.1 ATCO

failure to recognise

runway conflict;

5.9%

RB4.1.1.2.2 ATCO

misjudgement of

runway separation;

5.9%

RB4.1.1.1 Inadequate

aircraft position

information to

ATCO; 4.3%

RB4.1.2 Inadequate

communication with

pilot; 13.2%

RB6.1.2 Inadequate

communication with

pilot; 1.4%RB4.2.1 Pilot failure

to follow taxi route;

19.5%

RB6.2 Pilot failure

to follow take-off

instructions; 6.4%

RB4.2.2 Pilot failure

to follow runway

entry instructions;

41.5%

Runway entry procedures

Barrier Causes of barrier failure

Caution: Preliminary results only. Confidence ranges being determined.

12

IRP: Risk Model Overview

Taxiw

ay collision

Mid-air

collision

Runw

ay

collision

Wake

turbulence

CF

IT

Accident categories

Causal factors(technical failures, human errors)

Influences(safety management, operating environment, etc.)

Risks (frequencies of accidents)

� Fault tree model:

� Widely understood.

� Combination of multiple causes.

� Transparent quantification.

� Top-down approach:� Calibrated against accident &

incidents experience.� 5 main ATM-influenced

accident categories.

� Influence model

� Modifications of base events.

� Same for all accident categories.

13

Influence Model Structure

Task performance

Base event in fault tree

Quality of task inputs

Performance of other tasks

Performance of safety management

Performance of actors

Performance of equipment

PolicyPlanning

AchievementAssurancePromotion

ResourcesCompetence

HMIReliability

ProceduresTeamwork

FunctionalityIndependenceTransparencyRedundancy

MaintainabilityIntegrity

Operating environment

TrafficWeatherTerrain

etc

Quality of airport/airspace

design

14

0.0 0.2 0.4 0.6 0.8

Di r ect AT C causes

RB4.1.1.1.2.2 Inadequate ai r por t AT CO coor di nat i on

RB4.1.1.2.2 AT CO mi s j udgement of r unway separ at i on

A T C pr event i on f ai l ur es

RB4.1.1.1.1 Inef f ect i ve gr ound r adar sur vei l l ance

RB4.1.2 Inadequate communi cat i on wi th pi l ot

RB2.3 Cont r ol l er f ai l ur e to r espond to RIM CAS war ni ng

RB1.1.3 Rest r i cted vi ew f r om tower pr events conf l i c t detect i on

RB1.1.5 AT CO f ai l ur e to r esol ve conf l i ct i n t i me

AT M pr event i on oppor tuni t i es

RB5.2 Runway ent r y at i nter medi ate l ocat i on

RB2.1 RIM CAS not pr esent

RB1.1.2 Dar kness pr events conf l i c t detect i on

CONTRIBUTION

(maximum pot ent ial reduct ion as f ract ion of f at al runway collision f requency)

ATM Contribution to Runway Collisions

Direct causes

Failures to prevent accidents

Opportunities to prevent accidents

Caution: Preliminary results only. Confidence ranges being determined.

15

IRP for Safety Assurance of Future Systems

16

ATM CONOPS

CONOPS assessment…

ATM Direct cause of accidents pfh

Y/N

Step 6

If criteria cannot be met

(or is exceeded),

implementation

assumptions must be

changed.

17

TRANSITION TO SESAR END-STATE

SWIM

DLK

NAVIGATION

18

The Safety Target Achievement Roadmap (STAR)

Up

Risk Prediction for

Individual ATM

Change

OI # 2

OI # 3OI # i

THEN

OI # j

OR

j THEN i

Risk Prediction for

group of ATM

Changes

Risk Predictions for

different order of

implementation

19

Safety Target Achievement Roadmap (STAR): a potential safety monitoring tool

15%

5%

40%

10%

30%

Up

2005 2010 2015 2020

Accidents Frequency

OI # 1

OI # 2

OI # i

THEN

OI # j

OR

j THEN iOI # n

OI # 3

Risk Prediction for

Individual ATM

Change

Risk Prediction for

group of ATM

Changes

Risk Predictions for

different order of

implementation

When

OI # k?Risk Predictions for

different

implementation dates

Risk Predictions

considering

interactions

20

Summary - IRP – What it is, and what it is not

� How the IRP can help you

� Show overall safety target

compliance of a CONOPS

� Identify strategic directions for

safety improvements

� Determine specific safety

requirements

� Support the creation of a safe

implementation roadmap for a

CONOPS (STAR)

� Support safety performance

monitoring

�Where it needs to be

complemented…

�HAZID Method (IRP-ESSI

FAST White Paper)

�Substitute to SAM

(complementary)

�Dynamic Risk Modelling

�Substitute to advanced

Human Reliability

Assessment (HRA)

21

A coordinated & shared vision

22

Development till late 07 (started 2004)

� 2005 Initial IRP finished and being refined

� Uncertainty analysis (confidence ranges using Monte Carlo simulation)

� Sensitivity analysis

� 2012 IRP completed

� 2020 (SESAR) will follow the finalised Concept of Operation

(CONOPS)

� (STAR) Roadmap developed for end 2007

� STAR is sensitive to implementation timescale & sequence

� User Guidance will be developed to make the tools more

‘accessible’

23

Conclusions

� EUROCONTROL and FAA developing parallel toolsets to

� Model the safety of current operations

� Predict overall safety of future ATM system

� Identify safety vulnerabilities

� Identify safety opportunities

� Plan a safe way forward

� Enable monitoring of safety progress as new system

elements are implemented

Thank YouThank You

25

The need for a safe transition…

� In parallel with the SESAR CONOPS and Architecture

� Need for a roadmap for achieving the operational improvements

� I.e. showing how and when different pieces of the SESAR “system”

will fall into place.

� Safety “roadmap”:

� Highlights a transition to the end state and underscores the safety benefits along

the way

� Ensures transition remains within the safety envelope

� Roadmap for achieving operational improvements

� How & When different pieces fall into place – their ‘safety contribution’

� Need for new safety defenses

26

Integrated Risk Picture in Practice

Accidents

Airspace Design

Flow Management

Deconfliction

ATC Tactical

a/c Tactical

ATC Recovery

Pilot Recovery

Providence

Accidents

0.00 0.05 0.10 0.15 0.20 0.25 0.30

Airspace organisation & management

Air traffic flow & capacity management

ATC - traffic synchronisation

ATC - traffic separation

ATC - conflict resolution

ATC systems

Communications

Surveillance

ATM avionics - ACAS

Airport operations

Information management

ATM CONTRIBUTION(maximum potential reduction / total fatal accident frequency)

CFITMid-air collisionRunway collisionTaxiway collisionWake turbulence accident

27

END USERS

IRP Method

Risk R

esultsSa

fety R

equirements

Safety R

ecommendations

CONOPS Designers

System Designers

Safety Managers

28

0.0 0.2 0.4 0.6 0.8 1.0 1.2

AO&M

ATFCM

Traffic synchronisation

Traffic separation

Conflict resolution

ATC systems

Communications

Surveillance

ATM avionics

Airport operations

Flight planning

Information management

CONTRIBUTION (maximum potential reduction as fraction of fatal runway collision frequency)

Direct cause

Prevention failure

Prevention opportunity

Indirect influence

Contributions of ATM Elements to Runway accidents for Commercial Flights in 2005

Caution: Preliminary results only. Confidence ranges being determined.

29

SESAR Definition Phase

Air

Transport

Framework:

The

Current

Situation

Air

Transport

Framework:

The

Current

Situation

ATM

Performance

Targets

ATM

Performance

Targets

ATM

Target

Concept

ATM

Target

Concept

ATM

Deployment

Sequence

ATM

Deployment

Sequence

ATM

Master Plan

of Action

ATM

Master Plan

of Action

Work

Programme

for

2008-2012

Work

Programme

for

2008-2012

07/06 12/06 06/07 11/07 02/08 03/08

D 3 D 4

WP 2.2.2 / 2.2.4: Operational Concept

WP 2.3.1 / 2.3.2: Models & Validation Needs

D1 & D2: Bottlenecks & Performance Targets

30

Prediction and Validation

1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012

FA

TA

L A

CC

IDE

NT

FR

EQ

UE

NC

Y (

per

fligh

t)

IRP 2005:

� Top-down prediction from trended accident experience.

� Deduced base event probabilities.

IRP 2012:

� Bottom-up prediction from planned ATM changes.

� Judged changes to base events.

IRP 1990:

� Bottom-up prediction from historical ATM changes.

� Validated against accident experience.

31

EEC Objectives

To validate the SESAR target concept, i.e. providing the evidence, or otherwise, that the SESAR 2020 concept:

� Is specified to be acceptably safe (Holistic Risk modelling, and 1st steps of

SAM/ED78A safety analysis)

� Is operationally viable (through prototyping and R/T simulations)

� Can attain the required level of performances (ECAC wide modelling)

� Is environmentally efficient (Emission and Noise modelling)

Federating European Research actors to run this validation programme

based on the European Commission’s FP6 project – Episode 3).

RTA

AUTHORIZEDRBTA

Current Pos

EXECUTEDRBTX

PLANNEDRBTP

RTA

AUTHORIZEDRBTA

Current PosCurrent Pos

EXECUTEDRBTX

PLANNEDRBTP

32

Scope of Risk Estimates

Risks estimated

Risks not estimated

Fatal accident frequencies

Group risks of fatalities

External risks (people not on board aircraft)

Individual risks

ICAO-defined accident frequencies

Precursor incident

frequencies

Safety net reliability