Human Reliability AssessmentIn ATM: the CARA tool

Barry Kirwan, Huw Gibson, Brian HicklingEUROCONTROL Experimental CentreUniversity of Birmingham, UK

barry.kirwan@eurocontrol.int

Outline

Why HRA?Can a tool be developed

for ATM?What does it look like?What does it tell us?How does it fit with other

Human Factors & Safety approaches?

Are the numbers right?

Why do we need Human Reliability Assessment?

Safety AssuranceSafety Assurance

FHAFHAPSSA PSSA (HRA)(HRA)SSASSAHF CaseHF Case

Incident analysis, Incident analysis, TRM; CISM; NOSS, TRM; CISM; NOSS, etc.etc.

SafetySafetyAchievementAchievement

Ground-Based AugmentationSystem Safety Case

Do Safety Cases Need HRA?

ASAS (Airborne Separation Assurance System) Instruct wrong aircraft F Give ASAS instruction to non-ASAS equipped aircraft F/G1 Fail to detect ASAS aircraft deviation C2 Send incorrect spacing F Link wrong aircraft on screen G2 Fail to detect deceleration of aircraft C2 Pilot exceeds spacing P Pilot targets wrong aircraft P Pilot turns early P

RVSM (Reduced Vertical Separation Monitoring) Fail to detect level bust C2 Issue wrong flight level clearance F Fail to react in time to short term conflict alert B3 Fail to provide collision avoidance advice D2 Fail to use correct emergency terminology G1 Fail to detect non-RVSM aircraft C1 Fail to coordinate aircraft into sector with conflict-free trajectory E

Why Human Reliability Assessment?

The human is the critical component

Safety case target is quantitative

HRA puts HF in the risk equation

Human Factors

HRA

Safety

Human Factors mainly qualitative

What do we need?

Deriving the Generic Tasks - the Model Underlying CARA

Attention and Action Hierarchy

D. Solving conflicts

F. Manage routine traffic

Position take-over

Managing requests

G. Issuing instruction

s to ac

Monitoring

Confirming/ updating mental

picture

Sector plan

Workload monitoring

B. Checking

C. Monitoring for conflicts

Verbal InstructionDatalinkNon-communication

RadarFPSInfo displaysReminders

A. Offline tasks

E. Plan aircraft in/out of sector

E. Plan aircraft in/out of sector E. Plan aircraft in/out of sector. 0.01

F. Manage routine traffic

F. Routine element of sector management (e.g. rule-based selection of routine plan for an aircraft or omission of clearance).

0.003

G1. Verbal slips. 0.002 G. Issuing instructions G2. Physical slips (two simple choices). 0.002 M. Technical and support tasks

M3. Routine maintenance task. 0.004

Task Context Generic Task Type HEP Uncertainty Bounds

A. Offline tasks A. Offline tasks. 0.03 - B1. Active search of radar or FPS, assuming some confusable information on display. 0.005 0.002-0.02

B2. Respond to visual change in display (e.g. aircraft highlighted changes to low-lighted). 0.13 0.05-0.3 B. Checking

B3. Respond to unique and trusted audible and visual indication. 0.03 -

C1. Identify routine conflict. 0.02 Holding Value C. Monitoring for conflicts or unanticipated changes

C2. Identify unanticipated change in radar display (e.g. change in digital flight level due to aircraft deviation or corruption of datablock).

0.3 0.2-0.5

D1. Solve conflict which includes some complexity. Note, for very simple conflict resolution consider use of GTT F.

0.02 Holding Value D. Solving conflicts D2. Complex and time pressured conflict

solution (do not use time pressure EPC). 0.19 0.09-0.39

Quantification Sources Quantification Sources

Other HRA Other HRA techniquestechniques

Published HFPublished HFStudiesStudies

SimulatorSimulatorDataDataReal worldReal world

Data collectionData collection

Pilot actions

Error Producing Conditions

Sources of Error Producing ConditionsIncidents & ErrorIncidents & Error

Classification Schemes Classification Schemes ((egeg HERA)HERA)

Human FactorsHuman FactorsLiterature on ATMLiterature on ATM

PerformancePerformance

HRA ModelsHRA Models((egeg CREAM,CREAM,

NARA, HEARTNARA, HEARTSPARSPAR--HH

Traffic Traffic ComplexityComplexity

VigilanceVigilance

Poor equipmentfeedback

Cognitive Overload

Shift Handover

0n the job0n the jobTrainingTraining

Procedures

Unfamiliarity

Time Pressure

EPCs

Traffic Complexity

Vigilance

On the jobTraining

x11x11Error Producing Conditions (examples)

x10x10

x5x5

x20x20

x8x8

x6x6 x10x10

x5x5

x3x3

What does it look like,What does it tell us?

HRA (CARA) in practice: airport example

a/c1a/c1

a/c2a/c2

Aircraft 1 fails to exit runwayAircraft 1 fails to exit runwayATCO fails to identify a/c1 stoppedATCO fails to identify a/c1 stoppedOr ATCO fails to warn a/c2 in timeOr ATCO fails to warn a/c2 in time

Need to speed up airport throughputNeed to speed up airport throughputAircraft 1 vacates runway earlyAircraft 1 vacates runway earlyAircraft 2 anticipates this, lands soonerAircraft 2 anticipates this, lands sooner

CARA in practice

Task: Warn AC2 in time: Task: Warn AC2 in time: -- Audible and visual warning: Controller identifies AC 1 is stationary within the OFZ

Generic Task Type: Generic Task Type: FF: Routine instruction or clearance

Human Error ProbabilityHuman Error Probability: 0.003

Error Producing ConditionsError Producing Conditions: Time PressureMaximum AffectMaximum Affect x11Assessed Proportion of AffectAssessed Proportion of Affect: 0.2

Human Error ProbabilityHuman Error Probability: 0.009

Assumptions/ HF RequirementsAssumptions/ HF Requirements: Assume controllers trained to warn AC2 rather than talk to AC1 to identify the problem – procedures need to be developed/assessed and trained

An example of CARA in action

1.00E-03

1.00E-02

1.00E-01

Audible and visual alarm Visual alarm only No visual or audible alarm HEART assessment

HEP

Audio/Visual Visual Audio/Visual Visual NeitherNeither Original HEPOriginal HEP

1.01.0

0.10.1

0.010.01

0.0010.001

1.00E+00

Interface

Task HEP = 0.03 + 0.009 = 0.04Task HEP = 0.03 + 0.009 = 0.04

Outstanding Issues

The Barrier or Safety Nets Model (Reason’s ‘Swiss Cheese’ model)

PlanningATC

STCATCAS

See & Avoid

Trajectories through the Cheese

1,000,00050,0001,000

201ATCO

ResolvesRoutineConflictsATCO DetectsSTCA

ATCO Resolves STCA Conflict Late

TCAS;See & AvoidProvidence

d D1 or D2 c a B3 (0.02) C1 (0.02) b Late resolution Availability & Conflict geometry e C2 OR [D1 or D2] f

ATM influence +ve & -ve

Conflict occurs

Conflict managed early

STCA occurs b4 TCAS

ATCO detects STCA

ATCO resolves conflict

TCAS resolves conflict

Geometry favourable (providence)

Outcome

ATM Safe Safe LOSS LOSS Accident LOSS LOSS Accident Safe LOSS LOSS Accident

Time Ahead (minutes) Separation <= 0.5 0.5-1.0 1.0-2.0 2.0-3.0 3.0-4.0 4.0-5.0 5.0-7.5 7.5-10 Total <= 1.0 nm & 800 ft 3 1 3 4 6 11 11 39

1.0-2.5 nm & 800 ft 4 1 3 7 6 17 10 48

2.5-4.0 nm & 800 ft 7 1 5 2 1 16 12 44

4.0-5.0 nm & 800 ft 6 3 3 7 3 6 5 33

5.0-7.5 nm & 1000 ft 93 6 12 16 7 16 12 24 186

7.5-10 nm & 1000 ft 152 22 23 19 10 6 18 19 269

> 10 nm or > 1000 ft 57 101 133 54 57 37 70 101 610

Total 323 130 173 103 94 75 150 182 1229

Future workFuture workData collection on alarm responseData collection on alarm response

Internal Review 09Internal Review 09Manual Production (09)Manual Production (09)

Application in Macro Safety CaseApplication in Macro Safety Case(SESAR)(SESAR)

Questions?

26

0.0001

0.001

0.01

0.1

1

A1 A2 A3a A3b A4 B1 B2 B3 B4 B5 C2a C2b C4

HEART Value

CARA Value

Key Findings from GBAS study for CARA

Fewer EPCs Required - better match at GTT level

Identified improvements to CARA

GTT application more straightforward

Similar error probabilities

Elements of Quantification – the HEART template

Define the Generic Task Type

(GTT)

ConsiderError

Producing Conditions

(EPCs)

Calculate the Human Error Probability

(HEP) =GTTxEPCi..n

Assess how much each EPC affects task performance

Human Error Human Error Assessment &Assessment &

& Reduction & Reduction TechniqueTechnique

Williams, 1985Williams, 1985

Document & Feed forward

to Ops

Define Error

ReductionMeasures

Evaluatethe human contribution Consider

HumanDependence

Quantify Quantify Human ErrorHuman ErrorProbabilityProbability

Model the human in

the safetycase

Human Error

Identification

TaskAnalysis

HRAProcess

Where does CARA fit in HRA?