safety planning using applied systems theory · 2019-01-09 · scientia est virtus safety planning...

SCIENTIA EST VIRTUS

Safety Planning usingApplied Systems Theory

Dan MontesUSAF Test Pilot School

48th International SFTE Symposium

DISTRIBUTION AApproved for public release

Distribution is unlimited 412TW-PA-17664

SCIENTIA EST VIRTUS

Overview

WHY DO IT

WHAT IS IT

WHERE NEXT

2

SCIENTIA EST VIRTUS

Moscow, 2012• Red Wings Airlines

Flight 9268

• Fast approach in inclementweather resulted in groundeffect, delaying initial touchdown and shortening available runway

• Combination of soft (1.12g) touchdown and crosswind: weight-on-wheels switches did not trip, spoilers did not auto-deploy

• Thrust-reverse system would not activate

3

Tupolev Tu-204

SCIENTIA EST VIRTUS

5 of 8 occupants killed

Pilots believed the thrust reversers deployed like they always do. With the limited runway space, they quickly increased engine power to stop faster. Instead, this accelerated the aircraft, eventually colliding with the highway embankment

4

HUMANS? SOFTWARE?

Despite another serious landing incident involving a Tu-204 at Novosibirsk nine days earlier – which also

related to weight-on-wheels switches – no “timely preventative measures” were taken

SCIENTIA EST VIRTUS

Scenario not ID’d by reliability analysis

5

1940 20101980 202019901950 1960 1970 2000

FMEA FTAHAZOP

Bow Tie (CCA)

FTA + ETA

ETA➢ Introduction of computer control ➢ Exponential increases in complexity ➢ New technology ➢ Changes in human rolesAssumes accidents caused

by component failures

Leveson (2016)

“Intelligent machines seamlessly integrated with humans, maximizing mission performance in complex and contested environments”

––– AF Research Laboratory: 2013 Autonomy S&T Strategy

SCIENTIA EST VIRTUS 6

Scenarios involving FAILURES

Scenarios that are HAZARDOUS

No parts “broke”

Still Safe

Safety ≡ freedom from accidents (not reliability) Safety is an emergent property of the system

Safety vs Reliability

Leveson (2012)


Likelihood & Risk Estimation

Experience & Hindsight

“Use system-safety techniques, prior experience, legacy system research, and overall engineering judgment” to

identify hazards and populate the risk matrix

– AF Test Safety Policy (AFTCI 91-203)

Test-safety planning


Test planners have inconsistent expertise in new technologies

Expertise and hindsight

Even existing systems and ops/test procedures involve highly coupled

human and software behavior

SCIENTIA EST VIRTUS

Systems-Theoretic Process Analysis (STPA)

Leveson, 2012

Hazard-analysis technique that identifies causal scenarios based on a defined system, environment, accidents, and hazards

Consistent with MIL-STD-882 Defines hazards and accidents similarly to

Defense Dept

9

What we’re doing about it

SCIENTIA EST VIRTUS

What the systems approach gets us

• Act of explicit modeling

• Focus on control and coordination

• Top-down engineering traceability

10

SCIENTIA EST VIRTUS

Where it is most useful

11

Degree of Randomness

Degree of “Coupling”

Organized Simplicity (can use analytic reduction)

Unorganized Complexity (can use statistics and averages)

Organized Complexity (can use systems theory)

Weinberg (1975)

SCIENTIA EST VIRTUS

Traditional test-safety flow

12

Person 2: mental model of system



Word DocumentAgreed upon test hazards, mishaps, causes, MPs

Risk Assessment

SCIENTIA EST VIRTUS

STPA flow (systems-engineering)

13

Team’s ExplicitSystem Model

System-Under-Test Database- Design-based scenarios - Mitigations

Wing Database (Design Agnostic) - Accidents (Mishap) - Hazards

Word Document- Mishaps/Hazards - System Model - Scenarios/Mitigations

Risk Assessment

SCIENTIA EST VIRTUS

Study: HAVE Raider (TPS 14B)

14

Lead AircraftWingman Aircraft


Functional Relationships

Hierarchical Responsibility Accountability

Authority

Control-theoretic model


Sociotechnical system

SCIENTIA EST VIRTUS

Missing or wrongcommunicationwith anothercontroller

Process input missing or wrongConflicting control actions

Unidentified orout-of-rangedisturbance

Inadequate ControlAlgorithm

(Flaws in creation,Process changes,

Incorrect modificationor adaptation)

Component failures

Changes over time

Inadequateoperation

Actuator

Controlled Process

Sensor

Process Model(inconsistent,incomplete, orIncorrect)

OtherController

Inappropriate,ineffective or missing

control action

Control input orexternal informationwrong or missing

Inadequate ormissing feedback

Feedback delays

Incorrect or noInformation provided

Measurement inaccuracies

Feedback delays

Process outputcontributes tosystem hazard

Inadequateoperation

OtherControllerController

Delayedoperation

17

Model Analysis

Design-based analysis


Channel Name PRI ALT ALTControl 9a: Spatial Control Inputs Hand-StickThrottleForces-Cable

Feedback 13a: ACAS maneuver indicator Cable-HUDFeedback 13b: ACAS status Cable-DED Cable-HUDFeedback 11a: Pod status Cable-DED

Indirect Measure 1a: Motion PhysicsIndirect Measure 1b: Aerodynamic State ADC-Cable-HUDIndirect Measure 1c: TSPI and ranging EGI-Cable-Displays

Feedback 8a: Standard responses Voice-Radio-VoiceFeedback 8b: Contingency responses Voice-Radio-Voice HandSignal-Visual Geometry-Visual

Comm 3a: Sight of other vehicle in formation VisualComm 2a: Sight of other vehicle in airspace VisualComm 5a: Radar/Transponder range signals Xmitter-UHF-Receiver

Control 9a-1: Banked level turn

Not Providing Providing Wrong Order or Timing Wrong Duration or IntensityPilot does not bank Pilot banks Pilot banks Pilot continues bank

when wingman approaches from behind, above, or below

when wingman in direction of turn when wingman still in direction of turn when wingman in direction of turn

H1 H1 H1 H1

Pilot does not bank Pilot banks Pilot banks Pilot continues bank

when terrain in direction of flight when terrain in direction of turn when terrain still in direction of turn when terrain in direction of turn

H2A H2A H2A H2A

Database of scenarios

SCIENTIA EST VIRTUS

A. Developing Influences – Test/Safety Planning – Training and Qualifications, Flight/Test Manuals – Software coding

B. Settings and Configurations – Test Card Requirements – Briefing Requirements – Instrumentation and Item Configurations – O&M Considerations – ORM/Physio Considerations

C. Operating Procedures – Real time actions

19

If you can’t design it out, mitigate:


Accidents (Mishaps)

Hazards

Scenarios

Minimizing Procedures Corrective Actions Recovery ActionsMitigations

Model of the system

Traceability


Traditional STPA

2 Effects 6 Accidents

1 Test Hazard (actually a mishap) 4 System Hazards

3 Causes 392 Unsafe Scenarios

13 Minimizing Procedures

46 Minimizing Procedures - 14 developing influences - 10 settings/configurations - 22 operating procedures

Nothing identified to control hazard exposure (test hazard was a mishap) 8 Corrective Actions

1 Accident-Corrective Action 7 Recovery Actions

Study result


Traditional STPA

+

• More familiar and comfortable • Fast and convenient • Test hazards are easy to brief and keep in mind during the

mission • Easier for decision-makers to visualize test-specific

hazards and qualify risk

• Investigates the entire system; better for determining true risk

• Description of the system and boundary are more accurate and explicit

• Distinction between accidents and hazards is clear • Format is more straightforward to follow • Traceability is built-in

–

• Relies on experienced reviewers to catch any holes

• Repeats of multiple test-hazard sheets with overlapping information causes tuning out

• Unclear what belongs in the technical plan and the safety plan, often resulting in repeated information in both

• Requires an intricate control analysis and more time to perform

• Might be difficult to navigate for larger projects

• It requires more management involvement and teaching of the new method

What the community thought


Another study: sUAS @ EDW

SCIENTIA EST VIRTUS

• Safety Recommendations – Prior to testing, ensure airspace boundaries are accurate – Establish procedures and checklists to ensure SUAS operator

workload is appropriate so attention can be focused on flying the test

– Ensure SUAS software allows operator to check sensor status – SUAS geofences should still apply during payload delivery maneuvers

• Action Items – SUAS Operations must indicate the SUAV’s programmed lost link

procedures on the SPORT prebrief sheet, as well as any other details from the safety plan that SPORT may need to be aware of

– Develop a way for SPORT to see the position that the SUAV is reporting to the pilot

24

Study excerpts

SCIENTIA EST VIRTUS

Who else?

• Automobiles (about 60-80% of car companies)

• Aerospace

• Defense

• Medical Devices and Hospital Safety

• Chemical plants

• Oil and Gas

25

• Nuclear Power

• Finance

• Universities

• Mining

• Information Technology

SCIENTIA EST VIRTUS

Closer to home• FAA and INTA: aircraft certification

• Embraer: air management system

• U.S. Navy Vessels: dynamic positioning system

• Army Blackhawk: controls and displays

• U.S. Ballistic Missile Defense System

• AF Cyber Security and Mission Assurance - 53d Wing

• AF Cyber College

• Army’s 23d Cyber Protection Team (OLIVE HITCH)

26


• Larger sample size

• Cooperation between system SMEs and testers

• Full safety review boards (independent reviews)

Where we’re going: AFTC


• Consistent method that allows team to perform analysis on proven and unproven systems and consider worst cases pragmatically

• Team iterates on an explicit system model to communicate with reviewers and approvers

• Organizational influences can be considered in a analysis • Requires paradigm shift (e.g., control/feedback mindset

and traceability database)

Takeaways

SCIENTIA EST VIRTUS

Thank you• Acknowledgements

– Col Bill Young, 53 EWG/CC – Col Angie Suplisson, AFTC/CV – Maj Omar Moreno, AFTC/SE – Tom Hill, TPS – Nate Cook, 40 FLTS – LtCol Dan Javorsek, AWC – LtCol Kip Johnson, TPS – Maj Sarah Summers, ACSC – Lt Sarah Folse, AFRL/RW – Kerianne Hobbs, AFRL/RQ

29

safety planning using applied systems theory · 2019-01-09 · scientia est virtus safety planning...

Documents