safety assurance in software systems...safety-critical regulated redundancy for fault-tolerance...
TRANSCRIPT
Safety Assurance in Software Systems From Airplanes to Atoms
MDEP Conference on New Reactor Design Activities
Session 2 – Digital I&C: Current & Emerging Technical Challenges
12 September 2017
Dr. Darren Cofer
Rockwell Collins: Advanced Technology Center
Domains
• Avionics systems
• Commercial and military
• Manned and unmanned
• Safety and security
2
Trusted Systems Group
• Formal methods for verification
• Model-based development
• Practical and effective tools
• Certification (DO-178C)
Similar Concerns…
3
Safety-critical
Regulated
Redundancy for fault-tolerance
Software intensive
Fail-safe
Fail-op
Similar Challenges
• Increased use of software in safety-critical functions
• Complexity of software
• Incorporation of COTS hardware/software
• New technologies that challenge the existing certification process
• Limitations of testing for safety assurance
• Cybersecurity
4
“Houston, we have a problem.”
What can the nuclear industry learn from civil aviation experience?
Assertion 1: The nuclear industry can benefit from aerospace software development and verification practices.
5
De
sign
Ass
ura
nce
Certi
ficati
on
Pro
cess f
or
Civ
il A
via
tio
n
Safety Assessment Process Guidelines and Methods
(ARP 4761)
System Development Processes (ARP 4754A)
Guidelines for Integrated Modular Avionics
(DO-297)
Hardware Development Life-Cycle (DO-254)
Software Development Life-Cycle (DO-178C)
System Design Function, Failure, & Safety Information
Intended Aircraft
Function
Functional Aircraft
Hardware Requirements
Software Requirements
Implementation
Implementation Requirements
CFR Title 14 Part 25 Airworthiness Standards: Transport Category
6
Means of
Compliance
Operational Environment
DO-178 Principles
• Primarily a quality document, not safety
• Demonstrate that software implements requirements…
• …and nothing else
(no surprises)
7
• Requirements-based testing
• Traceability among requirements, test cases, code
• Structural coverage metrics to determine adequacy of testing
Assertion 2: To cope with software complexity, the aerospace industry is moving toward use of formal methods.
8
Formal Methods: Complete Exploration of Design
9
4
5
32 + 42 = 52
3
1
control_cmd
[V_s]
[V_cmd]
[theta_cmd]
[psi_cmd]
[phi]
[theta]
[psi]
[r]
[phi_cmd]
[h_cmd]
[p]
[h]
[q]
0
flaps
theta_cmd [rad]
theta [rad]
q [rad/sec]
r [rad/sec]
phi_cmd [rad]
phi [rad]
p [rad/sec]
elevator [rad]
rudder [rad]
aileron [rad]
baseline control
V_cmd [m/s]
V_s [m/s]
Throttle
Velocity Tracker
psi_cmd [rad]
psi [rad]
phi_cmd [rad]
Psi Tracker-1
[V_cmd]
[V_s]
[r][psi_cmd]
[psi]
[phi]
[h_cmd]
[q]
[theta]
[p]
[h]
h_cmd [m/s]
h [m]
theta_cmd [rad]
Altitude Tracker
2
ref_cmds
1
feedback
1
control_cmd
[V_s]
[V_cmd]
[theta_cmd]
[psi_cmd]
[phi]
[theta]
[psi]
[r]
[phi_cmd]
[h_cmd]
[p]
[h]
[q]
0
flaps
theta_cmd [rad]
theta [rad]
q [rad/sec]
r [rad/sec]
phi_cmd [rad]
phi [rad]
p [rad/sec]
elevator [rad]
rudder [rad]
aileron [rad]
baseline control
V_cmd [m/s]
V_s [m/s]
Throttle
Velocity Tracker
psi_cmd [rad]
psi [rad]
phi_cmd [rad]
Psi Tracker-1
[V_cmd]
[V_s]
[r][psi_cmd]
[psi]
[phi]
[h_cmd]
[q]
[theta]
[p]
[h]
h_cmd [m/s]
h [m]
theta_cmd [rad]
Altitude Tracker
2
ref_cmds
1
feedback
1
control_cmd
[V_s]
[V_cmd]
[theta_cmd]
[psi_cmd]
[phi]
[theta]
[psi]
[r]
[phi_cmd]
[h_cmd]
[p]
[h]
[q]
0
flaps
theta_cmd [rad]
theta [rad]
q [rad/sec]
r [rad/sec]
phi_cmd [rad]
phi [rad]
p [rad/sec]
elevator [rad]
rudder [rad]
aileron [rad]
baseline control
V_cmd [m/s]
V_s [m/s]
Throttle
Velocity Tracker
psi_cmd [rad]
psi [rad]
phi_cmd [rad]
Psi Tracker-1
[V_cmd]
[V_s]
[r][psi_cmd]
[psi]
[phi]
[h_cmd]
[q]
[theta]
[p]
[h]
h_cmd [m/s]
h [m]
theta_cmd [rad]
Altitude Tracker
2
ref_cmds
1
feedback
1
control_cmd
[V_s]
[V_cmd]
[theta_cmd]
[psi_cmd]
[phi]
[theta]
[psi]
[r]
[phi_cmd]
[h_cmd]
[p]
[h]
[q]
0
flaps
theta_cmd [rad]
theta [rad]
q [rad/sec]
r [rad/sec]
phi_cmd [rad]
phi [rad]
p [rad/sec]
elevator [rad]
rudder [rad]
aileron [rad]
baseline control
V_cmd [m/s]
V_s [m/s]
Throttle
Velocity Tracker
psi_cmd [rad]
psi [rad]
phi_cmd [rad]
Psi Tracker-1
[V_cmd]
[V_s]
[r][psi_cmd]
[psi]
[phi]
[h_cmd]
[q]
[theta]
[p]
[h]
h_cmd [m/s]
h [m]
theta_cmd [rad]
Altitude Tracker
2
ref_cmds
1
feedback
1
control_cmd
[V_s]
[V_cmd]
[theta_cmd]
[psi_cmd]
[phi]
[theta]
[psi]
[r]
[phi_cmd]
[h_cmd]
[p]
[h]
[q]
0
flaps
theta_cmd [rad]
theta [rad]
q [rad/sec]
r [rad/sec]
phi_cmd [rad]
phi [rad]
p [rad/sec]
elevator [rad]
rudder [rad]
aileron [rad]
baseline control
V_cmd [m/s]
V_s [m/s]
Throttle
Velocity Tracker
psi_cmd [rad]
psi [rad]
phi_cmd [rad]
Psi Tracker-1
[V_cmd]
[V_s]
[r][psi_cmd]
[psi]
[phi]
[h_cmd]
[q]
[theta]
[p]
[h]
h_cmd [m/s]
h [m]
theta_cmd [rad]
Altitude Tracker
2
ref_cmds
1
feedback
1
control_cmd
[V_s]
[V_cmd]
[theta_cmd]
[psi_cmd]
[phi]
[theta]
[psi]
[r]
[phi_cmd]
[h_cmd]
[p]
[h]
[q]
0
flaps
theta_cmd [rad]
theta [rad]
q [rad/sec]
r [rad/sec]
phi_cmd [rad]
phi [rad]
p [rad/sec]
elevator [rad]
rudder [rad]
aileron [rad]
baseline control
V_cmd [m/s]
V_s [m/s]
Throttle
Velocity Tracker
psi_cmd [rad]
psi [rad]
phi_cmd [rad]
Psi Tracker-1
[V_cmd]
[V_s]
[r][psi_cmd]
[psi]
[phi]
[h_cmd]
[q]
[theta]
[p]
[h]
h_cmd [m/s]
h [m]
theta_cmd [rad]
Altitude Tracker
2
ref_cmds
1
feedback
Testing can only show the presence of bugs (and only if you are lucky)
Analysis can show the absence of bugs (with evidence of correctness)
Proof
Formal Methods and Aircraft Certification
Theorem Proving demonstrated on the FGS design model
Model Checking demonstrated on the Mode Logic Simulink model
Abstract Interpretation demonstrated on the Heading Control Law source code
Flight Guidance System
Mode Logic
Heading Control
1
control_cmd
[V_s]
[V_cmd]
[theta_cmd]
[psi_cmd]
[phi]
[theta]
[psi]
[r]
[phi_cmd]
[h_cmd]
[p]
[h]
[q]
0
flaps
theta_cmd [rad]
theta [rad]
q [rad/sec]
r [rad/sec]
phi_cmd [rad]
phi [rad]
p [rad/sec]
elevator [rad]
rudder [rad]
aileron [rad]
baseline control
V_cmd [m/s]
V_s [m/s]
Throttle
Velocity Tracker
psi_cmd [rad]
psi [rad]
phi_cmd [rad]
Psi Tracker-1
[V_cmd]
[V_s]
[r][psi_cmd]
[psi]
[phi]
[h_cmd]
[q]
[theta]
[p]
[h]
h_cmd [m/s]
h [m]
theta_cmd [rad]
Altitude Tracker
2
ref_cmds
1
feedback
Formal methods reduce cost and increase confidence
through early detection and elimination of errors 10
DO-178C/DO-333 Case Study
Assertion 3: Formal methods can also address cybersecurity concerns for high-assurance systems.
11
12
High-Assurance Cyber Military Systems
13
Boeing Unmanned Little Bird Helicopter
High-Assurance Cyber Military Systems
• Final Demonstrations
– Boeing Unmanned Little Bird (ULB): Mesa AZ, Feb 2017
– Quadcopter: Sterling VA, Apr 2017
• Demonstrated cyber-resiliency of both vehicles
– “Before” and “after” flight demonstrations
– Attacked in-flight
– Comprehensive evaluation by “white hat” cyber-attackers
• Cyber-resiliency achieved through application of formal methods
– Model checking of architecture properties
– Synthesis/verification of software components
– Comprehensive proof of correctness of operating system
14
Formal methods are practical and effective for achieving cybersecurity in real aerospace systems
For More Information…
• HACMS final demo video
– https://insights.rockwellcollins.com/2017/07/06/video-high-assurance-cyber-military-systems-hacms/
• DARPA Blocks Cyberattacks on Unmanned Little Bird In Flight (Aviation Week)
– http://aviationweek.com/awindefense/darpa-blocks-cyberattacks-unmanned-little-bird-flight
• Cybersecurity Skeptics Now Embracing Formal Methods (ACM Ubiquity)
– http://ubiquity.acm.org/article.cfm?id=3081880
• DO-178C/333 Certification Case Studies Using Formal Methods
– http://loonwerks.com/projects/do333.html
15