1 system certification (safety assurance) of waas deane bunce sbas approval workshop 21-22 june 2005

54
1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

Upload: arnold-griffith

Post on 24-Dec-2015

219 views

Category:

Documents


3 download

TRANSCRIPT

Page 1: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

1

System Certification (Safety Assurance) of WAAS

Deane Bunce

SBAS Approval Workshop

21-22 June 2005

Page 2: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

2

System Certification

• Verify Performance Requirements Met– Accuracy– Integrity – Continuity– Availability

• GAO Report of 2000– FAA Underestimated Complexity of Proving the Integrity

Requirement Satisfied– FAA Did Not Closely Monitor the Contractor’s Efforts to

Demonstrate Integrity

• Recommendations– Establish Checkpoints To Determine Contractor’s

Approach For Meeting Performance Requirements – Have External Organization Evaluate Agency Progress At

These Checkpoints

Page 3: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

3

Approach for Assuring WAAS Integrity

• Elements to Consider– WAAS Hazard Assessment– WAAS Architecture– Need for Technical Oversight (Independent Verification)

Page 4: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

4

WAAS Hazard Assessment

Category Severity Causes Probability Limit

HMI • Hazardous/ Severe-Major (Cat 1 PA)

• Major (ER/NPA)

• Failure to Detect Broadcast of Misleading Information (MI)

• Failure to Alarm Detected Broadcast of MI Within the Specified TTA

1E-07

LOC • Minor • Hardware/Software Failure Resulting in Loss of Signal in Space

• Corrections or Monitor Trip Resulting in Transition or IGP to NM or DU State

• Detection of MI Resulting in User Alarm

1 - 5.5 x 10-5 per approach

Personnel • Minor to Catastrophic (per MIL-STD-882D)

• Miscellaneous (Electrical, Hazardous Materials, Physical Hazards)

N/A

Page 5: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

5

WAAS Architecture

• Correct – Then – Monitor Architecture– Trusted Processing for HMI Risk Limited to Safety Processors

– Data Verification and Integrity Monitoring Algorithms Designed to Detect and/or Bound Sources of Data Errors

• Must Verify System Architecture/Design Mitigates HMI Hazards or Reduces to Acceptable Risk

• Safe Behavior Cannot Be Verified Completely Through Testing for Complex/Non-Deterministic Algorithms

• System Integrity Performance Is Not Completely Quantifiable

• Process-Based Assurance Method Required to Assure all Threats to Integrity were Addressed

Corrections Processor

Safety Processor Uplink/GEO User

Monitors Data

(Level B)CRC protects dataGenerates data (Level D)

Satellite Signals

Receiver

Page 6: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

6

Process Based Assurance Method

• Safety Assurance: Assure that WAAS Design is capable of Meeting System Safety Requirements (Integrity and Continuity)

• Method of Assurance Based on Aircraft Certification Standards / Prior Navaid Applications– SAE ARP 4754/4761, RTCA DO-178B– Consists of Audits/Reviews of Contractor System

Development and Safety Assessment Activities

• Process Captured in Safety Assurance Process Requirements Document [AVR/ATS]

Page 7: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

7

Safety Assurance Process• Assurance Process: The Planned and Systematic Actions

Necessary to Provide Adequate Confidence That a Product or Process Satisfies Given Requirements

SafetyAssurance

System Safety Program Plan

MIL-STD-882

DO-178B

AVR

ARP 4761/4754

SafetyStandards

AssuranceGuidance

SafetyAssessment

SystemDevelopment

SystemDevelopment

Plan

Mission Need Statement

DesignImplementation

andIntegration

SystemDesign

RequirementsDefinition andMaintenance

ProgramRequirements

VerificationRequirements,

Design, andCode

Review

HazardAnalysis

andIdentification

VerificationReview

Integrity Threats

Requirements, Design , and Implementation

Design

Verification Methods and Results

Process Input

SafetyAssurance

Requirements

ARP 4761ARP 4754

Page 8: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

8

System Development Activities

• System and Component Development Activities to Assure Flow-down of Safety Requirements into the WAAS Implementation– System Engineering, Specification, and Architecture

Design– Component Design Specification– Implementation and/or Procurement

• Engineering Processes of Quality Assurance (QA), Configuration Management (CM) and Verification for All Development Activities

Page 9: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

9

Safety Assessment Activities

• System Hazard Assessment– Fault Tree Analysis – Algorithm P(HMI) Analysis– Safety Processor Input Analysis– Safety Directed Analysis– Exposure Time/Latency Analysis– Failure Modes and Effect Analysis (FMEA)– Common Cause Analysis

• Safety Risk Assessment – HMI Hazards Sufficiently Controlled

Page 10: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

10

Safety Assurance Process Requirements

Audits/Reviews

Artifacts

Reports/Deliverables

• Plans and Standards• System and Component

Specifications, Requirements, and Design Capture

• System Safety Assessments• Component Implementation• System Integration• Acceptance

Evidence Required

Vendor Vendor/FAA

Baseline

Page 11: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

11

Assurance Approval Process

WIPPApproval

Safety Assurance Complete

Monitor Development & Validation Complete

Safety Assessments & Requirements

Verification

Establish Baseline of

Safety Artifacts

Safety Design Approval

Page 12: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

12

Assurance Approval

• WAAS Program Office Establishes a WAAS Integrity Performance Panel (WIPP)– Role is to Establish Technical Approach and Solution for

Meeting Integrity

• WIPP Comprised of WAAS Experts … (FAA, Academia, Industry)

• WIPP Meetings with Contractor Held Monthly to Review and Evaluate Contractor’s Progress

Page 13: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

13

WIPP Tasks

SV 19

Iono Tropo WRS RFI,Multipath

Cycle Slips

GPSEphemeris

L1/L2Biases

WRS Clocks

A Great Stew of Feared Events Each Requiring:• Threat Model• Detection Algorithm• P(HMI) Analysis!

Target Levelof Safety

•Coding•Operating System•Computer HW•etc.Algorithm

Contribution to HMI

Level D SW

Page 14: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

WIPPGuidance

Ready for Test

WIPPApproval

WIPPApproval

WIPPApprovalWIPP

Guidance

HMI Analysis

Algorithm Development

ProvabilityAssessment

WIPPApproval

WIPP Role

DefineAnalysis

Methodology

ThreatModelMonitorDesign &Trades

Evaluate with FieldData + Threat

AlgorithmDefinitionDocument(ADD)Coding SpecHMI Plan &Prelim Results

Prototype& Validate

OperationalSoftware

Sys.Integ.

Prelim.Eval.

CollectData/

DevelopTools

Data-Driven

Analysis

TheoreticalAnalysis

HMI AnalysisReport

Page 15: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

15

WAAS Design Assurance

• WAAS Design Assurance can be Broken down into Three Major Areas– Software Assurance– Hardware Assurance– Algorithm (Monitor) Assurance

• Assurance that Hazards were Sufficiently Mitigated was Evaluated by Means of a Fault-Tree Analysis (FTA)

• Events in Fault Tree Broken Down by Hardware, Software and Algorithm (Monitor) Contributions

Page 16: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

16

Software Assurance Approach

• Assurance Level Requirement Assigned According to RTCA/DO-178B Guidelines

• Assigned According to the Hazard Severity Caused by Software Failure Based on the WAAS FTA– HMI – Assurance Level B (Where Software Failure is a

Single Point Cause of HMI Without Mitigation or Control; or Software Function Provides Mitigation/Control of an HMI Failure Condition)

– LOC – Minimum Assurance Level D (Where Software Failure is a Single Point Cause of LOC Without Mitigation or Control)

• Software PHMI Contribution is Assigned Based on Assurance Level Achieved– HMI – Assurance Level B Software is Assigned a Failure

Probability of Zero (0); Assurance Level D Software is Assigned a Failure Probability of One (1)

– LOC – Assurance Level B or D Software is Assigned a Failure Probability of Zero (0)

• Monitor Algorithms Failure to Detect Probability is Assessed Based on Probabilistic Analysis

Page 17: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

17

Software Assurance – Alternate Means

• Some Level D Functions Required Assurance • Developed a Process called Safety Directed

Analysis (SDA)– Software Tested and Analyzed to Assure Output Always

What is Expected– Result was Assignment of a Zero Probability for Failure in

Fault Tree– Process and Results were Coordinated with FAA

National Resource Specialist (NRS) for Software

Page 18: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

18

Hardware Assurance

• Assurance Completed at a Functional Level by means of a Failure Modes and Effects Analysis

• Failure Rates of Hardware Utilized were Based on Several Years of Empirical Data

• Exposure Times for Failures Captured as Part of Latency Analysis and Rolled into the Fault-Tree

• Un-trusted Communications Assured by Means of Cyclical Redundancy Checks (CRCs)

Page 19: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

19

Algorithm Assurance

• Algorithms Developed to Detect Both Internal and External Faults– Internal Faults:

Data Provided to Monitors from Un-trusted Components was Assumed to be Corrupted

Required Analysis of all Inputs to the Safety Processor

Analysis Approach called Safety Processor Input Analysis (SPIA) Considered Corrupted Data, Absent Data and Data Timeliness and the Algorithms (Monitors) ability to Detect Such Conditions

– External Faults or Environmental Conditions Evaluated by Means of an HMI Analysis Methodology

Page 20: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

20

COTS Use on WAAS

• Applied to Level D (or Lower) WAAS Functions– IBM AIX OS– Network Management Software– Compilers, CM Tools

• Required to Meet All Applicable RTCA/DO-178B Objectives

• Assured in Accordance with Allocated Functional Requirements Within the WAAS Application

Page 21: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

21

WAAS Network FTA

Any combination of active data at user

antenna causes MI and WAAS FTD within TTA

LNAV-001

WAAS GEO broadcasts a calculated underbound

UDRE/GIVE leading to MI

WAAS GEO broadcasts TSWM that got

corrupted between WMS CMP and GUS

WMP

WAAS GEO broadcasts TSWM that got

corrupted after Tandem CRC decoding

WAAS GEO broadcasts TSWM that got

corrupted after 24-bit CRC encoding and SLB

FTD

WAAS GEO broadcasts ranging signal that is MI

WAAS GEO broadcasts valid UDRE that

becomes MI

LNAV-001

LNAV-100 LNAV-200 LNAV-300 LNAV-400 LNAV-500 LNAV-700

Page 22: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

WAAS Network FTA

RFU

SGS (see figure D for details)

WSG

WMP2

WMP1

GUSP

GUS

TCN

ANH-ROUTER

BCN

TCS

ETHERNET HUB

ETHERNET

SWITCH

CP1

COMPARATOR

ETHERNET HUB

COMPARATOR

SP1

SP2

CP1

CP2

ETHERNET

SWITCH

WMS (see figure C for details)

TCN

BCN

TCS

WRS x 25 (see figure A1 for details)

DCP

DCP

WRE-A (see figure A2 for details)

WAAS RECEIVER

WAAS RECEIVER

GPS ANTENNA

GPS ANTENNA

WRE-B

ANH-ROUTER

32 BIT CRC 24 BIT CRC

GEO SatelliteGPS Satellite

LNAV-100 LNAV-200 LNAV-300 LNAV-400

LNAV-500

LNAV-700

Page 23: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

23

WAAS Network FTA – Data Transmission

WAAS GEO sent TSWM that got corrupted after

24-bit CRC encoding

WAAS GEO broadcasts TSWM that got

corrupted after 24-bit CRC encoding and SLB

FTD

WMPs short loopback (SLB) FTD TSWM that got corrupted after 24-

bit CRC encoding

LNAV-401

LNAV-400

LNAV-402

G= 7.95E-23

G= 2.12E-04 G= 3.75E-19

Page 24: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

24

WAAS Network FTA – SW Failure

Bias error exists leading to MI

Bias error exists and Range Domain Monitor

(RDM) FTD

RDM FTD bias errors

LNAV-192

LNAV-114

LNAV-193

G= 4.50E-08

G= 1.00E+00 G= 4.50E-08

Note: LNAV-192 in this example is the result of any one of satellite L1-L2 bias error, receiver L1-L2 bias error, or receiver clock bias error. Since the error can result from untrusted software or hardware in any of these sources, the worst-case probability is used.

Page 25: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

25

Safety Assurance Process Compliance Verification

• FAA and Contractor Assure System Safety Program Plans (SSPPs) Address Safety Assurance Process Requirements– Contractor SSPP

Captures When and How Requirements are Satisfied

– FAA SSPPSchedule of Reviews Contingent on Contractor Plan

Establish Audit/Review Teams and Define Roles/Responsibilities

Seek Plan Approval

• Safety Assurance Audit and Review Activity– Contractor Coordinates Artifact Schedule and Availability– FAA Verifies Artifact Compliance with SAPR– Completion of Activity Results in Safety Artifact Baseline

Page 26: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

26

Safety Assurance Documentation Baseline

• Established to Provide Evidence of Compliance with the WAAS Safety Assurance Process Requirements

• Includes:– Program Planning and Standards Definition– System Specification, Requirements Definition, and

Design– Component Specification, Requirements Definition and

Design– System Safety Assessment– Component Implementation– System Integration– Configuration Management– Quality Assurance– Safety Assurance Process Compliance

Page 27: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

27

Program Plans and Standards

– System Engineering: System Engineering Management Plan, Reliability Program Plan

– Software Engineering: Software Development Plan, PSAC, SAS and Supporting Process and Standards Notebooks

– System Test: Master Test Plan, Development Test Plan, Production Acceptance Test Plan

– System Safety: System Safety Program Plan, Supporting Process/Procedure Notebooks, Audit Records

Page 28: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

28

System Specification and Design

• System Specification• System Safety Architecture Description• Safety Requirements Traceability Matrix

Page 29: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

29

Component Specification and Design

• Prime Item Development Specifications• Software Requirement Specifications• Software Design Documents• Software Product Specifications• Algorithm Description Documents• WIPP Minutes

Page 30: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

30

System Safety Assessment

• System Hazard Analysis Report• Fault Tree Analysis• Algorithm Contribution to HMI• Safety Processor Input Analysis• Safety Directed Analyses• Qualitative Assessments• Failure Modes and Effects Analyses/Summaries

(FMEAs/FMESs)• Failure Rate Source Documentation• Common Cause Analysis• Hazard Closure Records

Page 31: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

31

Component Implementation

• Hardware Test Plans, Test Procedures, and Test Reports

• Software Test Plans, Test Procedures, and Test Reports

• Software Test Folders/Regression Test Folders

Page 32: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

32

System Integration

• Design Qualification Test Procedures and Reports

• Reliability Test Procedures and Report

Page 33: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

33

Configuration Management

– CM Plan– Configuration Audit Plan– Change Control Records– System Configuration Index

Page 34: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

34

Quality Assurance

– Quality Control Program Plan (or Quality System Plan)– Quality Audit/Review Logs

Page 35: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

35

Safety Assurance Process Compliance

• System Safety Program Plan• Safety Assurance Audit Process• Safety Audit Records• WAAS Safety Assurance Configuration Item

List (SACIL)– Identifies List of Artifacts Required for Proof of SAPR

Compliance– Defines Artifact Association to SAPR Requirements– Asserts Configuration Control over Artifacts Under Review– Creates and Maintains an Artifact Baseline Required for

Formal Approval

Page 36: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

36

SACIL Excerpt

SID Cat. Artifact Doc. # Rev Date Name StatusReleased Is it

CurrentFAA Approved

Audit/ Review Status

Discrepancies /Rationale

Other Comments Availability Change

181, 231, 251 PP A013-002 A 5/18/2004 FOC Systems Engineering Management Plan (SEMP)

X X Approved SEMP re-released for FOC. Replaces A013-001.

FOC

367, 370, 384, 395, 410, 460, 464

Tch A014-005 1/22/2001 ADD for WAAS Network Time Monitor X X Approved Accepted Reviewed 2/22/2002

367, 370, 384, 395, 410, 460, 464

Tch A014-006 B 5/7/2003 ADD for WAAS GIVE Monitor X X Accepted Electronic delivery

367, 370, 384, 395, 410, 460, 464

Tch A014-007 D 5/20/2003 ADD for WAAS Range Monitor X X Accepted Electronic delivery

367, 370, 384, 395, 410, 460, 464

Tch A014-008 F 1/16/2002 ADD for Code Noise and Multipath Monitor X X Approved Accepted Reviewed 2/22/2002

367, 370, 384, 395, 410, 460, 464

Tch A014-009 B 1/23/2002 ADD for UDRE Monitor X X Approved Accepted Reviewed 2/22/2002

367, 370, 384, 395, 410, 460, 464

Tch A014-010 A 9/12/2001 ADD for User Position Monitor X X Approved Accepted Reviewed 2/22/2002

337, 345, 354, 357, 360, 364, 374, 377, 380, 382, 383, 385, 390, 397, 404, 407, 420, 424, 460, 464, 951

Tch A014-011 B 4/28/2003 Algorithm Contribution to HMI for the Wide Area Augmentation System

X X Accepted Electronic delivery

Page 37: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

37

Maintenance of WAAS Safety Baseline

• Rationale for Safety Requirements a Must• Requirements Traceability • Baseline Definition and Control

Page 38: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

38

Questions?

Page 39: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

39

Backup Slides

Page 40: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

40

System Development and Safety Assurance Process Model

Program Plans and Standards Safety Assurance Reviews:-Plans and Standards-Verification Procedures-Verification Results-Configuration Conformance-QA Results Audits

Program Plans and Standards:-Configuration Control Plan-Quality Assurance Plan-Development Plans-Verification Plans-Acceptance Plan-Development Standards-Verification Procedures

Verification and Validation Reviews

= Contractor Activities = Safety Team Safety Assurance Activities

= Contractor Verification and/or Validation Activities

= Intersecting Paths are Connected

= Intersecting Paths are not Connected

Legend:Angled Arrows indicate data required for DO-178B Review

= Contractor Configuration Control and Quality Assurance Activities

System and Component Safety Assurance Reviews:-Specifications-System Architecture-Requirements-Designs-SW DO-178B Compliance-Procedures-Verification Procedures and Results-Validation Criteria-Configuration Conformance-QA Results Audits-Operating and Maintenance Procedures

Component Test Safety Assurance Reviews:-Test Procedures-Test Results-Validation Criteria-Configuration Conformance-QA Results Audits

System Integration Safety Assurance Reviews:-Test Procedures-Test Results-Validation Criteria-Configuration Conformance-QA Results Audits

Safety Assessment Safety Assurance Reviews:-Qualitative/Quantitative FTAs-Common Cause Analysis-FMEAs, FMESs, Failure Rates, Exposure Times-Procedures-Configuration Conformance-QA Results Audits

System and Component Specification, Requirement, and Design Capture:-System Specifications-System Architecture-HW and SW Requirements-HW and SW Designs-Component Verification Procedures-Operating and Maintenance Procedures-Validation and Verification Procedures

Component Implementation:-As-Built Components-Test Procedures

System Integration:-As-Built Ssytem-SystemTest Procedures

System Safety Assessments:-Qualitative/Quantitative FTAs-Common Cause Analysis-FMEAs, FMESs, Failure Rates, Exposure Times-Procedure Assessments

Acceptance:-Deliverables-Conformance-Demonstration-Delivery

Configuration Control

Quality Assurance

Page 41: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

41

Overview of Safety Assurance Activities

VV/CC/QA

Ver

if / V

alid

/CC

/QA

VV/CC/QA VV/CC/QA

VV/CC/QA

Verif / Valid/CC/QA

Verif/Valid/CC/QA

Algorithm failurerates

CRC failure rates

Quantitative HMI FTA

HMI FT base eventexposure times

HW HMI failure ratesfrom FMEA

System architecture & functional descriptions

Algorithm data

CRC data

HW componentdata

Other monitoringcheck data

QualitativeHMI FT structureSafety directed analysis & test

of critical level D SW functions

Quantitative HMI FTAassurance review

QualitativeHMI FT structure

assurance reviewSafety directed analysis & test

assurance review

Alg failure rates assurance review

CRC failure ratesassurance review

HW HMI FMEAassurance review

HMI FT base eventprobability calculations

HMI FT base eventexposure timesassurance review

SW componentdata

Designassurancereviews

HMI FT base eventprobability calculations

assurance reviews

= Data flow - intersecting pathes are connected.

= Data flow - Intersecting pathes are not connected.

Legend:

= Safety team safety assurance activites.

= Raytheon activites.

= Raytheon verification, validation, config control, & QA activities (placed behind corresponding Raytheon development activities)

Level B SWDO-178Bcompliancereview

Proceduralmitigationassessment

Proceduralmitigationcert review

Start

End

Page 42: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

42

PHMI Analysis - What Is It?

• Probability of Monitor Missing an Error that would cause HMI has to be less than the assurance level (on order of 10-7)

• DO-178B Level D inputs can NOT cause HMI when integrity monitor is operating as designed

• Test 30 Days of WAAS Data (UDREI & GIVEI) for Overbounding

• Evaluate Worst Case Convolutions of UDREI and GIVEI Histograms

Page 43: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

43

PHMI Analysis - How Does it Support FTA

HMI Faulty WAAS

SP receives Faulty Inputs

HMI occurs due to SP does not catch the Fault

2 3

HMI Fault-Free WAAS

1

HMI: Actual Error > Protection Level

Page 44: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

44

PHMI Analysis - How Does it Support FTA

10-7 P(HMIE) + P(HMIEC)

P (MD error > cause hazard error enters monitor) < Allocation to Algorithm HMI

From UDRE, GIVEmessage data determineprotection level

P(fail = 1) ~10-8

• Output of Algorithm PHMI Analysis Provides the Probability of Missed Detection based on a Fault free WAAS

• SPIA, SDA, FMECA and Latency Analysis Provide Probability of Missed Detection based on Faulted WAAS (SDA and SPIA qualitative)

Page 45: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

45

SPIA - What is it?

• Analysis that Determines the Erroneous Data Inputs that Could Result in HMI being Output from the Safety Processor (SP)– Define Inputs to each Monitor

– Identify any Input or Combination of Inputs that Could result in HMI

Input data Conditions include: corrupted data, absent data or data timeliness

Determine if the fault condition(s) is covered by existing monitors

Determine the Dependency or Relation to Other Monitors

– Backtracks the contributing data to the input source

– Assess the likelihood for the contributing data being corrupted

Page 46: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

46

SPIA - How does it support FTA?

• SPIA Complements and Validates the FT– Identifies Events that Cause HMI

Verifies Events are Captured by FT; orContributes to FT Completeness

– New Failure Event Identification

– Qualitative Event Probability Assessment

– Contributes to ResolutionNo Action for Extremely Low Event Probability; orSpecific Identification of Level D Software Functions

Which may be Assured by SDA; orNeed for Design Change to Mitigate Failure Event

Page 47: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

47

SDA - What is it?

• Qualitative Analysis of DO-178B Level D critical software functions identified in the WAAS fault tree – Critical Level D software functions are defined as those

that prevent satisfaction of WAAS safety performance requirements

For Fault Tree Analysis, Level D software has a failure probability of 1

– Safety Directed Analysis is applied to the Level D software that performs these critical functions

The SDA demonstrates through analysis and test that software threats can be assigned a failure probability of 0

Ensures that the Level D software is reliable enough to support the WAAS System safety performance

Page 48: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

48

SDA - How Does it Support FTA?

• Successful SDA Eliminates Software Failure Events from the Fault Tree

• Failed SDA assessments will identify Level D functions that must be mitigated– Software design change will be implemented to monitor

the Critical Level D function or rehost it to Level B

• Fault Tree will be modified to incorporate the updated design and reduced PHMI

Page 49: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

49

FMEA/FMES - What is it?

• Failure Modes and Effects Analysis/Summary• Systematic, bottom-up method of identifying:

– Failure modes of a system, component, or function and effects on next higher architecture level

– Failure rate or probability of occurrence of failure modes at each level (limited to hardware on WAAS)

– Also contains ability to detect severity of failure

• May be performed at any architectural level– For WAAS most done at LRU level

Page 50: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

50

FMEA/FMES - How Does it Support FTA?

ES hardware failures causeTSWM to be corrupted

NPA054ES

FTA

ENB 1-6-30(also doc’d in FMECA database)Event failure rate is the sum of the failure rates

of the failure modes that map to it. Eventprobability is calculated from the event failure rate. Mapping of failure modes to events

- implied ‘OR’ gate, sums failure ratesfor modes. Mapping documentedin ENB 1-6-30.

Failure mode failure rates are the sum of all the cause failure rates that map to the failure mode.

Corrupted or intermittent datafrom port 1 of ES 1 . . .

Corruption/noise in all external interfaces . . .

Estimated MI failure rate from HPR database . . .

FMECA

Failure Modes

Events in the FTAhave one or morefailure modes mappedto it. A group of suchfailure modes is called aFailure Mode Cluster.This cluster is CVES_WM:MI.

Data extracted from ENB 1-6-30CFigure 3.1-1

Page 51: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

51

Exposure Time/Latency Analysis - What is it?

• Analysis of Exposure Time:– e.g. 150 secs, PA; 1 Hr NPA

System element under analysis known good at beginning of exposure time.

• Latency accounts for components not continuously tested– The estimate of how long a failure mode can be

present in an equipment or function before it is mitigated (or removed).

Requires failure detection, reporting, and mitigationGround rules: Monitors must be in Level B software and

not already included in FTA at higher level.

Page 52: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

52

Exposure Time/Latency Analysis - How Does it Support FTA?

Reliability (Probability of Success) = e-FR*t

Where FR = Failure Rate = 1/MTBFt = Exposure Time or LatencyExposure time for PA = 150 seconds, other phases 1 hour

Probability of Failure (Pf) = 1- Probability of Success

Safety Computer Comparator Example:

Safety Computer (SC) failure rate = 5.08 x10-5 Failures/HourIf testing were continuous, t = 150 seconds (during PA),

Pf = 2.11 x10-6

Accounting for SC MTBF (19,704 Hours) Latency, Pf = 0.632Accounting for C&V MTBF (872 hours), Pf = 0.043

Lowest FTA element requires Probability of Failure

Source of failure rates (ENB 1-13-107)

Page 53: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

53

SSAD

• Provide System Architectural and Design Details

• Describes the Safety Architecture from System to Component to Capability Level

• Provides Requirements Rationale and Tracing• Identifies Analysis to Validate and Verify

Requirements at System and Component Level– System Satisfies Integrity Requirements– Failure Modes and Effects Mitigated

Page 54: 1 System Certification (Safety Assurance) of WAAS Deane Bunce SBAS Approval Workshop 21-22 June 2005

54

System Safety Architecture Description

SSAD A121

CapabilityRequirements

(CIs, SRSs)

ComponentCapabilities

System SafetyRequirement(PHMI, Cont’y)

System Safety ArchitectureComponents

System Safety Analyses

FTA

CCA

HEA

Hazards

Algorithm Design Documents

Verification Methods and Results

SDFs

System Test Procedures

System Test Results

OperatorSafety

Procedures

IETM/TIBSafety

Features

IETM/TIB Verification

Operational SequenceDiagrams

SoftwareSafety

Features

FeatureRequirements

(SRSs)

SAC Analyses

Alg PHMISPIA