do178c overview duncan brown - rolls royce nick tudor - qinetiq

36
DO178C Overview DO178C Overview Duncan Brown - Rolls Royce Duncan Brown - Rolls Royce Nick Tudor - QinetiQ Nick Tudor - QinetiQ

Upload: adela-barber

Post on 24-Dec-2015

215 views

Category:

Documents


0 download

TRANSCRIPT

DO178C OverviewDO178C Overview

Duncan Brown - Rolls RoyceDuncan Brown - Rolls Royce

Nick Tudor - QinetiQNick Tudor - QinetiQ

AgendaAgenda

Why we need a new documentWhy we need a new document Structure of the Special CommitteeStructure of the Special Committee Progress to dateProgress to date Specific overview of SG6 – Formal MethodsSpecific overview of SG6 – Formal Methods

Why we need a new documentWhy we need a new document

The DO-178B / ED-12B The DO-178B / ED-12B Takes into account the inputs, constraints, requirements from Takes into account the inputs, constraints, requirements from

all the stakeholders :all the stakeholders :– Consensus between Airframe Manufacturers, Equipment suppliers, Consensus between Airframe Manufacturers, Equipment suppliers,

Certification Authorities Certification Authorities

DO-178B / ED-12B was written as much as possible as a DO-178B / ED-12B was written as much as possible as a “requirements oriented” document“requirements oriented” document– try to stay not prescriptive on the means > less sensitive to technology try to stay not prescriptive on the means > less sensitive to technology

evolutionevolution

DO-178B / ED-12B is a “process oriented” document :DO-178B / ED-12B is a “process oriented” document :– In 1992, we could not imagine to get sufficient assurance on the software In 1992, we could not imagine to get sufficient assurance on the software

product just by assessing it product just by assessing it

More than 10 years of use do not reveal major safety flawsMore than 10 years of use do not reveal major safety flaws

So, why changing ?So, why changing ? Because FAA wants to...Because FAA wants to...

But also because we need it …But also because we need it … DO 178B / ED-12B was released in 1992 :DO 178B / ED-12B was released in 1992 :– In 1992, Software engineering was 24 years old...In 1992, Software engineering was 24 years old...

– In 2005, Software Engineering is 50% older, as compared to 1992… In 2005, Software Engineering is 50% older, as compared to 1992…

Changing for a better consensus, taking into account :Changing for a better consensus, taking into account :– Legacy from the clarification groupLegacy from the clarification group

– Lessons learnt in applying DO-178B / ED-12BLessons learnt in applying DO-178B / ED-12B

– Newly available industrial meansNewly available industrial means

A number of issues that need to be resolvedA number of issues that need to be resolved

EUROCAE WG 52/RTCA SC 190 main points : EUROCAE WG 52/RTCA SC 190 main points : – Should Should safety specific considerationssafety specific considerations be addressed in DO178 ? be addressed in DO178 ?– Configuration Control requirement Configuration Control requirement too high for tooltoo high for tool – Integration process vs.Integration process vs. integration integration testing testing – FindingFinding Common mode errorsCommon mode errors not really addressed not really addressed – Not enoughNot enough goal oriented goal oriented : :

DO-178B/ED-12B forces the applicant to address the objectives directly DO-178B/ED-12B forces the applicant to address the objectives directly which may not be applicable for a given technology which may not be applicable for a given technology

Objectives in Annex tables are not all objectives--some are specific means of Objectives in Annex tables are not all objectives--some are specific means of compliance (MC/DC), so an alternative means of compliance are not feasiblecompliance (MC/DC), so an alternative means of compliance are not feasible

– COTS issueCOTS issue not addressed in the same way in DO-178B & DO-278 not addressed in the same way in DO-178B & DO-278 Recent development shows us that these issues are not Recent development shows us that these issues are not

theoretical…theoretical…

Fall 2004:RTCA Ad-hoc SW meeting Fall 2004:RTCA Ad-hoc SW meeting Ad-hoc SW meeting convened by RTCA in October :Ad-hoc SW meeting convened by RTCA in October :

– US members : FAA, NASA, Boeing, Honeywell, Rockwell, P&W, United US members : FAA, NASA, Boeing, Honeywell, Rockwell, P&W, United Technologies, Avidyne, Consultants, Technologies, Avidyne, Consultants,

– European guests (D. Hawken, P. Heller, R. Hannan, G.Ladier)European guests (D. Hawken, P. Heller, R. Hannan, G.Ladier)

161 issues identified on DO-178B161 issues identified on DO-178B– 71 issues : clarification or consistency71 issues : clarification or consistency– 90 issues : new guidance. Helpful or needed for most of them90 issues : new guidance. Helpful or needed for most of them

Amongst the top 20 issues : Amongst the top 20 issues :

– Model based developmentModel based development

– Development tool qualification criteriaDevelopment tool qualification criteria

– Analyzing computer resource usage and WCETAnalyzing computer resource usage and WCET

– Separate document for CNS/ATMSeparate document for CNS/ATM

– Security GuidanceSecurity Guidance

– Static verificationStatic verification

Need method to validate models, need expressive requirements languages to be used to verify models. Do we now also need

guidance on how to build models, validate models, etc.

The criteria for development tool qualification is too rigorous.Memory and timing margins,

WCET, … difficult to analyze. No "objective" that addresses WCET

Consider adding the DO-278 criteria to DO-178[ ]Really needs to be addressed in

the safety assessment and ARP 4754/4761 - then flows down to

the software.

Section 6 of 178B requires "testing". There are now other techniques that are more exhaustive. Analyzing source code

using tools. Not clear how to address structural coverage and dead code issues

when using such tools.

Terms of referenceTerms of reference The ad-hoc SW group has proposed to : The ad-hoc SW group has proposed to :

– Maintain the current objective-based approach for SW assurance. Maintain the current objective-based approach for SW assurance. – Modify (Modify (minimum changesminimum changes) DO-178B/ED-12B > DO-178C/ED-12C) DO-178B/ED-12B > DO-178C/ED-12C. . – Develop Develop rationalerationale for each objective and package for each objective and package– Develop Develop guidelinesguidelines that provide that provide informationinformation ( ( DO-248B/ED-94BDO-248B/ED-94B))– Develop Develop supplementssupplements to document technology-specific or method- to document technology-specific or method-

specific specific guidanceguidance. .

In American Guidance is stronger than GuidelinesIn French Guidance is Directives, Guidelines are Recommandations

Supplements : Supplements :

–May May provide alternate means to satisfy DO-178C/ED-12C objectivesprovide alternate means to satisfy DO-178C/ED-12C objectives

–Possible supplements : Tool qualification, Model-based development, Possible supplements : Tool qualification, Model-based development, Object-oriented technology, Formal methodsObject-oriented technology, Formal methods,…,…

–On this basis, RTCA and EUROCAE agreed to commence new On this basis, RTCA and EUROCAE agreed to commence new committeescommittees

Structure of the Special Structure of the Special Committee/Working GroupCommittee/Working Group

Joint between Joint between

EUROCAE and RTCAEUROCAE and RTCA

WG-71/SC-205 StructureWG-71/SC-205 StructureJoint committee WG71/ SC205

Executive Committee

Joint Chair: Gerard Ladier [Airbus]Joint Chair: Jim Krodel [P&W]Joint Sec: Ross Hannan [Sigma]Joint Sec: Mike DeWalt [CSI]FAA Rep: Barbara Lingberg [FAA]

SG1: Sub Group Coordination

SG2: Issue Review & Rationale

SG3: Tools

SG4: Model Based Design

SG5: Object Oriented Technology

SG6: Formal Methods

SG7: Safety CNS/ATM

Membership from Airframers, avionics suppliers, certification authorities, engine manufacturers, CNS/ATM specialists, Space community, consultants

Progress to dateProgress to date

Joint between Joint between

EUROCAE and RTCAEUROCAE and RTCA

Sub Group 1 – Document IntegrationSub Group 1 – Document IntegrationChairsChairs Tom Ferrell (Ferrell and Associates Consulting) andTom Ferrell (Ferrell and Associates Consulting) and

Ron Ashpole (Bewicks Consulting)Ron Ashpole (Bewicks Consulting)

Focussing on issues within current documentFocussing on issues within current documente.g.e.g.– Annex A tables do not accurately reflect the real requirements in the main document. Annex A tables do not accurately reflect the real requirements in the main document.

Hence causes people who focus only on the Annex tables to have issues with the Hence causes people who focus only on the Annex tables to have issues with the DERDER

Information Paper system for managing work productsInformation Paper system for managing work products Technology Supplement TemplateTechnology Supplement Template ““Hooks” in DO-178C core document to link in Technology Hooks” in DO-178C core document to link in Technology

SupplementsSupplements Changes to core document such as ErrataChanges to core document such as Errata

Sub Group 2 - Issues & Rationale Sub Group 2 - Issues & Rationale ChairsChairs Will Struck (FAA)Will Struck (FAA)

Ross Hannan (Sigma Associates)Ross Hannan (Sigma Associates)

Believe it or not – they lost the rationale!Believe it or not – they lost the rationale!– Discussion on why we use MCDC (or not)Discussion on why we use MCDC (or not)

– Will coordinate activities relating to dead deactivated code (different for Will coordinate activities relating to dead deactivated code (different for some technologies?)some technologies?)

Rationale for objectives should be released to WG May 07Rationale for objectives should be released to WG May 07 Also new objectives will be set as a resultAlso new objectives will be set as a result

Sub Group 3 – Tool QualificationSub Group 3 – Tool QualificationChairsChairs Leanna Rierson (Digital Safety Consulting)Leanna Rierson (Digital Safety Consulting)

Frederick Pothon (ACG Solutions)Frederick Pothon (ACG Solutions)

Likely new approach for tool qualification. Likely new approach for tool qualification. – Aim is to develop a qualification approach that meets the needs of Aim is to develop a qualification approach that meets the needs of

development tools and keep the current approach for current development tools and keep the current approach for current classes of verification tools (as far as possible), classes of verification tools (as far as possible),

– enable re-use of tool qualification data, enable re-use of tool qualification data, – allow emerging tool technologies, allow emerging tool technologies, – identify users and developers, identify users and developers, – objective based approach, objective based approach, – used by multiple domains (system/software). used by multiple domains (system/software). – Propose a re-write of section 12.2 with the aim to assess the Propose a re-write of section 12.2 with the aim to assess the

impact of the tool rather than to determine the level directly. impact of the tool rather than to determine the level directly. – New DOXXX/EDXXX document : objective based, by levelNew DOXXX/EDXXX document : objective based, by level– Has some merit in that tool manufacturers don’t necessarily have to Has some merit in that tool manufacturers don’t necessarily have to

be software certification experts. be software certification experts.

Sub Group 4 – Model Based Design (& Sub Group 4 – Model Based Design (& Verification)Verification)ChairsChairs Mark Lillis (Hamilton Sundstrand)Mark Lillis (Hamilton Sundstrand)

Pierre Lionne (EADS)Pierre Lionne (EADS)

Split in agendasSplit in agendas– Some wish to do things because they have a technologySome wish to do things because they have a technology– Others wish to go back to first principles (as advised by Exec)Others wish to go back to first principles (as advised by Exec)

Opportunity is being lost as nothing abstract in what needs Opportunity is being lost as nothing abstract in what needs to be demonstrated by any MBD is being discussedto be demonstrated by any MBD is being discussed– Not addressing syntax/semanticsNot addressing syntax/semantics– Nothing said about relationship to existing objectivesNothing said about relationship to existing objectives– Diving into low level issuesDiving into low level issues

Biggest discussion topics to date have beenBiggest discussion topics to date have been– What is the difference between High-Level and Low-Level What is the difference between High-Level and Low-Level

requirements?requirements?– What is source code?What is source code?

Sub Group 5 – OO TechnologiesSub Group 5 – OO TechnologiesChairsChairs Jim Chelini (Verocel Inc)Jim Chelini (Verocel Inc)

Peter Heller (Airbus)Peter Heller (Airbus)

Unlikely to be much new because of FAA OOTiA workUnlikely to be much new because of FAA OOTiA work– Aim would be to withdraw this following completion of DO-178C Aim would be to withdraw this following completion of DO-178C

and supplementsand supplements– However not clear if an OO supplement is requiredHowever not clear if an OO supplement is required

Much initial work was creating FAQs/Discussion Papers for Much initial work was creating FAQs/Discussion Papers for DO-248 and minor changes for DO-178C – This was DO-248 and minor changes for DO-178C – This was challenged as the intent of the working group is to challenged as the intent of the working group is to consolidate guidance. consolidate guidance.

Will address some structural coverage issues specific to Will address some structural coverage issues specific to OO and polymorphismOO and polymorphism

Sub Group 6 – Formal MethodsSub Group 6 – Formal MethodsChairsChairs Kelly Hayhurst (NASA)Kelly Hayhurst (NASA)

Duncan Brown (Rolls-Royce)Duncan Brown (Rolls-Royce)

Established a definite need for a Formal Methods Established a definite need for a Formal Methods technology supplementtechnology supplement

Decided to separate the case studies and tutorial Decided to separate the case studies and tutorial information into a discussion paperinformation into a discussion paper

Proposed rewrite of Section 6 – Verification because it Proposed rewrite of Section 6 – Verification because it made the use of technology supplements easiermade the use of technology supplements easier

Sub Group 7 – CNS/ATM & SafetySub Group 7 – CNS/ATM & SafetyChairsChairs Don Heck (Boeing)Don Heck (Boeing)

David Hawken (National Air Traffic Services Limited)David Hawken (National Air Traffic Services Limited)

Issues surrounding merge of DO-278 (CNS/ATM) and DO-Issues surrounding merge of DO-278 (CNS/ATM) and DO-178 (Airborne)178 (Airborne)– Likely to happenLikely to happen– Domain specific sections where applicable – annexes?Domain specific sections where applicable – annexes?

Links to system safety considerationsLinks to system safety considerations Decided that there is not to be Level A+Decided that there is not to be Level A+

Why section 6 must changeWhy section 6 must change

The ProblemThe Problem

DO-178B Section 6 calls explicitly for Review, Analysis and DO-178B Section 6 calls explicitly for Review, Analysis and Test rather than setting out the objectives for verification Test rather than setting out the objectives for verification and leaving the applicant to create the appropriate and leaving the applicant to create the appropriate verification plan.verification plan.

However specific methods are deemed the only way to However specific methods are deemed the only way to meet specific objectives.meet specific objectives.

There are issues with specific technologies such as Formal There are issues with specific technologies such as Formal Methods, OO and MBD with respect to how the current Methods, OO and MBD with respect to how the current section 6 objectives can be met.section 6 objectives can be met.

The existing material can benefit from additional The existing material can benefit from additional clarification.clarification.

The ProposalThe Proposal

To re-arrange the DO-178B section 6 material according to To re-arrange the DO-178B section 6 material according to life cycle data.life cycle data.

To explicitly state the objectives for the verification of each To explicitly state the objectives for the verification of each life cycle data item.life cycle data item.

To retain the complete testing process under the To retain the complete testing process under the verification of executable object code.verification of executable object code.

To generalise the wording to use verification instead of To generalise the wording to use verification instead of specific methods (Review, Analysis & Test).specific methods (Review, Analysis & Test).

To ensure that DO-178C on its own is as forceful with To ensure that DO-178C on its own is as forceful with regard to testing as DO-178B.regard to testing as DO-178B.

The Verification ProcessThe Verification ProcessSystem

Requirements

High-LevelRequirements

SoftwareArchitecture

Source Code

ExecutableObject Code

(A-2: 3, 4, 5)

(A-2: 7)

A-3.1 Compliance A-3.6 Traceability

A-3.2 Accuracy & Consistency A-3.3 HW Compatibility A-3.4 VerifiabilityA-3.5 Conformance A-3.7 Algorithm Accuracy

A-4.9 ConsistencyA-4.10 HW Compatibility A-4.11 Verifiability A-4.12 Conformance A-4.13 Partition Integrity

A-4.2 Accuracy & ConsistencyA-4.3 HW Compatibility A-4.4 Verifiability A-4.5 Conformance A-4.7 Algorithm Accuracy

(A-2: 6)

A-5.3 Verifiability A-5.4 ConformanceA-5.6 Accuracy & Consistency

(A-2: 1, 2)

A-4.1 Compliance A-4.6 Traceability

A-4. 8 Architecture Compatibility

A-5.1 Compliance A-5.5 Traceability

A-5.2 Compliance

A-6.3 Compliance A-6.4 Robustness

A-6.1 Compliance A-6.2 Robustness

A-6.5 Compatible With TargetA-5. 7 Complete & Correct

Low-LevelRequirements

Compliance: with requirementsConformance: with standards

The Verification Process – Level AThe Verification Process – Level ASystem

Requirements

High-LevelRequirements

SoftwareArchitecture

Source Code

ExecutableObject Code

(A-2: 3, 4, 5)

(A-2: 7)

A-3.1 Compliance A-3.6 Traceability

A-3.2 Accuracy & Consistency A-3.3 HW Compatibility A-3.4 VerifiabilityA-3.5 Conformance A-3.7 Algorithm Accuracy

A-4.9 ConsistencyA-4.10 HW Compatibility A-4.11 Verifiability A-4.12 Conformance A-4.13 Partition Integrity

A-4.2 Accuracy & ConsistencyA-4.3 HW Compatibility A-4.4 Verifiability A-4.5 Conformance A-4.7 Algorithm Accuracy

(A-2: 6)

A-5.3 Verifiability A-5.4 ConformanceA-5.6 Accuracy & Consistency

(A-2: 1, 2)

A-4.1 Compliance A-4.6 Traceability

A-4.8 Architecture Compatibility

A-5.1 Compliance A-5.5 Traceability

A-5.2 Compliance

A-6.3 Compliance A-6.4 Robustness

A-6.1 Compliance A-6.2 Robustness

A-6.5 Compatible With TargetA-5. 7 Complete & Correct

Low-LevelRequirements

Compliance: with requirementsConformance: with standards

The Verification Process – Level BThe Verification Process – Level BSystem

Requirements

High-LevelRequirements

SoftwareArchitecture

Source Code

ExecutableObject Code

(A-2: 3, 4, 5)

(A-2: 7)

A-3.1 Compliance A-3.6 Traceability

A-3.2 Accuracy & Consistency A-3.3 HW Compatibility A-3.4 VerifiabilityA-3.5 Conformance A-3.7 Algorithm Accuracy

A-4.9 ConsistencyA-4.10 HW Compatibility A-4.11 Verifiability A-4.12 Conformance A-4.13 Partition Integrity

A-4.2 Accuracy & ConsistencyA-4.3 HW Compatibility A-4.4 Verifiability A-4.5 Conformance A-4.7 Algorithm Accuracy

(A-2: 6)

A-5.3 Verifiability A-5.4 ConformanceA-5.6 Accuracy & Consistency

(A-2: 1, 2)

A-4.1 Compliance A-4.6 Traceability

A-4.8 Architecture Compatibility

A-5.1 Compliance A-5.5 Traceability

A-5.2 Compliance

A-6.3 Compliance A-6.4 Robustness

A-6.1 Compliance A-6.2 Robustness

A-6.5 Compatible With TargetA-5. 7 Complete & Correct

Low-LevelRequirements

Compliance: with requirementsConformance: with standards

The Verification Process – Level CThe Verification Process – Level CSystem

Requirements

High-LevelRequirements

SoftwareArchitecture

Source Code

ExecutableObject Code

(A-2: 3, 4, 5)

(A-2: 7)

A-3.1 Compliance A-3.6 Traceability

A-3.2 Accuracy & Consistency A-3.3 HW Compatibility A-3.4 VerifiabilityA-3.5 Conformance A-3.7 Algorithm Accuracy

A-4.9 ConsistencyA-4.10 HW Compatibility A-4.11 Verifiability A-4.12 Conformance A-4.13 Partition Integrity

A-4.2 Accuracy & ConsistencyA-4.3 HW Compatibility A-4.4 Verifiability A-4.5 Conformance A-4.7 Algorithm Accuracy

(A-2: 6)

A-5.3 Verifiability A-5.4 ConformanceA-5.6 Accuracy & Consistency

(A-2: 1, 2)

A-4.1 Compliance A-4.6 Traceability

A-4.8 Architecture Compatibility

A-5.1 Compliance A-5.5 Traceability

A-5.2 Compliance

A-6.3 Compliance A-6.4 Robustness

A-6.1 Compliance A-6.2 Robustness

A-6.5 Compatible With TargetA-5. 7 Complete & Correct

Low-LevelRequirements

Compliance: with requirementsConformance: with standards

The Verification Process – Level DThe Verification Process – Level DSystem

Requirements

High-LevelRequirements

SoftwareArchitecture

Source Code

ExecutableObject Code

(A-2: 3, 4, 5)

(A-2: 7)

A-3.1 Compliance A-3.6 Traceability

A-3.2 Accuracy & Consistency A-3.3 HW Compatibility A-3.4 VerifiabilityA-3.5 Conformance A-3.7 Algorithm Accuracy

A-4.9 ConsistencyA-4.10 HW Compatibility A-4.11 Verifiability A-4.12 Conformance A-4.13 Partition Integrity

A-4.2 Accuracy & ConsistencyA-4.3 HW Compatibility A-4.4 Verifiability A-4.5 Conformance A-4.7 Algorithm Accuracy

(A-2: 6)

A-5.3 Verifiability A-5.4 ConformanceA-5.6 Accuracy & Consistency

(A-2: 1, 2)

A-4.1 Compliance A-4.6 Traceability

A-4.8 Architecture Compatibility

A-5.1 Compliance A-5.5 Traceability

A-5.2 Compliance

A-6.3 Compliance A-6.4 Robustness

A-6.1 Compliance A-6.2 Robustness

A-6.5 Compatible With TargetA-5. 7 Complete & Correct

Low-LevelRequirements

Compliance: with requirementsConformance: with standards

The Verification Process – Level EThe Verification Process – Level E

ExecutableObject Code

Comparison of Old -> NewComparison of Old -> New

6.06.0 SOFTWARE VERIFICATION PROCESS SOFTWARE VERIFICATION PROCESS 6.16.1 Software Verification Process Objectives Software Verification Process Objectives 6.26.2 Software Verification Process Activities Software Verification Process Activities 6.36.3 Software Reviews and AnalysesSoftware Reviews and Analyses6.3.1 Reviews and Analyses of the 6.3.1 Reviews and Analyses of the

High-Level Requirements High-Level Requirements a. a. Compliance with system requirementsCompliance with system requirementsb. b. Accuracy and consistencyAccuracy and consistencyc. c. Compatibility with the target computerCompatibility with the target computerd. d. VerifiabilityVerifiabilitye. e. Conformance to standardsConformance to standardsf. f. TraceabilityTraceabilityg. g. Algorithm aspectsAlgorithm aspects

6.3.2 Reviews and Analyses of the 6.3.2 Reviews and Analyses of the Low-Level Requirements Low-Level Requirements

a. a. Compliance with high-level requirementsCompliance with high-level requirementsb. b. Accuracy and consistencyAccuracy and consistencyc. c. Compatibility with the target computerCompatibility with the target computerd. d. VerifiabilityVerifiabilitye. e. Conformance to standardsConformance to standardsf. f. TraceabilityTraceabilityg. g. Algorithm aspectsAlgorithm aspects

6.06.0 SOFTWARE VERIFICATION PROCESSSOFTWARE VERIFICATION PROCESS6.16.1 Software Verification Process ObjectivesSoftware Verification Process Objectives6.26.2 Software Verification Process Activities Software Verification Process Activities 6.36.3 Detailed Guidance for Verification ActivitiesDetailed Guidance for Verification Activities6.3.1 Verification Activities for the 6.3.1 Verification Activities for the

High-Level RequirementsHigh-Level Requirementsa. Compliance with system requirementsa. Compliance with system requirementsb. Accuracy and consistencyb. Accuracy and consistencyc. Compatibility with the target computerc. Compatibility with the target computerd. Verifiabilityd. Verifiabilitye. Conformance to standardse. Conformance to standardsf. Traceabilityf. Traceabilityg. Algorithm aspectsg. Algorithm aspects

6.3.2 Verification Activities for the 6.3.2 Verification Activities for the Low-Level RequirementsLow-Level Requirements

a. Compliance with high-level requirementsa. Compliance with high-level requirementsb. Accuracy and consistencyb. Accuracy and consistencyc. Compatibility with the target computerc. Compatibility with the target computerd. Verifiabilityd. Verifiabilitye. Conformance to standardse. Conformance to standardsf. Traceabilityf. Traceabilityg. Algorithm aspectsg. Algorithm aspects

Comparison of Old -> NewComparison of Old -> New6.3.3 Reviews and Analyses of the 6.3.3 Reviews and Analyses of the

Software ArchitectureSoftware Architecturea. Compliance with high-level requirementsa. Compliance with high-level requirementsb. Consistencyb. Consistencyc. Compatibility with the target computerc. Compatibility with the target computerd. Verifiabilityd. Verifiabilitye. Conformance to standardse. Conformance to standardsf. Partitioning integrity f. Partitioning integrity

6.3.4 Reviews and Analyses of the 6.3.4 Reviews and Analyses of the Source Code Source Code a. Compliance with low-level requirementsa. Compliance with low-level requirementsb. Compliance with the software architectureb. Compliance with the software architecturec. Verifiabilityc. Verifiabilityd. Conformance to standardsd. Conformance to standardse. Traceabilitye. Traceabilityf. Accuracy and consistencyf. Accuracy and consistency

6.3.5 Reviews and Analysis of the Outputs of the 6.3.5 Reviews and Analysis of the Outputs of the Integration ProcessIntegration Process

6.3.3Verification Activities for the 6.3.3Verification Activities for the Software ArchitectureSoftware Architecturea. Compliance with high-level requirementsa. Compliance with high-level requirementsb. Consistencyb. Consistencyc. Compatibility with the target computerc. Compatibility with the target computerd. Verifiabilityd. Verifiabilitye. Conformance to standardse. Conformance to standardsf. Partitioning integrityf. Partitioning integrity

6.3.4 Verification Activities for the 6.3.4 Verification Activities for the Source CodeSource Codea. Compliance with low-level requirementsa. Compliance with low-level requirementsb. Compliance with the software architectureb. Compliance with the software architecturec. Verifiabilityc. Verifiabilityd. Conformance to standardsd. Conformance to standardse. Traceabilitye. Traceabilityf. Accuracy and consistencyf. Accuracy and consistency

6.3.5 Verification Activities for the Executable Object Code6.3.5 Verification Activities for the Executable Object Codea. Completeness and correctnessa. Completeness and correctnessb. Compliance with the high-level requirementsb. Compliance with the high-level requirementsc. Robustness for high and low-level requirementsc. Robustness for high and low-level requirementsd. Compliance with the low-level requirementsd. Compliance with the low-level requirementse. Compatibility with the target computere. Compatibility with the target computer

6.3.5.1 Software Testing6.3.5.1 Software Testing6.3.5.2 Test Environment6.3.5.2 Test Environment

Comparison of Old -> NewComparison of Old -> New

6.3.6 Reviews and Analyses of the Test Cases, 6.3.6 Reviews and Analyses of the Test Cases, Procedures, and Results Procedures, and Results

6.46.4 Software Testing Process Software Testing Process

6.4.1 Test Environment6.4.1 Test Environment

6.4.2 Requirements-Based Test Case Selection6.4.2 Requirements-Based Test Case Selection

6.4.2.1 Normal Range Test Cases6.4.2.1 Normal Range Test Cases

6.4.2.2 Robustness Test Cases6.4.2.2 Robustness Test Cases

6.4.3 Requirements-Based Testing Methods6.4.3 Requirements-Based Testing Methods

6.4.4 Test Coverage Analysis6.4.4 Test Coverage Analysis

6.4.4.1 Requirements-Based Test Coverage Analysis6.4.4.1 Requirements-Based Test Coverage Analysis

6.4.4.2 Structural Coverage Analysis6.4.4.2 Structural Coverage Analysis

6.4.4.3 Structural Coverage Analysis Resolution6.4.4.3 Structural Coverage Analysis Resolution

6.3.6 Verification Activities for the Analyses, Test Cases, 6.3.6 Verification Activities for the Analyses, Test Cases, Procedures and ResultsProcedures and Results

a. Analysis and Test casesa. Analysis and Test cases

b. Analysis and Test proceduresb. Analysis and Test procedures

c. Analysis and Test results c. Analysis and Test results

6.3.6.1 Coverage Analysis6.3.6.1 Coverage Analysis

6.3.6.1.1 Requirements Coverage Analysis6.3.6.1.1 Requirements Coverage Analysis

6.3.6.1.2 Structural Coverage Analysis6.3.6.1.2 Structural Coverage Analysis

6.3.6.1.3 Structural Coverage Analysis Resolution6.3.6.1.3 Structural Coverage Analysis Resolution

{Transferred to section 6.3.5.1}{Transferred to section 6.3.5.1}

{Transferred to section 6.3.5.2}{Transferred to section 6.3.5.2}

{Transferred to 6.3.5} {Transferred to 6.3.5}

{Transferred to 6.3.5} {Transferred to 6.3.5}

{Transferred to 6.3.5} {Transferred to 6.3.5}

{Transferred to 6.3.5} {Transferred to 6.3.5}

{Transferred to section 6.3.6.1}{Transferred to section 6.3.6.1}

{Transferred to section 6.3.6.1.1} {Transferred to section 6.3.6.1.1}

{Transferred to section 6.3.6.1.2} {Transferred to section 6.3.6.1.2}

{Transferred to section 6.3.6.1.3}{Transferred to section 6.3.6.1.3}

Major Comments RaisedMajor Comments Raised

The paper lowers the bar for testing significantly (To zero!)The paper lowers the bar for testing significantly (To zero!) Review and analysis are the only applicable methods for Review and analysis are the only applicable methods for

verification of higher level life cycle data.verification of higher level life cycle data. Testing is the only applicable method for meeting the Testing is the only applicable method for meeting the

verification of the executable object code.verification of the executable object code.

SummarySummary Latest Revision of Paper emphasises the reliance on testing Latest Revision of Paper emphasises the reliance on testing

where there are no other accepted means for verification.where there are no other accepted means for verification. DO-178C used alone needs to be as forceful in the need for DO-178C used alone needs to be as forceful in the need for

testing as DO-178Btesting as DO-178B It now says that “…testing is necessary to ensure that the It now says that “…testing is necessary to ensure that the

executable object code is compatible with the target computer”executable object code is compatible with the target computer” Only the use of approved guidance in conjunction with DO-178C Only the use of approved guidance in conjunction with DO-178C

could alter the amount of testing required.could alter the amount of testing required. Paper to be agreed by SG6 at interim meeting mid-year.Paper to be agreed by SG6 at interim meeting mid-year. Plenary consensus to be sought in Vienna in the Autumn.Plenary consensus to be sought in Vienna in the Autumn. There is “significant” backing for this paper.There is “significant” backing for this paper. This provides a way to use Formal Methods as part of This provides a way to use Formal Methods as part of

certification – The technology supplement will provide the “How”certification – The technology supplement will provide the “How”

IP 601 Rev B (Draft)IP 601 Rev B (Draft)

Microsoft Word Document