Is Proof More Cost-Is Proof More Cost-Effective Than Testing?Effective Than Testing?

Presented by Yin ShiPresented by Yin Shi

OverviewOverview

• IntroductionIntroduction

• The Application: SHOLISThe Application: SHOLIS

• The Programming Language: SPARKThe Programming Language: SPARK

• Proof in the SHOLIS Development Proof in the SHOLIS Development ProcessProcess

• Results, Experiences, and Lessons Results, Experiences, and Lessons LearnedLearned

• Analysis of the ResultsAnalysis of the Results

• Summary and ConclusionsSummary and Conclusions

IntroductionIntroduction• The use of formal development methods The use of formal development methods

on an industrial safety-critical applicationon an industrial safety-critical application• Controversy among software suppliers Controversy among software suppliers

on using formal methodson using formal methods• Developing software using formal Developing software using formal

techniques is indeed possibletechniques is indeed possible• Now becoming Common for project to Now becoming Common for project to

use formal notations to document use formal notations to document specifications and designsspecifications and designs

• Used Z for specification and design, the Used Z for specification and design, the SPARK for codeSPARK for code

The Application: SHOLISThe Application: SHOLIS• SHOLISSHOLIS

– The Ship Helicopter Operating Limits The Ship Helicopter Operating Limits Information SystemInformation System

– It is intended to be used on UK Royal Navy It is intended to be used on UK Royal Navy and Royal Fleet Auxiliary vesselsand Royal Fleet Auxiliary vessels

• Brief System DescriptionBrief System Description– SHOLIS contains a database of Ship SHOLIS contains a database of Ship

Helicopter Operating Limits (SHOLs) Helicopter Operating Limits (SHOLs) – One of the main safety-critical functions of One of the main safety-critical functions of

SHOLIS is to make continual comparisons of SHOLIS is to make continual comparisons of sensor information against a selected SHOLsensor information against a selected SHOL

– The SHOLIS functions are grouped on a The SHOLIS functions are grouped on a number of pages.number of pages.

The Application: SHOLISThe Application: SHOLIS

• Safety RequirementsSafety Requirements– Catastrophic hazardsCatastrophic hazards– Safety-critical and developed to SIL4Safety-critical and developed to SIL4– Non-safety criticalNon-safety critical

The Programming Language: The Programming Language: SPARKSPARK• The executable part of the language is a The executable part of the language is a subset of Adasubset of Ada

• DesignDesign– Logical soundnessLogical soundness– Simplicity of formal descriptionSimplicity of formal description– Expressive powerExpressive power– SecuritySecurity– VerifiabilityVerifiability– Bounded time and space requirementsBounded time and space requirements

• Several features of Ada have been removedSeveral features of Ada have been removed– Gotos, aliasing, default parameters for subprogramsGotos, aliasing, default parameters for subprograms– Side-effects in functions, recursion, tasks, user-Side-effects in functions, recursion, tasks, user-

defined exceptions, exception handlers, and defined exceptions, exception handlers, and generics.generics.

– Other featuresOther features

The Programming Language: The Programming Language: SPARKSPARK

• Annotations are processed by the SPARK Annotations are processed by the SPARK toolstools– The first group of annotations is concerned The first group of annotations is concerned

with data and information flow analysiswith data and information flow analysis•-- # global-- # global

•-- # derives-- # derives

•-- # own-- # own

•-- # inherit-- # inherit

– The second group of annotations is used for The second group of annotations is used for code verificationcode verification•-- # pre-- # pre

•-- # post-- # post

•-- # assert-- # assert

•-- # return-- # return

The Programming Language: The Programming Language: SPARKSPARK• The design of the SPARK ensures that The design of the SPARK ensures that

the Ada exceptions can not be raisedthe Ada exceptions can not be raised– Tasking_ErrorTasking_Error– Program_ErrorProgram_Error– Storage_ErrorStorage_Error

• Constraint_Error can only be caused by Constraint_Error can only be caused by a division check, an index check, a a division check, an index check, a range check, or an overflow checkrange check, or an overflow check

• Two possible routes for discharging the Two possible routes for discharging the VCsVCs– The SimplifierThe Simplifier– The Proof CheckerThe Proof Checker

Z notationZ notation• The formal specification notationThe formal specification notation• Useful for describing computer-based Useful for describing computer-based

systemssystems• Based on set theory and first order Based on set theory and first order

predicate logicpredicate logic• It has been developed by the It has been developed by the

Programming Research Group at the Programming Research Group at the Oxford University Computer Laboratory Oxford University Computer Laboratory since the late 1970ssince the late 1970s

• Z is now defined by an ISO standard and Z is now defined by an ISO standard and is public domainis public domain

• http://vl.fmnet.info/z/http://vl.fmnet.info/z/

Proof in the SHOLIS Development Proof in the SHOLIS Development ProcessProcess• The Development ProcessThe Development Process

– Requirements, written in EnglishRequirements, written in English– Software Requirement Specification (SRS), Software Requirement Specification (SRS),

written in Z and Englishwritten in Z and English– Software Design Specification (SDS), written Software Design Specification (SDS), written

in SPARK, Z, and Englishin SPARK, Z, and English– Code, written in SPARKCode, written in SPARK– TestingTesting

• Proof ActivitiesProof Activities– Z proofZ proof– SPARK proofSPARK proof

Z ProofZ Proof

• At SRS level proofAt SRS level proof– Consistency of global variables and constantsConsistency of global variables and constants– Existence of initial states and checking of Existence of initial states and checking of

preconditionspreconditions– The key safety properties of SHOLIS were The key safety properties of SHOLIS were

also formalized in Z, and proved.also formalized in Z, and proved.• In;Calc;OutIn;Calc;Out gives the correct warninggives the correct warning

• At DSD level proofAt DSD level proof– Further Z proofs were done to demonstrate Further Z proofs were done to demonstrate

the consistency and correctness of the part the consistency and correctness of the part of the design written in Zof the design written in Z•Assistance tools CADiZ tool for schema expansionAssistance tools CADiZ tool for schema expansion

SPARK proofSPARK proof• All of the work was carried out with All of the work was carried out with

machine assistancemachine assistance– The Examiner The Examiner – SimplifierSimplifier– Proof CheckerProof Checker

• Data and information flow analysis was Data and information flow analysis was carried out for all of the code in SHOLIScarried out for all of the code in SHOLIS

• SIL4 subprogram, SPARK pre and post SIL4 subprogram, SPARK pre and post annotations were produced from the Z annotations were produced from the Z descriptionsdescriptions

• The final group of SPARK proof activities The final group of SPARK proof activities concerned the run-time checks. concerned the run-time checks.

Proof in the SHOLIS Development Proof in the SHOLIS Development ProcessProcess• Proof personnelProof personnel

– EngineersEngineers• Two were for Z proofsTwo were for Z proofs• One also worked for other two engineers on generating SPARK One also worked for other two engineers on generating SPARK

proof annotations, and SPARK proof activities proof annotations, and SPARK proof activities • Two coders for Data and information flow analysisTwo coders for Data and information flow analysis

– Skills are necessary for such a projectSkills are necessary for such a project• Proof ValidationProof Validation

– Z proofs were subject to a formal peer-review processZ proofs were subject to a formal peer-review process– SPARK code proof also reviewed by IV&V teamSPARK code proof also reviewed by IV&V team– None of the proofs was inspected or reviewed by None of the proofs was inspected or reviewed by

customercustomer• Timing and Resource UsageTiming and Resource Usage

– Timings – static timing analysis tool was usedTimings – static timing analysis tool was used– Memory – simple static analysis of object code is Memory – simple static analysis of object code is

sufficientsufficient– I/O bandwidth – the available bandwidth to the I/O bandwidth – the available bandwidth to the

displays was a limiting factor.displays was a limiting factor.

Results, Experiences, and Lessons Results, Experiences, and Lessons LearnedLearned

• Quantitative ResultsQuantitative Results– Z proof workZ proof work

•150 proofs were carried out150 proofs were carried out– 130 were at SRS level130 were at SRS level– The remainder at the SDS levelThe remainder at the SDS level

•500 pages500 pages

– SPARK proof workSPARK proof work•9000 verification conditions were generated9000 verification conditions were generated

– 3100 were proofs of functional and safety properties3100 were proofs of functional and safety properties– Remaining 5900 came from the RTC generatorRemaining 5900 came from the RTC generator

•6800 were discharged automatically by the 6800 were discharged automatically by the simplifiersimplifier

•The remainder were discharged by the Proof The remainder were discharged by the Proof checker or by informal justificationchecker or by informal justification

• Faults found at different stagesFaults found at different stages

Results, Experiences, and Lessons Results, Experiences, and Lessons LearnedLearned

• Definition of a fault for these purposes is an Definition of a fault for these purposes is an error in the system developmenterror in the system development

Results, Experiences, and Lessons Results, Experiences, and Lessons LearnedLearned• The Z proof phase is effective at finding faults, with The Z proof phase is effective at finding faults, with relatively little effort, early in the development process.relatively little effort, early in the development process.

Results, Experiences, and Lessons Results, Experiences, and Lessons LearnedLearned• Types of Errors found by Z Precondition Types of Errors found by Z Precondition

ProofsProofs– Approximately 70 percent of the Z proofs Approximately 70 percent of the Z proofs

involved calculating preconditions, and they involved calculating preconditions, and they found about 75 percent of the total faults found about 75 percent of the total faults found by Z proof.found by Z proof.

– Major types of faultsMajor types of faults• Incorrect functionality specified (6)Incorrect functionality specified (6)

•Contradictory operations (11)Contradictory operations (11)

•Lack of mode/history information modeled (4)Lack of mode/history information modeled (4)

•Missing cases (7)Missing cases (7)

• Incorrectly loose specifications (4)Incorrectly loose specifications (4)

Results, Experiences, and Lessons Results, Experiences, and Lessons LearnedLearned

Results, Experiences, and Lessons Results, Experiences, and Lessons LearnedLearned• Subjective Feedback on the Use of ProofSubjective Feedback on the Use of Proof

– The “middle” part of the system could be neatly The “middle” part of the system could be neatly described by Z and SPARKdescribed by Z and SPARK

– At the very top level, experience showed that the At the very top level, experience showed that the proof annotations were often simply too large to be proof annotations were often simply too large to be manageable.manageable.

– At the “bottom” of the architectureAt the “bottom” of the architecture•Need to interface with other software, such as device Need to interface with other software, such as device

drivers, for which there was no formal specification at alldrivers, for which there was no formal specification at all

•o! = f(x)o! = f(x)

– Problems in ensuring that the SPARK code was Problems in ensuring that the SPARK code was both provable and obeyed the timing requirementsboth provable and obeyed the timing requirements

– In Z proofs, it was found that the choice of state In Z proofs, it was found that the choice of state invariants was particularly important for finding invariants was particularly important for finding errorserrors

Results, Experiences, and Lessons Results, Experiences, and Lessons LearnedLearned– Lessons learned about coding styles that made Lessons learned about coding styles that made

the proof task easierthe proof task easier•““Coding Style Guide”Coding Style Guide”

– It should be noted that the proportion of effort It should be noted that the proportion of effort spent producing the proofs was fairly lowspent producing the proofs was fairly low

– The use of a formal specification leads to simpler The use of a formal specification leads to simpler code that is easier to understand, and therefore to code that is easier to understand, and therefore to maintainmaintain

• SPARK 83 versus SPARK 95SPARK 83 versus SPARK 95– SHOLIS used SPARK 83SHOLIS used SPARK 83– New features of SPARK 95 would have made life New features of SPARK 95 would have made life

easier on the SHOLISeasier on the SHOLIS•Use type clausesUse type clauses•The ability to read out parametersThe ability to read out parameters•Moded globals and the changes to static Moded globals and the changes to static

expressionsexpressions

Analysis of the ResultsAnalysis of the Results• Our analysis is based on engineering judgment Our analysis is based on engineering judgment

drawn from the experiencedrawn from the experience

• The Z Proof was the most efficient phase in The Z Proof was the most efficient phase in finding faults, followed by the System finding faults, followed by the System Validation Test phase.Validation Test phase.

• Code Proof appears more efficient than Unit Code Proof appears more efficient than Unit TestingTesting

• Three questions:Three questions:– Are any additional cost incurred to make it possible Are any additional cost incurred to make it possible

to do proof or testing?to do proof or testing?– What are the severities of the faults found by proof What are the severities of the faults found by proof

or testing?or testing?– Is the sample size large enough to be statistically Is the sample size large enough to be statistically

significant?significant?

Analysis of the ResultsAnalysis of the Results• Additional CostsAdditional Costs

– Z proof, needed is the Z itself and statements of all the Z proof, needed is the Z itself and statements of all the properties to prove.properties to prove.

– The time taken to formulate the propertiesThe time taken to formulate the properties– SPARK code proof, data flow analysis must be SPARK code proof, data flow analysis must be

performed on the code and appropriate proof performed on the code and appropriate proof annotationsannotations

– Testing activities include all the time taken to write Testing activities include all the time taken to write test specifications, test scripts, run tests, etc.test specifications, test scripts, run tests, etc.

• Fault SeveritiesFault Severities– Not obvious how to compare the severity of faults Not obvious how to compare the severity of faults

found by different phasesfound by different phases– A possible measure of severity might be the effort A possible measure of severity might be the effort

taken to fix a faulttaken to fix a fault• Effort taken to fix fault found by Z proof was quit lowEffort taken to fix fault found by Z proof was quit low• The cost of correction rapidly increases the later the faults are The cost of correction rapidly increases the later the faults are

foundfound

Analysis of the ResultsAnalysis of the Results• Statistical SignificanceStatistical Significance

– The total number of faults is certainly not large The total number of faults is certainly not large enough to be conclusiveenough to be conclusive

– We believe SHOLIS is a large enough example for We believe SHOLIS is a large enough example for proof to be given serious consideration on other proof to be given serious consideration on other projectproject

• Analysis ConclusionsAnalysis Conclusions– The efficiency numbers in Fig.1 are a reasonable The efficiency numbers in Fig.1 are a reasonable

statement of the effectiveness of Z proof on SHOLISstatement of the effectiveness of Z proof on SHOLIS– Proving the code was free of run-time exceptions Proving the code was free of run-time exceptions

was practical and important for a safety-critical was practical and important for a safety-critical applicationapplication

– Only the systematic nature of proof could establish Only the systematic nature of proof could establish the absence of exceptionsthe absence of exceptions

Summary and Summary and Conclusions/QuestionsConclusions/Questions• The SHOLIS project made extensive use of formal The SHOLIS project made extensive use of formal

methodsmethods• Z proof phase was the most efficient phase of the Z proof phase was the most efficient phase of the

projectproject• Proofs at the SPARK code level were not as efficient at Proofs at the SPARK code level were not as efficient at

finding faultsfinding faults• The code proofs were still more efficient at error The code proofs were still more efficient at error

detection than unit testingdetection than unit testing• System validation testing was more efficient at finding System validation testing was more efficient at finding

faults than unit testingfaults than unit testing• Constraints to remember when attempting proof on a Constraints to remember when attempting proof on a

large-scalelarge-scale– SHOLIS’s simple system architectureSHOLIS’s simple system architecture– Limits of formality must be consideredLimits of formality must be considered

• Proof was an important part of the SHOLIS development Proof was an important part of the SHOLIS development processprocess

• Our success shows both the significant benefit and Our success shows both the significant benefit and practically of large-scale proof on projects of this kind.practically of large-scale proof on projects of this kind.