Model Checking and Planning

for Critical Software

Paolo Traverso

ITC/IRST, Via Sommarive 18, 38050 Trento

traverso@itc.it

htttp://sra.itc.it/

Motivations

Industrial Embedded Softwareo Functionality Issues: Safety Critical Systems,

Growing Complexityo Market Issues: Time to delivery, Costso Maintenance Issues: Requirements change over

time, Feature interaction problem Difficulties with Traditional Methodologies

o Ambiguous Specification (requirements, Analysis, Design)

o Errors in Specifications/Design Refinementso Limited Coverage by Tests

Consequenceso Expensive errors in the early design phaseo Infeasibility of achieving (ultra-high) reliability

requirementso Low Software Quality (hard to maintain)

Formal Methods: The Potentials

Key Ingredientso Formal Specification: unambiguous

description of the system and of the required properties

o Formal Validation and Verification Tools: exhaustive comparison of the formal description of the system against the formal properties

Potential Benefitso Find design bugs in early design stageso Achieve higher quality standardso Shorten Time to Market reducing manual

validation phase

Formal Methods: The Practice

Technical Problemso Formal Specification: hard to write, high

costs ...o Formal Validation and Verification Tools:

models of real industrial systems are often hard to analyze, tools often do not scale up or are not automatic, ...

Practical & Methodological Problems:o Introduction of a new technology that requires …o High expertise, long training, …o Modification of standard development process ... o Increase of development costs ...

Formal Methods @ IRST

Adapt the Application of Formal Methods to the Customers’

real needs

• In-house Tool Support and Development

• Integration with Standard Technologies

• Lightweight Approach

• Gentle Integration in the Development Process

A technology: Model Checking

Basic Ingredientso Systems Modeled as a Finite State Automata (FSA)o Requirement Expressed in Temporal Logico Formal V&V by exhaustive search over the state

space

Requirement

System Model

Yes!, the model satisfies the requirementsNo! Here’s a counterexample ...

Model Checker Always (if Signal=On then Engine=Off)

Signal

Engine

Model Checking

Powerful debugging capabilitieso Helps detect problems in early stages of the

development cycle, where they are more costlyo exhaustive, thus effective (often bugs are also in

scaled-down problems)o provides counterexamples

Easier to integrate within industrial development cycleo compilers for practical design languages (e.g.

VHDL, SDL, Statecharts)o although limited, expressiveness is often

sufficient in practice Does not require deep training (push-button

technology)

Model Checking: Problems

Problems:o Technical: Automata of 10,000,000,000,000,000,...

stateso Practical: Costs of the introduction the

Technologyo Methodological: How this affects

Development Process (e.g. testing)

Requirement

System Model

Yes!, the model satisfies the requirementsNo! Here’s a counterexample ...

Model Checker

... Is model checking still a dream?

Model Checking@IRST

Main features:o State-of-the art techniques to scale up to huge state

spaceso Open architecture (one techniques for one problem)o Open interface (“let me work with my own language!”)

Requirement

System Model

Yes!, the model satisfies the requirementsNo! Here’s a counterexample ...

NuSMV

Model Checker

The NuSMV Model Checker

Originally, joint project with CMU Academic version released in June 99 Over 200 installations worldwide High quality implementation Current interest by various industrial

partners Used in industrial technology transfer

projects

NuSMV is an Industrial Strength,

Open Architecture, Model Checker

... and it is open-source: http://sra.itc.it/tools/nusmv

Automated Synthesis of Controller

There exists some approaches ….

Automated Synthesis @ IRST by ….

Requirement

System Model

Controller that makes the system satisfy the requirements

No! There is no controller such that ...

Automated Synthesis

… by Planning as model checking

Basic Ingredientso Planning as Model Checking:

plan becomes a FSM, i.e., a controller goal becomes a requirement in temporal logic planning: generate plan s.t. system satisfies

the goal (similar to a model checking problem)

Requirement

System Model

Controller that makes the system satisfy the requirements

No! There is no controller such that ...

Automated Synthesis

Always (if Signal=On then Engine=Off)

Signal

Engine

MBP

MBP

The NuSMV Model Checker

Developed on top of NuSmv Academic version released in June 2001 Several installations worldwide High quality implementation Current interest by various industrial

partners Used in industrial technology transfer

projects

MBP: a Model Based Planner

... and it is available at http://sra.itc.it/tools/mbp

Some projects at IRST

Railways (Ansaldo, Union Switch & Signal, …) Avionics (Alenia, Airbus, Rockwell, …) Embedded Controllers (Invensys, …) MicroProcessors (Intell, …) Space (Nasa, Asi, …)

Application I: Interlocking System

Application I: Interlocking System

Difficultieso Safety Critical Systemo High-Complexity of functions and controlled deviceso Time-to-market: large amount of manual verification

Validation of Interlocking System for Control of Railway Stations

Application I: Interlocking System

Goalso Increased confidence in the correctness of

designo Automation of the verification tasko Improvement of time-to-market reducing manual

validation effort

Resultso Automated generation of formal specificationso Integrated, application specific verification

engineo Subtle bugs detected in simple configurations

Application II: Tool Certification

Requirementso Automation, Efficiency, Certification

Certification of COTS based, Safety Critical Compiler

Application II: Tool Certification

Approach (Run-Time Result Checking)o Certification Tool: validates each single

run of the compilero Efficiency: specialized, problem dependent

verification engineo Certification of the Certification Tool:

Logging + Checking

Resultso Certification Tool satisfies the requirementso Bug Found in Compiler Design

Application III: Comm. Protocol

A high complexity communicationprotocol for redundancy

Starting Pointo Incomplete, informal specificationso Existing Implementation (legacy code)o A history of expensive debugging on the field

Application III: Comm. Protocol

Approach Specification of Functional Requirements

with MSC Architectural and Formal Model in SDL Formal Validation using Model Checking

Subtle bugs detected after exchange of over 200 messages

Detailed Informal/Formal specification to code developers

Resultso First Implementation passed all tests

Application IV: RBC

Other RBC

Uncovered Area

Announce Area

overpass

rcvl …

Other RBC area

rtb

rtb rtb

Interlocking

Radio Block Centre

Radio Block Center

Application IV: RBC

Approacho Solution based on the integration of different

languages and notationso Simulation/Validation of the designo Simulation on a significant portion of the Italian

trial site

Resultso Increased confidence in the correctness of the

designo “Incremental” Architectureo Subtle communication issues detected (mainly

liveness issues)

Application V: Air Conditioning

ModelLibrary

Functions Specification

Plant Specification

Firmware

(on motherboard)

Design and Implementation of a Tool to Support Controllers’

Construction

Model Checker

Application V: Air Conditioning

Simulator

Configuration Tool Compiler

MBP

Application VI: ESACS

Enhanced Safety Analysis for Complex

SystemsA E R O N A U T I C A

Application Domain

Critical Points• Link between System Design and Safety Analysis.

• Growing complexity of systems.

System Design

Complex System

System Level Requirements

System Architecture

System Implementatio

n

Certification

Safety Analysis

Fault Hazard Analysis

PSSA

System Safety Analysis

Top Level Event

SPS_LH.GB.W_gb= 0

fault_cfg_1

description

fault_cfg_2

description

fault_cfg_3

description

fault_cfg_4

description

SPS_LH.GB_grippage

SPS_LH.GBgrippage

I E

r=0

SPS_LH.GB_broken_tra

SPS_LH.GBbroken_transmission

I E

r=0

ME_LH_grippage

ME_LHgrippage

I E

r=0

SPS_LH.PTO_fused

SPS_LH.PTOfused

I E

r=0

PRSOV_stuck

PRSOVstuck

I E

r=0

ME_LH_grippage

ME_LHgrippage

I E

r=0

SPS_LH.PTO_fused

SPS_LH.PTOfused

I E

r=0

SPS_LH.ATSM_broken

SPS_LH.ATSMbroken

I E

r=0

SPS_LH.FW_fail_disen

SPS_LH.FWfail_disengaged

I E

r=0

PRSOV_stuck

PRSOVstuck

I E

r=0

SPS_LH.FW_fail_disen

SPS_LH.FWfail_disengaged

I E

r=0

SPS_LH.ATSM_broken

SPS_LH.ATSMbroken

I E

r=0

Fault Tree Analysis

FMEA

Fault Probability

Intermediate Effect

Final Effect Severity

Undetected Fire in Bay Area

10e-8 Subsystem A fails

Loss of mechanical drive

5

… … … … …

The Problem

System Design

Safety Analysis

Complex System

System Level Requirements

System Architecture

System Implementation

Certification

Fault Hazard Analysis

PSSA

System Safety Analysis

Goal I. Development of a Platform (FSAP/NuSMV-SA) to Enhance Safety Analysis Process

FSAPNuSMV-SA

• Provides a formal link between System Design and Safety Analysis.

• Produces results useful both for System Design and for Safety Analysis (e.g. counterexamples and fault trees).

• Based on the NuSMV Tool.

• Implements novel techniques for dealing, e.g. with injection of failures and automatic construction of fault trees.

Goals

Goal II. Application to Case Studies• Application to various industrial case studies.

NASA Ames

NASA Ames Application areas: complex on-board

subsystemso Example: Shuttle Fuel management system

On-board model-based diagnosis executoro Keep Track of observations to detect faults

The diagnosability problem: o can on-board diagnosis detect ALL faults?

Goal: formal techniques for diagnosability: o Reduce diagnosability to model checkingo NuSMV used on real applications

Rockwell Collins

Rockwell Collins Application areas: complex on-board

subsystemso Example: Pilot cockpit control system

Enhance quality of requirements and designso Possibly contradictory requirementso Complex interaction between functions

Goal: formal techniques for verification o Map formal design language to NuSMVo NuSMV for requirement verification o NuSMV for design verification

Intel

Intel Application areas: circuit analysis

o ASICs, MicroProcessors subunitso Equivalence Checking, Property Checking

The problemo Boolean Verification techniques o Do not exploit system structure (data vs

control) Goal: Hybrid verification techniques

o Response to Academic Research Programo Boolean and non-boolean verification

The DOVES Project: Motivations

Domaino Deep Space missionso Small Scientific Missions program o International Space Station

Autonomous On-Board Softwareo Operate flexibily in unmanned and unstructured environmento Can carry out wide spectrum of complex functions

Problem: achieve higher degree of validation

o Example: deadlock in Deep Space 1 space probe softwareo Detect software problems and faults at design time

The DOVES project: Goals

Deliver effective methods and tools for

enabling production ofVERIFIED AUTONOMOUS ON-

BOARD SOFTWARE

Integrated platform to support:o Specification, Verification and Validation of Aos Requirementso Verification and Validation of Aos Designso Simulation of Aoso Compilation of Aoso Test planning

Advanced synthesis techniques:o From Specification of Aos Requirements...o … to Aos Designs…o ... guaranteed to satisfy requirements

Conclusions & Future Scenarios

Conclusionso There is a market need for model checking and

planningo There are technologies of potential significant

impacts o They have to be integrated and applied selectively

Futureo Automated synthesys for more complex controllers o “Hybrid” techniques to deal wider range of

problems o From off-line synthesis to on-board autonomy

A technology: Model Checking

Basic Ingredientso Systems Modeled as a Finite State

Automata (FSA)o Requirement Expressed in Temporal Logico Formal V&V by exhaustive search over the

state space

Requirement

System Model

Yes!, the model satisfies the requirementsNo! Here’s a counterexample ...

Model Checker