safe and secure dependable systems composing, analyzing and validating software models to assess /...

Safe and Secure Dependable SystemsComposing, Analyzing and Validating Software

Models to Assess / Develop / Validate Methods and Supporting Tools for the Creation of Dependable

Systems

Frederick T. Sheldon, Ph.D.Oak Ridge National Laboratory

Applied Software Engineering Research LaboratoryDirector – Software Engineering for Dependable Systems Research Lab

University of Tennessee

Knoxville, TN

2

Software Engineering for Dependable Systems Research Lab School of EECS, Washington State University

Def: Software Process

Structured set of activities required to develop a software systemSpecificationDesignValidationEvolution

Activities vary depending on the organization and the type of system being developed Must be explicitly modeled to be effectively managed

3


What is a software process?Activities – things that are doneProducts – inputs and outputsSequencing – relationship among the activities and

products

How does sequencing work?

PresentPhase

NextPhase

Adequate basis forsubsequent phases

Conformance to thestandards of this phase

Verification CheckPrevious

Phase

Compliance with previousphase requirements

4


Verification versus Validation

Verification determines if the products of a given phase of the SLC fulfill the requirements established during the previous phaseProof of transformation conformance

Did we build the product right ?

Validation checks if the program, as implemented, meets the expectations of the customer to ensure compliance with software requirementsDid we build the right product ?

5


Is the Software doingwhat it is supposed to do?

SoftwareVerification

Is the System doing whatit is supposed to do?

RequirementsDefinition

SystemDefinition

What the System issupposed to do?

Software Development

System ValidationTesting

SystemValidation

What the softwareis supposed to do?

SoftwareValidation

System Development

Coding &Component

Testing

SoftwareValidation

Testing

Hardware SoftwareIntegration

Integration & SWSystem Testing

SoftwareRequirements

Generation

SoftwareDesign

Verification & Validation in Context

6


Abundance of Process ModelsThe waterfall model

Separate and distinct phases of specification and development

Evolutionary development Specification and development are interleaved (e.g., Extreme Prgming)

Formal transformation Mathematical system model formally transformed to an implementation

Reuse-based development The system is assembled from existing components

Hybrid Methodologies Prototyping for high-risk specifications (e.g., Spiral Model) A heavyweight methodology has many rules, practices, and documents. A lightweight methodology few rules / practices easy to follow.

7


Evolutionary development

Initial Version

Intermediateversions

Final Version

ConcurrentActivities

Development

Validation

Specification

OutlineDescription

8


Transformational development

R2R1Formalspecification R3ExecutableprogramP1P2P3P4T1T2T3T4Proofs of transformation correctnessFormal transformationsFormal Transformations

Proof of Transformation Correctness

9


Spiral Model – Boehm

Prototype1

Prototype2

Prototype3

OperationalPrototype

RiskAnalysis

Requirementsand

Life Cycle PlanConcept ofOperation

SoftwareRequirements

SoftwareProductDesign

Plan Next Phases

Design Validationand Verification

Integration andTest Plan

DevelopmentPlan

RequirementsValidation

CommitmentPartitionReview

RiskAnalysisRisk

AnalysisRisk

Analy-sis

Progress Through Steps

CumulativeCost

Determine Objectives,Alternatives,and Constraints

Evaluate Alternatives;Identify and ResolveRisks

Develop, Verify NextLevel Product

Implementation

Integrationand Test

AcceptanceTest

UnitTest

DetailedDesign

Code

Simulations, Models,Benchmarks

RiskAnaly-sis Prototype

1


SoftwareRequirement

DevelopmentPlan

Requirements PlanLife-Cycle Plan Concept of

Operation

Prototype1

Prototype2

Prototype3


DevelopmentPlan

Requirements PlanLife-Cycle Plan Concept of

Operation

RiskAnalysisRisk

Analysis

Benchmarks

DetailedDesign

CodeUnitTest

Integrationand Test

AcceptanceTest

Implementation

Integrationand Test

Plan

DesignValidation &Verification

OperationalPrototype

RiskAnalysis

ProgressThrough

Steps

DetermineObjective,

Alternatives,Constraints

Commitment

Partition

CumulativeCost

Evaluate AlternativesIdentify, Resolve Risks

OperationalSystem

Simulations,Models, andBenchmarks

Prototype3

RiskAnalysisRisk

Analysis

Prototype2

SoftwareProductDesign

PrototypingEnvironment

ReviewTestbed/Fielded Prototypes

RiskAnaly-sis

SoftwareRequirement

PlanNext Phase

10


.

SystemRequire-mentsAnalysis

SystemDesign

HardwareRequirementsAnalysis

PreliminaryDesign

DetailedDesign

Coding andCSU Testing

CSC Integrationand Testing

CSCITesting

HWCITesting

Fabrication

FunctionalBaseline

AllocatedBaseline

ProductBaseline

SDRSSR

PDR

CDR

PDR

TRR

TestingAndEvaluation

ProductionandDeployment

SystemIntegrationAnd Testing

FQR

FCAPCA

FCAPCA

CDR

Reviews

SRR - System Requirements ReviewSDR - System Design ReviewSSR - Software Specification ReviewPDR - Preliminary Design ReviewCDR - Critical Design ReviewTRR - Test Readiness ReviewFCA - Functional Configuration AuditPCA - Physical Configuration AuditFQR - Formal Qualification Review

SRR †

†

† - May be multiple reviews and may be integrated with hardware reviews

DetailedDesign

PreliminaryDesign

SoftwareRequirementsAnalysis

SEI’s 2167 / 2167A

11


Indeed an Abundance of Process Models

Each has a particular place within the mosaic of computer science and engineering

However, when safety and/or significant cost are major driving requirements…

Its important to understand the maturity of your process in relation to those requirements!

The waterfall model Separate and distinct phases of specification and development

Evolutionary development Specification and development are interleaved (e.g., Extreme Prgming)

Formal transformation Mathematical system model formally transformed to an implementation

Reuse-based development The system is assembled from existing components

Hybrid Methodologies Prototyping for high-risk specifications (e.g., Spiral Model) A heavyweight methodology has many rules, practices, and documents. A lightweight methodology few rules / practices easy to follow.

12


CMM

Risk

Optimizing

Managed

Defined

Repeatable

Initial

• Focus on process improvement• Data gathering is automated and used to identify weakest process elements• Numerical evidence used to justify application of technology to critical tasks• Rigorous defect-cause analysis and detect prevention• Focus on process optimization to reduce errors

• Still human-intensive process• Maintain organization at optimizing level

(Quantitative)• Measured process: estimates/actuals, error-cause analysis• Minimum set of quality and productivity measurements established• Process database established & resources to analyze & maintain its data• Focus on technology management and insertion

• Changing technology• Problem analysis• Problem prevention

(Quantitative)• Process defined and institutionalized• Software Engineering Process Group established to direct improvements• Focus on Software design skills, design tracking• Focus on various types of traceability

• Process measurement• Process analysis• Quantitative quality plans

(Intuitive)• Process dependent on individuals• Established basic project controls with strength in doing similar work• Process faces major risk when presented with new challenges• Lacks orderly framework for improvement• Focus on collecting various types of "trend" data• Focus on management and tight project control

(Ad hoc/chaotic process)• No formal procedures, cost estimates, project plans• No management mechanism to ensure procedures are followed, tools not well integrated, and change control is lax• Senior management does not understand key issues

• Training• Technical practices (reviews, testing)• Process focus (standards, process groups)

• Project management• Project planning• Configuration management• Software quality assurance

Productivity& quality

Risk

Level Characteristics Key Challenges Result

5

4

3

2

1

13


Right process for the product to ensure … and no silver bullet!

High quality software with competitive cost and cycle time... Quality [Defects]

Cycle Time Cost

… we must shrink the triangle!

14


Quality Attribute(s) Versus Cost Relation

.

Product Attribute(s) (e.g., Reliability, Safety)

Moore’s law of Software Engineering

15


SLC Expenditure Profile

Without FormalSpecification

Specification

Cost

Validation

Design andImplementation

With FormalSpecification

Maintenance

Specification

Design andImplementation

Validation

MaintenanceNeed to validate effective SE methods

and tools…

16


Software Engineering is…A science of building software

Software V&V, the science of building the RIGHT piece of software finding errors as early as possible

Research issues Build tools for automatic software V&V Investigate the theories behind the tools

Formal methods are useful for software V&VSupported by a precise mathematical foundationTheorem-proving approachesModel-checking approaches (very short history)Many apps a must for mission/safety critical software

17


Projects AgendaVerifying Safety Properties for auto-coded software at

the model-based/ linguistic level

PCX Logical/ Stochastic formalism interoperability

SMSC Extending/ Integrating the MSC formalism into the Möbius framework

Model-based Requirements Specification Case study using Zed/Statecharts for guidance control software

Stochastic Modeling Framework (SMF) Case study using PNs/SANs for Anti-lock Braking System

2

1

3

4

5

18


Project OneVerifying Safety Properties for auto-coded software

at the model-based/ linguistic level





Project One

19


Goal: Verifying Safety Properties for Auto-coded Software at the Model level

Domain: X-by-wire driver assisted systems (steering, breaking, power, etc) generally embedded real-time control (hybrid).

Benefits: Higher confidence that defects will be detected earlier reducing the cost to ensure road vehicles are free of unsafe control software

Related work: http://www.ece.cmu.edu/~webk/checkmate/ overviews research sponsored by Ford Motor Company.

Problems/Results: Very complex system Status/Plans: Developing a Software Safety Process Guidebook

to recommend how to develop software in a more rigorous / systematic way taking into account new state-of-the-art standards and practices

20


Steer-by-wire: steering column disappears!

Steering functionality by electronics

Possibility for all new dynamicdriving control systems

Optimized crash behavior

No incomng steering column

Optimized comfort

Variable Steering gear ratio

More fun of driving

Vehicle behavior becomes designable

Problem:

Steering system is a safety critical in our vehicles...

Advantages:

A failure in the systemmust not endangerthe steerability of thevehicle!

Steer-by-wire: steering column disappears!

21


Fahrsituation: Fahrzeug fährt auf der Autobahn / v < 250 km/h

Funktion Versagensmodus Auswirkungen im schlimmsten Fall

Krit. Maßnahmen im heutigen Kfz

Erkennung, Gegen-, Schutzmaß-nahmen (Vorbeugung, Milderung)

Lenken Lenkung blockiert, Räder fixiert

Verlassen der Fahrspur, Unfall

5 Fehlertolerantes System, Lenkeingriff durch Bremsen, evtl. Komfortbremsung zur Gefahrenminderung

Räder floaten Fzg. wird instabil, Abflug in Botanik

5 Fehlertolerantes System, evtl. Räder über mech. Dämpfer in neutrale Position rückstellen, Lenkeingriff durch Bremsen, Fahrzeug muß zum Stillstand gebracht werden

Driftende Abweichung vom gewünschten Lenkwinkel

Verlassen der Fahrspur, Fahrer muß korrigieren, evtl. Unfall

4 Fahrerwarnung, Systemdegradierung, Onboard Monitoring

Safety-

Requirements (SR)Risk Analysis

Safety Requirements for vehicles with Steer by wire

1 Vehicle must be steerable at |vx| > 0 km/h. Actuators must be redundant and must not block each other

2 System must be fail tolerant3 Dormant Failures not allowed

A) Errors in the system must be recognized and displayed to the driver orB) Driver is not allowed to put the vehicle into operation

4 Vehicle must be steerable for delta t after motor stand still5 ...

Safety Requirements Electric System

Dual-circuit electric systemSeparation of the circuitsPower supply for actuators from different electric circuitsMonitoring of the electric system

SR for subsystems

Design proposals

System Safety is theability of a system to avoid critical situations andalso not to create critical situations

System Safety is theability of a system to avoid critical situations andalso not to create critical situations

The Process

Safety Analysis

22


Hybrid Systems

Continuous

Dynamics

Differential Equations/Inclusions

Stopwatch Timers etc.

Discrete

Dynamics

Finite State AutomataZed/ StatechartsPetri Nets etc.

23


Hybrid Systems are …

Found virtually everywhereResult of switching logic in many computer-

controlled applicationsExtremely difficult to analyze

Small perturbation can lead to drastically different behavior

No universally accepted framework for analysis and control

24


Focus: Verification ProblemVery important problem for safety-critical applications

All behaviors must be taken into account

25


MATLAB Tool

Overview

Slide based on defense of Alongkrit ChutinanCarnegie Mellon University, Dept of ECE

Polyhedral-Invariant Hybrid Automaton (PIHA)

Conversion

Simulink/Stateflow Front End(graphical editing, simulation)

Threshold-event-driven Hybrid Systems (TEDHS)

Flow PipeApproximations

Quotient Transition System

ACTL Verification

PartitionRefinement

Initial Partition


Threshold-event-driven Hybrid Systems (TEDHS)

g(.)zero

detector

y(t) v(t)

threshold event generator threshold

events

u(t) = h(u(t-),v(t))u(0-) = u0

finite state machine(event driven)

switched continuousdynamics

F(.,.)

x(t)u(t)

∫

x(0) ∈ X0

)(),( xFxuFx u=∈&


27


Sample Simulink Block Diagramx1

x2

x3

th1

th2

q1

q2

th3

SwitchedContinuous System 3



C*x <= d

PolyhedralThreshold 3

C*x <= d


C*x <= d


Mux

Mux2

MuxMux1

Mux

Mux

OR

LogicalOperator

c1

c2q

FiniteState Machine 2

c1

c2q

FiniteState Machine 1

28


Hybrid Automaton

u

continuous dynamics

invariant: hybrid automaton may remain in u as long as x I(u)

)(

)(

xFx

uIx

u∈∈

&

location (discrete state) u’

reset condition

guard conditionedge

0Xx∈

initial condition

)(

)(

xFx

uIx

u′∈′∈

&

)(: 11 eGxe ∈

),( 1 xeRx ∈+

),( 4 xeRx ∈+

)(: 44 eGxe ∈

)(: 22 eGxe ∈

)(: 33 eGxe ∈),( 3 xeRx ∈+

),( 2 xeRx ∈+


5

29


Project Two

Project Two

Verifying Safety Properties for auto-coded software at the model-based/ linguistic level





30


Goal: Logical/Stochastic Formalism Interoperability (SPNs Promela)

Problem/ Domain: Requirements specification and analysis focused on real-time embedded control

Challenges: Promela is more complex (expressive power) than Petri net formalism

Benefits: predict system performance and dependability (non-functional properties) so that certain structural and architectural features can be evaluated and considered within the context of Spin-verifiable properties.

Related work: Holzmann presented an approach for the translation of Petri Nets into a

PROMELA model [‘94] using the idea that Petri Nets can easily be represented with a small subset of PROMELA constructs.

Grahlmann developed the PEP tool (Programming Environment based on Petri Net) that incorporates a feature that translates PNs into PROMELA for analysis using Spin [‘98] based on the same idea.

31


Goal: Logical/Stochastic Formalism Interoperability (SPNs Promela)

Problems/Results: Small examples have been translated. Stochastic information is added during the translation. Only a small subset of the PROMELA language has been translated.

Status/Plans: Plan to incorporate more language constructs and run more test cases. GUI PN Editor will be integrated.

Example 1: Model created based on structural/operational characteristics including event/activity characteristics Model checking a stochastic model by decorating with logical /

relational properties and developing/refining safety assertions that must hold

Example 2: Model created based on domain specific properties like O/P agreement, variable relationships and dependencies, liveness, mutual exclusion and transition/function consistency Solving a logical model (written in Promela, CTL, etc.) to evaluate

performance and and reliability (transient and steady state behavior)

Example 2: Model created based on domain specific properties like O/P agreement, variable relationships and dependencies, liveness, mutual exclusion and transition/function consistency Solving a logical model (written in Promela, CTL, etc.) to evaluate

performance and reliability (transient and steady state behavior)

32


Promela/PCX/SPN Translation Process

Step Description of steps used in the approach

1PROMELA/Spin model for analyzing the logical consistency

2 Translate from PROMELA to Stochastic Petri Nets.

3 Assign performance and reliability parameters among subsystem components.

4 Analyze the SPN for stochastic properties [using SPNP] (validate performance

and reliability goals using stochastic system models).

5 Final design based on results from Spin and SPNP tool analysis

33


Petri Nets PROMELA

State Space State Space=

Petri Nets PROMELA

State Space State Space≠

Holzmann and Grahlmann

Sheldon and Wang

But remember… the model (encircled) is created based on domain specific properties such as O/P agreement, variable relationships and dependencies, liveness, mutual exclusion etc.

Solving a logical model (written in Promela, CTL, etc.) to evaluate performance and and reliability (transient and steady state behavior)

Petri Nets PROMELA

State Space State Space=

Petri Nets PROMELA

State Space State Space≠

Are They Equivalent …

≈

34


Sheldon, F.T. and Wang, S., "A Translation Tool (PCX) from PROMELA/Spin to C-Based Stochastic Petri Net Language (CSPL)," Wkshp PMCCS 2001, Erlangen, pp. 116-120, Sept. 2001.

Sheldon, F.T. and Wang, S., “PCX: A Translation Tool from PROMELA/Spin to the C-Based Stochastic Petri Net Language,” 2003 Illinois International Multiconference on Measurement, Modelling, and Evaluation of Computer-Communication Systems 9/2-5/2003 (Submitted Feb. 2003 to Tools’03)

SEDS Related Publications

35


Project ThreeVerifying Safety Properties for auto-coded software






Project Three

36


Goal: Extending Message Sequence Charts for Multi-formalism modeling in Möbius

Problem/Domain: modeling and performance analysis of distributed / message passing systems / inter-operability

Benefits: Analysis for models expressed in the MSC formalism. Integrating the extended MSC (SMSC) into the Möbius framework

facilitates the use of a feature rich toolkit to analyze SMSC models. MSC used with other formalisms for multi-formalism modeling.

Related work: ITU-T, Recommendation Z.120: Message Sequence Chart(MSC). 1999: Geneva D. Daly, et all, Möbius: An Extensible Framework For Performance and

Dependability Modeling, FTCS-29, Madison, June 15-18, 1999. Z. Zhou and F. Sheldon. Integrating the CSP Formalism into the Mobius

Framework for Performability Analysis, PMCCS'5, Erlangen, Springer 2001

37


Problems/Results: MSC are a popular formalism used in industry, but cannot

be used for performance analysis without including stochastic timing information.

How to map MSC model constructs to the Möbius entities (i.e., identify state variables, actions and action groups).

Status/Plans: Finished extending and mapping MSC to Möbius entities Implemented C++ base classes representing MSC models Implement the SMSC Möbius user interface (not done) Determine how to handle different MSC model

composition methods within Möbius (fundaments however are complete)

Goal: Extending Message Sequence Charts for Multi-formalism modeling in Möbius

38


MSC/Möbius IntegrationComputer communication systems

ApplicationFeedback

Hypothesis: Feasible/Useful/Valid

MSC

SMSC

Möbius entities

Möbius solvers

Validation

39


msc example1

i1 i2 i3

m0

m1

m2

m3a

Frame symbol

MSC name

Instance head

Instance axis

Instance end symbol

Message symbol

Local action

Graphical representation

Basic MSC – Graphical / Textual Formalism

40


msc example1

i1 i2 i3

m0

m1

m2

m3a

Graphical representation

msc example1;

i1: out m0 to env;

i1: out m1 to i2;

i1: action a;

i1: in m3 from i2;

i2: in m1 from i1;

i2: out m2 to i3;

i2: out m3 to i1;

i3: in m2 from i2;

endmsc;

Description of instance i1



Textual representation

Basic MSC – Graphical / Textual Formalism

41


Basic MSC – Vertical Composition

msc A

i j

m

msc B

j k

n

msc AB

i j

m

k

n

42


msc A

i j

m

msc B

j k

n

msc AB

i j

m

k

n

Basic MSC – Horizontal Composition

Co-region: where event order does

not matter

43


Basic MSC – Alternate Composition

msc A

i j

m

msc B

i j

n

44


msc Ai j

m

msc Bj k

n

msc Ci j

o

k

p

A

B

Basic MSC – Reference

45


msc example1;i1: out m0 to env withrate r1;i1: out m1 to i2 withrate r3;i1: action a withrate r; i1: in m3 from i2 withrate r8;i2: in m1 from i1 withrate r4;i2: out m2 to i3 withrate r5;i2: out m3 to i1 withrate r7;i3: in m2 from i2 withrate r6;endmsc;

Stochastic MSC – Annotated Messages and Actions

msc example1

i1 i2 i3

m0(r1, r2)

m1(r3, r4)

m2(r5,r6)

m3(r7,r8)

a(r)

46


Stochastic MSC – Example Showing Shareable Variables

msc example1(p1, p2)

Global int number; condition GO;

i1 i2 i3

m1(r3, r4)m2(r5,r6)

m3(r7,r8)

a(r): number++;

when number>0;

when GO;

SMSC Model

Möbius entities

Möbius Model

AFI

Möbius solvers State space generatorSimulatorAnalytic/numeric solvers

State variables Action groupsActions

Instances, Messages,Activities, Conditions

Model

SMSC Integration Makeup

Challenges of Predicting the Performance/ Dependability of Modern Computer Systems

Hardware Network OSApplication

Computer System

Fault Description Components Protocol Traffic Control/Data Flow Resource Contention

Fault Tree VHDLLOTOS,Estelle

QueuingModel

Block DiagramLanguage

Petri Nets,SANs

?Slide based on presentation of Bill SandersUniversity of Illinois at Urbana/Champagne

DSPN, GSPN, Markov chain, Queuing Network, SAN, SAN, SPA, other SPN extensions,Domain-specific formalism

Graph interconnectionKronecker Composition (SAN), Replicate/Join, SPA Domain-specific formalism

Rate/Impulse reward variables Path-based reward variablesDomain-specific formalism

Fixed-point governorAcyclic model composer

Range and Set VariationNon-linear optimizer

Example Formalisms

Model Specification

Submodel Interaction

Model Categories in the Möbius Framework

Slide based on presentation of Bill SandersUniversity of Illinois at Urbana/Champagne

Atomic Model

Composed Model

Solvable Model

Connected Model

Study Specifier(generates multiple

models)

Framework Component

Model Construction Process

Models expressed in different framework components can be combined to form larger models One or more atomic and/or

composed models form a composed model

A atomic, composed, or solvable model, together with reward variables, form a solvable model

One or more solvable or connected models, together with reward variables, form a connected model

The model construction process retains the structure of each constituent model, facilitating efficient solution.

ConnectedModel

RewardVariables

SolvableModel

ComposedModel

AtomicModel


Model Support of the Abstract Functional Interface: State Variables, Actions, & Groups

Formally, a model in the Möbius framework is a set of “state variables,” a set of “actions,” and set of “action groups”

State variables “contain” information about the state of the system being modeledThey have a type, which defines their “structure”They have a value, which defines the “state” of the variable

Actions prescribe how the value of state variables may change as a function of time

Action groups define sets of actions and/or action groups that cooperate or compete to change state

Other models and solvers may request state information or change a model’s state variables, actions, and groups via the abstract functional interface


Möbius Tool Architecture

Linker

Executable Model

Atomic Model Object Code

Composed Model Object

Code

Study Editor Object Code

Model Conn. Object Code

Atomic Editor Reward Editor `Study Editor`Model ConnectorComposer

Formalism Libraries

Solver Libraries

Reward Object Code


Project Manager

53


Sheldon, F.T. and Zhou, Z, "Integrating the CSP formalism into Möbius Framework for Performability Analysis," Fifth Int’l Wkshp on Performability Modeling of Computer and Communication Systems PMCCS 2001, Erlangen, pp. 86-89, Sept. 2001.

Sheldon, F.T., Xie, Gaoyan, Pilskalns, Orest and Zhou, Zhihe, "Survey of Rigorous Software Specification and Design Tools," Software Focus Jr., John Wiley and Sons, London, 2:4 Winter 2001.

Sheldon, F.T. and Zhou, Z., "Extending Message Sequence Charts for Multi-level and Multi-formalism Modeling in Möbius," Submit Feb. 15 2003, PNPM 2003).


54


Project Four

Project FourVerifying Safety Properties for auto-coded software






55


Goal: Zed/ Statecharts analysis for validating software requirements

Problem/Domain: Specification / analysis for the purpose of V&V (testing requirements)

Challenges: Validity of abstract / translation : Zed NL, S/Cs Zed Choosing test cases to ensure complete coverage.

Benefits: One is able to … Ensure the completeness, consistency, and fault-tolerance

of a software requirements specification (SRS), Avoid the problems that force corrective rework, … to minimize risk of costly errors propagated into

downstream activities.

56


From NL-based SRS Statecharts, and beyond …

NL-based SRS

Z Specification

Statecharts

Testing and Fault Injection

NASA Spacecraft Guidance and Control Software

ApplicationFeedback

57


Related Work/ Results/ Plans Related work:

(1) Hierons RM, Sadeghipour S, Singh H. Testing a system specified using Statecharts and Z. Info and SW Tech; 43: 137-149 2001

(2) Bussow R, Geisler R, Klar M. Specifying Safety-Critical Embedded Systems with Statecharts and Z: A Case Study. LNCS 1382 1998

Problems/Results: Guidance Control NL-based SRS Case Study Showed/ detected multiple ambiguities, Verified completeness, consistency, and fault-tolerance, Approach is extensible to embedded control systems, See below for more specific results.

Status/Plans: Develop concrete (mechanizable) translation rules between methods. Plan to evolve/ revise the approach to x-by-wire composed system (e.g.,

Steer/Brake/Power/Traction-by-wire for collision avoidance and automatic functions like parking)

58


Landing Vehicle During Descent

X

v

x

p

p

y

p

Z

y

v

z

v

59


Illustration of Vehicle

Axial Engine (3)

Foot Pad (3)

Roll Engine (3)

Bottom View

(x out of page)

positive

roll

thrust z

v

y

v

+p

(roll)

Side View

x

v

(z into page)

y

v

+r

(yaw)

positive

axial thrust

Side View

x

v

(y into page)

z

v

+q

(pitch)

60


Planned Terminal Descent Trajectory

61


Engines On Altitude

Altitude

Velocity-Altitude Contour

62


GCS System Architecture

SENSOR_OUTPUT

GUIDANCE_STATE

CRCP AECLP RECLP

CONTROL AND TELEMETRY

OUTPUTS CP

SENSOR DATA

ASP GSP TSP ARSP TDLRSP ASP

GP RUN_PARAMETERS

PACKET

63


Natural Language based SRSZed Specification and Proof

Statechart Visualization and Simulation

64


GCS Schema Hierarchy

65


GCS Module Chart

66


Actual GCS Module Chart

67


Methodology Summary Interpretation from NL to Zed

Clarified ambiguities

Interpretation from Zed to Statecharts Iterative process that clarified misinterpreted Zed

specification *

Simulation and testing with fault injectionReveals fault-prone system statesFault-tolerance validation

* Several versions of Statecharts were developed. During the testing phase a better understanding of the specification/model caused revision of the Zed specification

68


Two Formalisms Combining Strengths

ZedClarify ambiguities and identify assumptions

Correctness checking tool-based hand-guided proofs

In-depth understanding of the requirements

StatechartsVisual formalism with diagrammatic grammar made up

of activity and Statecharts

Symbolic simulation enables testing and evaluation

Providing e.g., predevelopment evaluation via fault

injection

69


Summary of Case Study Findings

From NL to Zed discovered ambiguities Rotation variable definition are ambiguous

AR_COUNTER variable type & scope are ambiguous

Undefined 3rd order polynomial.

Zed to Statecharts revealed a deficiency must include 1 ≤ AR_FREQUENCY ≤ AR_COUNTER*75000

Or,… AR_COUNTER =

–1 (0≤AR_COUNTER≤AR_FREQUENCY/75000)

70


Sheldon, F.T., Kim, H.Y., and Zhou, Z, "A Case Study: Validation of Guidance Control Software Requirements for Completeness, Consistency and Fault Tolerance," IEEE Proc. Pacific Rim Dependability Conference PRDC 2001, Seoul, Dec. 2001.

Sheldon, F.T., Kim, H.Y., and "Validation of Guidance Control Software Requirements for Reliability and Fault-Tolerance," IEEE Proc. Reliability and Maintainability Symp. RAMS 2001, Seattle, Jan. 2002.

Kim, H.Y. and Sheldon, F.T., "Testing Software Requirements with Z and Statecharts Applied to an Embedded Control System," Springer-Verlag, Software Quality Jr. Kluwer Submitted Dec. 2002


71


Project Five

Verifying Safety Properties for auto-coded software at the model-based/ linguistic level




Stochastic Modeling Framework Case study using PNs/SANs for Anti-lock Braking System

Project Five

72


Goal: Stochastic Modeling Framework Case Study: Anti-lock Braking System

Problem/Domain: Model road vehicle ABS emphasizing failure severity, coincident failures and usage profiles using SPNs and SANs formalisms.

Challenges: Need to handle large state space –complex systems often include many layers

of complexity and numerous constituent components For realistic results we must model components to a sufficient level of detail Models should be scalable and extensible to accommodate the larger context

Benefits: Greater insight about the contribution of different components and non-functional factors to the overall system reliability. Establishes a framework for studying important factors that determine system

reliability Related work:

F.T. Sheldon, S. Greiner and M. Benzinger, “Specification, safety and reliability analysis using Stochastic Petri Net models”, in Proc. of 10 th Int. Wkshp on Software Specification and Design, San Diego, 2000, pp. 123-132.

73


Goal: Stochastic Modeling Framework Case Study: Anti-lock Braking System

Problems/Results: Transient analysis of SPNs (using Stochastic Petri Net Package v. 6) and Stochastic Activity Network (UltraSAN v. 3.5) models was carried out and the results compared for validation purposes. Results emphasized the importance of modeling failure severity, coincident

failures and usage-profiles for measuring system reliability. Status/Plans:

Carry out the sensitivity analysis for the models developed to gain an insight into which components affect reliability more than others.

Model the entire system. ABS is a small part of the Dynamic Driving Regulation system and shares components with the ESA (Electronic Steer Assistance) and TC (Traction Control).

Use simulation to analyze the model of the entire system. The model of the system would be too complex to allow numerical means of analysis.

Validate the results of the analysis against real data (if such data becomes available to us).

74


ESA Electronic Steer Assistance

TC Traction Control

DDR (Dynamic Driving Regulation System)

Sensitivity Analysis

Simulation

PT Power Transmission

ABS (Anti-lock Braking System)

Experiments (Monitoring of real system)

Problem Identification & Requirements Analysis

Modeling using Stochastic Petri Nets

Analysis using Stochastic Petri Net Package v. 6

Comparison of results (Semi-validation)

Complete validation by comparison against real data

feedback feedback

Modeling using Stochastic Activity Networks

Analysis using UltraSAN v. 3.5

Further Details

75


SEDS Related Publications Sheldon, F.T. Greiner, S.A., and Benzinger, M., "Specification, Safety and

Reliability Analysis Using Stochastic Petri Net Models," ACM Proc. 10th Int'l Wkshp on SW Spec. and Design, San Diego, pp. 123-132 Nov. 5-7 2000.

Sheldon, F.T. and Jerath, K., "Reliability Analysis of an Anti-lock Braking System Using Stochastic Petri Nets," 5th Int’l Wkshp on Performability Modeling of Comp. / Comm. Sys PMCCS 2001, Erlangen, pp. 56-60, Sept. 2001.

Sheldon, F.T., Jerath, K., and Greiner, S.A., “Examining Coincident Failures and Usage-Profiles in Reliability Analysis of an Embedded Vehicle Sub-System,” Proc. 9th Int’l. Conference on Analytical and Stochastic Modeling

Techniques ASMT 2002, Darmstadt, June 3-5, 2002. Sheldon, F.T. and Jerath, Kshamta, “Specification Modeling and Stochastic

Analysis of Embedded Systems with Emphasis on Coincident Failures and

Usage-Profiles,” Submit June 2002, IEEE Trans. On Reliability

76


Key SE Issues in Software Safety I

Provide readier access to formal methods for developers of safety-critical systems by further integration of informal and formal methods †.

Develop better methods for safety analysis of product families and safe reuse of Commercial-Off-The-Shelf software †.

Improve the testing and evaluation of safety-critical systems through the use of requirements-based testing, evaluation from multiple sources, model consistency, and virtual environments †.

†R. Lutz, NASA JPL

77


Ensure SW technology transfer from the lab to practice with case studies to enable earlier adoption of potentially cost savings / safety ensuring advances.

Advance the use of runtime monitoring to detect faults and recover to a safe state, as well as to profile system usage to enhance safety analyses.

Promote collaboration with related fields in order to exploit advances in areas such as security and survivability, software architecture, theoretical computer science, human factors engineering, and software engineering education.

Key SE Issues in Software Safety II

78


Research Summary and Plans Experience with numerous formalisms and tools

We will continue to …

Assess, develop and validate methods and supporting tools for the creation of safe and dependable systems

Develop an integrated framework for composing, analyzing and validating system and software models

Verification mission / safety critical hybrid systems

Extend to security related verification and validation

Publish …

79


Contact Information

Frederick T. Sheldon, Ph.D. and Tom Potok, Ph.D.

Software Engineering for Dependable for Systems

Applied Software Engineering Laboratory

Rick: 865-576-1339Tom: 865-574-0834Fax: 865-574-6275

URL: http://www.csm.ornl.gov/~sheldon

http://computing.ornl.gov/cse_home/acer.shtml

80


The end…

81


Other SEDS Research Recent publications

Sheldon, F.T., Jerath, Kshamta and Chung, Hong, "Metrics for Maintainability of Class Inheritance Hierarchies," Jr. of Software Maintenance and Evolution, John Wiley and Sons, London, Summer 2002.

Sheldon, F.T., Kwon, Y-J., Baik, Young-Wook and Jerath, K., “Case Study: Implementing a Web Based Auction System using UML and Components,” Proc. Int’l. Annual Computer Software and Applications Conference COMPSAC 2002, Oxford, Aug. 26-29, 2002

Developing CGE (PN Graphical Editor) with Java using various graph layout algorithms for model visualization

Home page: http://www.csm.ornl.gov/~sheldon

Publications: http:// www.csm.ornl.gov/~sheldon /publications.html Username = batman, Password = just ask me ([email protected])

82


Making Verification Tools Useful for Industry

safe and secure dependable systems composing, analyzing and validating software models to assess /...

Documents