bounded probabilistic model checking with the murphi verifier

Bounded Probabilistic Model Checking with the Murphi Verifier

Enrico TronciComputer Science Department, University of Rome “La Sapienza”, Via Salaria 113,

00198 Roma, Italy, [email protected], http://www.dsi.uniroma1.it/~tronci

Joint work with:

G. Della Penna, B. Intrigila, I. Melatti, M. Venturini Zilli

FMCAD 2004: Formal Methods in Computer-Aided Design, Austin, Texas, USA, November 14-17, 2004

mailto:[email protected]



http://www.dsi.uniroma1.it/~tronci



2

Goal

Given

•a stochastic system (e.g. a randomized protocol) M and •a property for M,

check automatically that M satisfies .

For example, a Probabilistic Model Checker can be used to verify properties like:

The probability that a message is not delivered after 0.1 seconds is less than 0.8

3

Example

BufferLS HS

Statistically modulated ACK

Low System(e.g. public info)

High System(e.g. private info)

Data Data

ACK

NRL PUMP

The NRL Pump is a special purpose-device that forwards data from a low (security) level system LS to a high (security) level system HS, but not conversely.

Idea: LS ACK delay is probabilistically based on a moving average of HS ACK delays.

Ready to send

Waiting ACK

Got ACK Send Data

LS

Ready to receive

Done

Read DataSend ACK

HSMinimize information flow from HS to LS.

Enforce reasonable performances, i.e.: <average ACK delay as seen from LS> = <average HS ACK delay>

NRL Pump (Probabilistic) Properties:

4

Probabilistic Model Checking

Probabilistic Model Checking (PMC) typically uses:

•Markov Chains to represent the system M to be verified;

•A probabilistic logic (e.g. PCTL) to represent the property to be verified.

1

2

0.6

0.5

M : the probability of reaching (from the initial state 1) state 3 is less than 0.2 (TRUE).PCTL: [tt U (s = 3)] 0.20.4

0.1

3

1.0

: the probability of reaching (from the initial state 1) state 3 is less than 0.1 (FALSE).PCTL: [tt U (s = 3)] 0.1

0.4

In fact, the probability of reaching state 3 is exactly 0.16.

5

Bounded Probabilistic Model CheckingBounded Probabilistic Model Checking (BPMC) uses:

•Markov Chains to represent the system M to be verified;•A probabilistic logic with bounded Untils (e.g. BPCTL) to represent the property to be verified.

1

2

0.6

0.5

M : the probability of reaching state 3 in at most 2 steps is less than 0.2 (TRUE). PCTL: [tt U2 (s = 3)]0.2

0.4

0.1

3

1.0: the probability of reaching state 3 in at most 1 step is at least 0.1 (FALSE). PCTL: [tt U1 (s = 3)]

0.1

0.4

In fact, the probability of reaching state 3 in at most 2 steps is exactly 0.16, whereas state 3 is not reachable in 1 step.

: the probability of reaching state 3 in at most 2 steps is at least than 0.1 (TRUE). PCTL: [tt U2 (s = 3)]

0.1

As for BMC, BPMC is less expressive (and may be more efficient) than PMC.

6

State of the ArtNot surprising the main obstruction to probabilistic model checking is State Explosion

•Linear Algebra and Sparse Matrices, e.g. as in ETMCC;•Symbolic, OBDD based, e.g. PRISM;•Symbolic, MTBDD(Multi Terminal Binary Decision Diagrams) based, e.g. ProbVerus;•Hybrid, using MTBDDs allowing reals in [0, 1] on terminal nodes to represent the transition matrix and linear algebra algorithms to carry out the computation (PRISM).

Typically, PRISM hybrid approach is faster than a probabilistic model checker based only on MTBDDs and avoids the state explosion of a probabilistic model checker based only on sparse matrices.

Since PRISM implements all of the above approaches we will compare our results with PRISM best performances.

Roughly speaking there are 4 approaches:

7

Hybrid Systems Analysis

On (Discrete Time) Hybrid Systems we run quite soon into state explosion using ANY of the previously mentioned approaches (i.e. using PRISM).

Intuitively, here is why:

• the number of states is too large to be handled using a linear algebra approach

• the complex math computations entailed by the system dynamics makes life very hard for OBDDs or MTBDDs.

8

Finite Horizon Probabilistic Murphi (FHP-Murphi)

As for FHP-Murphi performances:

• when PRISM can handle the verification task PRISM is faster than FHP-Murphi,•however there are many verification tasks (typically those involving hybrid systems) feasible with FHP-Murphi and out of reach for PRISM because of state explosion.

We have shown (CHARME’03) that an explicit approach can overcome such limitations. Namely, we have extended the Murphi verifier as follows (FHP-Murphi):

• Added finite precision real numbers and probabilities;

• Replaced the set of the initial states with a distribution of initial states;

• Labeled Murphi rules with probabilities, so as to define Markov Chains;

• Replaced Murphi deterministic invariant with a BPCTL formula of form [tt Uk p] stating that that the probability of reaching an error state in k steps is less than a given threshold .

9

Nested Untils Are InterestingFHP-Murphi can only handle finite horizon safety properties, that is BPCTL formuals of form [tt Uk p].

However there are many interesting properties that are not definable using formulas of form [tt Uk p], and thus cannot be handled by FHP-Murphi.

For example, robustness and reliability properties are very useful when carrying out requirements analysis. Typically such properties have the following form:

More formally: There is a low probability of reaching (within k steps) an undesired state s, such that there is not an high probability of reaching (within h steps) a non-undesired state from s.

The probability of getting into an undesired system state and staying there for a long time, is low.

We need nested Until’s to define reliability properties: ( defines undesired states) [tt Uk ( [tt Uh ] 1)]0

Example: if a (probabilistic dining) philosopher has not eaten since a long time (undesired state), then he will soon eat, thus avoiding perishing for starvation.

10

Our GoalUsing nested untils it is possible to express interesting properties (e.g. robustness, reliability). Moreover such properties are particularly important for hybrid systems.

Remark 1.Of course PRISM can handle full BPCTL, however when handling hybrid systems it dies quite soon because of state explosion.

Thus it is worthwhile extending FHP-Murphi to full BPCTL.

Remark 2.Of course explicit PCTL (no bound on path lengths) would also be interesting, however it is computationally harder than BPCTL.

Here we will show how FHP-Murphi can be extended so as to handle full BPCTL.

11

FHP-Murphi Verification Algorithm

.

No-nested-untils FHP-Murphi uses a BF visit to carry out its computations.

However, a BF visit is not effective when dealing with nested untils.

Example.

Suppose we want to check if s |= F holds, where:

F = [tt Uk ] , = [tt Uh ] and is an atomic proposition.

Using the no-nested-untils FHP-Murphi algorithm this requires to check that for all t reachable from s in at most k steps it holds t |= .

Since = [tt Uh ] , this means that for all t reachable from s in at most k steps we need to start a (nested) BF visit to check if t |= [tt Uh ] holds.

We will show how, using a Depth First (DF) visit and a cache we can avoid most of these nested visits.

12

Easy Case: No Nested Untils

We want to verify if s0 |= [tt U2 ]0.5 . Atomic formula holds in states: s1 , s4 , s7.

After visitig s0, the probability of reaching a state in which holds is: 0

s0

s1

s2

s5

s6

s4

s3

s7

s8

1/3

1/3

1/3

1

1/2

1/2

1

1

11

1

1

13

We want to verify if s0 |= [tt U2 ]0.5 holds. Atomic formula holds in states: s1 , s4 , s6.

After visitig s1 the probability of reaching a state in which holds is: (1/3)*1 = (1/3).

s0

s1

s2

s5

s6

s4

s3

s7

s8

1/3

1/3

1/3

1

1/2

1/2

1

1

11

1

1


14

After visitig s2 the probability of reaching a state in which holds is: (1/3) + (1/3)*[…]

s0

s1

s2

s5

s6

s4

s3

s7

s8

1/3

1/3

1/3

1

1/2

1/2

1

1

11

1

1



15


s0

s1

s2

s5

s6

s4

s3

s7

s8

1/3

1/3

1/3

1

1/2

1/2

1

1

11

1

1

After visitig s4 the probability of reaching a state in which holds is: (1/3) + (1/3)*[(1/2)*1 + …]


16


s0

s1

s2

s5

S6

s4

s3

s7

s8

1/3

1/3

1/3

1

1/2

1

1

11

1

1

After visitig s3 the probability of reaching a state in which holds is: (1/3) + (1/3)*[(1/2)*1 + (1/2)*0]

1/2


17


s0

s1

s2

s5

s6

s4

s3

s7

s8

1/3

1/3

1/3

1

1/2

1/2

1

1

11

1

1

After visitig s5 the probability of reaching a state in which holds is: (1/3) + (1/3)[(1/2)1 + (1/2)0] + (1/3)[…]


18


s0

s1

s2

s5

s6

s4

s3

s7

s8

1/3

1/3

1/3

1

1/2

1/2

1

1

11

1

1

After visitig s5 the probability of reaching a state in which holds is: (1/3) + (1/3)[(1/2)1 + (1/2)0] + (1/3)[1]


19


s0

s1

s2

s5

s6

s4

s3

s7

s8

1/3

1/3

1/3

1

1/2

1/2

1

1

11

1

1

Finally, the probability of reaching a state in which holds is: (1/3) + (1/3)[(1/2)1 + (1/2)0] + (1/3)[1] = (1/3) + (1/6) + (1/3) = (5/6) 0.5. Thus s0 |= [tt U2 ]0.5 holds.


Never reached, since holds in s1.

Never reached, since at distance 3 from s0.

20

Nested Untils (Use Cache)

s0

s5

s1

s4

s2

s3

1/2

1/2

1/2

1/2

1/2

1/2

1

1

.. .

…..

We want to verify if s0 |= F holds, where: F [ Uk ]0.4 , [tt Uh ]0.9 , atomic formula.

CACHE

NotationP(s, [ Uk ]) = the probability that ( Uk ) holds on a path starting from s.

21

Nested Untils (Use Cache)CACHE

b3 = P(s3, [tt Uh ])

p3 = P(s3, [ Uk-3 ])s0

s5

s1

s4

s2

s3

1/2

1/2

1/2

1/2

1/2

1/2

1

1

.. .

…..

DF visit of s3 completed. We first compute P(s3, [tt Uh ]) and then P(s3, [ Uk-3 ]).Both probabilities are stored in cache.

F [ Uk ]0.4 , [tt Uh ]0.9 , atomic formula.

22


s0

s5

s1

s4

s2

s3

1/2

1/2

1/2

1/2

1/2

1/2

1

1

.. .

…..

CACHE

b3 = P(s3, [tt Uh ])

p3 = P(s3, [ Uk-3 ])

p2 = P(s2, [ Uk-2 ])

DF visit of s2 completed. P(s2, [ Uk-2 ]) = 1*p3 computed and stored in cache.

RemarkNo need to revisit s3, we can use p3 in cache.


23


s0

s5

s1

s4

s2

s3

1/2

1/2

1/2

1/2

1/2

1/2

1

1

.. .

…..

CACHE

b3 = P(s3, [tt Uh ])

p3 = P(s3, [ Uk-3 ])

p2 = P(s2, [ Uk-2 ])

Visiting s1. P(s1, [ Uk-1 ]) = (1/2) P(s2, [ Uk-2 ]) + (1/2) P(s4, [ Uk-2 ]) = (1/2)p2 + …

RemarkNo need to revisit s2, we can use p2 in cache. We need P(s4, [ Uk-2 ]).


24


s0

s5

s1

s4

s2

s3

1/2

1/2

1/2

1/2

1/2

1/2

1

1

.. .

…..

CACHE

b3 = P(s3, [tt Uh ])

p3 = P(s3, [ Uk-3 ])

p2 = P(s2, [ Uk-2 ])

p4 = P(s4, [ Uk-2 ])

Completed DF visit of s4. P(s4, [ Uk-2 ]) = 1* P(s3, [ Uk-3 ]) = 1*p3.

RemarkNo need to revisit s3, we can use p3 in cache.


25


s0

s5

s1

s4

s2

s3

1/2

1/2

1/2

1/2

1/2

1/2

1

1

.. .

…..

CACHE

b3 = P(s3, [tt Uh ])

p3 = P(s3, [ Uk-3 ])

b3 = P(s3, [tt Uh ])

p2 = P(s2, [ Uk-2 ])

p4 = P(s4, [ Uk-2 ])

p1 = P(s1, [ Uk-1 ])

Completed DF visit of s1. P(s1, [ Uk-1 ]) = (1/2)* P(s2, [ Uk-2 ]) + (1/2) P(s4, [ Uk-2 ]) = (1/2)p2 + (1/2)p4.

RemarkNo need to revisit s2, s4, we can use p2, p4 in cache.


26


s0

s5

s1

s4

s2

s3

1/2

1/2

1/2

1/2

1/2

1/2

1

1

.. .

…..

CACHE

b3 = P(s3, [tt Uh ])

p3 = P(s3, [ Uk-3 ])

p2 = P(s2, [ Uk-2 ])

p4 = P(s4, [ Uk-2 ])

p1 = P(s1, [ Uk-1 ])

Visiting s0. P(s0, [ Uk ]) = (1/2)* P(s1, [ Uk-1 ]) + (1/2) P(s5, [ Uk-1 ]) = (1/2)p1 + ….

RemarkNo need to revisit s1, we can use p1 in cache. We need P(s5, [ Uk-1 ]).


27


s0

s5

s1

s4

s2

s3

1/2

1/2

1/2

1/2

1/2

1/2

1

1

.. .

…..

CACHE

b3 = P(s3, [tt Uh ])

p3 = P(s3, [ Uk-3 ])

p2 = P(s2, [ Uk-2 ])

p4 = P(s4, [ Uk-2 ])

p1 = P(s1, [ Uk-1 ])

Visiting s5. P(s5, [ Uk-1 ]) = (1/2)* P(s4, [ Uk-2 ]) + (1/2) P(s3, [ Uk-2 ]) = (1/2)p4 + ….

RemarkNo need to revisit s4, we can use p4 in cache. We need P(s3, [ Uk-2 ]) (not in cache)


28


s0

s5

s1

s4

s2

s3

1/2

1/2

1/2

1/2

1/2

1/2

1

1

.. .

…..

CACHE

b3 = P(s3, [tt Uh ])

p3 = P(s3, [ Uk-3 ])

p2 = P(s2, [ Uk-2 ])

p4 = P(s4, [ Uk-2 ])

p1 = P(s1, [ Uk-1 ])

q3 = P(s3, [ Uk-2 ])

Visit s3 and compute P(s3, [ Uk-2 ]).

RemarkThis computation may use P(s3, [tt Uh ]) (b3) in cache.


29


s0

s5

s1

s4

s2

s3

1/2

1/2

1/2

1/2

1/2

1/2

1

1

.. .

…..

CACHE

b3 = P(s3, [tt Uh ])

p3 = P(s3, [ Uk-3 ])

p2 = P(s2, [ Uk-2 ])

p4 = P(s4, [ Uk-2 ])

p1 = P(s1, [ Uk-1 ])

q3 = P(s3, [ Uk-2 ]) p5 = P(s5, [ Uk-1 ])

Completed DF visit of s5. P(s5, [ Uk-1 ]) = (1/2)* P(s4, [ Uk-2 ]) + (1/2) P(s3, [ Uk-2 ]) = (1/2)p4 + (1/2)q3.

RemarkNo need to revisit s4, s3, we use p4, q3 in cache.


30


s0

s5

s1

s4

s2

s3

1/2

1/2

1/2

1/2

1/2

1/2

1

1

.. .

…..

CACHE

b3 = P(s3, [tt Uh ])

p3 = P(s3, [ Uk-3 ])

p2 = P(s2, [ Uk-2 ])

p4 = P(s4, [ Uk-2 ])

p1 = P(s1, [ Uk-1 ])

p6 = P(s6, [ Uk-2 ]) q3 = P(s3, [ Uk-2 ]) p5 = P(s5, [ Uk-1 ])

p0 = P(s0, [ Uk ])

Completed DF visit of s0. P(s0, [ Uk ]) = (1/2)* P(s1, [ Uk-1 ]) + (1/2) P(s5, [ Uk-1 ]) = (1/2)p1 + (1/2)p5.Remark 1. No need to revisit s1, s5, we use p1, p5 in cache.

Remark 2. The sketched procedure is carried out on all U-subformulas of the BPCTL formula to be verified.


31

Experimental Results

To assess effectiveness of our approach we run experiments with

• Randomized protocols

• Hybrid Systems

32

Randomized ProtocolsFor such experiments we used:• PZ: Pnueli-Zuck probabilistic dining philosophers protocol, as found in PRISM distribution;• MPZ: PZ modified so as to meet quality-of-service requirements; • LR: Lehmann-Rabin probabilistic dining philosophers protocol;• MLR: LR modified so as to meet quality-of-service requirements.

Property: When an undesired state s is reached, then the system almost always reaches, from s

and in a few steps, a non-error state.More formally:There is a low probability that we reach in d steps an undesired state s, and there is not a high probability of reaching in k steps from s, a non-error state.BPCTL form: [tt Uk (und [tt Uh err] 1)]0 (PZ, LR, MPZ, MLR).

FHP-Murphi options: -500m (use 500 MB RAM)

PRISM options: default

All experiments run on a Linux 500 MHz INTEL Pentium III with 2GB RAM.

33

Randomized Protocols ResultsNPHIL Result Murphi MB PRISM MB Murphi T (s) PRISM T (s)

5 False 500 1.70 3411 1.318

6 False 500 14.30 > 300,000 12.60

PZ

NPHIL Result Murphi MB PRISM MB Murphi T (s) PRISM T (s)

3 True 500 1.42 1792 2.018

4 True 500 23.56 142,337 103.414

LR

WAIT NPHIL Result Murphi MB PRISM MB Murphi T (s) PRISM T (s)

4 5 False 500 917 12,838 1197

5 5 False 500 N/A 12,737 N/A

MPZ

WAIT NPHIL Result Murphi MB PRISM MB Murphi T (s) PRISM T (s)

4 3 True 500 70 5006 536

3 4 True 500 N/A 111,481 N/A

MLRPRISM better on PZ, LR. FHP-Murphi better on MPZ, MLR

34

Hybrid SystemsWe experimented with verification of a turbogas control system (HSCC’03), given a probability distribution on the user demand. Because of state explosion, this verification is feasible only with FHP-Murphi (i.e. PRISM runs out of memory).

Property: If the system reaches an undesired state, then with high probability it is able to return to a non-undesired state.

More formally:There is a low probability that we reach an undesired state s, such that there is not a high probability of reaching a non-undesired state from s.

BPCTL form: [tt Uk ( [tt Uh ] 1)]0 ( defines undesired states).

All experiments run on a Linux 500 MHz INTEL Pentium III with 2GB RAM.

FHP-Murphi options: -500m (use 500 MB RAM)

35

Gas Turbine System

ControllerGas Turbine(Turbogas)

Disturbances: electric users, param. var, etc

Vrot: Turbine Rotation speedTexh: Exhaust smokes TemperaturePel: Generated Electric PowerPmc: Compressor Pressure

Settings Fuel Valve OpeningFG102

Vrot, Texh, Pel, Pmc

36

Hybrid System ResultsMAX_D_U Visited

StatesRules Fired Horizon (k) CPU T (s) Probability

35 1.159e+05 3.477e+05 800 3.702e+03 4.105e-03

45 4.098e+04 1.229e+05 700 1.314e+03 1.793e-02

50 4.067e+04 1.220e+05 700 1.309e+03 3.825e-02

Probability: P(q, [tt Uk ( [tt Uh ] 1)]), q initial state, h = (k/100).

MAX_D_U: Max speed of variation of user demand (KW/sec).

37

ConclusionsPerformances

When PRISM terminates it is faster than FHP-Murphi.

However, PRISM is not suitable for verification tasks involving (complex) math computations. Examples of such tasks are: analysis of hybrid systems, analysis of randomized protocols, performance analysis, etc. FHP-Murphi can handle such verification tasks when the horizon is not too large.

Expressivness

PRISM can handle BPCTL formulas and much more. Hard to model hybrid systems using PRISM.

FHP-Murphi only handles BPCTL formulas. Hybrid systems are easily modeled using FHP-Murphi (since all C functions on real number available).

Moral: If you have to handle (complex) math use FHP-Murphi, else use PRISM.

38

Future Work

• Infinite Horizon

• Continuous Markov Chains

• …..

bounded probabilistic model checking with the murphi verifier

Documents

probabilistic model

initial state

verifieda probabilistic

tt u s

tt u2 s

tt u1 s

stochastic system

high security level