StatisticalProbabilistic Model Checking

Håkan L. S. YounesCarnegie Mellon University

2

Introduction

Model checking for stochastic processes Stochastic discrete event systems Probabilistic time-bounded properties

Model independent approach Discrete event simulation Statistical hypothesis testing

3

Example:Tandem Queuing Network

q1 q2

arrive route depart

q1 = 0q2 = 0q1 = 0q2 = 0

q1 = 1q2 = 0

q1 = 1q2 = 1

q1 = 2q2 = 0

q1 = 1q2 = 0

t = 0 t = 1.2 t = 3.7 t = 3.9 t = 5.5

With both queues empty, is the probability less than 0.5that both queues become full within 5 seconds?

q1 = 1q2 = 0

q1 = 2q2 = 0

q1 = 1q2 = 1

q1 = 1q2 = 0

4

Probabilistic Model Checking

Given a model M, a state s, and a property , does hold in s for M? Model: stochastic discrete event

system Property: probabilistic temporal logic

formula

5

Continuous Stochastic Logic (CSL)

State formulas Truth value is determined in a single

state Path formulas

Truth value is determined over a pathDiscrete-time analogue: PCTL

6

State Formulas

Standard logic operators: , 1 2, …

Probabilistic operator: P≥ () Holds in state s iff probability is at

least that holds over paths starting in s

P< () P≥1– ()

7

Path Formulas

Until: 1 U ≤T 2

Holds over path iff 2 becomes true in some state along before time T, and 1 is true in all prior states

8

CSL Example

With both queues empty, is the probability less than 0.5 that both queues become full within 5 seconds? State: q1 = 0 q2 = 0

Property: P<0.5(true U ≤5 q1 = 2 q2 = 2)

9

Model Checking Probabilistic Time-Bounded Properties

Numerical Methods Provide highly accurate results Expensive for systems with many states

Statistical Methods Low memory requirements Adapt to difficulty of problem

(sequential) Expensive if high accuracy is required

10

Statistical Solution Method [Younes & Simmons 2002]

Use discrete event simulation to generate sample paths

Use acceptance sampling to verify probabilistic properties Hypothesis: P≥ () Observation: verify over a sample

pathNot estimation!

11

Error Bounds

Probability of false negative: ≤ We say that is false when it is true

Probability of false positive: ≤ We say that is true when it is false

12

Performance of Test

Actual probability of holding

Pro

bab

ility

of

acc

ep

tin

gP

≥ (

) as

tru

e

1 –

13

Ideal Performance of Test

Actual probability of holding

Pro

bab

ility

of

acc

ep

tin

gP

≥ (

) as

tru

e

1 –

False negatives

False positives

Unrealistic!

14

Realistic Performance of Test

Actual probability of holding

Pro

bab

ility

of

acc

ep

tin

gP

≥ (

) as

tru

e

1 –

p1 p0

Indifference region

False negatives

False positives

2

15

SequentialAcceptance Sampling [Wald 1945]

True, false, or another observatio

n?

16

Graphical Representation of Sequential Test

Number of observations

Nu

mb

er

of

posi

tive

ob

serv

ati

on

s

17

Graphical Representation of Sequential Test

We can find an acceptance line and a rejection line given , , , and acceptance line

rejection line

reject

accept

continue

Number of observations

Nu

mb

er

of

posi

tive

ob

serv

ati

on

s

Start here

Verify oversample paths

Continue untilline is crossed

18

Special Case

p0 = 1 and p1 = 1 – 2 Reject at first negative observation Accept at stage m if p1

m ≤ Sample size at most dlog / log p1e

“Five nines”: p1 = 1 – 10–5

Maximum sample size

10–2 460,515

10–4 921,030

10–8 1,842,059

19

Case Study:Tandem Queuing Network

M/Cox2/1 queue sequentially composed with M/M/1 queue

Each queue has capacity n State space of size O(n2)

1 2

a……

1 − a

20

Tandem Queuing Network (results) [Younes et al. 2004]V

eri

fica

tion

tim

e (

seco

nds)

Size of state space101 102 103 104 105 106 107 108 109 1010 1011

10−2

10−1

100

101

102

103

104

105

106 T=500 (numerical)T=50 ( " )T=5 ( " )T=500 (statistical)T=50 ( " )T=5 ( " )

P≥0.5(true U≤T full)

= 10−6

= = 10−2

= 0.5·10−2

21

Tandem Queuing Network (results) [Younes et al. 2004]

n=255 (numerical)n=31 ( " )n=3 ( " )n=255 (statistical)n=31 ( " )n=3 ( " )

Veri

fica

tion

tim

e (

seco

nds)

T

10−2

10−1

100

101

102

103

104

105

106

101 102 103 104

= 10−6

= = 10−2

= 0.5·10−2

P≥0.5(true U≤T full)

22

Case Study:Symmetric Polling System

Single server, n polling stations Stations are attended in cyclic

order Each station can hold one message State space of size O(n·2n)

Server

…Polling stations

23

Symmetric Polling System (results) [Younes et al. 2004]

T=40 (numerical)T=20 ( " )T=10 ( " )T=40 (statistical)T=20 ( " )T=10 ( " )

Veri

fica

tion

tim

e (

seco

nds)

Size of state space

10−2

10−1

100

101

102

103

104

105

106

102 104 106 108 1010 1012 1014

serv1 P≥0.5(true U≤T poll1)

= 10−6

= = 10−2

= 0.5·10−2

24

Symmetric Polling System (results) [Younes et al. 2004]

n=15 (numerical)n=10 ( " )n=5 ( " )n=15 (statistical)n=10 ( " )n=5 ( " )

Veri

fica

tion

tim

e (

seco

nds)

T

10−2

10−1

100

101

102

103

104

105

106

101 102 103

= 10−6

= = 10−2

= 0.5·10−2

serv1 P≥0.5(true U≤T poll1)

25

Symmetric Polling System (results) [Younes et al. 2004]

numerical (=10−6)==10−2

==10−4

==10−6

==10−8

==10−10

Veri

fica

tion

tim

e (

seco

nds)

10−1

100

101

102

10−4 10−210−3

n = 10T = 40

serv1 P≥0.5(true U≤T poll1)

26

Tandem Queuing Network: Distributed Sampling Use multiple machines to generate samples

m1: Pentium IV 3GHz m2: Pentium III 733MHz m3: Pentium III 500MHz

% samples % samples m1 only

n m1 m2 m3 time m1 m2 time time

63 70 20 10 0.46 71 29 0.50 0.58

2047 60 26 14 1.28 70 30 1.46 1.93

65535 65 21 14 26.29 67 33 33.89 44.85

27

Summary

Acceptance sampling can be used to verify probabilistic properties of systems

Sequential acceptance sampling adapts to the difficulty of the problem

Statistical methods are easy to parallelize

28

Other Research

Failure trace analysis “failure scenario” [Younes & Simmons 2004a]

Planning/Controller synthesis CSL goals [Younes & Simmons 2004a]

Rewards (GSMDPs) [Younes & Simmons 2004b]

29

Tools

Ymer Statistical probabilistic model

checking Tempastic-DTP

Decision theoretic planning with asynchronous events

30

ReferencesWald, A. 1945. Sequential tests of statistical hypotheses. Ann.

Math. Statist. 16: 117-186.Younes, H. L. S., M. Kwiatkowska, G. Norman, and D. Parker.

2004. Numerical vs. statistical probabilistic model checking: An empirical study. In Proc. TACAS-2004.

Younes, H. L. S., R. G. Simmons. 2002. Probabilistic verification of discrete event systems using acceptance sampling. In Proc. CAV-2002.

Younes, H. L. S., R. G. Simmons. 2004a. Policy generation for continuous-time stochastic domains with concurrency. In Proc. ICAPS-2004.

Younes, H. L. S., R. G. Simmons. 2004b. Solving generalized semi-Markov decision processes using continuous phase-type distributions. In Proc. AAAI-2004.