the challenges of probabilistic thinking (keynote talk at icfem 2017)

The Challenges of Probabilis1c Thinking

David S. Rosenblum Na1onal University of Singapore

National University of Singapore

Last year at ASE 2016 …

National University of Singapore

– Norman Vincent Peale, The Power of Posi,ve Thinking

“Do not build up obstacles in your imagina1on.”

Three Key Challenges for Probabilis1c Methods

1. Where do the probabili0es come from?

2. What if the probabili0es are wrong?

3. What if uncertainty masks faulty behavior?

Acknowledgments

Guoxin Su

University of Wollongong

Yamilet Serrano

Na1onal University of Singapore

Genaína Rodrigues Universidade de

Brasília

LeFcia Duboc

E1cas Research & Consul1ng

and Sebas1an Uchitel, Yuan Feng, Taolue Chen, P.S. Thiagarajan,Giordano Tamurrelli, MaS Dwyer, David Shirver, Tony Wicks

Sebas0an Elbaum

University of Nebraska - Lincoln

Where Do the Probabili1es Come

From?

Quan1fying Uncertaintyself-inflicted uncertainty: randomiza1on deliberately introduced into a system for some desirable effect

coin flipping, symmetry breaking (Leader Elec,on), …

epistemic uncertainty: reducible systema1c uncertainty that is too difficult to resolve or quan1fy more precisely

choice of node address (Zeroconf), user’s input commands, ...

aleatoric uncertainty: irreducible sta1s1cal noise that varies from execu1on to execu1on

latency of a message delivery, collision on an Ethernet link, …

… in both a model and its proper,es

Assigning Probabili1es Requirements Engineering

Le^cia Duboc, Emmanuel Le1er and David S. Rosenblum, “Systema1c Elabora1on of Scalability Requirements through Goal-Obstacle Analysis”, IEEE Transac,ons on SoGware Engineering, Jan. 2013.

Par,al KAOS Goal Model for IEF Financial

Transac,on Fraud Detec,on System

Assigning Probabili1es Requirements Engineering

Le^cia Duboc, Emmanuel Le1er and David S. Rosenblum, “Systema1c Elabora1on of Scalability Requirements through Goal-Obstacle Analysis”, IEEE Transac,ons on SoGware Engineering, Jan. 2013.

Example KAOS Goal Specifica,on

for IEF

Assigning Probabili1es Distribu1ons MaSer Too!

Ana Le^cia de Cerqueira Le1e Duboc, “A Framework for the Characterisa1on and Analysis of Sofware Systems Scalability”,PhD thesis, University College London, 2010.

U1lity of Two Hypothe1cal Designs as a Func1on of Number of IEF En11es

pdf 1 pdf 2

Assigning Probabili1es Run1me Sampling

Guoxin Su, David S. Rosenblum and Giordano Tamburrelli, “Reliability of Run-Time Quality-of-Service Evalua1onusing Parametric Model Checking”, Proc. ICSE 2016.

What If the Probabili1es Are Wrong?

Probabilis1c Model Checking Changing the Model Probabili1es

! ¬p→◊q( )∧"( )

Model Checker

✓

State Machine Model

Temporal Property

ResultsSystem

Requirements

P≥0.95 [ ]

0.4

0.6

Quan1ta1ve Results

0.9732Probabilis1c

Probabilis1c

Probabilis1c Model Checking Changing the Model Probabili1es

! ¬p→◊q( )∧"( )

Model Checker

✕

State Machine Model

Temporal Property

Results

Counterexample Trace

System

Requirements

P≥0.95 [ ]

Quan1ta1ve Results

Probabilis1c

Probabilis1c0.41

0.59

0.6211

Perturbed Probabilis1c Systems Current Research

Overall approach

• Compute asympto1c bounds for efficiency • Apply the bounds for non-asympto1c es1ma1on

Star@ng Point • Parametric Discrete-Time Markov Chains • “Small” perturba1ons of probability parameters • Reachability proper1es P≤p [ ] • Linear bounds for es1ma1ng verifica1on impact

S? U S!

Example The Zeroconf Protocol

s1 s0 s2 s3

q

1

1

{ok} {error}

{start} s4

s5

s6

s7

s8

1

1-q

1-p

1-p

1-p 1-p

p p p

p

1

DTMC model from the PRISM group(Kwiatkowska et al.)

P=? [ true U error ]

0.1 0.9

0.5

0.5

0.10.10.1

0.90.9

0.9

S?

S!

Perturba1on Distance•Perturba1on is captured in distribu1on parameters,

a vector x of probability parameters xi

•The norm of total variance measures the amount of perturba1on

•Perturba1on distance is computed with respect to reference values r

v = vi∑

x − r ≤ Δ

Guoxin Su and David S. Rosenblum, “Asympto1c Bounds for Quan1ta1ve Verifica1on of Perturbed Probabilis1c Systems”, Proc. ICFEM 2013.


s1 s0 s2 s3

q

1

1

{ok} {error}

{start} s4

s5

s6

s7

s8

1

1-q

1-p

1-p

1-p 1-p

p p p

p

1


0.1 0.9

0.5

0.5

0.10.10.1

0.90.9

0.9


s1 s0 s2 s3

q

1

1

{ok} {error}

{start} s4

s5

s6

s7

s8

1

1-q

1-p

1-p

1-p 1-p

p p p

p

1


0.1 0.9

0.5

0.5

0.10.10.1

0.90.9

0.9

x1

1-x1

x2

1-x2 1-x3

x3

x4

1-x4

• Perturba1on Func1on captures effect of perturba1on on verifica1on result p

where i? is the ini1al state distribu1on and A is the transi1on probability sub-matrix for S? and b is the vector of one-step probabili1es from S? to S!and h•(x-r) is the linear approxima1on of ρ near r

• Condi1on Number provides asympto1c bound of ρ

• Predicted varia1on to verifica1on result p due to perturba1on Δ

ρ x( ) = ι? i A x( )i ib x( )( )− A r i ib r( )( )i=0

∞

∑ ≈ h i x − r( )

κ = limΔ→0

sup ρ(x)δ

: x − r ≤ Δ,0 < δ ≤ Δ⎧⎨⎩

⎫⎬⎭≈ 1

2max h( )− min h( )( )

p̂ = p ±κΔ

Guoxin Su and David S. Rosenblum, “Asympto1c Bounds for Quan1ta1ve Verifica1on of Perturbed Probabilis1c Systems”, Proc. ICFEM 2013.

Asympto1c Perturba1on Bounds on Verifica1on Impact

Case Study Results Noisy Zeroconf (35000 Hosts)x

Probability of Reaching error StateActual (PRISM) Predicted (via κ)

0.095 -19.8% -21.5%0.096 -16.9% -17.2%0.097 -12.3% -12.9%0.098 -8.33% -8.61%0.099 -4.23% -4.30%0.100 1.8567 ✕ 10-4 —

0.101 +4.38% +4.30%0.102 +8.91% +8.61%0.103 +13.6% +12.9%0.104 +18.4% +17.2%0.105 +23.4% +21.5%

Guoxin Su and David S. Rosenblum, “Perturba1on Analysis of Stochas1c Systems with Empirical Distribu1on Parameters”, Proc. ICSE 2014.

Addi1onal Results[ICSE 2014] ω-regular proper1es and quadra1c perturba1on bounds

[CONCUR 2014] Asympto1c and non-asympto1c bounds forthree addi1onal perturba1on distance norms

[ATVA 2014] Interval approxima1ons for reachability proper1es withnested P operators

[FSE 2014 Doctoral Symposium]

Heuris1cs for Markov Decision Processes (MDPs) (Yamilet Serrano’s PhD)

[FASE 2016] Applica1on to decision making in self-adap1ve systems

[ICSE 2016] Applica1on to run1me QoS evalua1on

[IEEE TSE 2016] Integra1on of previous results and new case studies

[ICSE 2017] Con1nuous-Time Markov Chains (CTMCs)

[ESEC/FSE 2017] Markov Decision Processes (MDPs)

MDP Example Cloud Service Migra1on in a Mobile Network

adversaries resolve nondeterminis1c choices

The Challenge for MDPs Iden1fying Relevant Adversaries

0 1Pmin Pmax

op1mal adversaries

The Challenge for MDPs Iden1fying Relevant Adversaries

0 1Pmin Pmax

op1mal adversaries

Op1mal adversaries may not induce minimum and maximum probabili1es in presence of perturba1on

ApproachesIden1fying Relevant Adversaries

✓Approximate: ignore the problem, find the maximum condi1on number over the op1mal adversaries

✓Heuris1c: apply a range of brute-force adversary enumera1on schemes to find the maximum condi1on number

✓Algorithmic: efficiently find the maximum condi0on number over the most relevant adversaries

Case Study Results Cloud Migra1on (5 & 8 Rings, 3 Proper1es)

Yamilet R. Serrano Llerena, Guoxin Su and David S. Rosenblum, “Probabilis1c Model Checking of Perturbed MDPswith Applica1ons to Cloud Compu1ng”, Proc. ESEC/FSE 2017.

Model States Transi0ons Adversaries PropertyMaximumCondi0on Number

Time (seconds)

Exhaus0ve Our Algorithm

5 Rings 9 17 32

P1 0.1111 3.28 0.15P2 0.5 4.02 0.21P3 0.5 2.95 0.10

8 Rings 15 32 2048

P1 0.0102 197.10 141.06P2 0.5 288.97 202.42P3 0.1111 199.05 139.75

What If Uncertainty in the Modeled System

Masks Faulty Behavior?

Challenge Pinpoin1ng the Root Cause of Uncertainty

“There are known knowns; there are things we know we know. We also

know there are known unknowns; that is to say, we know there are some

things we do not know. But there are also unknown unknowns – the ones

we don’t know we don’t know.”

— Donald Rumsfeld

Known Unknowns in Modern Systems

✓ Autonomous Vehicles

✓ Cyber Physical Systems

✓ Internet of Things

✓ Extensive reliance on machine learning

see

Deep Learning and Understandability versusSoGware Engineering and Verifica,on

by Peter Norvig, Director of Research at Google

hSp://www.youtube.com/watch?v=X769cyzBNVw

Uncertainty in Tes1ng Current Research

TestExecu1on

System Under Test

Result Interpreta1on


TestExecu1on

System Under Test


Unacceptable

Acceptable✓

✕


TestExecu1on

System Under Test


Unacceptable

Acceptable

Acceptable

✓

✕

✕

Acceptable misbehaviors can mask real faults!

One Possible Solu1on Distribu1on Fi�ng

System Under Test

TrainingData WEKA

Sebas1an Elbaum and David S. Rosenblum, “Known Unknowns: Tes1ng in the Presence of Uncertainty”, Proc. FSE 2014.


System Under Test

WEKA



System Under Test


TestExecu1on



System Under Test


Unacceptable

Acceptable

Inconclusive

p < 0.99

TestExecu1on p < 0.37

p < 0.0027


Characterizing Recommenda1on Systems

David Shriver, MaShew B. Dwyer, Sebas1an Elbaum and David S. Rosenblum, “Characteris1c Proper1es of Recommenda1on Systems”,under review, 2017.

Summary

The Challenges of Probabilis1c Thinking

David S. Rosenblum Na1onal University of Singapore

the challenges of probabilistic thinking (keynote talk at icfem 2017)

Technology