Program Verification using Probabilistic Techniques

Sumit GulwaniMicrosoft Research

Invited Talk: VSTTE WorkshopAugust 2006

Joint work with George Necula and Nebojsa Jojic

2

Probabilistic Techniques

• Used successfully in several areas of computer science.• Yields more efficient, precise, even simpler algorithms.• Technique 1: Random Interpretation

– Discovers program invariants– Monte Carlo Algorithm: May generate invalid invariants

with a small probability. Running time is bounded.– “Random Testing” + “Abstract Interpretation”

• Technique 2: Simulated Annealing– Discovers proof of validity/invalidity of a Hoare triple.– Las Vegas Algorithm: Generates a correct proof. Running

time is probabilistic.– “Forward Analysis” + “Backward Analysis”

3

Random Interpretation

= Random Testing + Abstract Interpretation

Random Testing:• Test program on random inputs• Simple, efficient but unsound (can’t prove absence of bugs)

Abstract Interpretation:• Class of deterministic program analyses• Interpret (analyze) an abstraction (approximation) of program

• Sound but usually complicated, expensive

Random Interpretation:• Class of randomized program analyses• Almost as simple, efficient as random testing• Almost as sound as abstract interpretation

4

a := 0; b := i;

a := i-2; b := 2;

c := b – a; d := i – 2b;

assert(c+d = 0); assert(c = a+i)

c := 2a + b; d := b – 2i;

True False

FalseTrue

*

*

Example 1

5

a := 0; b := i;

a := i-2; b := 2;

c := b – a; d := i – 2b;

assert(c+d = 0); assert(c = a+i)

c := 2a + b; d := b – 2i;

True False

FalseTrue

*

*

Example 1: Random Testing

• Need to test blue path to falsify second assertion.

• Chances of choosing blue path from set of all 4 paths are small.

• Hence, random testing is unsound.

6

a+b=i

a+b=i, c=-d

a=i-2, b=2

a+b=i c=2a+b, d=b-2ia+b=i

c=b-a, d=i-2b

a=0, b=i

a := 0; b := i;

a := i-2; b := 2;

c := b – a; d := i – 2b;

assert(c+d = 0); assert(c = a+i)

c := 2a + b; d := b – 2i;

True False

FalseTrue

*

*

Example 1: Abstract Interpretation

• Computes invariant at each program point.

• Operations are usually complicated and expensive.

7

a := 0; b := i;

a := i-2; b := 2;

c := b – a; d := i – 2b;

assert(c+d = 0); assert(c = a+i)

c := 2a + b; d := b – 2i;

True False

FalseTrue

*

*

Example 1: Random Interpretation

• Choose random values for input variables.

• Execute both branches of a conditional.

• Combine values of variables at join points.

• Test the assertion.

8

Random Interpretation: Outline

• Random Interpretation

Linear arithmetic (POPL 2003)

– Uninterpreted functions (POPL 2004)

– Inter-procedural analysis (POPL 2005)

9

Linear relationships in programs with linear assignments

• Linear relationships (e.g., x=2y+5) are useful for– Program correctness (e.g. buffer overflows)– Compiler optimizations (e.g., constant and copy

propagation, CSE, Induction variable elimination etc.)

• “programs with linear assignments” does not mean inapplicability to “real” programs– “abstract” other program stmts as non-

deterministic assignments (standard practice in program analysis)

10

Basic idea in random interpretation

Generic algorithm:

• Choose random values for input variables.

• Execute both branches of a conditional.

• Combine the values of variables at join points.

• Test the assertion.

11

Idea #1: The Affine Join operation

w = 7

a = 2b = 3

a = 4b = 1

a = 7(2,4) = -10b = 7(3,1) = 15

• Affine join of v1 and v2 w.r.t. weight w

w(v1,v2) ´ w v1 + (1-w) v2

• Affine join preserves common linear relationships (a+b=5)

• It does not introduce false relationships w.h.p.

12

Idea #1: The Affine Join operation

• Affine join of v1 and v2 w.r.t. weight w

w(v1,v2) ´ w v1 + (1-w) v2

• Affine join preserves common linear relationships (a+b=5)• It does not introduce false relationships w.h.p.• Unfortunately, non-linear relationships are not preserved

(e.g. a £ (1+b) = 8)

w = 5

a = 5(2,4) = -6b = 5(3,1) = 11

w = 7

a = 2b = 3

a = 4b = 1

a = 7(2,4) = -10b = 7(3,1) = 15

13

Geometric Interpretation of Affine Join

a

ba + b =

5

b = 2

(a = 2, b = 3)

(a = 4, b = 1)

: State before the join

: State after the join

satisfies all the affine relationships that are satisfied by both (e.g. a + b = 5)

Given any relationship that is not satisfied by any of (e.g. b=2), also does not satisfy it with high probability

i=3, a=0, b=3

i=3

a := 0; b := i;

a := i-2; b := 2;

c := b – a; d := i – 2b;

assert (c+d = 0); assert (c = a+i)

i=3, a=-4, b=7

i=3, a=-4, b=7c=23, d=-23

c := 2a + b; d := b – 2i;

i=3, a=1, b=2

i=3, a=-4, b=7c=-1, d=1

i=3, a=-4, b=7 c=11, d=-11

False

False

w1 = 5

w2 = 2

True

True*

*

Example 1

• Choose a random weight for each join independently.

• All choices of random weights verify first assertion

• Almost all choices contradict second assertion

15

Correctness of Random Interpreter R

• Completeness: If e1=e2, then R ) e1=e2

– assuming non-det conditionals

• Soundness: If e1e2, then R e1 = e2

– error prob. ·

• j : number of joins• d: size of set from which random values are

chosen• k: number of points in the sample

– If j = 10, k = 4, d ¼ 232, then error ·

16

Proof Methodology

Proving correctness was the most complicated part in this work. We used the following methodology.

• Design an appropriate deterministic algorithm (need not be efficient)

• Prove (by induction) that the randomized algorithm simulates each step of the deterministic algorithm with high probability.

17

Random Interpretation: Outline

• Random Interpretation

– Linear Arithmetic (POPL 2003)

Uninterpreted functions (POPL 2004)

– Inter-procedural analysis (POPL 2005)

18

Problem: Global value numbering

a := 5;x := a*b;y := 5*b;z := b*a;

a := 5;x := F(a,b);y := F(5,b);z := F(b,a);

Abstraction

• x=y and x=z• Reasoning about multiplication is undecidable

• only x=y• Reasoning is decidable but tricky in presence of joins

• Axiom: If x1=y1 and x2=y2, then F(x1,x2)=F(y1,y2)

• Goal: Detect expression equivalence when program operators are abstracted using “uninterpreted functions”

• Application: Compiler optimizations, Translation validation

19

Random Interpretation: Outline

• Random Interpretation

– Linear arithmetic (POPL 2003)

– Uninterpreted functions (POPL 2004)

Inter-procedural analysis (POPL 2005)

20

Example 1

a := 0; b := i;

a := i-2; b := 2;

c := b – a; d := i – 2b;

assert (c + d = 0); assert (c = a + i)

c := 2a + b; d := b – 2i;

True False

False

•The second assertion is true in the context i=2.

•Interprocedural Analysis requires computing procedure summaries.

True

*

*

i=2

a=0, b=i

a := 0; b := i;

a := i-2; b := 2;

c := b – a; d := i – 2b;

assert (c+d = 0); assert (c = a+i)

a=8-4i, b=5i-8

a=8-4i, b=5i-8c=21i-40, d=40-21i

c := 2a + b; d := b – 2i;

a=i-2, b=2

a=8-4i, b=5i-8c=8-3i, d=3i-8

a=8-4i, b=5i-8 c=9i-16, d=16-9i

False

False

w1 = 5

w2 = 2

Idea: Keep input variables symbolic

•Do not choose random values for input variables (to later instantiate by any context).

• Resulting program state at the end is a random procedure summary.

a=0, b=2c=2, d=-2

True

True

*

*

22

Experimental measure of error

The % of incorrect relationships decreases with increase in • S = size of set from which random values are chosen.• N = # of random summaries used.

2 95.5 95.5 95.5

3 64.3 3.2 0

4 0.2 0 0

5 0 0 0

6 0 0 0

S

N

The experimental results are better than what is predicted by theory.

210 216 231

23

Simulated Annealing

Problem: Given a program with a pre/post conditions, discover proof of validity/invalidity.

• Proof is in the form of an invariant at each program point that can be locally verified.

• Key Idea:– Initialize invariants at all program points to anything.– Pick a random program point whose invariant is not

locally consistent and update it to make it less inconsistent.

24

Simulated Annealing: Outline

• Simulated Annealing

Inconsistency Measure & Penalty Function

– Algorithm

– Experiments

25

Inconsistency Measure for an Abstract Domain

• Let A be an abstract domain with ) as the partial order and as the concretization function.

• An inconsistency measure IM: A £ A ![0,1] satisfies:– IM(1,2) = 0 iff 1 ) 2

– IM is monotonically decreasing in its first argument– IM is monotonically increasing in its second argument

• IM is a monotonic (increasing) measure of (1) - (2) [set of states that violate 1 ) 2]. The more strictly monotonic IM is, the more smooth it is.

26

Example of a Smooth Inconsistency Measure

Let A be the abstract domain of Boolean formulas (with the usual implication as the partial order).

Let 1 ´ a1 Ç … Ç an in DNF

and 2 ´ b1 Æ … Æ bm in CNF

IM(1, 2) = IM(ai,bj)

where IM(ai,bj) = 0, if ai ) bj

= 1, otherwise

27

Penalty Function

Penalty(I,) is a measure of how much inconsistent is I with respect to the invariants at neighbors of .

Penalty(I,) = IM(Post(), I) + IM(I,Pre()) • Post() is the strongest postcondition of the

invariants at the predecessors of at .• Pre() is the weakest precondition of the invariants

at the successors of at .

28

Example of Penalty Function

• Penalty(I, 2) = IM(Post(2), I) + IM(I, Pre(2))

I

Q

P

R

c

1

• Post(2) = StrongestPost(P,s)

• Pre(2) = (c ) Q) Æ (: c ) R)

s

Since Post() and Pre() may not belong to A, we define:• IM(Post(), I) = Min {IM(I1,I) | I12A, I1 overapproximates Post()}

• IM(I, Pre()) = Min {IM(I,I2) | I22A, I2 underapproximates Pre()}

29

Simulated Annealing: Outline

• Simulated Annealing

– Inconsistency Measure & Penalty Function

Algorithm

– Experiments

30

Algorithm

• Search for proof of validity and invalidity in parallel.

• Same algorithm with different boundary conditions.

• Proof of Validity

– Ientry = Pre

– Iexit = Post

• Proof of Invalidity

– Ientry Æ Pre is satisfiable

– Iexit = : Post

– This assumes that program terminates on all inputs.

31

Algorithm (Continued)

• Initialize invariant Ij at program point j to anything.

• While penalty at some program point is not 0:

– Choose j randomly s.t. Penalty(Ij, j) 0.

– Update Ij s.t. Penalty(Ij,j) is minimized.

• More precisely, Ij is chosen randomly with probability inversely proportional to Penalty(Ij,j).

32

Interesting Aspects of the Algorithm

• Combination of Forward & Backward Analysis

• No distinction between forward & backward information

• Random Choices– Program point to update– Invariant choice

33

Simulated Annealing: Outline

• Simulated Annealing

– Inconsistency Measure & Penalty Function

– Algorithm

Experiments

34

Example 2

y := 50;

y = 100

False

x := x +1;

x := x +1;y := y +1;

x < 50

x <100

True

True False

1

2

3

4

5

6

7

8

x = 0

Prog. Point

Invariant

1 x=0 Æ y=50

2 x·50 )y=50 Æ 50·x )x=y Æ x·100

3 x·50 )y=50 Æ 50·x )x=y Æ x<100

4 x<50 Æ y=50

5 x·50 Æ y=50

6 50·x<100 Æ x=y

7 50<x·100 Æ x=y

8 x·50 )y=50 Æ 50·x )x=y Æ x·100

Proof of Validity

35

Stats: Proof vs Incremental Proof of Validity

• Black: Proof of Validity• Grey: Incremental Proof of Validity• Incremental proof requires fewer updates

36

Stats: Different Sizes of Boolean Formulas

• Grey: 5*3, Black: 4*3, White: 3*2• n*m denotes n conjuncts & m disjuncts• Larger size requires fewer updates

37

Example 3

x := 0; m := 0;

n· 0 Ç 0· m < n

False

m := x;

x := x +1;

*

x < n

True

1

2

3

4

6

5

7

8

true

Prog. Point

Invariant

1 x=0 Æ m=0

2 n· 0 Ç (0·x Æ 0·m<n)

3 n· 0 Ç (0·x<n Æ 0·m<n)

4 n· 0 Ç (0·x<n Æ 0·m<n)

5 n· 0 Ç (0·x<n Æ 0·m<n)

6 n· 0 Ç (0·x<n Æ 0·m<n)

7 n· 0 Ç (0·x<n Æ 0·m<n)

8 n· 0 Ç (0·x·n Æ 0·m<n)

Proof of Validity

38

Stats: Proof of Validity

• Example 2 is “easier” than Example 1.• Easier example requires fewer updates.

39

Example 2: Precondition Modified

Prog. Point

Invariant

0 x¸100

1 x¸100 Æ y=50

2 x¸100 Æ y=50

3 false

4 false

5 false

6 false

7 false

8 false

Proof of Invalidity

y := 50;

y = 100

False

x := x +1;

x := x +1;y := y +1;

x < 50

x <100

True

True False

1

2

3

4

5

6

7

8

true

40

Stats: Proof of Invalidity

Conclusion

Lessons Learned

• Randomization buys efficiency and simplicity.

• Randomization suggests ideas for deterministic algorithms.

• Combining randomized and symbolic techniques is powerful.

Summary

• Random Interpretation:

•Linear Arithmetic: Affine Joins

•Uninterpreted Functions: Random Linear Interpretations

•Interprocedural Analysis: Symbolic Input Variables

• Simulated Annealing:

• Smooth Inconsistency Measure for an abstract domain