SAT Based Abstraction/Refinement in Model-Checking
Joint work with E. Clarke* A. Gupta* J. Kukula**
*Carnegie Mellon University**Synopsys
Ofer Strichman*
2
Model Checking
I
Add reachable states until reaching a fixed-point
3
Model Checking
Too many states to handle !
I
4
Abstraction
h h hh h
Abstraction Function h : S ! S’
S
S’
5
Abstraction Function
Partition variables into visible(V) and invisible(I) variables.
The abstract model consists of V variables. I variables are made inputs.
The abstraction function maps each state to its projection over V.
6
Abstraction Function
0 0
0 0 0 00 0 0 10 0 1 00 0 1 1
h
x1 x2 x3 x4
x1 x2
Group concrete states with identical visible part to a single abstract state.
7
Building Abstract Model
M’ can be computed efficiently if M is in functional form, e.g. sequential circuits.
Abstract
x1 x2 x3 x4 x1 x2
i1 i2 i1 i2 x3 x4
8
Existential Abstraction
I
I
9
Model Checking Abstract Model
Preservation Theorem
The counterexample may be spurious
Converse does not hold
10
Current trends… (1/3)
PFrontier
Inputs
Invisible
Visible
(R. Kurshan, 80’s)Localization
11
Abstraction-Refinement Loop
Check Counterexample
Refine
Model CheckAbstract
M’, pM, p, hNo Bug
Pass
Fail
BugRealSpurious
h’
12
Why spurious counterexample?
I
I
Deadend states
Failure State
Bad States
f
13
Refinement
Problem: Deadend and Bad States are in the same abstract state.
Solution: Refine abstraction function.
The sets of Deadend and Bad states should be separated into different abstract states.
14
Refinement
h’
h’
h’
h’
h’
Refinement : h’
h’
h’
15
Abstraction
Check Counterexample
Refine
Model CheckAbstract
M’, pM, p, hNo Bug
Pass
Fail
BugRealSpurious
h’
16
Checking the Counterexample
Check Counterexample
Refine
Model CheckAbstract
M’, pM, p, hNo Bug
Pass
Fail
BugRealSpurious
h’
17
Checking the Counterexample
Counterexample : (c1, …,cm) Each ci is an assignment to V.
Simulate the counterexample on the concrete model.
18
Checking the Counterexample
Concrete traces corresponding to the counterexample:
(Initial State)
(Unrolled Transition Relation)
(Restriction of V to Counterexample)
19
Refinement
Check Counterexample
Refine
Model CheckAbstract
M’, pM, p, hNo Bug
Pass
Fail
BugRealSpurious
h’
20
RefinementDeadend States
21
RefinementDeadend States
Bad States
22
Refinement as Separation
0 1 0 1 0 1 0
0 0 1 0 0 1 0
0 1 1 1 0 1 0
d1
b1
b2
I
V
0
1
1
1
0
1
23
Refinement as Separation
0 1 0 1 0 1 0
0 0 1 0 0 1 0
0 1 1 1 0 1 0
d1
b1
b2
0
1
1
I
V
Refinement : Find subset U of I that separates between all pairs of deadend and bad states. Make them visible.
Keep U small !
24
Refinement as Separation
The state separation problemInput: Sets D, BOutput: Minimal U I s.t.: d D, b B, u U. d(u) b(u)
The refinement h’ is obtained by adding U to V.
25
Two separation methods
ILP-based separation Minimal separating set. Computationally expensive.
Decision Tree Learning based separation. Not optimal. Polynomial.
26
Separation with ILP (Example)
27
Separation with ILP
One constraint per pair of states. vi = 1 iff vi is in the separating set.
28
Decision Tree learning (Example)
Separating Set : {v1,v2,v4}
D B
B D BD
10 0 1
b1d2d1b2
v1
v4v2
0 1{d1,b2} {d2,b1}
DB
Classification:
29
Decision Tree Learning
Input : Set of examples Each example is an assignment of
values to the attributes. Each example has a classification.
a1
a5a2
c0 c2c1c1
0
0
0 11
1
Output : Decision Tree Each internal node is a test
on an attribute. Each leaf corresponds to a
classification.
30
Separation using Decision Tree Learning
Attributes : Invisible variables I Classifications : ‘D’ and ‘B’ Example Set : Deadend Bad
Separating Set : The variables on the nodes of the decision tree.
31
Refinement as Learning
For systems of realistic size Not possible to generate D and B. Expensive to separate D and B.
Solution: Sample D and B Infer separating variables from the samples.
The method is still complete: counterexample will eventually be eliminated.
32
Efficient Sampling
D Bd b
Let (D,B) be the smallest separating set of D and B.
Q: Can we find it without deriving D and B ?
A: Search for smallest d,b such that (d,b) = (D,B)
33
Efficient Sampling
Direct search towards samples that contain more information.
How? Find samples not separated by the current separating set (Sep).
34
Efficient Sampling
Recall: D characterizes the deadend states B characterizes the bad states D B is unsatisfiable
Samples that agree on the sep variables:
Rename all vi B to
vi’
35
Efficient Sampling
Sep = {}d,b = {}
Run SAT solveron (Sep)
STOPunsat
Compute Sep:= (d,b)
Add samples to d and b
sat
Sep is the minimal separating set of D and B
36
The Tool
NuSMV CadenceSMV
MC
Chaff
SAT
LpSolve
Dec Tree
Sep
37
ResultsProperty 1
38
ResultsProperty 2
Efficient Sampling together with Decision Tree
Learning performs best.
Machine Learning techniques are useful in
computing good refinements.
39
Current trends… (1/3)
PFrontier
Inputs
Invisible
Visible
(Barner, Geist, Gringauze, CAV’02)
Check counterexample incrementally (‘layering’).Find small set of variables in Sf for which it is impossible to find an assignment consistent with the counterexample.
(Originally: R. Kurshan, 80’s)Localization
40
Current trends… (2/3)
Generate all counterexamples.Prioritize variables according to their consistency in the counterexamples.
X1 x2 x3 x4
(Glusman et al., 2002) Intel’s refinement heuristic
41
Current trends… (3/3)
Simulate counterexample on concrete model
with SAT
If the instance is unsatisfiable, analyze conflict
Make visible one of the variables in the clauses
that lead to the conflict
(Chauhan, Clarke, Kukula, Sapra, Veith, Wang, FMCAD 2002) Abstraction/refinement with conflict analysis
42
Current trends… (3/3)
Remove clauses gradually, until instance becomes satisfiable.Choose invisible variables from the removed set.
43
Current trends (3/3)
(Chauhan, Clarke, Kukula, Sapra, Veith, Wang, FMCAD 2002) Abstraction/refinement with conflict analysis
44
Future Work
Currently: Sometimes we find too many equally ‘good’ refinements to choose from.We need more criteria for a good refinement (not just # latches).
Number of gates, number of clauses Distance from property Fan-in degree
45
Future work
Currently we restart with a refined transition relation
T T T
T ‘ T ‘ T ‘
46
Future work
T ’
T ’
A different approach: restart from the previous state.
TT T
An abstraction/refinement backtrack algorithm What intermediate BDD’s should we save ? How can BDDs be altered rather than recomputed ?
47
The End
48
Generating Samples
Execute Sat Solver
satisfiable sample
Add clause negating assignment to I in
failure state
STOP
“enough samples”/
unsatisfiable
Initialize SAT solver with ( or )