automated extraction of inductive invariants to aid model checking mike case des/chess seminar eecs...
Post on 22-Dec-2015
220 views
TRANSCRIPT
Automated Extraction of Inductive Invariants to Aid Model Checking
Mike CaseDES/CHESS Seminar
EECS Department, UC BerkeleyApril 10, 2007
April 10, 2007 Mike Case, DES Seminar 2
Motivation
• Want to build the fastest unbounded model checker– Can get significant speedup from knowledge of a few
local properties (inductive invariants)• Want to find and prove inductive invariants
– Not all invariants are useful for model checking• In previous work, we have a way to quickly
prove many local properties– Can these be limited to properties that are useful for
verification?– Is there an easy way to incorporate this information
into a model checker?
April 10, 2007 Mike Case, DES Seminar 3
ApproximateApproximateReachable StatesReachable States
SequentialSequentialSynthesisSynthesis
Graph TheoryGraph Theory
Better VerificationBetter Verification
InterpolationInterpolation
SequentialSequentialSimulationSimulation
• Known-unreachable states are sequential don’t-cares
• IWLS ’06• Upcoming IBM Internship
• Quickly find and prove small properties
• Approximation to reachability• EECS 290a, IWLS ’06
• Efficiently store candidate properties
• IWLS ’06, WG ’06, WG ’07
• EECS 219C
• Build reachability approximation• Interpolation explores a smaller state space• EECS 219C
• Only help where reachability info needed most• Motivated by work at Calypto• IWLS ’07
• Quickly refute candidate properties
• Under investigationFinding InductiveFinding Inductive
InvariantsInvariants
SynthesisVerification
April 10, 2007 Mike Case, DES Seminar 4
ApproximateApproximateReachable StatesReachable States
SequentialSequentialSynthesisSynthesis
Graph TheoryGraph Theory
Better VerificationBetter Verification
InterpolationInterpolation
SequentialSequentialSimulationSimulation
Finding InductiveFinding InductiveInvariantsInvariantsSynthesis
Verification
Outline
April 10, 2007 Mike Case, DES Seminar 5
Approximating the Reachable States
• Prove local properties hold reachable states• Conjunction gives reachability approximation
I
April 10, 2007 Mike Case, DES Seminar 6
Quickly Proving Local Properties
• Use simple induction to prove the properties– “Sequential Equivalence Checking without State Space
Traversal,” van Eijk, DATE ‘98• Biggest obstacle is an overly large set of candidate
properties– Candidates discovered through random simulation, but
inadequate for large designs– Candidates are Boolean implications
• Lots of expressive power• Can minimize the number of implications under test by applying a
reduction technique on the implication graph– Can also window the candidate set to only prove small subsets
at a time• Can hurt results because sometimes we need multiple implications
to be proved in parallel
April 10, 2007 Mike Case, DES Seminar 7
ApproximateApproximateReachable StatesReachable States
SequentialSequentialSynthesisSynthesis
Graph TheoryGraph Theory
Better VerificationBetter Verification
InterpolationInterpolation
SequentialSequentialSimulationSimulation
Finding InductiveFinding InductiveInvariantsInvariantsSynthesis
Verification
Outline
April 10, 2007 Mike Case, DES Seminar 8
Motivation for interpolation
• Desire to experiment with a state-of-the-art model checker
• Chose interpolation because it is complete and fast – “Interpolation and SAT-Based Model
Checking,” McMillan, CAV ’03– “An Analysis of SAT-based Model Checking
Techniques in an Industrial Environment,” Amla, CHARME ‘05
April 10, 2007 Mike Case, DES Seminar 9
Fixed Point?
Bad state reached?
Property Verified
Property Falsified
frontier := initial states
frontier += image(frontier)
unsat
sat
yes
no
I
Image 1
B
Image 2
Reachability-Based Verification
April 10, 2007 Mike Case, DES Seminar 10
Fixed Point?
Bad state reached?
Property Verified
Property Falsified
frontier := initial states
frontier +=approxImage(frontier)
Initialize approximationparameters
Cex reachedon a BMC from the
initial state?
Tighten approximationparameters
unsat
sat
no
yes
yes
no
I BImage 1
Image 2
Interpolation
Image 2
Image 1
I BS
Reachability:
Interpolation:
April 10, 2007 Mike Case, DES Seminar 11
Problems With Interpolation
• Can explore unreachable states– No control over the approximate image and can
contain unreachable states– If an unreachable state enters start, many other
unreachables will follow– Can lead to an unreachable bad state being explored
• Requires frequent model refinements– Refining the approximation parameters and restarting
is the most expensive operation– Discards all prior work
April 10, 2007 Mike Case, DES Seminar 12
ApproximateApproximateReachable StatesReachable States
SequentialSequentialSynthesisSynthesis
Graph TheoryGraph Theory
Better VerificationBetter Verification
InterpolationInterpolation
SequentialSequentialSimulationSimulation
Finding InductiveFinding InductiveInvariantsInvariantsSynthesis
Verification
Outline
April 10, 2007 Mike Case, DES Seminar 13
Enhanced Interpolation
• Interpolation may explore unreachable states
• Approximate reachable states to help bound number of interpolation iterations
Interpolate
Property Verified
Quickly approximatereachable states
Property Falsified
April 10, 2007 Mike Case, DES Seminar 14
Key Observations From Experimental Results
• Preprocessing imposes a runtime penalty– Can be minimized by resource thresholding– Need to give it sufficient time to prove properties
• Ignoring overhead…– Faster in 92% of designs, can solve 4% that
previously timed out– Slower in 4% of the benchmarks
• Not all invariants help the model checker• Adding a constraint to a SAT solver might slow it down
• We can do better!
April 10, 2007 Mike Case, DES Seminar 15
ApproximateApproximateReachable StatesReachable States
SequentialSequentialSynthesisSynthesis
Graph TheoryGraph Theory
Better VerificationBetter Verification
InterpolationInterpolation
SequentialSequentialSimulationSimulation
Finding InductiveFinding InductiveInvariantsInvariantsSynthesis
Verification
Outline
April 10, 2007 Mike Case, DES Seminar 16
Image
Image
I B
1
2
S
A Better Way to Enhance Interpolation
• Abstraction refinement most expensive• Show either S or B unreachable
– No other constraints matter
• Suppose we had a tool to find invariants to do this
April 10, 2007 Mike Case, DES Seminar 17
Targetted Invariant Tool
• Given a state S that we want to prove unreachable
• Find {P} such that– Implies that S is unreachable– Can be proved with simple induction
April 10, 2007 Mike Case, DES Seminar 18
Can wefind invariants?
Fixed Point?
Bad state reached?
Property Verified
Property Falsified
frontier := initial states
frontier +=approxImage(frontier)
Initialize approximationparameters
Cex reachedon a BMC from the
initial state?
Tighten approximationparameters
unsat
sat
no
yes
yes
no
yes
no
April 10, 2007 Mike Case, DES Seminar 19
Another Application
• We’ve helped interpolation– Short-circuited expensive refinement
• Can we help other applications?– Consider simple induction– Technique used to prove properties– Is often incomplete. Can we fix this?
April 10, 2007 Mike Case, DES Seminar 20
Simple Induction Can Also Be Helped
Is there a way toviolate thebase case?
Remaining candidates hold reachable states
Remove violatedcandidates
Obtain a set ofcandidate properties
Is there a way toviolate the
inductive step?
Remove violatedcandidates
yes
no
yes
no
Base Case
Inductive Step
“Sequential Equivalence Checking without State Space Traversal,” van Eijk, DATE ‘98
Do all properties hold in all initial states?
For all states where the properties hold, do they hold in all next states also?
April 10, 2007 Mike Case, DES Seminar 21
¬pppS XS
Enhancing Simple Induction
• Simple induction is fast, but often fails to prove properties that are true
• If can show that S or XS unreachable, then this structure can’t disrupt the proof
April 10, 2007 Mike Case, DES Seminar 22
Proving Properties by Induction
Remove violatedcandidates
Can wefind invariants?
yes
no
Is there a way toviolate thebase case?
Remaining candidates hold reachable states
Remove violatedcandidates
Obtain a set ofcandidate properties
Is there a way toviolate the
inductive step?
Remove violatedcandidates
yes
no
yes
no … with possibly reachable counterexample states
yes
April 10, 2007 Mike Case, DES Seminar 23
Proving That A State Is Unreachable
• Multiple areas could benefit from a tool that could prove 1 state unreachable– Interpolation– Simple Induction
• Previous work proves a large set of states unreachable– Proves many small properties– Can we limit the properties to target states of
interest?
April 10, 2007 Mike Case, DES Seminar 24
{ P }
S { P }
S
The Proof Graph
• Every property in the set is violated in S
• Proving any such property implies that S is unreachable
• {P} are how we will prove S unreachable
• S is the reason the inductive proof of the properties does not succeed– S is the counterexample in the
inductive step of the proof• Proving S unreachable is a
necessary condition for proving any property in the set
• S is why we can’t prove {P}
(a state)
(a set of properties)
(a set of properties)
(a state)
April 10, 2007 Mike Case, DES Seminar 25
Proof Graph ExampleS0
{ P0 }1{ P0 }2
{ P0 }3
S1 S3S2
{ P1 }
{ P3 }{ P2 }
• Input S0
• Find properties violated in S0
• Prove {P0}
• Cover the new states with properties
• Prove {P3}
• Prove {P03}
April 10, 2007 Mike Case, DES Seminar 26
S0
{ P0 }1{ P0 }2
{ P0 }3
S1
S2 S3
{ P1 }
{ P2 } { P3 }
Proof Graph Notes
• Proof of a property set implies that all parent states are unreachable
• Proof attempt on leaves only
• Leaves can be proved independently
• Select shallowest leaf for next proof
April 10, 2007 Mike Case, DES Seminar 27
Special Case: Cycles
• If a cycle develops…
• Cannot prove either property set independently
• Might be able to prove them together
• Successful proof implies both states unreachable{ P2 } = { P0 }{ P1 }
S0
S1
{ P1 }
{ P0 }
April 10, 2007 Mike Case, DES Seminar 28
Can wefind invariants?
Fixed Point?
Bad state reached?
Property Verified
Property Falsified
frontier := initial states
frontier +=approxImage(frontier)
Initialize approximationparameters
Cex reachedon a BMC from the
initial state?
Tighten approximationparameters
unsat
sat
no
yes
yes
no
yes
no
April 10, 2007 Mike Case, DES Seminar 29
Interpolation Results
• Solves some problems that previously timed out
• Needs work
0.1
1
10
100
1000
10000
0.1 1 10 100 1000 10000
Interpolation Time (sec)
Assi
sted
Tim
e (s
ec)
April 10, 2007 Mike Case, DES Seminar 30
Notes on Performance
• What is “good performance” for a verification tool?– Only meaningful statistics are time, memory, and
whether or not verification completed– Industry very concerned with completion on a large
set of problems– Slower on average is ok, if we complete on most
benchmarks• We verify a few designs that previously timed
out, but not enough– Lack a powerful simulator– Lots of false properties in the candidate set– This is a work in progress
April 10, 2007 Mike Case, DES Seminar 31
ApproximateApproximateReachable StatesReachable States
SequentialSequentialSynthesisSynthesis
Graph TheoryGraph Theory
Better VerificationBetter Verification
InterpolationInterpolation
SequentialSequentialSimulationSimulation
Finding InductiveFinding InductiveInvariantsInvariantsSynthesis
Verification
Outline
April 10, 2007 Mike Case, DES Seminar 32
Simulation Motivation
• Safety property in S420– “Small” design– Timeout for me
• 50% of candidates remain after 10 minute sim– Initial candidate set
is quite poor– Can we refine it in
30 seconds?
0
10000
20000
30000
40000
50000
60000
70000
0 100 200 300 400 500 600
Seconds
Cand
idat
e Pr
oper
ties
April 10, 2007 Mike Case, DES Seminar 33
Future Work
• Continue improving my implementation– Refine candidates with more/better simulation– Try other property domains – maybe
implications are the wrong choice
• Apply my tool in other applications– Could synthesis benefit from knowing that 1
interesting state is unreachable?– Can another state of the art model checker
(UCSB) be fitted to use my tool?
April 10, 2007 Mike Case, DES Seminar 36
ApproximateApproximateReachable StatesReachable States
SequentialSequentialSynthesisSynthesis
Graph TheoryGraph Theory
Better VerificationBetter Verification
InterpolationInterpolation
SequentialSequentialSimulationSimulation
Finding InductiveFinding InductiveInvariantsInvariantsSynthesis
Verification
Outline
April 10, 2007 Mike Case, DES Seminar 37
Sequential Synthesis
• Over-approximate reachable states
• Under-approximate unreachables
• Safe to use as sequential don’t cares
• Sequential don’t cares from 2 minutes of pre-processing give same synthesis results as exact seqdc set
April 10, 2007 Mike Case, DES Seminar 38
ApproximateApproximateReachable StatesReachable States
SequentialSequentialSynthesisSynthesis
Graph TheoryGraph Theory
Better VerificationBetter Verification
InterpolationInterpolation
SequentialSequentialSimulationSimulation
Finding InductiveFinding InductiveInvariantsInvariantsSynthesis
Verification
Outline