![Page 1: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/1.jpg)
1
IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007
PROPERTIES INCOMPLETENESS EVALUATION BY FUNCTIONAL VERIFICATION
![Page 2: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/2.jpg)
2MAIN
CONTRIBUTION
A coverage methodology based on a combination of static and dynamic verification that allows us to reduce the evaluation time with respect to pure formal approaches
![Page 3: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/3.jpg)
3 INTRODUCTION
Simulation-based techniques
Lack of exhaustiveness
Formal verification
Overcome the exhaustiveness problem
Properties are derived from informal design specifications.
Model checking: prove the presence of bugs, but not their absence
![Page 4: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/4.jpg)
4VERIFICATION FLOW BASED
ON MODEL CHECKING
![Page 5: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/5.jpg)
5INTRODUCTION – MODEL CHECKING
To increase the effectiveness of model checking
Vacuity detection: look for properties that hold in a model and can be strengthened
without causing them to fail
Property coverage: address the question of whether enough properties have been
defined
How many properties should be defined to completely check the implementation? Coverage metric!
![Page 6: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/6.jpg)
6
Mutation-based
ACTL, LTL, and CTL
State coverage path coverage transition-
based coverage
Implementation-based
State explosion problem
Cannot precisely reflect the
completeness of properties
INTRODUCTION – PREVIOUS WORK
•How about use mutation coverage jointly with dynamic verification to address the quality of the model checking process?
![Page 7: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/7.jpg)
7 BACKGROUND
Kripke structure K = {S, S0 , R, L}
FSM M = {I, O, S, s0 , R}
Product machine MP = M1 XP M2
Retroactive network
Ιε
![Page 8: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/8.jpg)
8METHODOLOGY
OVERVIEW
![Page 9: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/9.jpg)
9GENERATION OF FAULTY
IMPLEMENTATIONS
The proposed methodology is independent of the adopted fault model Different fault models can provide different estimations of
the property completeness
Functional fault model Bit coverage has been proved to be related to design errors
Bit coverage fault model assumptions
Bit failure: stuck-at 0 or stuck-at 1
Condition failure: stuck-at true or stuck-at false
Single fault: A faulty implementation is generated for each fault
![Page 10: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/10.jpg)
10GENERATION OF FAULTY
IMPLEMENTATIONS(CONT.) Detectable faults
![Page 11: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/11.jpg)
11GENERATION OF FAULTY
IMPLEMENTATIONS(CONT.) A non-optimized algorithm
If fail then f is ε-detectable
Time-consuming and very likely state explosion
![Page 12: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/12.jpg)
12ESTIMATION OF GOLDEN
MODEL INCOMPLETENESS
Ƥ-detectable and Ƥ-det
Property coverage
![Page 13: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/13.jpg)
13
ESTIMATION OF GOLDEN MODEL
INCOMPLETENESS(CONT.)
CP = 1 formal properties are complete w.r.t. a particular fault model
Non-optimized algorithm
![Page 14: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/14.jpg)
14
ESTIMATION OF GOLDEN MODEL
INCOMPLETENESS(CONT.)
Witnesses and counterexamples Tools can provide witnesses and counterexamples for
CTL and LTL properties
Input witness and input counterexample
![Page 15: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/15.jpg)
15 WITNESS COVERAGE
Property coverage can be estimated by using input witnesses
Under some conditions, CP = Cw
![Page 16: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/16.jpg)
16 PROOF OF CP = CW
Consider the safety and liveness properties separately
![Page 17: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/17.jpg)
17 PROOF OF CP = CW (CONT.)
![Page 18: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/18.jpg)
18 PROOF OF CP = CW (CONT.)
![Page 19: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/19.jpg)
19INCREMENTAL PROPERTY
COVERAGE COMPUTATION
![Page 20: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/20.jpg)
20COVERAGE ACCURACY
COMPARISON
Combining static and dynamic verification makes this methodology can deal with real industrial circuits.
The methodology presented in this paper covers faults rather than states.
Can estimate coverage more accurate (compare with previous works)
![Page 21: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/21.jpg)
21EXPERIMENTAL
RESULTS
![Page 22: IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1](https://reader031.vdocuments.site/reader031/viewer/2022012914/5a4d1b767f8b9ab0599b74ec/html5/thumbnails/22.jpg)
22INSPIRE TO IC/CAD
CONTEST
Functional fault model
Estimate coverage by fault instead of properties