case study of end to end formal verification methodology
TRANSCRIPT
Case Study of End-to-End Formal Verification MethodologyJake MaasNirabh RegmiKrishnan PalaniswamiAshish Kulkarni
Methodology for verifying a block completely using formal methods• Alternative for traditional UVM based IP verification
environment• Avoid high-cost development of custom agents, sequence
libs, checkers for UVM test benches with custom interfaces• UVM agents or VIPs are replaced by “proof-kits”
IP functionality is fully described using formal properties• All properties derived from spec and reviewed with
arch/design teams• Includes covers, black-box and white-box assertions, for all
control/data/debug interfacesWork can start after specifications are ready. • First bugs are available almost immediately after the first RTL drop
Registers are checked with assertions generated from an IPXACT XML source
IP is refined as assertions are implemented & proofs are debugged
IP is fully verified when all the assertions and proof kits are proven
What is End-to-End Formal Verification?
• Experiment with end-to-end formal methods to achieve same quality as traditional UVM test benches, especially for certain types of control-path designs
• Reduce verification effort by deploying a formal verification only environment due to resource/schedule pressures on IP verification
• Develop an end-to-end formal verification methodology for custom IP blocks
• Identify areas of improvement with the EDA vendors on metrics used for formal end-to-end IP verification signoff
Motivation Behind FE2E
Benefits• Early feedback on RTL due to formal methodology• Exhaustive verification compared to simulation based bug-
hunting methodologies. Multiple SEED regressions not required to close coverage• Shorter DV Cycle compared to UVM • Portability of assertions to other verification environments
(subsystem/chip level test benches)• Uses proof-kits, which are lower cost (effort) than traditional
UVM VIP
Challenges• Relating formal verification “metrics” to traditional metrics
used for simulation based test benches (e.g. code/functional coverage)• Reviews were more time consuming to ensure the
completeness of the assertions and assumptions to correctly constrain the design• Partitioning of proofs to achieve a reasonable run time• Trade-offs between automatic vs manual generation of
assertions
Impact of FE2E
Interrupt Controller (INTC)
IP Architecture Overview
Maskable Level Interrupts
Interrupt Routing
Interrupt Bit-Vectors
Bus Slave
Maskable Output Interrupts
MMIO
Crossbar (XB)
IP Architecture OverviewU
nit 0
Unit 2
Bank Modules
Uni
t 1
Hierarchical Strategy• When an IP has repetitive components, total runtime can be
reduced by separately verifying the IP’s sub-blocks and connections. Proofs can focus on unique interactions and ignore repeated subblocks, or repeated subblocks can be blackboxed with additional constraints.
Divide and Conquer• Dividing large bodies of assertions into separate jobs,
keeping existing limits, abstractions, et al. the same between jobs, yielded a reduction in total runtime.
Abstractifying • Large families of assertions to be reduced to equivalent, single
assertions with limited addition of Verilog in the glue logic. This reduction in property count significantly improved runtime, despite checking identical functionality.
Traditional Strategies to Improve Formal Verification• Tool specific optimizations• Constraining inputs
Managing Runtime Explosion
INTC XBAR Average-UVM-Simple0
20
40
60
80
100
120
140
70 57 71.5
1725
23.50 0
23
Effort Comparison - PersonDays
Effort - RTL 0.5 Effort - RTL 0.8 Effort - RTL 1.0
INTC XBAR Average-UVM-Simple0.001.002.003.004.005.006.007.008.009.00
8.46
3.40 2.79
Total Bugs found by DV per 1K LOC-DV
INTC XBAR Average-UVM-Simple0
5
10
15
20
25
16
3
19.5
Time to First Design Bug (First Bug - First Verif Checkin) Calendar Days
Item INTC XBAR Average-UVM-SimpleEffort - RTL 0.5 70 57 72Effort - RTL 0.8 17 25 24Effort - RTL 1.0 0 0 23LOC-RTL 21000 4500 3100LOC-DV 1300 5000 5350
Total Bugs found by IP Verif 11 17 17Gate Count (Est) 835734 60186 58502Escaped Bugs from L1 1* 1 1
*Bug found after new logic post RTL 1.0
Shallow Cone of Logic• Do any possible states of the IP require more cycles to reach,
than can be tolerated in FE2E?
High Symmetry• The IP reuses sub-blocks or has repeating patterns
Low Code Complexity• If the IP can reasonably be fully specified in SVA, it is likely a
good candidate for FE2E
Avoid Blocks with Known Large State Spaces• Signal Processing Blocks
Recommendations for Selecting IP for FE2E
Make Formal coverage reporting more like simulation based metrics• Add features to improve compatibility with existing coverage
tracking. e.g. merging reports from hierarchical and partitioned FE2E environments.
Provide tools to help appraise how fit an IP is for FE2E
Automate specific methods to reduce runtime that are normally done manually• Automate Divide and Conquer• Provide suggestions to user to blackbox repeated modules
Warnings for properties which are anticipated to not converge in a reasonable amount of time based on certain heuristics
Provide more proofkits for standard interfaces
Suggestions for Future Improvement
Summary• End-To-End Formal IP Verification is well suited for certain types of symmetric and control path intensive designs.
• FE2E is worth the savings in verification effort & resources, compared to a traditional simulation based approach
• We achieve the same quality of verification with FE2E as with traditional UVM testbenches.
• The main challenge is to determine if an IP can be verified using this methodology and manage the proof run times.