a game changer for vhdl verification: - advanced vhdl ... · dma intr ctrl sequential testing find...

1

110001011010011110100111011011010011110011

www.bitvis.no Your partner for SW and FPGA

A game changer for VHDL verification:

- Advanced VHDL Verification - Made simple - For anyone

NMI FPGA Network: Design tools & methodologies

October 11, 2016

www.bitvis.no A game changer for VHDL Verification 2

The 2016 Wilson Research Group Functional Verification Study (1)

2

www.bitvis.no

The 2016 Wilson Research Group Functional Verification Study (2)

A game changer for VHDL Verification 3

www.bitvis.no

Quality and Efficiency enablers


Structure & Architecture

Overview, Readability, Simplicity

Modifiability, Maintainability, Extendibility

Debuggability

Reusability

3

www.bitvis.no

UVVM Universal VHDL Verification Methodology


Free and Open source code

• Downloadable from GitHub or Bitvis.no

Two levels of functionality:

• Utility Library (and BFMs) : May be used stand-alone

• VVC Framework (and VVCs) : Uses Utility Library

Structure & Architecture

Overview, Readability, Simplicity

Modifiability, Maintainability, Extendibility

Debuggability

Reusability

UVVM: Targets exactly these challenges

www.bitvis.no

UVVM Utility Library

Previously 'Bitvis Utility Library'. Free & open source since 2013

A basic VHDL testbench infrastructure

• Logging and verbosity control

• Alert handling incl. optional positive acknowledge

• Extended string handling

• Simplified randomisation

• check_value(), check_value_in_range()

• check_stable(), await_stable()

• await_change(), await_value()

• report_alert_counters()

• BFM support and examples

• etc....

Very user friendly

Extremely low user threshold

Advanced funct. when needed

Up and running in less than an hour

Well documented


4





5


UVVM Utility Library - BFMs

www.bitvis.no

BFMs to handle interfaces

Handle transactions at a higher level

E.g. Read, Write, Send packet, Config, etc

BFM: - A model or model set (or API) for handling transactions on a physical interface.

- Models the environment - e.g. a bus master


6

www.bitvis.no

BFM: - A model or model set (or API) for handling transactions on a physical interface.

- Models the environment - e.g. a bus master




Example: BFM procedure for a CPU access to a module's register E.g. write 0xF0 into a register at address 0x22

Replaced by:

write(x”22”, x”F0”);

cs <= ’1’;

we <= ’1’;

addr <= x”22”;

data <= x”F0”;

wait until rising_edge(clk);

wait until falling_edge(clk);

cs <= ’0’;

we <= ’0’;

or:

sbi_write(x”22”, x”F0”);

SBI: Simple Bus Interface - Single cycle - Optional ready - Dead simple


www.bitvis.no




More understandable for anyone

Simpler code & Improved overview

Uniform style, method, sequence, result

Easy to add several very useful features

Replaced by:

write(x”22”, x”F0”);

cs <= ’1’;

we <= ’1’;

addr <= x”22”;

data <= x”F0”;

wait until rising_edge(clk);

wait until falling_edge(clk);

cs <= ’0’;

we <= ’0’;

or:

sbi_write(x”22”, x”F0”);


7

www.bitvis.no

What about more complex DUTs?

More interfaces

More simultaneous activity on interfaces

Value related corner cases

Cycle related corner cases

May need structured Constrained Random?

May need structured Functional Coverage?

Need some kind of Scoreboarding

Reusable IP/Libraries will save time


www.bitvis.no

What about more complex DUTs?

More interfaces

More simultaneous activity on interfaces


Cycle related corner cases

May need structured Constrained Random?

May need structured Functional Coverage?

Need some kind of Scoreboarding

Reusable IP/Libraries will save time

Must have a good architecture and interface control


8

www.bitvis.no

Chip level functional scenario


PIF

SPI

P3

ETH ETH

P1

P2

uart

DMA

Intr ctrl

Sequential testing

Find some bugs?

Then behaves fine

Value related corners?

Parallel operation

Multi-end

cycle related corners

Bugs are hard to find

www.bitvis.no

Typical testbench approaches


How do designers handle Cycle related corner cases

Test Controller

Clock Generator

Testcase Sequencer Adding threads CCL

Ad hoc "structure"

Lab ?

?

PIF

SPI

P3

ETH ETH

P1

P2

uart

DMA

Intr ctrl

- Fixed structure

Testcase Sequencer

???

- Overview??? - Reuse??? - Bad synchronisation

Good architecture, overview, reuse & sync.

9

www.bitvis.no

Wouldn't it be nice if we could ...

handle any number of interfaces in a structured manner?

reuse major TB elements between module TBs?

reuse major module TB elements in the FPGA TB?

read the test sequencer almost as simple pseudo code?

recognise the verification spec. in the test sequencer?

understand the sequence of event

- just from looking at the test sequencer

Is that feasible at all?

Wishful thinking?


www.bitvis.no

BFM procedures only allow sequential operation (in test sequencer)

Parallelism requires BFMs in separate processes or entities

• But - processes may only have one single thread of execution

Entities / Components

Allow multiple active threads (processes or more entities)

May add dedicated processes for any extended functionality

Enables encapsulation

Makes re-use far more efficient

Using Components is the best solution - exactly as for Design

'VHDL Verification Components' : VVC

Parallelism & Encapsulation


10

www.bitvis.no

VHDL Verification Component - VVC


VHDL entity / component

Allows multiple thread(s)

• May include multiple processes or sub-components

Autonomous operation

Controllable via commands

Encapsulated complete verification functionality

More structured

Better overview

Easy to extend

Easy to reuse

A standard interface towards a bus structure / sequencer?

All issues exactly as for Design !!!

www.bitvis.no

VVC – In its simplest form


DUT (UART VC)

“Ext.” I/O Other

Ports

Clocks

Bus

interface

Clock

Generation

Testcase

Sequencer

SBI_

VVC

Going from BFM to VVC

Using Bus access (SBI) as example - E.g. write to a register in DUT

Sequencer command using BFM:

sbi_write(C_ADDR_TX, x"2A");

Minimum VVC 1. Interpret command from sequencer in zero time

2. Execute respective BFM towards DUT

Sequencer command using VVC: sbi_write(SBI_VVCT, C_ADDR_TX, x"2A");

Results in above BFM being executed from VVC towards DUT

Very simple VVC – already allows simultaneous execution of BFMs on different interfaces

11

www.bitvis.no

TB implementation & understanding - Three main areas


Clock

Generation

Testcase

Sequencer

SBI_VVC

UART_TX_VVC

UART (DUT)

RX Other

Ports

Clocks

Bus

interface

TX UART_RX_VVC

1: The complete Testbench with Test Harness

2: The Verification Components

3: The Central Test Sequencer

www.bitvis.no

1:The UVVM testbench


UVVM is LEGO-like Testbench

Test harness

Build test harness

• Instantiate DUT and VVCs

• Connect VVCs to DUT

Build TB with test sequencer

• Instantiate test harness

• Include VVC methods pkg Connections included

• No additional connections

• May have VVCs anywhere

DUT

VVC A

VVC B

VVC C

Test seq.

VVC A library

VVC B library

VVC C library

VVC D

12

www.bitvis.no

2: VVC: VHDL Verification

Component


SBI_VVC

Testcase Sequencer SBI_VVC

UART (DUT)

RX Other Ports

Clocks

Bus interface TX

Interpreter

- Is command for me?

- Is it to be queued?

- If not: Case on what to do

Executor

- Fetch from queue

- Case on what to do

- Call relevant BFM(s) & Execute transaction

Command Queue

Same main architecture in every VVC

• >90% same code in Interpreters

• Same command queue

• 90% same code in Executors - apart from BFM calls

VVC Generation

UART BFM to UART_VVC:

in less than 30 min

*_VVC

Clock

Generation

Testcase

Sequencer

SBI_VVC

UART_TX_VVC

UART (DUT)

R

X

Other

Ports

Clocks

Bus

interfac

e

T

X

UART_RX_VVC

UART_RX_VVC

Baudrate Checker

(1:Testbench : Easy to implement & understand by anyone)

Now - what about these VVCs?


2: VVC: VHDL Verification

Component

*_VVC

(1:Testbench : Easy to implement & understand by anyone)

Now - what about these VVCs?

Interpreter

- Is command for me?

- Is it to be queued?

- If not: Case on what to do

Executor

- Fetch from queue

- Case on what to do

- Call relevant BFM(s) & Execute transaction

Command Queue

Bit-rate checker

Frame-rate checker

Gap checker

Queue

Response-Executor

Clock

Generation

Testcase

Sequencer

SBI_VVC

UART_TX_VVC

UART (DUT)

R

X

Other

Ports

Clocks

Bus

interfac

e

T

X

UART_RX_VVC

UART_RX_VVC

Baudrate Checker

13

www.bitvis.no

(Based on very structured TB and VVCs)

The sequencer is the most important part of the Testbench

Most man-hours will be (or should be) spent here

MUST be easy to understand, modify, maintain, ....

3: The test sequencer


Clock

Generation

Testcase

Sequencer

SBI_VVC

UART_TX_VVC

UART (DUT)

RX Other

Ports

Clocks

Bus

interface

TX UART_RX_VVC

UART_RX_VVC

Baudrate Checker

Next: For each verification plan step - Check UVVM code in test sequencer

www.bitvis.no

Verification plan - In plain English (1)


1. First Apply data to UART RX, then Wait for RX interrupt

then Read reg RX_DATA. (to show standard serial sequencer)

Pure sequential - Using Sequential BFMs only

uart_transmit(x"A1", "First byte on UART RX");

await_value(rx_empty, '0', 0, 12*bit_period, ERROR, message);

sbi_check(C_ADDR_RX_DATA, x"A1", "UART RX first byte");

Clock

Generation

Testcase

Sequencer

SBI_VVC

UART_TX_VVC

UART (DUT)

R

X

Other

Ports

Clocks

Bus

interfac

e

T

X

UART_RX_VVC

UART_RX_VVC

Baudrate Checker

Pure sequential - using UVVM VVC Framework + VVCs

uart_transmit(UART_VVCT,1, x"A1", "First byte on UART RX");

await_value(rx_empty, '0', 0, 12*bit_period, ERROR, message);

sbi_check(SBI_VVCT,1, C_ADDR_RX_DATA, x"A1", "UART RX first byte");

[ await_completion(SBI_VVCT,1, C_TIMEOUT_SBI_CHECK); ]

14

www.bitvis.no

2. Apply data to UART RX and read reg RX_DATA at the same time



Parallel operation - using UVVM VVC Framework + VVCs



Clock

Generation

Testcase

Sequencer

SBI_VVC

UART_TX_VVC

UART (DUT)

R

X

Other

Ports

Clocks

Bus

interfac

e

T

X

UART_RX_VVC

UART_RX_VVC

Baudrate Checker

www.bitvis.no

3. Read reg RX_DATA when this has just been up-loaded





insert_delay(SBI_VVCT,1, C_FRAME_TIME + 2 * C_CLK_PERIOD);


N

Clock

Generation

Testcase

Sequencer

SBI_VVC

UART_TX_VVC

UART (DUT)

R

X

Other

Ports

Clocks

Bus

interfac

e

T

X

UART_RX_VVC

UART_RX_VVC

Baudrate Checker

15

www.bitvis.no



4. Extend the range in 3. to include all possible cycle corner cases

Similarly for Write on reg TX_DATA wrt. transmitting on TX

Similarly to check TX vs RX independence


for i in -C_CYCLES_BEFORE to C_CYCLES_AFTER loop

uart_transmit(UART_VVCT,1, my_data(i), "New byte on UART RX");

insert_delay(SBI_VVCT,1, C_FRAME_TIME + i * C_CLK_PERIOD);

sbi_check(SBI_VVCT,1, C_ADDR_RX_DATA, my_data(i-1), message);

await_completion(UART_VVCT,1, "Finish before next transmit");

end loop;

Make your own super-procedure

uart_transmit_sbi_check(

C_START_VALUE, -C_CYCLES_BEFORE, C_CYCLES_AFTER);

Even better: Make the VVCs handle the sequences

sbi_write(SBI_VVCT,1, C_ADDR_TX_DATA, RANDOM_TO_BUFFER, 1, 256);

uart_expect(UART_VVCT, 1, RX, FROM_BUFFER, 1, 256);

i

i

Clock

Generation

Testcase

Sequencer

SBI_VVC

UART_TX_VVC

UART (DUT)

R

X

Other

Ports

Clocks

Bus

interfac

e

T

X

UART_RX_VVC

UART_RX_VVC

Baudrate Checker

www.bitvis.no



• Direct testing

• Constrained random

• Use std VHDL, Utility Library or OSVVM to generate values

• Apply from test sequencer as previously shown :

• 1. As individually initiated single accesses - as shown earlier

• 2. As a chunk of single accesses distributed "simultaneously"

• 3. As a single command to initiate multiple accesses from VVC

• Functional Coverage

• OSVVM is currently the best system for functional coverage

• Control from test sequencer as follows:

uart_transmit(UART_VVCT,1,TX, RANDOM, C_BUFFER_2, 256, msg);

uart_transmit(UART_VVCT,1,TX, RANDOM, C_BUFFER_1, DATA_COVERAGE, msg);

Clock

Generation

Testcase

Sequencer

SBI_VVC

UART_TX_VVC

UART (DUT)

R

X

Other

Ports

Clocks

Bus

interfac

e

T

X

UART_RX_VVC

UART_RX_VVC

Baudrate Checker

16

www.bitvis.no

Value related corner cases - Full solution


UVVM vs OSVVM (Randomisation and Coverage)

UVVM & OSVVM are Complementary

Unified VHDL Verif. Meth.

Plug-in: OSVVM Random+Coverage - (with UVVM Utility Library?) - inside UVVM BFMs - inside UVVM VVCs - inside UVVM Test Sequencers


Logging, Alerts, Check value,

Stability, Awaits

UVVM VVC Framework TB Architecture, VVC architecture, Commands, Debug

OSVVM Randomisation

and Functional Coverage

BFMs & VVCs AXI4-lite/stream, Avalon, UART, I2C, SPI, SBI, GPIO, ...

www.bitvis.no

Debugging (1) - Verbosity ctrl


• Important for both the DUT, TB and VVCs

• UVVM built-in messaging simplifies TB/VVC debugging

Showing only the distribution of commands from the test sequencer to the VVCs

2045ns TB seq.(uvvm) ->uart_transmit(UART_VVC,1,TX, x"AA"): . [15]

2045ns TB seq.(uvvm) ->await_completion(UART_VVC,1,TX, 2080 ns): . [16]

4005ns TB seq.(uvvm) ->sbi_check(SBI_VVC,1, x"00", x"AA"): RX_DATA. [17]

4005ns TB seq.(uvvm) ->await_completion(SBI_VVC,1, 2080 ns): . [18]

Showing only the completed commands (executed BFMs)

3805ns UART_VVC,1,TX uart transmit(x"AA") completed. [15]

4017ns SBI_VVC,1 sbi_check(x"00", x"AA")=> OK, read data = x"AA". RX_DATA [17]

Time stamp

Scope

Message

Showing distribution and execution - together


2045ns TB seq.(uvvm) ->await_completion(UART_VVC,1,TX, 2080 ns): . [16]


4005ns TB seq.(uvvm) ->sbi_check(SBI_VVC,1, x"00", x"AA"): RX_DATA. [17]

4005ns TB seq.(uvvm) ->await_completion(SBI_VVC,1, 2080 ns): . [18]

4017ns SBI_VVC,1 sbi_check(x"00", x"AA")=> OK, read data = x"AA". RX_DATA [17]

Clock

Generation

Testcase

Sequencer

SBI_VVC

UART_TX_VVC

UART (DUT)

R

X

Other

Ports

Clocks

Bus

interfac

e

T

X

UART_RX_VVC

UART_RX_VVC

Baudrate Checker

17


Debugging (2) - Follow command

Showing all log messages, but for uart_transmit() only


2045ns UART_VVC,1,TX uart_transmit(UART_VVC,1,TX, x"AA"). Command received [15]

2045ns TB seq.(uvvm) ACK received [15]

2045ns UART_VVC,1,TX uart_transmit(UART_VVC,1,TX, x"AA") Will be executed [15]


3805ns UART_VVC,1,TX ..Executor: Waiting for command

3805ns UART_VVC,1,TX ..Interpreter: Waiting for command

Debugging the Testbench or VVC (Only showing uart_transmit)

Normal view: Showing executed commands only


4017ns SBI_VVC,1 sbi_check(x"00", x"AA")=> OK, read data = x"AA" RX_DATA [17]

Command index

Clock

Generation

Testcase

Sequencer

SBI_VVC

UART_TX_VVC

UART (DUT)

R

X

Other

Ports

Clocks

Bus

interfac

e

T

X

UART_RX_VVC

UART_RX_VVC

Baudrate Checker

www.bitvis.no

Scoreboards


Check that the result is as expected

Simplest Scoreboard is just a FIFO

Complex Scoreboards may include

• Out of order transactions

• One to one, one to many, many to one, many to many

May include modelling -including generation of the expected result

18

www.bitvis.no

Scoreboards in UVVM


Level 0: Entry Level:

• Centr. seq. direct words

• Centr. seq. direct block

Level 1

• Centr. seq. indirect block

Level 2

• Centr. seq. requests local CR

• Centr. seq. requests local Coverage

Level 3 (roadmap)

• Introducing independent Monitors

• Introducing central scoreboard and modelling

Clock

Generation

Testcase

Sequencer

SBI_VVC

UART_TX_VVC

UART (DUT)

R

X

Other

Ports

Clocks

Bus

interfac

e

T

X

UART_RX_VVC

UART_RX_VVC

Baudrate Checker

Word or Block of words from Sequencer to VVC-A + VVC-B

Reference to block of words from Sequencer to VVC-A + VVC-B

CR and Functional Coverage request from Sequencer

to VVC-A + VVC-B

Monitors and scoreboards outside VVCs

www.bitvis.no

UVVM is just a step-by-step evolution on VHDL (2008)

UVVM can run on Modelsim, Questa, Active HDL, Riviera Pro

UVM is far more comprehensive and complex than UVVM

• But UVVM is sufficient for almost all testbenches

UVVM user threshold is a fraction of the UVM threshold

UVVM allows a gentle introduction to modern verification

UVVM allows the use of low cost tools

UVVM vs UVM


Using VHDL for FPGA verification is:

• more cost efficient for almost all VHDL designers

• more time efficient for almost all VHDL designers

19

www.bitvis.no

Wouldn't it be nice if we could ...

handle any number of interfaces in a structured manner?

reuse major TB elements between module TBs?

reuse major module TB elements in the FPGA TB?

read the test sequencer almost as simple pseudo code?

recognise the verification spec. in the test sequencer?

understand the sequence of event

- just from looking at the test sequencer

Wishful thinking - Recap


UVVM

Already available Open Source BFMs & VVCs:

AXI4-lite, AXI4-stream, Avalon MM, SBI, SPI, I2C, UART, GPIO

www.bitvis.no

UVVM is gaining momentum


UVVM VVC Framework - Released February 2016

Great feedback on LinkedIn

Really good feedback from users (industrial and academia)

Presented at various conferences - with huge interest: - 'FPGA-Kongress' in Munich, July - 'FPGAworld' in Stockholm, September Next: 'DVCon-Europe in Munich, October

Next course

- Dec 6-8, München: Accelerating FPGA VHDL Verification (In Cooperation with Trias Mikroelektronik Gmbh)

Will soon set up more courses for 2017.

20

www.bitvis.no

Conclusions


VHDL is the best language for Verification for almost all VHDL designers.

Understanding, Maintainability & Reuse are critical

Testbench architecture is key

No standard exists, but is really needed

UVVM handles the critical issues

UVVM yields a very good architecture at all levels

UVVM significantly improves Efficiency and Quality

UVVM allows the best reuse possible for VHDL verification

www.bitvis.no

Your partner for Embedded software and FPGA

Quality in every bit

A game changer for VHDL verification:

- Advanced VHDL Verification - Made simple - For anyone

Thank you

a game changer for vhdl verification: - advanced vhdl ... · dma intr ctrl sequential testing find...

Documents