Jaeyong Chung

System-on-Chips (SoC) Laboratory

Incheon National University

Digital Integrated Circuits

Term Project v0.95

Term Project

Dealing with real-world designs/problems

Analyze and learn well-designed systems and codes

Point to what you should learn even after this course

Make what you can show off (for your interview)

Do not write from scratch

Modify existing modules and integrate

Taking advantage of open-source hardware

Dealing with more real-world designs/problems

Mix & Match

Chung EPC6055 2

Term Projects

A team consists of up to 3 people

Choose one of the given topics

Team’s project should be standalone

If done, move on Mix & Match phase

Mix & Match phase

More than two teams work together based on their individual

projects, creating a more complex system

A large, extra credit will be awarded

Grading

Projects will be ranked within the same topic

Difference of difficulty between topics will be adjusted

Chung EPC6055 3

Deliverables

Deliverables

Proposal Presentation Slides

Source codes

Final reports

Chung EPC6055 4

Topics

(difficult) RISC-V core & System bus (new in 2017)

(difficult) Matrix Multiplication (new in 2017)

(moderate) CIS Controller (FPGA needed, up to 10

teams)

(moderate) VGA Controller (FPGA needed)

(moderate) UART (FPGA needed)

(easiest) LAVA2

Chung EPC6055 5

CIS Controller

OmniVision OV7670 Camera Module

http://www.eleparts.co.kr/EPX37F6M

Reading materials

http://embeddedprogrammer.blogspot.kr/2012/07/hacking-

ov7670-camera-module-sccb-cheat.html

Reference code

https://github.com/romovs/xula-lib-verilog

Camera/CameraSetup.v

Camera/OV7670Init.v

Camera/RGB565Receive.v

Camera/SCCBCtrl.v

Chung EPC6055 6

CIS Controller

Chung EPC6055 7

Examples

FPGA

CISctrl

fndctrl

Initialize via SCCB

Image Data

OV7670 Pin Numbers

Chung EPC6055 8

##### Camera #####NET "ca_vdd" LOC = "C26" | IOSTANDARD=LVCMOS33 ; NET "ca_sioc" LOC = "E23" | IOSTANDARD=LVCMOS33; NET "ca_vsync" LOC = "D26" | IOSTANDARD=LVCMOS33; NET "ca_pclk" LOC = "E26" | IOSTANDARD=LVCMOS33 | SLEW=SLOW | CLOCK_DEDICATED_ROUTE = FALSE;NET "ca_data<7>" LOC = "G21" | IOSTANDARD=LVCMOS33 ; NET "ca_data<5>" LOC = "F24" | IOSTANDARD=LVCMOS33 ; NET "ca_data<3>" LOC = "G23" | IOSTANDARD=LVCMOS33 ; NET "ca_data<1>" LOC = "G25" | IOSTANDARD=LVCMOS33 ; #NET "ca_pwdn" LOC = “H20" | IOSTANDARD=LVCMOS33 ; ##### put 1'b0 to pwdn

NET "ca_gnd" LOC = “F22" | IOSTANDARD=LVCMOS33 ; NET "ca_siod" LOC = "E24" | IOSTANDARD=LVCMOS33 | PULLUP ; NET "ca_href" LOC = "E25" | IOSTANDARD=LVCMOS33; NET "ca_xclk" LOC = "G20" | IOSTANDARD=LVCMOS33 ; NET "ca_data<6>" LOC = "F23" | IOSTANDARD=LVCMOS33 ; NET "ca_data<4>" LOC = "G22" | IOSTANDARD=LVCMOS33 ; NET "ca_data<2>" LOC = "F26" | IOSTANDARD=LVCMOS33 ; NET "ca_data<0>" LOC = "G26" | IOSTANDARD=LVCMOS33 ; #NET "ca_rst" LOC = “H21" | IOSTANDARD=LVCMOS33 ;

#### The OV7670 which have 18 pins need to set "ca_pwdn" and "ca_rst"

To set OV7670 on Ext. port of FPGA

Chung EPC6055 9

FND Pin Map

Chung EPC6055 10

RISC-V Core

RISC-V

Recommended for who took computer architecture

https://riscv.org/

Reading Materials:

https://github.com/sifive/freedom

Should be interested in computer architecture

Should be familiar with Linux

Chung EPC6055 11

RISC-V Core

Things you can do

1) Generate your CPU cores with the tools provided

2) Write a simple test program in assembly or C

3) Run RTL Simulation

4) Try to synthesize verilog using vivado and run a program

on FPGA

Chung EPC6055 12

Matrix Multiplication

Chung EPC6055 13

=

Matrix Multiplication is a fundamental operation in

artificial intelligence (AI)

You are to design an efficient MM hardware unit

Broadcast-based

Chung EPC6055 14

Row vector 1

Row vector 2

Row vector n

col vector 1 col vector 2 col vector n

Inner Product

Systolic Array

Chung EPC6055 15

Row vector 1

Row vector 2

Row vector n

col vector 1

col vector 2

col vector n

Google’s TPU

Chung EPC6055 16

References

Chung EPC6055 17

Design and Analysis of a Hardware CNN Accelerator

http://cs231n.stanford.edu/reports/2017/pdfs/116.pdf

Improving the Performance of OpenCL-based FPGA

Accelerator for Convolutional Neural Network

https://wicil.ece.wisc.edu/wp-

content/uploads/2017/02/JZhang_FPGA17_CNN.pdf

VGA Controller

VGA controller

Reading material and source codes:

FPGA PROTOTYPING BY VERILOG EXAMPLES, Pong P.

Chu (You can find PDF by googling)

Chapter 13, VGA Controller I

Chung EPC6055 18

VGA Controller

Examples

Chung EPC6055 19

VGA Pin Numbers

Chung EPC6055 20

##### VGA ######NET "B<0>" LOC = "G9";NET "B<1>" LOC = "F9";NET "B<2>" LOC = "E9";NET "B<3>" LOC = "D9";NET "B<4>" LOC = "F5";NET "B<5>" LOC = "F6";NET "B<6>" LOC = "E3";NET "B<7>" LOC = "E4";NET "G<0>" LOC = "D2";NET "G<1>" LOC = "G6";NET "G<2>" LOC = "G7";NET "G<3>" LOC = "E1";NET "G<4>" LOC = "E2";NET "G<5>" LOC = "F3";NET "G<6>" LOC = "F4";NET "G<7>" LOC = "G4";NET "R<0>" LOC = "G5";NET "R<1>" LOC = "F1";NET "R<2>" LOC = "F2";NET "R<3>" LOC = "H6";NET "R<4>" LOC = "G1";NET "R<5>" LOC = "G2";NET "R<6>" LOC = "J6";NET "R<7>" LOC = "H3";NET "VGA_PCLK" LOC = "R3";NET "h_sync" LOC = "M6";NET "v_sync" LOC = "M8";NET "VGA_PWM" LOC ="H2"; ### put 1'b1 (like enable)NET "VGA_DEN" LOC ="K7"; #### put 1'b1 (like enable)

UART

Universal Asynchronous Receiver Transmitter

(UART)

Common/simple way to communicate with CPUs

Reading material and source codes:

FPGA PROTOTYPING BY VERILOG EXAMPLES, Pong P.

Chu (You can find PDF by googling)

Chapter 8, UART

Terminal Programs

http://realterm.sourceforge.net

Chung EPC6055 21

UART

Examples

Chung EPC6055 22

UART

CPU

VGA ctrl

PS2ctrl

BUS

FPGA

fndctrl

UARTkeypad

ctrl

rs232 cable

Terminal program

UART Pin Numbers

Chung EPC6055 23

##### UART #####NET "UART_RX" LOC = "P2";NET "UART_TX" LOC = "P3";

ICE2

ICE2

Recommended for who took digital signal processing

Not required to run it on FPGA

Enhance ICE as you want

Chung EPC6055 24

ICE2

Examples Detect a specific pattern and draw a box enclosing it as in face

defectors (Study 2D Matched Filter)

http://www.engr.colostate.edu/ECE513/SP09/lectures/lectures19_20.pd

f

Detect objects with a specific color and draw a box enclosing it as

in face detectors

http://www.mathworks.com/matlabcentral/fileexchange/28512-simple-

color-detection-by-hue

https://www.kickstarter.com/projects/254449872/pixy-cmucam5-a-fast-

easy-to-use-vision-sensor

Chung EPC6055 25

LAVA2

LAVA2

Recommended for who took computer architecture

Not required to run it on FPGA

Enhance LAVA to be more like a real-cpu

Add new instructions such as branch

Analyze Educational 16-bit MIPS Processor

Reference Code:

http://opencores.org/project,mips_16

Chung EPC6055 26

Mix & match examples

ARM core + UART

LAVA2 + UART

Camera Controller + UART

VGA Controller + UART

CIS Controller + VGA Controller

CIS Controller + VGA Controller + ICE2

Chung EPC6055 27

Dual-Port Memory on FPGA

Chung EPC6055 28

To generate memory(block memory)

right click menu

Dual-Port Memory on FPGA

Chung EPC6055 29

*You should remember your Location path.

Dual-Port Memory on FPGA

Chung EPC6055 30

Dual-Port Memory on FPGA

Chung EPC6055 31

*For this example, depth is 256. -> address will be 8bits

Dual-Port Memory on FPGA

Chung EPC6055 32

* Page 2,3 and 4 are not important. Our memory is simple.* If you don’t need to initialize the memory, click the “Generate” and pass the next two slides.* If not, refer to next slide.

Dual-Port Memory on FPGA

Chung EPC6055 33

To initialize memory

* Make a file to initialize memory. File extension is “coe” * Refer the example file as “pp_test_init.coe”* Check the “Load Init File” and select made file.

Dual-Port Memory on FPGA

Chung EPC6055 34

• You can check the initialized memory.

Dual-Port Memory on FPGA

Chung EPC6055 35

Dual-Port Memory on FPGA

Chung EPC6055 36

Dual-Port Memory on FPGA

Chung EPC6055 37

• Open the “your_memory.v”

Dual-Port Memory on FPGA

Chung EPC6055 38

• Refer this file, you can exactly instantiate your memory.

You don’t care after this line.

Dual-Port Memory on FPGA

Chung EPC6055 39

• We made 2 ports memory.• Lastly, fill the blank for your project.