fpga implementation image processing using verilog

FPGA IMPLEMENTATION IMAGE PROCESSING USING

VERILOG

E2MATRIX RESEARCH LABOpp Phagwara Bus Stand, Backside Axis Bank, Parmar ComplexPhagwara, Punjab ( India ). Contact : +91 9041262727Web : www.e2matrix.com Email : [email protected]

http://www.e2matrix.com/







PRESENTATION OBJECTIVES

Prerequisites Motivations for using FPGAs in RC and HPC HPC and RC FPGA systems hardware and infrastructure

Objectives HPC algorithms and Considerations for Reconfigurable Computing (RC) Share a perspective on the State-of-the-Art for C-based HW design Describe the C to FPGA Flow Illustrate with code examples … Look forward to some critical debate…

AGENDA

Reconfigurable Computing Considerations, core algorithm relationships, commercial

applications C-based design

The solution space (its place in EDA) Nature of C for HW design

The Design Flow Summary JPEG2000 Design Example




AGENDA

Reconfigurable Computing (RC) Considerations, core algorithm relationships, commercial applications

C-based design The solution space (its place in EDA) Nature of C for HW design

The Design Flow Summary “RC = Using FPGAs for (algorithmic) computation”

1. Embedded: Well established – body of knowledge/experience 2. Enterprise: Some3. HPC: Starting Out

RECONFIGURABLE COMPUTING

Promised Opportunities Algorithm Acceleration

Exploit parallelism to increase performance with custom HW implementation Algorithm Offload

Free CPU resource by offloading bottleneck processes

BIG Challenges Development complexity

Design framework and methods, deployment and integration/middleware Coupling to coprocessor/data bandwidth Price/Performance/Power! Choosing the right applications!

1980 1990 2000 20X0?

Commercial C-to-FPGA tools

FPGAs

Closely Coupled SystemsPartitioning Frameworks

Intimately Coupled SystemsAdvanced CompilersFirst RC Successes

FPGA COMPUTING AND METHODOLOGY High Performance Embedded and Reconfigurable Computing

Why FPGA Computing? Moore’s Law showing signs of strain Ability to parallelize in HW Price/GOPS coming down rapidly Hard IP blocks – excellent density

Example: Floating Point Performance Maximum for Virtex-4 – 50 GFLOPS (Courtesy of Dave Bennett, Xilinx Labs) Maximum for Virtex-2 – 17.5 GFLOPS “ “ “ “ “ “ “Can fit 10’s of FPUs on 2 Xilinx Virtex-4’s” (Courtesy of Justin Tripp, LANL) Use of hard macros for functions is mandatory (example DSP48 on Virtex-4)

C-based design for FPGAs Several offerings on commercial marketplace or in research

Commercial – Celoxica, Mentor Graphics, Impulse Technologies, Mitrion… Research – Sandia, UC Riverside, LANL

RTL/HDL is the most widely used way to get to FPGAs but is not usable by SW engineers

CONVENTIONAL WISDOM FOR RC

1. Small data objects Data transfer overhead to coprocessor, High operation to byte ratio

2. Modest arithmetic Difficult to design and implement complex algorithms in HW Integer/fixed precision calculations Floating point too resource expensive

3. Data-parallelism Parallelism essential - FPGA clocks order of magnitude slower than CPUs Fine grain - wide data widths Medium grain - operation/function routine Course grain - multiple instantiations of application processes

4. Pipeline-ability Streaming Applications – most successful

5. Simple Control Difficult to design complex scheduling schemes in Parallel HW

FURTHER CONSIDERATIONS

6. Exploiting “Soft” programmable HW Configurable Applications

Schedule and load HW content prior to HW execution Reconfigurable Applications

Dynamically change HW content during HW execution

COMMERCIAL RC APPLICATIONS Well established in embedded systems:

Digital Video Technology and Image Processing “PROCESSING AT THE SENSOR” versus local and/or remote processing 3D LCD display development and test Real-time verification of HDTV image processing algorithms Robust image matching - product tracking and production line control

Digital Signal Processing Engine control unit for 3-phase motors Radar and sonar beamforming and spatial filtering Computer aided tomography security system

Communications and Networking Internet reconfigurable multimedia terminal, MP3, VoIP etc. Ground traffic simulation testbed for broadband satellite network communications Satellite based Internet data tracking system

Rapid Systems Prototyping Automotive safety system incorporating sensor fusion Robotic vision system for object detection and robot guidance

Defense & Security

Consumer Automotive & Industrial

…using C-based design

COMMERCIAL RC APPLICATIONS Enterprise Computing

Content processing solutions XML parsing, virus checking Packet/Pattern Matching/Filtering Compression/decompression Security/Encryption – DES/3-DES, SHA, MD5, AES/Rijndael

High Performance Computing Image processing

CT scan analysis, 3D modeling, Ray Tracing Finite element analysis and simulation Custom Vector Engines Genome calculations Seismic data processing

…using C-based design

CFD

FourierMethods

n-body

GraphTheoretic

RasterGraphics

DiscreteEvents

PatternMatching

SymbolicProcessing

MonteCarlo

Transport

PDE

ODE

Fields

BasicAlgorithms

&NumericalMethods

Combustion

Structural Mechanics

Multibody Dynamics

Electromagnetics

Geophysical Fluids

Weather and Climate

Aerodynamics

Reservoir Modelling

Ecosystems

CVD

Plasma Processing

Astrophysics

Seismic Processing

Cloud Physics

Chemical Reactors

Boilers

Chemical Reactors

Magnet Design

Economics Models

Phylogenetic Trees

Electrical Grids

Pipeline Flows

Distribution Networks Biosphere/Geosphere

Neural NetworksCrystallographyTomographic Reconstruction

MRI ImagingDiffractionInversionProblems

Signal Processing

Condensed MatterElectronic Structure

RationalDrug Design

Biomolecular Dynamics

Nanotechnology

DataAssimilation

Chemical Dynamics Atomic

Scattering

ActinideChemistry

FractureMechanics

CosmologyAstrophysics

Orbital Mechanics

MilitaryLogistics

Manufacturing Systems

Population Genetics

Air TrafficControl

TransportationSystems Economics

VLSI Design

QCD Nuclear StructureNeutronTransport

VirtualReality

VirtualPrototypes

ComputationalSteering

ScientificVisualization

MultimediaCollaborationTools

GenomeProcessing

ComputerVision

Databases

Data Mining

Cryptography

IntelligentSearch

ComputerAlgebra

Number TheoryAutomatedDeductionIntelligent

AgentsCAD

Molecular Modeling

Electronic Structure

Quantum Chemistry

Flow in Porous Media

Radiation Reaction-Diffusion

Multiphase Flow

Source: Rick Stevens - ANL

Core Algorithm Relationships in HPC

CFD

FourierMethods

n-body

GraphTheoretic

RasterGraphics

DiscreteEvents

PatternMatching

SymbolicProcessing

MonteCarlo

Transport

PDE

ODE

Fields

BasicAlgorithms

&NumericalMethods

Combustion

Structural Mechanics

Multibody Dynamics

Electromagnetics

Geophysical Fluids

Weather and Climate

Aerodynamics

Reservoir Modelling

Ecosystems

CVD

Plasma Processing

Astrophysics

Seismic Processing

Cloud Physics

Chemical Reactors

Boilers

Chemical Reactors

Magnet Design

Economics Models

Phylogenetic Trees

Electrical Grids

Pipeline Flows

Distribution Networks Biosphere/Geosphere

Neural NetworksCrystallographyTomographic Reconstruction

MRI ImagingDiffractionInversionProblems

Signal Processing

Condensed MatterElectronic Structure

RationalDrug Design

Biomolecular Dynamics

Nanotechnology

DataAssimilation

Chemical Dynamics Atomic

Scattering

ActinideChemistry

FractureMechanics

CosmologyAstrophysics

Orbital Mechanics

MilitaryLogistics

Manufacturing Systems

Population Genetics

Air TrafficControl

TransportationSystems Economics

VLSI Design

QCD Nuclear StructureNeutronTransport

VirtualReality

VirtualPrototypes

ComputationalSteering

ScientificVisualization

MultimediaCollaborationTools

GenomeProcessing

ComputerVision

Databases

Data Mining

Cryptography

IntelligentSearch

ComputerAlgebra

Number TheoryAutomatedDeductionIntelligent

AgentsCAD

Molecular Modeling

Electronic Structure

Quantum Chemistry

Flow in Porous Media

Radiation Reaction-Diffusion

Multiphase Flow

Source: Rick Stevens - ANL

Core Algorithm Relationships in HPC

How do we map out the right Apps?

EXPLOITING FPGA IN HPC Hardware:

“Enterprise Quality” co-processor system products (Cray XD1, SGI RASC) Robust PCI/PCIx/VME-based FPGA card solutions for development

A software design methodology is essential:

SW dominated application sector Target developers have a SW background Register Transfer Level (RTL), Hardware Description Languages (HDL) are foreign

Complete designs can be specified in a C environment Porting to HW implementations simplified

Platform abstractions through API’s and Libraries Simplified Specification, Development, Deployment

How do we select and benchmark?

AGENDA



The solution space (its place in EDA – Electronic Design Automation) Nature of C for HW design


EMBEDDED HARDWARE (HW) DESIGN

Function

Architecture

Implementation

Physical Design

Algorithm Design

Block Design

RTL

ArchitectureExploration

Specification

Design Analysis

Interface Synthesis

Custom Processors

Fast Mixed Simulation

HW Accelerated Simulation

Fixed Point extraction

HLL Synthesis

Implementation IP ModelsTLM Frameworks

DSP IP

Reconfigurable Prototypes

Emulation Platforms

Implementation IP

RTL Verification

Algorithm Design

Block Design

RTL


Design Analysis

Interface Synthesis

Custom Processors

Mixed Simulation

HW Accelerated Simulation

Fixed Point extraction

HLL Synthesis

Implementation IP ModelsTLM Frameworks

DSP IP

Reconfigurable Prototypes

Emulation Platforms

Implementation IP

RTL Verification

Algorithm Design


C-Based Synthesis

API’s/Libraries

FPGA/SoPC

C to FPGA/SoPC

C TO FPGA ACCELERATED SYSTEM

Algorithm Design

EDIF

FPGA

Function & Architecture

Implementation

Mixed Simulation

C for HWCAC/C++AL

API’s/Libraries

OBJ

Processor

SoftwareModel

Specification Model

TestbenchDesign

HW SW

Partitioning

System Model

Design Analysis Optimization

P&RSynthesis

RTL

C-Based Synthesis


BSPBSP

COMMS

CHALLENGES FOR C-BASED SYNTHESIS

Concurrency (Parallelism) Compiler-determined (behavioral synthesis) Explicit

Timing Constraints Explicit Rules-based

Data Types Annotations, additional or C++

Communication Additional or C-like

TWO APPROACHES TO C-BASED DESIGN

ANSI/ISO C Language Standard

par{…}, seq{…}, Interfaces, Channels,

Bit Manipulation,RAM & ROM

Single cycle assignment

Bits and bit-vectorsArbitrary width integers

Signals

Core LibrariesTLM (PAL/DSM), Fixed/Floating point …

Handel-C

ANSI/ISO C++ Language Standard

Modules, Ports, Processes, Events,

Interfaces, ChannelsEvent Driven Sim Kernel

4-valued logic/vectorsBits and bit-vectors

Arbitrary width integersFixed-point

C++ user-defined types

Signal, Timer, Mutex, Semaphore, FIFO, etcPrimitive Channels

Kahn Process Networks, Static Dataflow…Standard Channels for Various MOC

Core LibrariesSCV, TLM, Master/Slave …

SystemC

Core Language Core Language Data TypesData Types

C Algorithm to FPGA SoC (System-on-a-Chip) Prototyping/Verification

AGENDA





SYSTEM DESIGN REFINEMENTFunction

par{ processA(…); processB(…); processC(…); processD(…); }

void processD(…){ unsigned 9 a,b,c; par{ a=1; b=2; } c=3; };

A B

C D CP

CA

C/C++

Handel-C

• System Function• Course grain parallelism

• Parallel algorithm design• Fine-grain parallism • Bit/cycle true processes• Algorithm Testbench

AL

Handel-C

Architecture• Add interfaces• Signal/cycle accurate test

A B

C D

void main(){ interface port_in… interface port_out… … }

CA Handel-C

EDIF/RTL

A B

C D

SYSTEMS INTEGRATIONImplementation• Complete system design• Interface to pins• Multi-Clock domain• IP Integration

A

C

RTL from HDL IP

A B

C D

D

CLKRST

Data

B

EDIF (Electronic Design Interface Format)

set clock = external “CLK”;set reset = external “RST”;interface Data(…)…void main() { par{ processA(…); processB(…); processC(…); processD(…); }}

{ interface processD(…)…};

{ interface processB(…)…};

EDIF/RTL

PARALLEL DEBUG IN C ENVIRONMENT

Algorithm Design

RESOURCE USAGE/SPEED ESTIMATIONS


FPGA SUPPORT

Technology mappingOptimizations

HANDEL-C TEMPLATE MULTIPLIERset clock = external "clk";void main(){

…while(1) par{

… process();}

}

void process(){ unsigned W A, B, C;

while(1) par { … Multiply(A, B, &C); … }}

void Multiply(unsigned W A, unsigned W B, unsigned W *C){ static unsigned W a[W], b[W], c[W]; par{ a[0] = A; b[0] = B; c[0] = a[0][0] == 0 ? 0 : b[0]; par (i = 1; i < W; i++) { a[i] = a[i-1] >> 1; b[i] = b[i-1] << 1; c[i] = c[i-1] + (a[i][0] == 0 ? 0 : b[i]); } *C = c[W-1]; }}

Pipelined

AGENDA





SUMMARY Commercial C-based design is a reality For the HPC and RC communities it offers:

Fastest route to accelerating SW designs in FPGA Lower barrier to adoption than RTL technologies Greater customization and productivity than block based approaches Complete integration with RTL/block based approaches for “Power users”

Deterministic and quality results State of the art tools used by embedded systems designers

RC platforms for rapid prototyping Simple migration, development to deployment with full library support

DESIGN EXAMPLEJPEG2000 Image Compression Algorithm

EXAMPLE DESIGNFive Steps to HW Platform:

1. Specification Model Algorithm Profiling

2. Functional System Model System Estimations

3. Architecture and Communication Model Optimization

4. Implementation Model Direct Synthesis C to EDIF

5. HW Platform Board level integration

JPEG 2000 Compressor

Pre processing

RGB to YUVconversion

DWT

Quantization

Tier-2 Encoder

Rate Control

Original Image

Coded Image

Tier-1 Encoder

1. SPECIFICATION MODELFunction & Architecture

SoftwareModel

Specification Model

TestbenchDesign

Pre processing


DWT

Quantization

Tier-2 Encoder

Rate Control

Original Image

Coded Image

Tier-1 Encoder

0

1

2

3

4

5

6

Memory Usage (x86) MB

CurrentSum

Algorithm Profiling- Memory- Processing Time- Data Flow

22 *.c and *.h files1468 lines of code

DWT/Tier1 are the compute intensive blocks

C/C++AL

2. FUNCTIONAL SYSTEM MODELFunction & Architecture

SoftwareModel

Specification Model

TestbenchDesign

HW SW

Partitioning

System ModelPre processing


quantization

Tier-2 Encoder

Rate Control

Original Image

Coded Image

Tier-1 Encoder

DWT

Handel-CCAC/C++AL

/* C */void sw_block(…){

…

}

/*Handel-C*/extern “C” sw_block(…);

void main(void){ while(1) par{

sw_block(…);hw_block(…);

} }

void hw_block(…){ … } Cycles/speed/area…

3. ARCHITECTURE AND COMMUNICATION MODEL

Function & Architecture

Pre processing


quantization

Tier-2 Encoder

Rate Control

Original Image

Coded Image

DWT

Handel-CCAC/C++AL

FIFO

FIFO

DsmPortH2S

DsmRead(…)DsmWrite(…)DsmFlush(…)

Dataflow/Cycles/speed/area…

Tier-1 Encoder

4. IMPLEMENTATION MODEL

EDIFRTL

Implementation

EDIF

Device Family

A B

C D

void main(){ interface port_in… interface port_out… … }

ESTIMATIONS FROM SYNTHESIS DWT ~ 6% VII1000

5. HARDWARE PLATFORM

Implementation

uP HW

RAM

• Microblaze + Xilinx FPGA• Nios + Altera FPGA• Xilinx V2Pro• Toshiba MeP + FPGA• PowerPC + PLB + FPGA• PC + FPGA PCI Card• …etc

uP HW

uP HW

RAM

HW

uP

EDIF

FPGA

P&R

DWTSlices: 758Device utilization : 7%Speed (MHz): 151Lines of code: 395

Implementation Model Estimations

DWT ~6%

From P&R Report for VII1000-4A B

C D

Board Level IntegrationSpecific I/O ImplementationsPin Location constraints

JPEG2000 DWT IMPLEMENTATION Example taken from a “Xilinx Design Challenge”

Comparison made with HDL approach See Article in Xcell Volume 46

http://www.xilinx.com/publications/xcellonline/xcell_46/xc_celoxica46.htm

ObservationsComparable

Using C faster Using C quicker

Expert vs Novice

HDL 8007%12843520*+6 hours

* Doesn’t include partitioning spec.

development

C-Based Design 1st passSlices

646Device utilization

6%Speed (MHz) 110Lines of code 386Design time (days) 6Simulation time

5 mins

2nd pass5465%1303867 (6+1)5 mins

Final7587%1513957 (6+1)20 mins

* Lena used as testbench throughout, input bit width12, max 1K image width

http://www.xilinx.com/publications/xcellonline/xcell_46/xc_celoxica46.htm

> Celoxica 1st PassSlices 1.347Device utilization 12%Speed (MHz) 89.5Lines of code 310Design time (days) 10Simulation time for Lena jpeg 5 mins

JPEG2000 MQ coder ImplementationObservations

HDL Smaller

HC FasterHC Quicker

Expert vs Novice

Celoxica Final1,99918%115.533012 (10+2)5 mins

HDL6206%7680030*Hours

* Doesn’t include partitioning spec.

development

> Common language base eased porting to hardware of the MQ coder source & DSM allowed partition, co verification & data to be moved between hardware & software

> Optimizations included adding parallelism, replacing for() loops with while() loops, & simplifying loop control.

> Design developed in a unified design environment

E2MATRIX RESEARCH LABOpp Phagwara Bus Stand,

Backside Axis Bank, Parmar ComplexPhagwara, Punjab ( India ).Contact : +91 9041262727Web : www.e2matrix.com

Email : [email protected]




fpga implementation image processing using verilog

Education