4/6/2012
1
ĐẠI HỌC QUỐC GIA TP.HỒ CHÍ MINHTRƯỜNG ĐẠI HỌC BÁCH KHOA
KHOA ĐIỆN-ĐIỆN TỬ BỘ MÔN KỸ THUẬT ĐIỆN TỬ
11
VIDEO AND IMAGE PROCESSING USING DSP AND PFGA
Chapter 5: Video and image processing system design on FPGA
5.1 Design Process for video and image processing system
5.2 Hardware architecture for video and image processing
5.3 Video and image processing on FPGA
Bộ môn Kỹ Thuật Điện Tử
5.1 Design Process
• Hardware design image processing requires two skills
– Algorithm development
– Hardware design
2
Problem
Specification
Algorithm
Development
System
Implementation
Architecture
Selection
The four main steps of design process
4/6/2012
2
Bộ môn Kỹ Thuật Điện Tử
Problem Specification• A problem specification should clearly describe
– System functionality
• What the system needs to be able to do
• Specify the desired result of image processing.
– Performance of the system
• how well must it perform the functions
• Specify the maximum allowable latency and the number of images
or frames that must be processed every second
• If the decision is binary, the allowable failure rate should be
specified
– System environment
• Lighting, optics
• Input/output interfacing, user interface
• Other important aspects: encoding, decoding, pre-processing
3
Bộ môn Kỹ Thuật Điện Tử
Problem Specification
• Example: Design a video denoising system
4
No. Specification Description
1 System functionality - Function: remove noise in video source,
receive high quality pictures at output
- Algorithm: LMS filtering
2 Performance of the
system
-Process video in real-time
-Throughput: 30 CIF frames per second
-PSNR > 30dB
3 System environment -Input: capture video from S-video
-Output: display video at LCD through VGA port
-Video format: color video 4:2:2 YCbCr
-Including a deinterlacing unit
-User interface: switch ON/OFF for denoising
4/6/2012
3
Bộ môn Kỹ Thuật Điện Tử
Algorithm Development
• Algorithm development
– formulate the process
– consider preprocessing algorithm
– test the algorithm on different images.
• Algorithm structure
– how the operations work together to give a robust algorithm
– Specify physical or environmental constraints
• Hardware development issues
– convert algorithm-level design to hardware-level design
– consider timing constraints
– Specify system-level architecture
5
Bộ môn Kỹ Thuật Điện Tử
Algorithm Development• Example for pre-processing
6
4/6/2012
4
Bộ môn Kỹ Thuật Điện Tử
Algorithm Development
• Example for architecture using CPU embedded within the
FPGA
7
Bộ môn Kỹ Thuật Điện Tử
Assignments
1. Describe problem specification for the design of
text recognition system
2. Describe problem specification for the design of
color enhancing system
8
4/6/2012
5
Bộ môn Kỹ Thuật Điện Tử
Hardware architecture for video and
image processing
• Hardware architecture
– Serial processing
• CPU-based architecture
• DSP-based architecture
– Parallel processing
• Massively parallel architecture
• Pipelining architecture
• Systolic architecture
– Hybrid processing
• DSP-FPGA based architecture
• System-on-chip (SoC) architecture
9
Bộ môn Kỹ Thuật Điện Tử
Massively parallel architecture
• Characteristics:
– highest throughput
– small delay
– heavy resource
10
Operation 1
Operation 2
Operation 3
+
Input Output
4/6/2012
6
Bộ môn Kỹ Thuật Điện Tử
Pipelining architecture
• Pipelining technique
– splits the logic into smaller blocks,
– spread over multiple clock cycles,
– reducing the propagation delay in any one clock cycle.
– This allows the design to be clocked faster in order to meet timing
constraints.
11
one-stage pipeline two-stage pipeline
Bộ môn Kỹ Thuật Điện Tử
Pipelining architecture
• Some problems in pipelining architecture
– imbalance in the propagation delay
– high logic depth due to multiplication and division
– the input data is requested asynchronously
12
4/6/2012
7
Bộ môn Kỹ Thuật Điện Tử
Systolic Architecture
• Systolic architecture is also called Systolic Arrays
• A network of logic blocks compute and pass data through the
system
• Used as a coprocessor in combination with a host computer
and the behavior is analogous to the flow of blood through the
heart; thus named as systolic.
13
Bộ môn Kỹ Thuật Điện Tử
Systolic Architecture
14
4/6/2012
8
Bộ môn Kỹ Thuật Điện Tử
Systolic Architecture
• Systolic Array Design Methodology
15
Bộ môn Kỹ Thuật Điện Tử
Systolic Architecture• Example: design systolic architecture for a FIR filter
16
Dependence graph
Space-time representation
4/6/2012
9
Bộ môn Kỹ Thuật Điện Tử
Assignments
1. Design a median filter using systolic architecture
2. Design an edge detection filter using pipelining architecture
3. Design a Gaussian filter using pipelining architecture
4. Design a motion estimation unit using systolic architecture
5. How to solve the asynchronous problem in pipelining
architecture?
17
Bộ môn Kỹ Thuật Điện Tử
5.3 Video and Image Processing on FPGA
• FPGA: Field Programmable Gate Array
18
Types of programmable logic array.
4/6/2012
10
Bộ môn Kỹ Thuật Điện Tử
FPGA• FPGA manufactures
– Xilinx
– Altera
– Lattice
– Cypress
19
Altera FPGA board Xilinx FPGA board
Bộ môn Kỹ Thuật Điện Tử
Video and image processing system design
using Xilinx FPGA
20
4/6/2012
11
Bộ môn Kỹ Thuật Điện Tử
Video and image processing system design
using Xilinx FPGA
• Automatically generates
HDL/optimized algorithms
– Shortens learning curve
– HW redesign eliminated
– Optimal implementation
21
Bộ môn Kỹ Thuật Điện Tử
Video and image processing system design
using Xilinx FPGA
• Video system design with IP cores
22
4/6/2012
12
Bộ môn Kỹ Thuật Điện Tử
Video and image processing system design
using Xilinx FPGA
• Video AllianceCOREs
23
Bộ môn Kỹ Thuật Điện Tử
Video and image processing system design
using Altera FPGA
• General design flow
24
4/6/2012
13
Bộ môn Kỹ Thuật Điện Tử
Video and image processing system design
using Altera FPGA
• IP-based design flow
25
Bộ môn Kỹ Thuật Điện Tử
Video and image processing system design
using Altera FPGA
• DSP Builder:
– integrates the algorithm development, simulation, and verification
capabilities of The MathWorks MATLAB and Simulink system-level
design tools with the Altera Quartus II software
26
4/6/2012
14
Example design: Smart Camera
Hardware Architecture
• ALTERA Stratix EP1S60F1020C7
• 4Mpixels LUPA-400 image sensor
• (2) 2d accelerometers
• (3) gyroscopes
• 10Mb SRAM
• 64Mb SDRAM
Smart Camera Hardware Architecture
4/6/2012
15
Bộ môn Kỹ Thuật Điện Tử
Assignments
1. Design median filter in Matlab Simulink and convert the
design to HDL by using Altera DSP Builder
2. Design a color correction filter in Matlab Simulink and
convert the design to HDL by using Altera DSP Builder
3. Build a Gaussian filter IP core and integrate it to Altera SOPC
Builder
4. Build a VGA controller IP core and integrate it to Altera SOPC
Builder
5. Build a SOPC system which performs DCT transform for input
image from ROM.
29