![Page 1: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/1.jpg)
Design of image
denoising circuit
成功大學 資訊工程系
陳 培 殷
![Page 2: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/2.jpg)
Outline
Embedded System
What CS Students Can Do in Embedded
System Design
Example -Image Denoising
![Page 3: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/3.jpg)
PCB
PC: a general-purpose computing system
PC
Pentium
![Page 4: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/4.jpg)
PCBEmbedded System: a special-purpose computing system
Most embedded systems are designed for
1. special purpose (customizing and non-programmable)
2. real time applications 3. stable applications
4. automatic applications
Embedded System (1/2)
![Page 5: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/5.jpg)
uPUART
MPEG
ROM
RAM
Embedded System (2/2)
Traditional embedded systems use low-level processors only.
ARM PCI
MPEGUSBFLASH
ROM
RAM
DSPAMBA
Advanced embedded systems
Multi-core
![Page 6: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/6.jpg)
Application Everywhere!
![Page 7: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/7.jpg)
Applications
Information Appliances (IA):
1. Smart phone, VOIP
2. Digital TV, set-top box
3. PlayStation
4. PDA, mp3 player
5. Camera, DV
6. Air-conditioner, microwave oven, refrigerator,
vacuum cleaner, sensor network
7. Motorcycles
8. Car (abs, engine firing, air bag) >100 processors
9. … Ubiquitous computing (many computers for everyone)
![Page 8: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/8.jpg)
Requirements
1. Friendly user interface
2. Short time-to-market
3. Real-time (Speed)
4. Cost
5. Power consummation/dissipation
(cooling strategy and battery life)
![Page 9: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/9.jpg)
Synthesis
Applications
傳輸距離: 100 m功率消耗: 2 W功能:影像傳輸、 語音傳輸
螢 幕: 176 x 220 pixel 65535 色 1.8 吋 TFT
其他
傳輸距離: 100 m功率消耗: 2 W功能:影像傳輸、 語音傳輸
螢 幕: 176 x 220 pixel 65535 色 1.8 吋 TFT
其他
Specifications
Hardware:CPU,
RAM, I/O…
Hardware:CPU,
RAM, I/O…
Software:C,C++
Software:C,C++
System Design
always @(posedge clk) begin if (sel1) begin out=in1; else out=in2; endend
always @(posedge clk) begin if (sel1) begin out=in1; else out=in2; endend
Component Design
Layout
Placement & RoutingFabrication
Marketing
System Development Flow
Testing
![Page 10: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/10.jpg)
What CS students can do in
Embedded System Design?
My personal Views: 1. Hardware Design - Chip
2. Hardware Design - System
3. Related Tools Design
4. Firmware Design
5. Software Design
6. ESL Design
![Page 11: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/11.jpg)
Design Flow
Specification
System Architecture
Hardware Design Software Design
System Integration
System Verification/Testing
Hardware/software partition is
very difficult !!! (cost time)
![Page 12: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/12.jpg)
IC Industry in Taiwan
晶粒測試及切割
晶圓切割
設計
導線架
測試 封裝 製造 光罩
晶圓
邏輯設計 封 裝
化學品
成品測試光罩設計
長晶
![Page 13: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/13.jpg)
ARM PCI
MPEGUSBFLASH
ROM
RAM
DSPAMBA
Hardware Design -- Chip (1/3)
ASIC
The basic design flow for digital cell-based ASIC
Describe circuits with hardware description language
(HDL 硬體描述語言 ) VHDL and Verilog
Synthesis ( 合成 ) the circuits ….
application specific integrated circuit (ASIC 晶片 ) such as USB, MPEG, ….
![Page 14: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/14.jpg)
Hardware Design -- Chip (2/3)
always @(IN)begin OUT = (IN[0] | IN[1]) &
(IN[2] | IN[3]);end
OUT
IN[0]IN[1]
IN[2]IN[3]
Example:
….
….
![Page 15: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/15.jpg)
Hardware Design -- Chip (3/3)
always @(…) if (a==b) if (c==1) d=f; else d=1; else d=0; a
b
c
f
d
fca
b
d
Translate into Boolean Representation
Optimize + MapHDL Source
Target Technology
Synthesis = Translation+Optimization+Mapping
Process of logic synthesis
![Page 16: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/16.jpg)
Hardware Design -- System
ARM PCI
MPEGUSBFLASH
ROM
RAM
DSPAMBA
ASIC
Input devices: keyboard, touch screen, switch, button, ..
Output devices: monitor, LCD, LED, …
Extended devices: compact flash card (CF), PCMCIA, SD
(for storage, wireless communication, I/O)
Power system:
Transmission Interface: PCI, USB, IEEE 1394, UART, bluetooth…
Bus: AMBA (Advanced Microcontroller Bus Architecture)
Input devices
Output devices
![Page 17: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/17.jpg)
Related Tools Design
晶粒測試及切割
晶圓切割
設計
導線架
測試 封裝 製造 光罩
晶圓
邏輯設計 封 裝
化學品
成品測試光罩設計
長晶
![Page 18: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/18.jpg)
Firmware Design
ARM PCI
MPEGUSBFLASH
ROM
RAM
DSPAMBA
ASIC
Devices drivers for I/O devices, extended devices, transmission interface
Assembly codes and C codes for some dedicated CPUs (ARM, 8086,..)
Architectures and instruction sets of different CPUs, DMA,…
Input devices
Output devices
![Page 19: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/19.jpg)
Software Design
ARM PCI
MPEGUSBFLASH
ROM
RAM
DSPAMBA
ASIC
Input devices
Output devices
Embedded OS: WinCE, Palm OS, uC/OS, Linux, JAVA
Real time OS (time) as small as possible (memory)
Distributed embedded system (+ fault tolerance)
Application Software:
wireless communication, network, multimedia,
health, convenience, Web, ….
Porting a customized embedded system to different
machines is very difficult (need large modification)
![Page 20: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/20.jpg)
FutureChip: tens of millions of transistors or more (.35, .25, .18, .09)
Design shifts from ASIC/board to system
System on a Board(printed circuit board)
System on a chip
uP FPGA
MPEGASIC
ATMROM
ROM
SW SW
SWSWPCB
uP Core SRAM
ROM
ATMMPEG
ROM
FPGA
Glue Logic
A/D Block
PCB
SOCSystem-on-a chip is possible
(the whole system is
built in a single chip)
![Page 21: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/21.jpg)
SOC is industry trend
![Page 22: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/22.jpg)
Example: Mobile Phone
Voice only; 2 processors 4 year product life cycle Short talk time
Yesterday
Voice, data, video, SMS <12 month product life cycle Lower power; longer talk time
Today
• 5~8 Processors
• Memory• Graphics• Bluetooth• GPS• Radio• WLAN
Single Chip
DSP
Radio
FlashMemory
Processor
Source: EI-SONICS
![Page 23: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/23.jpg)
What’s ESL tools ?
Describe a SOC design in a more abstract way.
Provide virtual prototypes for HW and SW implementation.
Help system integration in early design phase.
Help to verify the result after job partition.
Claim that design and verification can be done in minutes.
Electronic System Level (ESL) Tools
Suitable for students in CS
![Page 24: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/24.jpg)
Hardware Software Co-Simulation
SoC Designer
![Page 25: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/25.jpg)
Example: Image Denoising
Noise arises During image acquisition
CCD camera, light levels, sensor temperature During image transmission
Interference in the transmission channel
NoiseDetection
NoiseRemoval
Noise Suppression Process
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
Hardware Design – Chip !
![Page 26: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/26.jpg)
Drawbacks of conventional algorithms Heavy computation loading Iterations
Large computation loading Additional memory space
Our goals: Cost-effective Good performance Suitable for VLSI implementation Real-time requirement
Motivation
![Page 27: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/27.jpg)
Low-Complexity Algorithms
Embedded System
1. Power consumption
2. Computing time (real time vs. high quality)
VLSI Implementation
1. Circuit cost
2. Circuit speed
3. Power consumption
Low-Complexity algorithms are necessary for
embedded systems and VLSI implementations.
![Page 28: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/28.jpg)
Issues for VLSI Implementation
Circuit Cost fixed point vs. floating point
less complexity, moderate quality vs. high complexity, high quality
+, - vs. *, / 2*x left shift x 5*x 4*x + x
Functional units sharing
Circuit Speed Pipeline architecture
Delay reduction
Power Consumption Sleep
Shut down partially
![Page 29: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/29.jpg)
Cost-Effective Denoising Method (1/4)
1.Find Min & Max in the mask
3.Check Suspected Noise
4.Choose eligible & representativeedge directions
5.Find the minimum difference
6.Reconstruct the corrupted pixel
2.Utilize Min & Max info.to detect noise
Noise Detection Noise Removal
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
![Page 30: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/30.jpg)
Check suspected noise
Case 1 Case 2 Case 3 Case 4
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
Cost-Effective Denoising Method (2/4)
![Page 31: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/31.jpg)
Choose eligible & representative edge directions
Case 1 Case 2 Case 3 Case 4
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
Cost-Effective Denoising Method (3/4)
![Page 32: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/32.jpg)
Find the minimum difference and reconstruct the noisy pixel
Case 1
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
Cost-Effective Denoising Method (4/4)
![Page 33: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/33.jpg)
Architecture of the Denoising Chip
Line Buffer 1
Line Buffer 2
ShiftRegister
Edge Detection Unit
Min & Max Computation
Unit
Noise Detection
Unit
Image Out
Image In
Output Selection Unit
![Page 34: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/34.jpg)
Shift Register Bank
R0R1R2
R3R4R5
R6R7R8
Line Buffer
Line Buffer1st Raw2nd Raw
1st Raw 1st Raw2nd Raw
1st Raw
3rd Raw
![Page 35: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/35.jpg)
DataflowMax & Min
Computation Unit
Noise Detection Unit
Edge Detection Unit Output Selection Unit
![Page 36: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/36.jpg)
Stage 1
Stage 2
Stage 3
Stage 4
Pipeline Stages
![Page 37: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/37.jpg)
less flexible, long design cycle, larger-scale production to reduce price
more flexible, shorter design cycle, suitable for smaller production
FPGA or CPLD
Real ASIC chipStandard cellStandard cell
PLDPLD
Fab (TSMC, UMC, ..)
Xilinx, Altera
ASIC vs. FPGA
Circuit Realization
![Page 38: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/38.jpg)
Demo (1/3)
Source ImageSending Image
Denoising Circuit
FPGA Board
Receiving Image
Result Image
Software Program
Hardware PlatformPC
Hardware Software Co-Simulation/Verification
USB
PC
![Page 39: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/39.jpg)
Demo (2/3)
![Page 40: Design of image denoising circuit 成功大學 資訊工程系 陳 培 殷](https://reader038.vdocuments.site/reader038/viewer/2022102718/56649d6b5503460f94a4a85d/html5/thumbnails/40.jpg)
Demo (3/3)