community works for muli core embedded image processing
DESCRIPTION
TRANSCRIPT
multi-core embedded image processing
Nov. 2, 2013 KELP
Jeong-pyo Kong / ntekLogic [email protected]
Community works for
Agenda
1. Why we do this?
2. Technical backgrounds
3. Coworking with ETRI, KESSIA
4. Current our works
5. Future of our works
OpenSEED imageA 2
1. WHY WE DO THIS?
OpenSEED imageA 3
IT trends are changing quite quickly
OpenSEED imageA 4
OpenSEED imageA 5
http://www.wired.com/magazine/2013/04/convergence/
40 Gadgets
8 Imaging
6 Gaming
α
Convergence
Connected world
OpenSEED imageA 6
http://press.teleinteractive.net/media/blogs/tialife/InternetofThingsVector.svg
IoT/M2M Big data
Wearable
Computational world
OpenSEED imageA 7
UAV Robot
Autonomous Vehicle
OpenSEED imageA 8
http://www.slideshare.net/robman/divergence-what-happens-after-convergence
Divergence
Appropriate Technology for IT business environment
OpenSEED imageA 9
Problem solving
• Complex problem -> Complex solution
• Complex problem -> Simple solution
• Simple problem -> Complex solution
• Simple problem -> Simple solution
OpenSEED imageA 10
OpenSEED imageA 11
http://www.youtube.com/watch?v=nrWfLAh2T3k
What we thought
• Interesting image processing and connected world
• Let’s assume if we can do below things – Using improved hardware
– Improving (for example) • Response time to 30%
• Power saving to 15%
• Accurate rate to 10%
OpenSEED imageA 12
Isn’t it a big deal?
Is it a chance for us?
OpenSEED imageA 13
• Many changes are
hidden from
ordinary people,
usually
• Next generation infra
• Eco system
So, what is the result of our works?
OpenSEED imageA 14
2. TECHNICAL BACKGROUNDS
OpenSEED imageA 15
1. why we do this?
Requirements for embedded image processing
OpenSEED imageA 16
System components
• Input – Image sensor – Video files on local storage – Network streaming
• Processing – ISP – Low/high level image processing application
• Output – Display, Signal – Network streaming
OpenSEED imageA 17
Requirements
• Low power
• High performance
• Easy to release : Productivity
• Easy to use for user : UX/UI
• Compatibility
OpenSEED imageA 18
How can we meet these requirements
OpenSEED imageA 19
Hardware based approach(1)
• Silicon gate length : e.g. 20nm
• Architecture : e.g. RISC
• Instruction processing : e.g. ARM NEON
OpenSEED imageA 20
Hardware based approach(2)
• CPU trend : Operating Frequency
OpenSEED imageA 21
Hardware based approach(3)
• Multi-core
OpenSEED imageA 22
Processor 1
Processor 3 Processor 4
Processor 2
General Purpose
Processor 1
General Purpose
Processor 2 DSP
GPU
Homogeneous
Architecture
Heterogeneous
Architecture
Software based approach
• Efficient algorithm
• Multi-core framework
– OpenMP
– OpenCL(WebCL)
– CUDA
• Efficient application design with multi-
core framework
OpenSEED imageA 23
Heterogeneous multi-core system application
OpenSEED imageA 24
For desktop
• Games
• Web acceleration
• UHD video : 4K, 8K
• Computer graphics, 3D, authoring tools
• Financial
OpenSEED imageA 25
For embedded system
• General purpose – Games
– Web acceleration
– UHD video : 4K, 8K
• Specialized purpose – Computer vision
– Robot
– Autonomous vehicles, UAV
– Medical imaging
OpenSEED imageA 26
Embedded multi-core image processing
application framework
OpenSEED imageA 27
•
OpenSEED imageA 28
HSA (Heterogeneous System Architecture)
Current Future
Desktop, smart device application
http://www.tomshardware.com/reviews/fusion-hsa-opencl-history,3262-8.html
OpenCL(1)
• Low level hardware abstraction
• Supporting two parallelism methods
– Data parallelization
– Task parallelization
• User is in charge of optimization
– Engineer needs to know big picture
– Regression testing…
OpenSEED imageA 29
• Programming Model
OpenCL(2)
OpenSEED imageA 30
• Memory
model
OpenCL(3)
OpenSEED imageA 31
OpenCV
• Image processing library
• OpenCV is being worked on supporting discrete GPU(nVidia, AMD) with x86 only, currently
• OpenCV doesn’t support ARM OpenCL (Planed to release in near future)
OpenSEED imageA 32
3. COWORKING WITH ETRI, KESSIA
OpenSEED imageA 33
2. technical backgrounds
ETRI providing
• Platform hardware
– Target board
• Platform software
– Kernels
– Middleware
– Development tools
• Community base
– OpenSEED
OpenSEED imageA 34
ETRI roadmap
OpenSEED imageA 35
ETRI
KESSIA providing
• Making interconnection between ETRI
and community
• Community operation
– OpenSEED
OpenSEED imageA 36
4. CURRENT OUR WORKS
OpenSEED imageA 37
3. coworking with ETRI, KESSIA
imageA = 이미자?
OpenSEED imageA 38
imageA = image processing Acceleration
OpenSEED imageA 39
First Trial(1)
• From ETRI – Hardware : Exynos 4412 based development
board. Mali-400 GPU (http://www.hardkernel.com/renewal_2011/products/prdt_info.php?g_code=G133888637376)
– Kernel : high performance
resource management
kernel
• Long journey for the works OpenSEED imageA 40
First Trial(2)
• Our activities
– Referring user manuals in OpenSEED
– Kernel porting
– OpenCV library porting
– OpenCV test application development
OpenSEED imageA 41
First Trial(3)
• How about now?
OpenSEED imageA 42
Second Trial
• From ETRI
– Hardware : x86 based AMD SBC
– Kernel
• Our activities (~2013.11)
– OpenCV library
– OpenCV test application development
– Performance test
OpenSEED imageA 43
5. FUTURE OF OUR WORKS
OpenSEED imageA 44
4. current our woks
First Trial
• From ETRI – Hardware : x86 based AMD SBC
– Kernel
– Development tools
• Our activities – OpenCV test application development
– OpenCL test application
– Performance test
OpenSEED imageA 45
Second Trial
• From ETRI – Hardware : ARM based development board
– Kernel
– Development tools
• Our activities – OpenCL test application
– OpenCV test application development
– Performance test
OpenSEED imageA 46
Further Trial
• From ETRI – Hardware : ARM based development board
– Kernel
– Development tools
• Our activities – Existing activities
– + Specific application
– + For smart devices
– + Connected application : streaming and so on
OpenSEED imageA 47
Agenda, again
1. Why we do this?
2. Technical backgrounds
3. Co-working with ETRI, KESSIA
4. Current our works
5. Future of our works
OpenSEED imageA 48
Thank you!