community works for muli core embedded image processing

multi-core embedded image processing

Nov. 2, 2013 KELP

Jeong-pyo Kong / ntekLogic jpkong@gmail.com

Community works for

Agenda

1. Why we do this?

2. Technical backgrounds

3. Coworking with ETRI, KESSIA

4. Current our works

5. Future of our works

OpenSEED imageA 2

1. WHY WE DO THIS?

OpenSEED imageA 3

IT trends are changing quite quickly

OpenSEED imageA 4

OpenSEED imageA 5

http://www.wired.com/magazine/2013/04/convergence/

40 Gadgets

8 Imaging

6 Gaming

α

Convergence

Connected world

OpenSEED imageA 6

http://press.teleinteractive.net/media/blogs/tialife/InternetofThingsVector.svg

IoT/M2M Big data

Wearable

Computational world

OpenSEED imageA 7

UAV Robot

Autonomous Vehicle

OpenSEED imageA 8

http://www.slideshare.net/robman/divergence-what-happens-after-convergence

Divergence

Appropriate Technology for IT business environment

OpenSEED imageA 9

Problem solving

• Complex problem -> Complex solution

• Complex problem -> Simple solution

• Simple problem -> Complex solution

• Simple problem -> Simple solution

OpenSEED imageA 10

OpenSEED imageA 11

http://www.youtube.com/watch?v=nrWfLAh2T3k

What we thought

• Interesting image processing and connected world

• Let’s assume if we can do below things – Using improved hardware

– Improving (for example) • Response time to 30%

• Power saving to 15%

• Accurate rate to 10%

OpenSEED imageA 12

Isn’t it a big deal?

Is it a chance for us?

OpenSEED imageA 13

• Many changes are

hidden from

ordinary people,

usually

• Next generation infra

• Eco system

So, what is the result of our works?

OpenSEED imageA 14

2. TECHNICAL BACKGROUNDS

OpenSEED imageA 15

1. why we do this?

Requirements for embedded image processing

OpenSEED imageA 16

System components

• Input – Image sensor – Video files on local storage – Network streaming

• Processing – ISP – Low/high level image processing application

• Output – Display, Signal – Network streaming

OpenSEED imageA 17

Requirements

• Low power

• High performance

• Easy to release : Productivity

• Easy to use for user : UX/UI

• Compatibility

OpenSEED imageA 18

How can we meet these requirements

OpenSEED imageA 19

Hardware based approach(1)

• Silicon gate length : e.g. 20nm

• Architecture : e.g. RISC

• Instruction processing : e.g. ARM NEON

OpenSEED imageA 20

Hardware based approach(2)

• CPU trend : Operating Frequency

OpenSEED imageA 21

Hardware based approach(3)

• Multi-core

OpenSEED imageA 22

Processor 1

Processor 3 Processor 4

Processor 2

General Purpose

Processor 1

General Purpose

Processor 2 DSP

GPU

Homogeneous

Architecture

Heterogeneous

Architecture

Software based approach

• Efficient algorithm

• Multi-core framework

– OpenMP

– OpenCL(WebCL)

– CUDA

• Efficient application design with multi-

core framework

OpenSEED imageA 23

Heterogeneous multi-core system application

OpenSEED imageA 24

For desktop

• Games

• Web acceleration

• UHD video : 4K, 8K

• Computer graphics, 3D, authoring tools

• Financial

OpenSEED imageA 25

For embedded system

• General purpose – Games

– Web acceleration

– UHD video : 4K, 8K

• Specialized purpose – Computer vision

– Robot

– Autonomous vehicles, UAV

– Medical imaging

OpenSEED imageA 26

Embedded multi-core image processing

application framework

OpenSEED imageA 27

•

OpenSEED imageA 28

HSA (Heterogeneous System Architecture)

Current Future

Desktop, smart device application

http://www.tomshardware.com/reviews/fusion-hsa-opencl-history,3262-8.html

OpenCL(1)

• Low level hardware abstraction

• Supporting two parallelism methods

– Data parallelization

– Task parallelization

• User is in charge of optimization

– Engineer needs to know big picture

– Regression testing…

OpenSEED imageA 29

• Programming Model

OpenCL(2)

OpenSEED imageA 30

• Memory

model

OpenCL(3)

OpenSEED imageA 31

OpenCV

• Image processing library

• OpenCV is being worked on supporting discrete GPU(nVidia, AMD) with x86 only, currently

• OpenCV doesn’t support ARM OpenCL (Planed to release in near future)

OpenSEED imageA 32

3. COWORKING WITH ETRI, KESSIA

OpenSEED imageA 33

2. technical backgrounds

ETRI providing

• Platform hardware

– Target board

• Platform software

– Kernels

– Middleware

– Development tools

• Community base

– OpenSEED

OpenSEED imageA 34

ETRI roadmap

OpenSEED imageA 35

ETRI

KESSIA providing

• Making interconnection between ETRI

and community

• Community operation

– OpenSEED

OpenSEED imageA 36

4. CURRENT OUR WORKS

OpenSEED imageA 37

3. coworking with ETRI, KESSIA

imageA = 이미자?

OpenSEED imageA 38

imageA = image processing Acceleration

OpenSEED imageA 39

First Trial(1)

• From ETRI – Hardware : Exynos 4412 based development

board. Mali-400 GPU (http://www.hardkernel.com/renewal_2011/products/prdt_info.php?g_code=G133888637376)

– Kernel : high performance

resource management

kernel

• Long journey for the works OpenSEED imageA 40

First Trial(2)

• Our activities

– Referring user manuals in OpenSEED

– Kernel porting

– OpenCV library porting

– OpenCV test application development

OpenSEED imageA 41

First Trial(3)

• How about now?

OpenSEED imageA 42

Second Trial

• From ETRI

– Hardware : x86 based AMD SBC

– Kernel

• Our activities (~2013.11)

– OpenCV library

– OpenCV test application development

– Performance test

OpenSEED imageA 43

5. FUTURE OF OUR WORKS

OpenSEED imageA 44

4. current our woks

First Trial

• From ETRI – Hardware : x86 based AMD SBC

– Kernel

– Development tools

• Our activities – OpenCV test application development

– OpenCL test application

– Performance test

OpenSEED imageA 45

Second Trial

• From ETRI – Hardware : ARM based development board

– Kernel

– Development tools

• Our activities – OpenCL test application

– OpenCV test application development

– Performance test

OpenSEED imageA 46

Further Trial

• From ETRI – Hardware : ARM based development board

– Kernel

– Development tools

• Our activities – Existing activities

– + Specific application

– + For smart devices

– + Connected application : streaming and so on

OpenSEED imageA 47

Agenda, again

1. Why we do this?

2. Technical backgrounds

3. Co-working with ETRI, KESSIA

4. Current our works

5. Future of our works

OpenSEED imageA 48

Thank you!